Python 爬蟲利器三之 Xpath 語法與用法2022-04-04 18:07:27

安裝

pip install lxml

初步使用

首先我們利用它來解析 HTML 代碼，先來一個小例子來感受一下它的基本用法。

from lxml import etree
text = '''
<div>
    <ul>
         <li class="item-0"><a href="link1.html">first item</a></li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-inactive"><a href="link3.html">third item</a></li>
         <li class="item-1"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a>
     </ul>
 </div>
'''
html = etree.HTML(text)
result = etree.tostring(html)
print(result)

首先我們使用 lxml 的 etree 庫，然后利用 etree.HTML 初始化，然后我們將其打印出來。其中，這里體現了 lxml 的一個非常實用的功能就是自動修正 html 代碼，大家應該注意到了，最后一個 li 標簽，其實我把尾標簽刪掉了，是不閉合的。不過，lxml 因為繼承了 libxml2 的特性，具有自動修正 HTML 代碼的功能。所以輸出結果是這樣的

<html><body>
<div>
    <ul>
         <li class="item-0"><a href="link1.html">first item</a></li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-inactive"><a href="link3.html">third item</a></li>
         <li class="item-1"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
</ul>
 </div>

</body></html>

不僅補全了 li 標簽，還添加了 body，html 標簽。

文件讀取

除了直接讀取字符串，還支持從文件讀取內容。比如我們新建一個文件叫做 hello.html，內容為

<div>
    <ul>
         <li class="item-0"><a href="link1.html">first item</a></li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-inactive"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
 </div>

利用 parse 方法來讀取文件。

from lxml import etree
html = etree.parse('hello.html')
result = etree.tostring(html, pretty_print=True)
print(result)

同樣可以得到相同的結果。

XPath 實例測試

依然以上一段程序為例（1）獲取所有的

標簽

from lxml import etree
html = etree.parse('hello.html')
print type(html)
result = html.xpath('//li')
print result
print len(result)
print type(result)
print type(result[0])

運行結果

<type 'lxml.etree._ElementTree'>
[<Element li at 0x1014e0e18>, <Element li at 0x1014e0ef0>, <Element li at 0x1014e0f38>, <Element li at 0x1014e0f80>, <Element li at 0x1014e0fc8>]
5
<type 'list'>
<type 'lxml.etree._Element'>

可見，etree.parse 的類型是 ElementTree，通過調用 xpath 以后，得到了一個列表，包含了 5 個

元素，每個元素都是 Element 類型（2）獲取

標簽的所有 class

result = html.xpath('//li/@class')
print result

運行結果

['item-0', 'item-1', 'item-inactive', 'item-1', 'item-0']

（3）獲取

標簽下 href 為 link1.html 的標簽

1 2	result = html.xpath('//li/a[@href="link1.html"]') print result

運行結果

1	[<Element a at 0x10ffaae18>]

（4）獲取

標簽下的所有標簽注意這么寫是不對的
1
result = html.xpath('//li/span')
因為 / 是用來獲取子元素的，而并不是

的子元素，所以，要用雙斜杠

1 2	result = html.xpath('//li//span') print result

運行結果

1	[<Element span at 0x10d698e18>]

（5）獲取

標簽下的所有 class，不包括

1 2	result = html.xpath('//li/a//@class') print result

運行結果

['blod']

（6）獲取最后一個

的的 href
1
2
result = html.xpath('//li[last()]/a/@href')
print result
運行結果
1
['link5.html']
（7）獲取倒數第二個元素的內容
1
2
result = html.xpath('//li[last()-1]/a')
print result[0].text
運行結果
1
fourth item
（8）獲取 class 為 bold 的標簽名
1
2
result = html.xpath('//*[@class="bold"]')
print result[0].tag
運行結果
1
span
通過以上實例的練習，相信大家對 XPath 的基本用法有了基本的了解。也可以利用 text 方法來獲取元素的內容。大家多加練習！

結語
XPath 是一個非常好用的解析方法，同時也作為爬蟲學習的基礎，在后面的 selenium 以及 scrapy 框架中都會涉及到這部分知識，希望大家可以把它的語法掌握清楚，為后面的深入研究做好鋪墊。

=

[ 取li text

result = html.xpath('/html/body/div[1]/div/div[1]/ul[1]/li[3]')

print (result[0].xpath('string(.)').strip())

]

=

[取a text

result = html.xpath('/html/body/div[1]/div/div[1]/ul[1]/li[3]/a/text()')

print (result)

]

=

[取a href

result = html.xpath('/html/body/div[1]/div/div[1]/ul[1]/li[3]/a/@href')

print (result)

]

文件自動加https2022-03-28 17:28:25

PHP laravel框架實現SMTP發送郵箱2022-03-21 15:54:50

首先在.env配置里配置信息

MAIL_DRIVER=smtp//主機

MAIL_HOST=smtp.163.com//主機網址

MAIL_PORT=465//端口號

MAIL_USERNAME=用戶名

MAIL_PASSWORD=授權碼(不是密碼)

MAIL_ENCRYPTION=ssl//安全碼

在控制器里引入

use Illuminate\Support\Facades\Mail;
再寫一個方法,直接走一個方法里的程序就ok
class WyController extends BaseController
{
    public function r(){
        Mail::raw("范志超超超超超",function ($mesage){
            $mesage->from("17601079705@163.com","范志超");
            $mesage->subject("一二三四");
            $mesage->to("904712081@qq.com");
        });
    }
}

or

public function send()

{

$name = '學院君';

$flag = Mail::send('emails.test',['name'=>$name],function($message){

$to = '1072155122@qq.com';

$message ->to($to)->subject('測試郵件');

});

if($flag){

echo '發送郵件成功，請查收！';

}else{

echo '發送郵件失敗，請重試！';

}

html中如何使一個table或者div永遠處于中間位置2022-03-21 15:51:43

tml中使一個table或者div永遠處于中間位置,首先需要理解塊元素的概念，如div、table都是width和height屬性，然后在通過css中的一個margin屬性，來實現居中，具體用法看下圖：

不管瀏覽器的寬度是多少，這個屬性就會將div居中，具體看下代碼：

<html>

<head>

<style>

#round{

width:400px; //通過給divwidth和heigh值，在通過margin屬性來定位

height:200px;

margin:0 auto;

}

table{ //通過給divwidth和heigh值，在通過margin屬性來定位

width:300px;

height:50px;

margin:0 auto;

}

</style>

</head>

<body>

</div>

<table>

<tr>

</tr>

</table>

</body>

</html>

div中內容上下居中小結2022-03-21 15:45:51

雖然div中內容上下居中的問題已經是一個比較古老的話題，但是最近發現還是有很多前端開發者在詢問如何實現。其實網絡上已經有很多資料和案例了，我這里再總結一下幾個比較常見的處理方式。

情形一：div限高，內容長度限一行

<style>
.v-align {
margin: 0 auto;
width: 200px;
height: 80px;
text-align: center;
line-height: 80px;
border: 1px solid #ddd;
}
</style>
<!-- html -->
<div class="v-align">我的內容只能有一行。</div>

情形二：div限高，內容不限

.v-mult {
margin: 0 auto;
width: 200px;
height: 100px;
border: 1px solid #ddd;
overflow: hidden;
}
.v-mult .empty,
.v-mult .text {
display: inline-block;
*display: inline;
*zoom: 1;
vertical-align: middle;
}
.v-mult .empty {
height: 100%;
}
<!-- html -->
<div class="v-mult">
<span class="empty"></span>
<span class="text">我的內容不限，多高都行
換行照常</span>
</div>

情形三：div高度不定，內容高度一定

.v-auto {
position: relative;
margin: 0 auto;
width: 200px;
border: 1px solid #ddd;
}
.v-auto .text {
position: absolute;
top: 50%;
margin-top: -50px;
height: 100px;
border: 1px dashed #ddd;
}
<!-- html -->
<div class="v-auto">
<div class="text">
我的高度是固定的，只有100px高，但是我的父及高度不定，我怎么垂直居中呢？
</div>





</div>

情形四：div高度不定，內容高度不定

.v-auto-out { position: relative; margin: 0 auto; width: 200px; border: 1px solid #ddd; } .v-auto-out .auto-in { position: absolute; top: 50%; border: 1px dashed #ddd; /* 這里有兼容性問題 */ -webkit-transform: translateY(-50%); -ms-transform: translateY(-50%); -o-transform: translateY(-50%); transform: translateY(-50%); }  <div class="v-auto-out"> <div class="auto-in">我的高度不定，我的父及高度也不定，這下要上下居中，該如何是好？我們一起來瞧瞧吧。</div> </div>

DigitalOcean忘記root密碼怎么辦？2022-03-17 13:30:00

DigitalOcean忘記root密碼怎么辦？在這個時候我們可以選擇重置.

方法1

Access->

Reset root password

Clicking Reset Root Password below will immediately shut down your Droplet and set a new root password.

The new root password will be emailed to you within a few minutes. If the email doesn't arrive or the new password doesn't work, try using the recovery environment.

Do you wish to proceed?

方法2

Access->

Recovery Console

Use the Recovery Console if you need to use the recovery ISO or you can't connect to your Droplet with the Droplet Console. To use the recovery console, you must enable password authentication. If necessary, you can reset your root password below.

Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to conn2022-03-16 10:42:16

====

[2022-03-16 02:32:13] local.ERROR: Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to connect to ssl://smtp.gmail.com:465 (Connection timed out) {"exception":"[object] (Swift_TransportException(code: 0): Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to connect to ssl://smtp.gmail.com:465 (Connection timed out) at /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php:261)

[stacktrace]

#0 [internal function]: Swift_Transport_StreamBuffer->{closure}(2, 'stream_socket_c...', '/www/wwwroot/ca...', 264, Array)

#1 /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php(264): stream_socket_client('ssl://smtp.gmai...', 110, 'Connection time...', 30, 4, Resource id #679)

#2 /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php(58): Swift_Transport_StreamBuffer->establishSocketConnection()

#3 /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(143): Swift_Transport_StreamBuffer->initialize(Array)

#4 /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Mailer.php(65): Swift_Transport_AbstractSmtpTransport->start()

#5 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Mail/Mailer.php(521): Swift_Mailer->send(Object(Swift_Message), Array)

#6 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Mail/Mailer.php(288): Illuminate\\Mail\\Mailer->sendSwiftMessage(Object(Swift_Message))

#7 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Mail/MailManager.php(484): Illuminate\\Mail\\Mailer->send('emails.test', Array, Object(Closure))

#8 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(261): Illuminate\\Mail\\MailManager->__call('send', Array)

#9 /www/wwwroot/card.inkstarbox.com/app/Http/Controllers/DoController.php(159): Illuminate\\Support\\Facades\\Facade::__callStatic('send', Array)

#10 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): App\\Http\\Controllers\\DoController->personal_info(Object(Illuminate\\Http\\Request))

#11 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\\Routing\\Controller->callAction('personal_info', Array)

#12 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Route.php(254): Illuminate\\Routing\\ControllerDispatcher->dispatch(Object(Illuminate\\Routing\\Route), Object(App\\Http\\Controllers\\DoController), 'personal_info')

#13 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Route.php(197): Illuminate\\Routing\\Route->runController()

#14 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(693): Illuminate\\Routing\\Route->run()

#15 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))

#16 /www/wwwroot/card.inkstarbox.com/app/Http/Middleware/SetLangMiddleware.php(30): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#17 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\\Http\\Middleware\\SetLangMiddleware->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#18 /www/wwwroot/card.inkstarbox.com/vendor/laravel/jetstream/src/Http/Middleware/ShareInertiaData.php(61): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#19 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Laravel\\Jetstream\\Http\\Middleware\\ShareInertiaData->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#20 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(50): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#21 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#22 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(78): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#23 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\VerifyCsrfToken->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#24 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#25 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\View\\Middleware\\ShareErrorsFromSession->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#26 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(121): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#27 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(63): Illuminate\\Session\\Middleware\\StartSession->handleStatefulRequest(Object(Illuminate\\Http\\Request), Object(Illuminate\\Session\\Store), Object(Closure))

#28 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Session\\Middleware\\StartSession->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#29 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#30 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#31 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(67): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#32 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Cookie\\Middleware\\EncryptCookies->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#33 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#34 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(695): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))

#35 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(670): Illuminate\\Routing\\Router->runRouteWithinStack(Object(Illuminate\\Routing\\Route), Object(Illuminate\\Http\\Request))

#36 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(636): Illuminate\\Routing\\Router->runRoute(Object(Illuminate\\Http\\Request), Object(Illuminate\\Routing\\Route))

#37 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(625): Illuminate\\Routing\\Router->dispatchToRoute(Object(Illuminate\\Http\\Request))

#38 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(166): Illuminate\\Routing\\Router->dispatch(Object(Illuminate\\Http\\Request))

#39 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}(Object(Illuminate\\Http\\Request))

#40 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#41 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#42 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#43 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#44 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#45 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#46 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(86): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#47 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#48 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#49 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(141): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))

#50 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(110): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter(Object(Illuminate\\Http\\Request))

#51 /www/wwwroot/card.inkstarbox.com/public/index.php(52): Illuminate\\Foundation\\Http\\Kernel->handle(Object(Illuminate\\Http\\Request))

#52 {main}

"}

[2022-03-16 02:32:56] local.ERROR: Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to connect to ssl://smtp.gmail.com:465 (Connection timed out) {"exception":"[object] (Swift_TransportException(code: 0): Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to connect to ssl://smtp.gmail.com:465 (Connection timed out) at /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php:261)