Python3 + Selenium + ChromeDriver 初探

简介

Selenium是一个用于Web应用程序测试的工具。它直接运行在浏览器中,就像真正的用户在操作一样。运用这种特性,可以用于简单爬虫。

配置环境

ChromeDriver

下载相应版本的ChromeDriver,版本对应请看文件夹里面的note.txt文件。

Selenium

1
sudo pip3 install Selenium

模拟登录实现

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
from selenium import webdriver
import os
import csv
import time

ID = ""
PASSWORD = ""

chromedriver = "../Chrome_Driver/2.40/chromedriver"
os.environ["webdriver.chrome.driver"] = chromedriver

driver = webdriver.Chrome(chromedriver)
driver.get("http://jwxt.zufe.edu.cn/jwglxt/xtgl/login_slogin.html?language=zh_CN&_t=1547536545893")

inputID = driver.find_element_by_id("yhm")
inputPASSWORD = driver.find_element_by_id("mm")
clickButton = driver.find_element_by_id("dl")

inputID.clear()
inputPASSWORD.clear()

inputID.send_keys(ID)
inputPASSWORD.send_keys(PASSWORD)
clickButton.click()

driver.get("http://jwxt.zufe.edu.cn/jwglxt/cjcx/cjcx_cxDgXscj.html?gnmkdm=N305005&layout=default&su=%s" %ID)

time.sleep(10)
driver.find_element_by_id("search_go").click()
table_rows = driver.find_element_by_tag_name("tbody").find_elements_by_tag_name("tr")
rowsLen = len(table_rows)

CSVPATH = "../Data/Score.csv"

with open(CSVPATH,'w') as f:
csv_write = csv.writer(f)
csv_head = ["学年","学期","课程编号","课程名","课程类型","学分","成绩"]
csv_write.writerow(csv_head)
for i in range(1,rowsLen):
csv_write.writerow(table_rows[i].text.split(" ")[1:8])

Socks Proxy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
from selenium import webdriver

PORT = 1088

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=socks5://localhost:%s' %PORT)
browser = webdriver.Chrome(chrome_options=chrome_options, executable_path='../Chrome_Driver/2.40/chromedriver')
proxy.new_har('youtube')

browser.get('http://www.google.com')
browser.find_element_by_name("q")
inputContent = browser.find_element_by_name("q")
inputContent.clear()
inputContent.send_keys("asmire.site")
submit = browser.find_elements_by_name("btnK")
submit[-1].click()

即可访问外网。