목록으로 출력 인쇄
Nov 19 2020
다음 코드는 정상적으로 실행됩니다. LinkedIn의 목록별로 정보를 수집합니다.
(계정 정보가 제공되며 테스트 계정이므로 무료로 사용 가능)
그러나 출력은 자체 필드가있는 각 필드 대신 데이터를 결합합니다.
출력은 자체 열에있는 사전 (이름, 회사, 위치)의 각 필드와 함께 Excel로 인쇄되고 출력은 자체 셀에 있습니다.
예상 출력의 예는 첨부를 참조하십시오.

나는 beautifulSoup을 시도했지만 그것이 효과가 있다고 생각하지 않는다.
import time
import pandas as pd
from selenium import webdriver
from bs4 import BeautifulSoup
import requests
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from webdriver_manager.chrome import ChromeDriverManager
test1=[]
options = Options()
driver = webdriver.Chrome(ChromeDriverManager().install())
url = "https://www.linkedin.com/uas/login?session_redirect=https%3A%2F%2Fwww%2Elinkedin%2Ecom%2Fsearch%2Fresults%2Fpeople%2F%3FcurrentCompany%3D%255B%25221252860%2522%255D%26geoUrn%3D%255B%2522103644278%2522%255D%26keywords%3Dsales%26origin%3DFACETED_SEARCH%26page%3D2&fromSignIn=true&trk=cold_join_sign_in"
driver.get(url)
time.sleep(2)
username = driver.find_element_by_id('username')
username.send_keys('[email protected]')
password = driver.find_element_by_id('password')
password.send_keys('Applesauce1')
password.submit()
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(3)
elementj=(WebDriverWait(driver,10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,".subline-level-2.t-12.t-black--light.t-normal.search-result__truncate"))))
place1=[j.text for j in elementj]
elementk=WebDriverWait(driver,10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,".subline-level-1.t-14.t-black.t-normal.search-result__truncate")))
compan=[c.text for c in elementk]
element1 = driver.find_elements_by_class_name("actor-name")
title=[t.text for t in element1]
diction={"Location":place1,"Company":compan,"Title":title}
test1.append(diction)
print(test1)
답변
1 PaulBrennan Nov 19 2020 at 19:06
나는 당신의 코드를 실행할 수 있습니다.
다음은 Pandas DataFrame에서 여러 목록 열의 중첩을 해제 (폭발)하는 효율적인 방법의 도움으로 얻은 것입니다.
import time
import pandas as pd
import numpy as np
from selenium import webdriver
from bs4 import BeautifulSoup
import requests
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from webdriver_manager.chrome import ChromeDriverManager
test1=[]
options = Options()
driver = webdriver.Chrome(ChromeDriverManager().install())
url = "https://www.linkedin.com/uas/login?session_redirect=https%3A%2F%2Fwww%2Elinkedin%2Ecom%2Fsearch%2Fresults%2Fpeople%2F%3FcurrentCompany%3D%255B%25221252860%2522%255D%26geoUrn%3D%255B%2522103644278%2522%255D%26keywords%3Dsales%26origin%3DFACETED_SEARCH%26page%3D2&fromSignIn=true&trk=cold_join_sign_in"
driver.get(url)
time.sleep(2)
username = driver.find_element_by_id('username')
username.send_keys('[email protected]')
password = driver.find_element_by_id('password')
password.send_keys('Applesauce1')
password.submit()
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(3)
elementj=(WebDriverWait(driver,10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,".subline-level-2.t-12.t-black--light.t-normal.search-result__truncate"))))
place1=[j.text for j in elementj]
elementk=WebDriverWait(driver,10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,".subline-level-1.t-14.t-black.t-normal.search-result__truncate")))
compan=[c.text for c in elementk]
element1 = driver.find_elements_by_class_name("actor-name")
title=[t.text for t in element1]
diction={"Location":place1,"Company":compan,"Title":title}
test1.append(diction)
print(test1)
df = pd.DataFrame(test1)
def explode(df, lst_cols, fill_value=''):
# make sure `lst_cols` is a list
if lst_cols and not isinstance(lst_cols, list):
lst_cols = [lst_cols]
# all columns except `lst_cols`
idx_cols = df.columns.difference(lst_cols)
# calculate lengths of lists
lens = df[lst_cols[0]].str.len()
if (lens > 0).all():
# ALL lists in cells aren't empty
return pd.DataFrame({
col:np.repeat(df[col].values, df[lst_cols[0]].str.len())
for col in idx_cols
}).assign(**{col:np.concatenate(df[col].values) for col in lst_cols}) \
.loc[:, df.columns]
else:
# at least one list in cells is empty
return pd.DataFrame({
col:np.repeat(df[col].values, df[lst_cols[0]].str.len())
for col in idx_cols
}).assign(**{col:np.concatenate(df[col].values) for col in lst_cols}) \
.append(df.loc[lens==0, idx_cols]).fillna(fill_value) \
.loc[:, df.columns]
explode(df,['Location','Company','Title'])
그리고 그 결과
Location Company Title
0 Dayton, Ohio Area National Account Executive LinkedIn Member
1 Dayton, Ohio Area Currently seeking permanent employment LinkedIn Member
2 Dayton, Ohio Area Account Manager at LexisNexis LinkedIn Member
3 Greater Denver Area Currently seeking new opportunities in managem... LinkedIn Member
4 Dayton, Ohio Area Advertising Sales Representative at AMOS MEDIA LinkedIn Member
5 Dayton, Ohio Area Territory Manager at Huntington Outdoor, LLC LinkedIn Member
6 Vandalia, Ohio, United States Cintas LinkedIn Member
7 Dayton, Ohio Area Outside Sales Representative at Carter Lumber. LinkedIn Member
8 Dayton, Ohio Area Actively Searching LinkedIn Member
9 Corpus Christi, Texas Area Currently looking for sales position LinkedIn Member