Beautiful Soup으로 테이블 헤더 긁기

Question

테이블을 긁어 내려고합니다.

<!DOCTYPE html>
<html>
<head>
    <title></title>
</head>
<body>

    <table class="table ajax">
    <thead>
        <tr>
            <th scope="col">
                <span>NO.</span>
            </th>
            <th scope="col" data-index="1">
                    <span>Year of initiation</span>
                              
            </th>
            <th scope="col" data-index="2">
                
                    <span>Short case name</span>
                    
                
            </th>
            <th scope="col" data-index="3" style="display: none;">
                
                    <span>Full case name</span>
                    
                
            </th>
            <th scope="col" data-index="4">
               
                    <span>Applicable IIA</span>
                    
                
        </tr>
    </thead>
    <tbody>
            <tr>
                <th scope="row">1</th>
                <td data-index="1">
                    2019
                </td>
                <td data-index="2">
                   Alcosa v. Kuwait</a>
                </td>
                <td data-index="3" style="display: none;">
                    Alcosa v. The State of Kuwait
                </td>
                <td data-index="4">
Kuwait - Spain BIT(2005)</a>                </td>
                <td data-index="5"> UNCITRAL
               </td>
</tbody>
</table>

</body>
</html>

다음 코드로 :

html = driver.page_source
bs=BeautifulSoup(html, "lxml")
table = bs.find('table', { 'class' : 'ajax' })
table_body=table.find('tbody')
rows = table_body.findAll('tr')

with open('son.csv', "wt+") as f:
    writer = csv.writer(f)
    for row in rows:
        cols = row.find_all('td')
        cols = [x.get_text(strip=True, separator='|') for x in cols]
        writer.writerow(cols)

테이블 행을 가져올 수 있지만 테이블 헤더를 가져올 수 없습니다.

이것은 내가 얻고 싶은 출력입니다.

NO. Year of initiation  Short case name Applicable IIA
1   2019    Alcosa v. Kuwait    Kuwait - Spain BIT(2005)    UNCITRAL

내가 어떻게 해? 감사.

Answer 1

이 스크립트를 사용하여 테이블을 csv에 저장할 수 있습니다.

import csv
from bs4 import BeautifulSoup    


txt = '''<!DOCTYPE html>
<html>
<head>
    <title></title>
</head>
<body>

    <table class="table ajax">
    <thead>
        <tr>
            <th scope="col">
                <span>NO.</span>
            </th>
            <th scope="col" data-index="1">
                    <span>Year of initiation</span>
                              
            </th>
            <th scope="col" data-index="2">
                
                    <span>Short case name</span>
                    
                
            </th>
            <th scope="col" data-index="3" style="display: none;">
                
                    <span>Full case name</span>
            </th>
            <th scope="col" data-index="4">               
                    <span>Applicable IIA</span>
                    
             </th>   
        </tr>
    </thead>
    <tbody>
            <tr>
                <th scope="row">1</th>
                <td data-index="1">
                    2019
                </td>
                <td data-index="2">
                   Alcosa v. Kuwait
                </td>
                <td data-index="3" style="display: none;">
                    Alcosa v. The State of Kuwait
                </td>
                <td data-index="4">
                    Kuwait - Spain BIT(2005)
                </td>
                <td data-index="5"> UNCITRAL
               </td>
            </tr>
</tbody>
</table>

</body>
</html>'''


soup = BeautifulSoup(txt, 'html.parser')

headers = [th.get_text(strip=True) for th in soup.select('table.ajax thead th')]
rows = []
for row in soup.select('table.ajax tbody tr'):
    data = [d.get_text(strip=True) for d in row.select('th, td')]
    rows.append(data)
   
with open('son.csv', "wt+") as f:
    writer = csv.writer(f)
    writer.writerow(headers)
    for row in rows:
        writer.writerow(row)

쓰기 son.csv(LibreOffice의 스크린 샷) :