python爬虫用bs4或者叫BeautifulSoup
- 创业
- 2025-09-18 11:12:02
Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库.它能够通过你喜欢的转换器实现惯用的文档导航,查找,修改文档的方式.Beautiful Soup会帮你节省数小时甚至数天的工作时间.
Beautiful Soup 4.4.0 文档 — Beautiful Soup 4.2.0 中文 文档
安装 ：
pip install bs4
下面的一段HTML代码将作为例子被多次用到.这是 爱丽丝梦游仙境的 的一段内容(以后内容中简称为 爱丽丝 的文档):
html_doc = """ <html><head><title>The Dormouse's story</title></head> <body> <p class="title"><b>The Dormouse's story</b></p> <p class="story">Once upon a time there were three little sisters; and their names were <a href="http://example /elsie" class="sister" id="link1">Elsie</a>, <a href="http://example /lacie" class="sister" id="link2">Lacie</a> and <a href="http://example /tillie" class="sister" id="link3">Tillie</a>; and they lived at the bottom of a well.</p> <p class="story">...</p> """使用BeautifulSoup解析这段代码,能够得到一个 BeautifulSoup 的对象,并能按照标准的缩进格式的结构输出:
from bs4 import BeautifulSoup soup = BeautifulSoup(html_doc, 'html.parser') print(soup.prettify()) # <html> # <head> # <title> # The Dormouse's story # </title> # </head> # <body> # <p class="title"> # <b> # The Dormouse's story # </b> # </p> # <p class="story"> # Once upon a time there were three little sisters; and their names were # <a class="sister" href="http://example /elsie" id="link1"> # Elsie # </a> # , # <a class="sister" href="http://example /lacie" id="link2"> # Lacie # </a> # and # <a class="sister" href="http://example /tillie" id="link2"> # Tillie # </a> # ; and they lived at the bottom of a well. # </p> # <p class="story"> # ... # </p> # </body> # </html>几个简单的浏览结构化数据的方法:
soup.title # <title>The Dormouse's story</title> soup.title.name # u'title' soup.title.string # u'The Dormouse's story' soup.title.parent.name # u'head' soup.p # <p class="title"><b>The Dormouse's story</b></p> soup.p['class'] # u'title' soup.a # <a class="sister" href="http://example /elsie" id="link1">Elsie</a> soup.find_all('a') # [<a class="sister" href="http://example /elsie" id="link1">Elsie</a>, # <a class="sister" href="http://example /lacie" id="link2">Lacie</a>, # <a class="sister" href="http://example /tillie" id="link3">Tillie</a>] soup.find(id="link3") # <a class="sister" href="http://example /tillie" id="link3">Tillie</a>从文档中找到所有<a>标签的链接:
for link in soup.find_all('a'): print(link.get('href')) # http://example /elsie # http://example /lacie # http://example /tillie从文档中获取所有文字内容:
print(soup.get_text()) # The Dormouse's story # # The Dormouse's story # # Once upon a time there were three little sisters; and their names were # Elsie, # Lacie and # Tillie; # and they lived at the bottom of a well. # # ...只学了这么多
python爬虫用bs4或者叫BeautifulSoup由讯客互联创业栏目发布，感谢您对讯客互联的认可，以及对我们原创作品以及文章的青睐，非常欢迎各位朋友分享到个人网站或者朋友圈，但转载请说明文章出处“python爬虫用bs4或者叫BeautifulSoup”
