¹School of Information Engineering, China University of Geosciences, Beijing, 100083, China
²Tencent, Inc., Beijing, 100083, China

---

Abstract—In the context of comparative analysis of common web information retrieval technologies, this article discusses the principles and applications of Beautiful Soup, a vertical information search technology based on DOM tree structure. Supported by actual system examples and centering on the system architecture and core technology, this article discusses how to use Beautiful Soup to conduct deep information retrieval for partially structured webpage data, obtain directional information, reorganize the information, and then send the information to users via text message. The test results demonstrate that the web crawler achieved over 95% accuracy, satisfying the needs for commercial application.

Index Terms—DOM, information collection, vertical information retrieval, beautiful soup, customize.

[PDF]

Cite: Chunmei Zheng, Guomei He, Zuojie Peng, "A Study of Web Information Extraction Technology Based on Beautiful Soup," Journal of Computers vol. 10, no. 6, pp. 381-387, 2015.