Structured Lists Directory

INI

Web Scraping Tools - INI

Web scraping tools are software and libraries for automatically collecting structured data from websites on the internet. Various tools exist to accommodate different approaches and use cases, from static HTML parsing to dynamic JavaScript rendering and browser automation. Implemented in multiple programming languages including Python, JavaScript, and Java, they are widely used for everything from small-scale data collection to large-scale crawling projects.

web scraping data collection crawling automation Python JavaScript 
[item.scrapy]
code=01
slug=scrapy
name=Scrapy
description=A high-level Python web crawling and scraping framework
githubUrl=https://github.com/scrapy/scrapy
javascriptSupport=false
language=Python
officialUrl=https://scrapy.org/
type=Framework

[item.beautifulsoup]
code=02
slug=beautifulsoup
name=BeautifulSoup
description=A Python library for parsing HTML and XML documents
githubUrl=
javascriptSupport=false
language=Python
officialUrl=https://www.crummy.com/software/BeautifulSoup/
type=Library

[item.selenium]
code=03
slug=selenium
name=Selenium
description=A cross-platform tool for browser automation
githubUrl=https://github.com/SeleniumHQ/selenium
javascriptSupport=true
language=Multi-language
officialUrl=https://www.selenium.dev/
type=Framework

[item.playwright]
code=04
slug=playwright
name=Playwright
description=Microsoft's end-to-end testing and automation framework
githubUrl=https://github.com/microsoft/playwright
javascriptSupport=true
language=Multi-language
officialUrl=https://playwright.dev/
type=Framework

[item.puppeteer]
code=05
slug=puppeteer
name=Puppeteer
description=Google's Node.js library for Chrome and Firefox automation
githubUrl=https://github.com/puppeteer/puppeteer
javascriptSupport=true
language=JavaScript/Node.js
officialUrl=https://pptr.dev/
type=Library

[item.octoparse]
code=06
slug=octoparse
name=Octoparse
description=A no-code visual web scraping tool
githubUrl=
javascriptSupport=true
language=N/A
officialUrl=https://www.octoparse.com/
type=No-code Tool

[item.apify]
code=07
slug=apify
name=Apify
description=A cloud-based web scraping and automation platform
githubUrl=https://github.com/apify
javascriptSupport=true
language=JavaScript/Node.js
officialUrl=https://apify.com/
type=Cloud Platform

[item.parsehub]
code=08
slug=parsehub
name=ParseHub
description=A machine learning-powered cloud-based scraping tool
githubUrl=
javascriptSupport=true
language=N/A
officialUrl=https://www.parsehub.com/
type=Cloud Tool