25.2k
Stars
1.8k
Forks
1 month ago
Last Commit
146
Open Issues
Details
LanguageGo
LicenseApache-2.0
Category: Web Scraping & Crawling
OSS Maturity:Strong
Build note: Scrapy and Crawlee are production-grade crawling frameworks. Firecrawl turns websites into LLM-ready data. Puppeteer/Playwright handle dynamic pages.
More in Web Scraping & Crawling
ApifyCommercial
Web scraping and automation cloud platform
axiosPackage
Promise-based HTTP client for Node.js
beautifulsoup4Package
HTML/XML parser; de facto standard for Python scraping
Bright DataCommercial
Proxy network + scraping infrastructure; enterprise
cheerioPackage
Fast jQuery-like HTML parser for Node.js
CrawleeOpen Source
Web scraping and browser automation library; Apache 2.0
DiffbotCommercial
AI-powered web data extraction; knowledge graph
FirecrawlOpen Source
Turn websites into LLM-ready markdown data; AGPL-3.0