All projects
Repositories in this selection 24 · UTC 2026-11-04
CompareRankProjectStarFork30-day trend7-day net changeLast pushSign in to save
1—Sscrapy / scrapyScrapy, a fast high-level web crawling & scraping framework for Python.PythonBSD-3-Clause · PermissiveInsufficient rank history65,19012,095——2026-11-02
2—Ssoxoj / maigret🕵️♂️ Collect a dossier on a person by username from 6K websitesPythonMIT · PermissiveInsufficient rank history39,6203,150——2026-11-02
3—SScrapeGraphAI / Scrapegraph-aiPython scraper based on AIPythonMIT · PermissiveInsufficient rank history32,4373,313——2026-10-06
4—Ggocolly / collyElegant Scraper and Crawler Framework for GolangGoApache-2.0 · PermissiveInsufficient rank history25,5741,860——2026-10-16
5—Ppsf / requests-htmlPythonic HTML Parsing for HumansTMPythonMIT · PermissiveInsufficient rank history13,797995——2024-04-16
6—Ddaijro / camoufox🦊 Anti-detect browserPythonMPL-2.0 · Share derivative sourceInsufficient rank history13,0091,104——2026-11-02
7—Uultrafunkamsterdam / undetected-chromedriverCustom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)PythonGPL-3.0 · Share derivative sourceInsufficient rank history12,8961,347——2025-07-05
8—Jjo-inc / camofox-browserStealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.JavaScriptMIT · PermissiveInsufficient rank history12,1191,225——2026-11-02
9—Ccode4craft / webmagicA scalable web crawler framework for Java.JavaApache-2.0 · PermissiveInsufficient rank history11,6714,085——2025-12-20
10—Aapify / crawlee-pythonCrawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.PythonApache-2.0 · PermissiveInsufficient rank history9,637820——2026-11-02
11—Llorien / awesome-web-scrapingList of libraries, tools and APIs for web scraping and data processing.JavaScriptNOASSERTION · Restricted / unknownInsufficient rank history8,207969——2026-10-27
12—Aalirezamika / autoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for PythonPythonMIT · PermissiveInsufficient rank history8,068833——2026-07-29
13—Ttabulapdf / tabulaTabula is a tool for liberating data tables trapped inside PDF filesCSSMIT · PermissiveInsufficient rank history7,493698——2025-03-14
14—Aautoscrape-labs / pydollPydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.HTMLMIT · PermissiveInsufficient rank history7,132413——2026-10-09
15—Aadbar / trafilaturaPython & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XMLPythonApache-2.0 · PermissiveInsufficient rank history7,065487——2026-10-30
16—Pproxifly / free-proxy-list🚀 Free HTTP, SOCKS4, & SOCKS5 proxy list * Updated every 5 minutes * and rotating proxy API (100+ countries)UnknownGPL-3.0 · Share derivative sourceInsufficient rank history7,000802——2026-11-03
17—Yyujiosaka / headless-chrome-crawlerDistributed crawler powered by Headless ChromeJavaScriptMIT · PermissiveInsufficient rank history5,632403——2023-04-29
18—Bbjesus / pipetSwiss-army tool for scraping and extracting data from online assets, made for hackersGoMIT · PermissiveInsufficient rank history4,773222——2024-10-12
19—Dd60 / twikitTwitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter BotPythonMIT · PermissiveInsufficient rank history4,762575——2026-03-10
20—Ssparklemotion / mechanizeMechanize is a ruby library that makes automated web interaction easy.RubyMIT · PermissiveInsufficient rank history4,445477——2026-10-26
License tiers are derived from metadata. Check obligations for your use; unknown does not mean prohibited.