All projects
Repositories in this selection 31 · UTC 2026-11-03
CompareRankProjectStarFork30-day trend7-day net changeLast pushSign in to save
1—Uunclecode / crawl4aiOpen-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtfPythonApache-2.0 · PermissiveInsufficient rank history86,7779,043——2026-11-02
2—NNaiboWang / EasySpiderA visual no-code/code-free web crawler/spider: a visual browser automated testing/data collection/web crawler software that can graphically design and execute crawler tasks without code. Alias: ServiceWrapper is an intelligent service packaging system for Web applications.JavaScriptAGPL-3.0 · Share derivative sourceInsufficient rank history44,7655,393——2026-11-02
3—Iiawia002 / lux👾 Fast and simple video download library and CLI tool written in GoGoMIT · PermissiveInsufficient rank history31,8183,329——2026-03-29
4—Ccheeriojs / cheerioThe fast, flexible, and elegant library for parsing and manipulating HTML and XML.TypeScriptMIT · PermissiveInsufficient rank history30,5391,745——2026-11-03
5—Ggocolly / collyElegant Scraper and Crawler Framework for GolangGoApache-2.0 · PermissiveInsufficient rank history25,5741,860——2026-10-16
6—EEvil0ctal / Douyin_TikTok_Download_API🚀 Self-hosted Douyin, TikTok, Bilibili data collection and watermark-free video download API, supports AI agent calling through MCP and offline voice-to-text, and one-click deployment with Docker. | Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.PythonApache-2.0 · PermissiveInsufficient rank history21,1122,938——2026-11-02
7—Ccodelucas / newspapernewspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:PythonMIT · PermissiveInsufficient rank history15,1822,103——2026-10-26
8—Ppwxcoo / chinese-xinhua:orange_book: Chinese Xinhua Dictionary Database. Including idioms, idioms, words, and Chinese characters.PythonMIT · PermissiveInsufficient rank history11,7512,714——2023-12-26
9—NNopeCHALLC / nopecha-extensionAutomated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.UnknownMIT · PermissiveInsufficient rank history11,089234——2026-10-02
10—Gguyueyingmu / avbookAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video DatabasePHPUnknown · Restricted / unknownInsufficient rank history10,1001,992——2024-06-01
11—Aapify / crawlee-pythonCrawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.PythonApache-2.0 · PermissiveInsufficient rank history9,637820——2026-11-02
12—Aalirezamika / autoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for PythonPythonMIT · PermissiveInsufficient rank history8,068833——2026-07-29
13—BBruceDone / awesome-crawlerA collection of awesome web crawler,spider in different languagesUnknownMIT · PermissiveInsufficient rank history7,361752——2024-06-16
14—Ggo-rod / rodA Chrome DevTools Protocol driver for web automation and scraping.GoMIT · PermissiveInsufficient rank history7,143489——2026-08-11
15—Mmishushakov / llm-scraperTurn any webpage into structured data using LLMsTypeScriptMIT · PermissiveInsufficient rank history6,951453——2026-10-26
16—AAhmadIbrahiim / Website-downloader💡 Download the complete source code of any website (including all assets). [ Javascripts, Stylesheets, Images ] using Node.jsHTMLMIT · PermissiveInsufficient rank history6,0651,404——2026-08-12
17—Yyujiosaka / headless-chrome-crawlerDistributed crawler powered by Headless ChromeJavaScriptMIT · PermissiveInsufficient rank history5,632403——2023-04-29
18—JJustAnotherArchivist / snscrapeA social networking service scraper in PythonPythonGPL-3.0 · Share derivative sourceInsufficient rank history5,468782——2023-11-15
19—Nniespodd / browser-fingerprintingAnalysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web?JavaScriptUnknown · Restricted / unknownInsufficient rank history5,156277——2026-07-27
20—Bbjesus / pipetSwiss-army tool for scraping and extracting data from online assets, made for hackersGoMIT · PermissiveInsufficient rank history4,773222——2024-10-12
License tiers are derived from metadata. Check obligations for your use; unknown does not mean prohibited.