GitHub topic

web-crawler GitHub repositories

Explore 7 tracked GitHub repositories tagged web-crawler. Compare their recorded star activity and implementation languages, then follow related topics to find other projects.

Snapshot: 2026-10-28. The collection gained 505 stars since the previous daily snapshot.

Tracked repositories: 7
Total stars: 324,420
Stars gained · 7 days: +3,366
Stars gained · 30 days: +13,738

Compare star activity

Up to five projects on this page, ordered by daily star growth. Open a project for its description and recorded star history.

Swipe to compare all star periods →

Star activity for web-crawler repositories on this page
Repository	Total stars	Today	7 days	30 days
firecrawl/firecrawl	196.1K	+375	+2,589	+10,532
unclecode/crawl4ai	86.4K	+85	+480	+2,048
ScrapeGraphAI/Scrapegraph-ai	32.3K	+28	+243	+924
0xMassi/webclaw	2.4K	+8	+14	+32
KnockOutEZ/wigolo	5.5K	+5	+17	+95

Explore repositories

Showing 1–7 of 7 tracked repositories.

firecrawl/firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf

★ 86.4K+85 todayPython

llm ai

View ↗

ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

0xMassi/webclaw

Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

KnockOutEZ/wigolo

The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.

fastcrw/crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

★ 1.2K+4 todayRust