GitHub topic

web-scraping GitHub repositories

Explore 21 tracked GitHub repositories tagged web-scraping. Compare their recorded star activity and implementation languages, then follow related topics to find other projects.

Snapshot: 2026-10-28. The collection gained 731 stars since the previous daily snapshot.

Tracked repositories: 21
Total stars: 562,944
Stars gained · 7 days: +4,955
Stars gained · 30 days: +21,477

Compare star activity

Up to five projects on this page, ordered by daily star growth. Open a project for its description and recorded star history.

Swipe to compare all star periods →

Star activity for web-scraping repositories on this page
Repository	Total stars	Today	7 days	30 days
firecrawl/firecrawl	196.1K	+375	+2,589	+10,532
browser-use/browser-use	119.3K	+97	+651	+2,800
unclecode/crawl4ai	86.4K	+85	+480	+2,048
JCodesMore/ai-website-cloner-template	37.5K	+48	+441	+2,137
MODSetter/SurfSense	16.6K	+38	+107	+379

Explore repositories

Showing 1–21 of 21 tracked repositories.

firecrawl/firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

browser-use/browser-use

Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.

★ 119.3K+97 todayPython

llm ai-agents

View ↗

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf

★ 86.4K+85 todayPython

llm ai

View ↗

JCodesMore/ai-website-cloner-template

Clone any website with one command using AI coding agents

★ 37.5K+48 todayTypeScript

ai ai-agents

View ↗

MODSetter/SurfSense

Air-gapped, privacy-focused, open-source NotebookLM alternative. AI agent that researches, transforms, and edits your documents. Join our Discord: https://discord.gg/ejRNvftDp9

★ 16.6K+38 todayPython

ai agents

View ↗

ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

browserbase/stagehand

The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.

★ 25.8K+15 todayTypeScript

ai ai-agents

View ↗

getmaxun/maxun

Turn any website into a structured API. Extract, automate, search and monitor the web.

★ 18.1K+12 todayTypeScript

agents self-hosted

View ↗

0xMassi/webclaw

Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

karust/openserp

Self-hosted web search for AI agents and SEO. Google, Bing, Yandex, Baidu, DuckDuckGo & Ecosia, rendered in a real browser: ranked results and clean page Markdown. API, CLI, MCP 🎉

★ 1.5K+7 todayGo

llm ai-agents

View ↗

KnockOutEZ/wigolo

The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.

fastcrw/crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

★ 1.2K+4 todayRust

llm ai

View ↗

brightdata/brightdata-mcp

A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.

★ 2.7K+4 todayJavaScript

llm ai-agents

View ↗

adbar/trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

vibheksoni/stealth-browser-mcp

Stealth browser automation for AI agents over MCP and the Chrome DevTools Protocol: navigation, network hooks, DOM extraction, and pixel-accurate UI cloning.

only-cli/oc

Turn any website into a compact CLI tailored for AI agents. Browse the web in hundreds of tokens, not tens of thousands.

★ 547+1 todayJavaScript

llm ai-agents

View ↗

Johell1NS/browser-search

A skill for AI agents: search the web with SearXNG, browse with Camofox, bypass protections with CloakBrowser. Anti-hallucination by design. Self-hosted, free, unlimited.

Anil-matcha/awesome-agent-apis

660+ muapi-hosted generative-media models plus community-submitted third-party API tools (SEO, enrichment, social, scraping) — one YAML file per entry, browsable by capability.

tinyfish-io/agentql

AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.

★ 1.5K+0 todayPython