firecrawl
firecrawl/firecrawlThe context API to search, scrape, and interact with the web at scale. 🔥
Pushed today.GNU Affero General Public License v3.0TypeScriptDeveloper tools201.3k+2.4k
Book · a repository
crawl4ai
unclecode/crawl4ai
Not yet reviewedDeveloper tools
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Topics
Understand it first
Don’t install what you can’t explain. Each of these free tools reads unclecode/crawl4ai when you swap github.com in its link for another address.
- See the architecturegitdiagram.com/unclecode/crawl4ai
An interactive diagram of how the parts fit together. Generated by AI.
- Read the manualdeepwiki.com/unclecode/crawl4ai
A written guide to the codebase, with a chat you can ask questions. Generated by AI.
- Browse in an editorgithub.dev/unclecode/crawl4ai
Opens the code in VS Code in your browser. Needs a GitHub sign-in. Same as pressing . on any GitHub repo.
- Get it as one filegitingest.com/unclecode/crawl4ai
The whole project as a single text file to paste into Claude or another assistant. Large repos may not fit in one chat.
Or build it on your computer
npx techandria digest unclecode/crawl4aiBuilt from the current code, which Techandria hasn’t reviewed. The file says so at the top. Add
--only srcfor part of a large repo. - Let your agent read itgitmcp.io/unclecode/crawl4ai
A remote MCP server your agent can connect to and read the docs and code live while it works.
Add it to Claude Code
claude mcp add --transport http crawl4ai-docs https://gitmcp.io/unclecode/crawl4ai
The linked sites are independent services, not part of Techandria. They read the default branch as it is now, not the reviewed commit, and AI-written explanations can be wrong. Repository text you paste into an assistant can contain instructions aimed at it, so treat it as data. None of this is a security review.
Install
Not available through Techandria. A library has no Techandria install path. Running an arbitrary program is a different risk from adding a reviewed skill.
You can still get the code from the source and read it yourself:
git clone https://github.com/unclecode/crawl4aiWhat we checked
Not yet reviewedDiscovery metadata only. Nothing has been scanned or approved, so this is a listing — not a recommendation to run the code.
Nothing has been scanned for this repository yet, so there is nothing to report. Listing here is not permission or a recommendation to run the code.
Maintain this project? Add the review badge to your README
The badge shows this project's current review state and updates on its own — including if the state changes. It links back to this page.
[](https://techandria.com/project/unclecode/crawl4ai)README
No README could be fetched from GitHub for this repository.
More in Developer tools
Scrapegraph-ai
ScrapeGraphAI/Scrapegraph-aiPython scraper based on AI
Pushed 1 months ago.MIT LicensePythonDeveloper tools32.7k+223
Scrapling
D4Vinci/Scrapling🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
Pushed yesterday.BSD 3-Clause "New" or "Revised" LicensePythonPlugins102.1k+4.6k
haystack
deepset-ai/haystackOpen-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.
Pushed today.Apache License 2.0PythonDeveloper tools27.0k+67