libraryAgent Tools & Data
Open-source web crawler and scraper that turns pages into clean Markdown or structured JSON for RAG and AI agents. Run it yourself with Python or Docker, or use the hosted API.
Every tool tagged βcrawlerβ, grouped by category.
Open-source web crawler and scraper that turns pages into clean Markdown or structured JSON for RAG and AI agents. Run it yourself with Python or Docker, or use the hosted API.
Web data API that lets AI agents search, scrape, map and crawl the web. The core is open source.
Python web scraping framework for everything from a single request to a full crawl, with stealth fetching and a built-in MCP server.