brainstate

discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens

alirezamika/autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Python★ 8.1kMITGitHub ↗

History and source changes · Reports and owner responses · Compare

Similar

  1. scrapy/scrapy

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python★ 65.2kBSD-3-ClauseGitHub ↗

    Why this match

    topics crawler, python, scraping; license differs: MIT → BSD-3-Clause

  2. D4Vinci/Scrapling

    Scrapling — Python repository. Updated description pending.

    Python★ 95.8kBSD-3-ClauseGitHub ↗

    Why this match

    topics ai, automation, crawler; license differs: MIT → BSD-3-Clause

  3. firecrawl/firecrawl

    Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

    TypeScript★ 198.4kAGPL-3.0GitHub ↗

    Why this match

    topics ai, crawler, scraper; language differs: Python → TypeScript; license differs: MIT → AGPL-3.0

  4. apify/crawlee-python

    Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

    Python★ 9.6kApache-2.0GitHub ↗

    Why this match

    topics automation, crawler, python; license differs: MIT → Apache-2.0

  5. binux/pyspider

    A Powerful Spider(Web Crawler) System in Python.

    Python★ 16.8kApache-2.0archivedGitHub ↗

    Why this match

    topics crawler, python; license differs: MIT → Apache-2.0

  6. apify/crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

    TypeScript★ 26.3kApache-2.0GitHub ↗

    Why this match

    topics automation, crawler, scraper; language differs: Python → TypeScript; license differs: MIT → Apache-2.0

  7. gocolly/colly

    Elegant Scraper and Crawler Framework for Golang

    Go★ 25.6kApache-2.0GitHub ↗

    Why this match

    topics crawler, scraper, scraping; language differs: Python → Go; license differs: MIT → Apache-2.0

  8. getmaxun/maxun

    Turn any website into a structured API. Extract, automate, search and monitor the web.

    TypeScript★ 18.1kAGPL-3.0GitHub ↗

    Why this match

    topics automation, crawler, scraper; language differs: Python → TypeScript; license differs: MIT → AGPL-3.0