brainstate

discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens

codelucas/newspaper

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

Python★ 15.2kMITGitHub ↗

History and source changes · Reports and owner responses · Compare

Similar

  1. gocolly/colly

    Elegant Scraper and Crawler Framework for Golang

    Go★ 25.6kApache-2.0GitHub ↗

    Why this match

    topics crawler, crawling, scraper; language differs: Python → Go; license differs: MIT → Apache-2.0

  2. binux/pyspider

    A Powerful Spider(Web Crawler) System in Python.

    Python★ 16.8kApache-2.0archivedGitHub ↗

    Why this match

    topics crawler, python; license differs: MIT → Apache-2.0

  3. scrapy/scrapy

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python★ 65.2kBSD-3-ClauseGitHub ↗

    Why this match

    topics crawler, crawling, python; license differs: MIT → BSD-3-Clause

  4. apify/crawlee-python

    Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

    Python★ 9.6kApache-2.0GitHub ↗

    Why this match

    topics crawler, crawling, python; license differs: MIT → Apache-2.0

  5. apify/crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

    TypeScript★ 26.3kApache-2.0GitHub ↗

    Why this match

    topics crawler, crawling, scraper; language differs: Python → TypeScript; license differs: MIT → Apache-2.0

  6. newsnext/newsnow

    Elegant reading of real-time and hottest news

    TypeScript★ 22.3kMITGitHub ↗

    Why this match

    topics news; language differs: Python → TypeScript

  7. injetlee/Python

    Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

    Python★ 10.9kGitHub ↗

    Why this match

    topics crawler, python

  8. getmaxun/maxun

    Turn any website into a structured API. Extract, automate, search and monitor the web.

    TypeScript★ 18.1kAGPL-3.0GitHub ↗

    Why this match

    topics crawler, crawling, scraper; language differs: Python → TypeScript; license differs: MIT → AGPL-3.0