discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens

lorien/awesome-web-scraping · source

Snapshot 0 · 2026-11-04T12:47:23Z · JavaScript ·

GitHub description at capture

List of libraries, tools and APIs for web scraping and data processing.

README at capture

# Awesome Web Scraping

List of packages, services, and manuals related to web scraping.

## Topics

* [Python](https://github.com/lorien/web-scraping/blob/master/python.md) - Python packages
* [PHP](https://github.com/lorien/web-scraping/blob/master/php.md) - PHP packages
* [Ruby](https://github.com/lorien/web-scraping/blob/master/ruby.md) - Ruby packages
* [JavaScript](https://github.com/lorien/web-scraping/blob/master/javascript.md) - JavaScript packages
* [Go](https://github.com/lorien/web-scraping/blob/master/golang.md) - Go packages
* [Command Line Tools](https://github.com/lorien/web-scraping/blob/master/cli.md) - tools with a command line interface
* [Web-Scraping Manuals](https://github.com/lorien/awesome-web-scraping/blob/master/manuals.md) - list of articles and books teaching web scraping
* [dhamaniasad / HeadlessBrowsers](https://github.com/dhamaniasad/HeadlessBrowsers) - list of (almost) all headless web browsers in existence
* [DNS over HTTPS providers](https://github.com/curl/curl/wiki/DNS-over-HTTPS) - list of DNS over HTTPS providers
* [Awesome Pastebins](https://github.com/lorien/awesome-pastebins) - list of pastebin sites
* [Awesome Tunneling](https://github.com/anderspitman/awesome-tunneling) - List of tunneling software and services. Focus on self-hosting.

## Captcha Solving Services

* [https://2captcha.com](https://2captcha.com/?from=3019071)

## Proxy Server Marketplaces

* https://www.blackhatworld.com/forums/proxies-for-sale.112/
* https://forum.antichat.com/forums/147/

## Telegram Discussion Groups
* [@grablab_ru](https://t.me/grablab_ru) - web-scraping talks in Russian

## How to Contribute to This List

Please see the [CONTRIBUTING.md](CONTRIBUTING.md) document.

## Credits

The list is initially based on partial data from these sources [awesome-python](https://github.com/vinta/awesome-python), [awesome-php](https://github.com/ziadoz/awesome-php), [awesome-ruby](https://github.com/markets/awesome-ruby), [ruby-nlp](https://github.com/diasks2/ruby-nlp), [awesome-javascript](https://github.com/sorrycc/awesome-javascript)