crawler
Tested projects carrying the crawler topic, with their current verdicts. Topics come from each project's own metadata and Argusic's pool tags. Spellings of the same topic are gathered here under one name, so this page holds every project tagged any of them.
Measured by Argusic on a fresh machine every time. Every number links to its evidence.
What Argusic measured in crawler
Argusic installed and ran 83 crawler projects on clean machines: 63 reached a running state, 17 only ran against stand-in services, 3 did not run. Last tested November 6, 2026.
Every tested crawler project
| subject | verdict | language | runs | last tested |
|---|---|---|---|---|
| node-crawler | Runs with mocks | TypeScript | 2 | |
| trafilatura | Runs | Python | 2 | |
| JMComic-Crawler-Python | Runs | Python | 2 | |
| crawlee-python | Runs | Python | 2 | |
| webmagic | Runs | Java | 2 | |
| crawlab | Runs | Go | 2 | |
| katana | Runs | Go | 2 | |
| proxy_pool | Runs | Python | 2 | |
| colly | Runs | Go | 2 | |
| scrapy | Runs | Python | 2 | |
| crawl4ai | Runs | Python | 2 | |
| Scrapling | Runs | Python | 2 | |
| ghs | Runs with mocks | Java | 1 | |
| crawler | Runs | Java | 1 | |
| ticket-monitoring | Runs | Python | 1 | |
| go-movies | Runs | Go | 1 | |
| laravel-seo-scanner | Runs with mocks | PHP | 1 | |
| xvideos | Runs | TypeScript | 1 | |
| gplay-scraper | Runs with mocks | Python | 1 | |
| 4chan-downloader | Runs | Python | 1 | |
| extractor | Runs with mocks | TypeScript | 1 | |
| Laravel-Crawler-Detect | Runs | PHP | 1 | |
| aliexpress-product-scraper | Runs | JavaScript | 1 | |
| hQuery.php | Runs | PHP | 1 | |
| scrapy-zyte-smartproxy | Runs | Python | 1 | |
| crawler | Runs | PHP | 1 | |
| nebula | Runs | Go | 2 | |
| news-crawl | Runs with mocks | Java | 1 | |
| top-user-agents | Runs | JavaScript | 1 | |
| ir-search | Runs | Python | 1 | |
| Hydra | Runs | Java | 1 | |
| sparkler | Runs | Python | 1 | |
| catbus | Runs with mocks | TypeScript | 1 | |
| fundus | Runs | Python | 1 | |
| scrapple | Runs with mocks | Python | 1 | |
| reader | Runs | TypeScript | 1 | |
| rebrowser-patches | Runs | JavaScript | 2 | |
| freeproxy | Runs | Python | 1 | |
| NetDiscovery | Runs | Java | 1 | |
| Krawl | Could not verify | Python | 2 | |
| xxl-crawler | Runs | Java | 1 | |
| jvppeteer | Runs | Java | 2 | |
| seonaut | Runs | Go | 1 | |
| spidr | Runs with mocks | Ruby | 1 | |
| Scavenger | Runs with mocks | Python | 1 | |
| scrapyrt | Runs | Python | 1 | |
| chatWeb | Runs with mocks | Python | 1 | |
| TikHub-API-Python-SDK | Runs | Python | 1 | |
| icrawler | Runs | Python | 1 | |
| siteone-crawler | Runs | Rust | 1 |
All tested projects: subjects.