开源 Web 抓取工具，已安装并运行

使用 AI coding agents 通过一条命令克隆任何网站

测试发现：Next.js 16.3.0 project builds, typechecks, and serves HTTP 200 on the root route via npm run dev. 4 分钟。

用于 Golang 的优雅抓取与爬虫框架

测试发现：The Colly module builds, vets, passes its full unit and race test suites, its CLI prints help, and its scraper successfully crawled a local mock site end to end. 14 分钟。

新一代爬虫框架。

测试发现：katana v1.7.0 is installed, resolves from a fresh login shell, crawls HTTP sites in standard and headless modes, and its unit, functional, and integration suites pass. 7 分钟。

将文档网站、GitHub 仓库和 PDF 转换为 Claude AI 技能，并自动检测冲突

测试发现：skill-seekers 3.10.0.dev0 installed and all 3857 unit/adaptor/scraper tests pass without external API keys or network dependencies. 6 分钟。

newspaper3k 是一个用于 Python 3 的新闻、全文和文章元数据提取工具。高级文档：

测试发现：newspaper3k 0.3.0 installs in a virtualenv on Python 3.12, all 40 unit tests pass, and core Article download/parse/nlp pipeline produces correct output from static HTML. 4 分钟。

无限制的浏览器自动化与测试。SeleniumBase 的 CDP Mode 可轻松绕过机器人检测。查看示例！

测试发现：SeleniumBase 4.54.7 is installed in a Python 3.12 venv; 8 framework unit tests, 7 offline HTML tests, and 2 real web tests passed using headless Chrome 154. 5 分钟。

分布式爬虫管理平台，用于管理任何语言和框架的爬虫。

测试发现：Crawlab v0.6.3 backend compiles, starts as master node, connects to MongoDB, and serves its REST API on port 8000 (system-info, login, nodes, spiders, tasks, users, stats endpoints confirmed reachable). 67 分钟。

用于 Java 的可扩展 Web 爬虫框架。

测试发现：The webmagic project compiles and passes all 108 tests with JDK 22 and Maven 3.9.9. Tests exercise real HTTP downloads from github.com (e.g. GithubRepoPageProcessorTest downloads and processes 1 page), SSL compatibility checks, XPath/CSS/regex selectors, file pipeline, proxy providers, duplicate-removal scheduling... 13 分钟。

Crawlee：用于构建可靠爬虫的 Python 网页抓取和浏览器自动化库。提取数据以用于 AI、LLM、RAG 或 GPT。从网站下载 HTML、PDF、JPG、PNG 和其他文件。支持 Parsel、BeautifulSoup、Playwright 和原生 HTTP。支持有头和无头模式。支持代理轮换。

测试发现：Dependencies installed with uv sync --all-extras, Playwright browsers (chromium + firefox) downloaded, all 2480 unit tests pass, lint and type-check pass, and the package builds and imports successfully at version 1.10.3. 8 分钟。

更轻量的 Python Web 自动化

测试发现：Helium 7.0.3 installed from source via pip install -e . Selenium 4.49.0 with Chrome 154 headless via Selenium Manager; all 332 tests pass with 2 regressions fixed (date input JS fallback, subprocess module discovery). 33 分钟。

JMComic 的 Python API | 提供 Python API 访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂 GitHub Actions 下载器

测试发现：Package jmcomic 2.7.7 installed in a Python 3.12.3 venv with all dependencies; 238/240 tests pass including real network calls against JMComic API servers; CLI commands jmcomic and jmv produce correct help output. 10 分钟。

Python 与命令行工具，用于在 Web 上收集文本和元数据：爬取、抓取、提取，输出为 CSV、JSON、HTML、MD、TXT、XML

测试发现：Trafilatura 2.2.0 is installed in editable mode in a .venv, all 428 tests pass, and both CLI and Python extraction work correctly on sample HTML input. 6 分钟。

Python binding for curl-impersonate fork via cffi. 一个能够模拟浏览器 TLS/JA3/HTTP2 指纹的 HTTP 客户端。

测试发现：After installing the pre-built curl-cffi wheel (v0.16.3) into a fresh venv, the library imports, the CLI runs (curl-cffi get, doctor, --help), real HTTP requests with browser impersonation return 200, and the unit test suite passes 600+ tests across all 20 test modules including sync/async HTTP, websockets... 38 分钟。

用于结构化提取工作流的声明式数据自动化语言和 Go 运行时。

测试发现：Go 1.27.1 installed, all dependencies resolved, all 68 test packages pass with race detection, CLI binary compiles and runs FQL queries end-to-end. 7 分钟。

基于 Redis 的 Scrapy 组件。

测试发现：scrapy-redis 0.9.1 is installed in a venv at /work/repo/.venv with real Redis 6.2 running on localhost:6379; all 102 tests pass covering connection pooling, dupefilter, queue (FIFO/LIFO/priority), scheduler, spider setup, idle backoff, multi-key consumption, and utility modules. 9 分钟。

面向人类的 Node.js 爬虫。

测试发现：scrape-it installs, builds, and passes its full test suite on Node 18.19.1 with cheerio pinned to 1.0.0. 3 分钟。

每个网站都提供 API。

测试发现：toapi 2.2.4 installed from source, all 7 tests pass, CLI help works, Flask server answers 200 with JSON, version 2.2.4 confirmed. 2 分钟。

面向 AI agents 的隐身浏览器自动化，基于 MCP 和 Chrome DevTools Protocol：导航、网络钩子、DOM 提取和像素级准确的 UI 克隆。

测试发现：Stealth Browser MCP server runs on stdio transport, spawns a headless Chromium 157.0.8086.0, navigates to data URLs, extracts page HTML content, and closes cleanly -- all via MCP protocol over stdio. 15 分钟。

PHP Curl Class 让发送 HTTP 请求和集成 Web API 变得简单。

测试发现：php-curl-class repository is fully installed with PHP 8.5.8 static binary and all Composer dependencies; the full test suite passes with 264 tests and 23390 assertions and zero failures. 43 分钟。

用于从 Google Play 获取数据的 Node.js 爬虫

测试发现：The google-play-scraper npm package installs, all 84 tests pass against live Google Play, and the library's app/list/search/suggest/developer/reviews/similar/permissions/datasafety/categories methods all fetch and parse real data correctly. 8 分钟。

owllook：小说搜索引擎

测试发现：The owllook web server launches on port 8001 and responds with 200 HTTP status on /, /novels/search, /novels/top, /api/search, and /health routes, with real HTML content, connecting to running Redis and MongoDB instances. The existing pytest test suite passes (1 passed, 0 failed). 15 分钟。

https://spatie.be/docs/crawler

测试发现：PHP 8.4.23 and Composer 2.7.1 installed from static binaries, all dependencies resolved via `composer update --prefer-stable`, and the full test suite passes (229/229) including both fake-HTTP unit tests and real-HTTP integration tests against a local PHP server. 6 分钟。

基于 Rust 的基础性低延迟 Web 数据收集

测试发现：Rust 1.99.0 installed, cargo build succeeds, spider CLI binary crawls and scrapes example.com returning status 200 and expected page content, all 608 spider core tests pass, workspace-wide crate tests all pass. 56 分钟。

Geziyor，专为 Go 打造的极速网络爬虫与抓取框架。支持 JS 渲染。

测试发现：The Geziyor web scraping framework builds and passes every test across all 7 packages, and a compiled demo successfully scraped a live website. 16 分钟。

易用的轻量化网络爬虫

测试发现：Build succeeded (mvn compile, package, test-compile all exit 0); demo crawler successfully fetched and parsed https://www.baidu.com/, producing JSON output. 5 分钟。

CrawlerDetect 是一个 PHP 类，用于通过 user agent 检测机器人/爬虫/蜘蛛。

测试发现：The CrawlerDetect PHP library installs, builds, and passes its full test suite of 28 tests with 170,886 assertions on PHP 8.0.30. 5 分钟。

Website Cloner - 利用强大的 Go routines 在几秒钟内将网站克隆到你的计算机上。

测试发现：goclone binary builds from source, all 9 test suites across 3 packages pass, the built binary's help and version commands print correct output. 11 分钟。

基于 asyncio 的 Web 爬虫框架。

测试发现：All 7 tests pass, ruff lint passes, library imports correctly, and item attribute extraction from HTML produces expected results. 1 分钟。

灵活的 Node.js 爬虫库

测试发现：x-crawl library builds successfully; its crawlData, crawlHTML, async/sync modes, baseUrl, timeout, errorCollect, proxy rotation, and Puppeteer browser APIs work correctly against a real HTTP server and CONNECT proxy. 34 分钟。

基于 asyncio 的异步 Python 3.6+ 网页抓取微框架

测试发现：Ruia 0.8.5 is installed in a Python 3.12 venv with uvloop; pip install -e .[uvloop] succeeds; 66 of 67 tests pass including all field extraction, item parsing, middleware, request fetching, and spider orchestration tests against live httpbin.org and news.ycombinator.com. 22 分钟。

Modest 和 Lexbor 引擎的 Python 绑定，是一个支持 CSS 选择器的快速 HTML5 解析器。

测试发现：selectolax C extension successfully built, installed in a venv, all 279 tests passing, and basic HTML parsing + CSS selector usage verified. 4 分钟。

Playwright 测试和自动化库的无检测 Python 版本

测试发现：Patchright 1.63.0 is installed, Chromium browser is downloaded and launchable, the CLI reports its version, and 2237 of 2239 Chromium tests pass (sync + async). 37 分钟。

易用的、注重隐私的 Python HTTP 客户端

测试发现：wreq Python package builds, installs, and passes all 57 unit tests against a local httpbin server, supporting HTTP/1.1 and HTTP/2 requests, cookie jars, redirect policies, TLS emulation (JA3/JA4), multipart uploads, streaming, and compression. 15 分钟。

轻量级 Ruby 网页爬虫/抓取工具，提供优雅的 DSL，可从页面提取结构化数据。

测试发现：Wombat 3.3.0 builds and installs as a gem under Ruby 3.2.3, all 56 RSpec tests pass, and a live Wombat.crawl scrape of a local WEBrick page returns correctly parsed structured data. 12 分钟。

Beanbun 是用 PHP 编写的多进程网络爬虫框架，具有良好的开放性、高可扩展性，基于 Workerman 和 vi。

测试发现：Beanbun 1.0.4 with vendored workerman and guzzle dependencies. PHP 8.3.6 extracted from apt. All source and vendor files parse without errors. Framework class instantiation, configuration, and all Helper methods (randUserAgent, formatUrl, getUrlByHtml) verified working. 16 分钟。

Newspaper4k 是备受喜爱的 Newspaper3k 的一个分支。用于从新闻网站提取文章、标题和元数据。

测试发现：newspaper4k 0.9.6 is pip-installed and editable in /work/repo/venv; CLI runs (help + real article extraction into JSON), 234 tests pass with 2 deselected stale summarization fixtures and 5 external-site failures; package state is clean working tree. 38 分钟。

一个基于 Apache Storm 的可扩展、成熟且多功能的网络爬虫

测试发现：Apache StormCrawler 4.0.0-SNAPSHOT builds from source (mvn clean install) and its full test suite passes 983 tests with zero errors and zero failures on Java 26 Temurin with Maven 3.10.0. 15 分钟。

SiteOne Crawler 是一个用于 SEO、安全、可访问性和性能优化的跨平台网站爬虫与分析工具，适用于开发者、DevOps、QA 工程师和顾问。支持 Windows、macOS 和 Linux（x64 和 arm64）。

测试发现：siteone-crawler 2.7.0.20260925-dev builds cleanly with cargo build --release, its offline test suite passes with 1673 passed and 0 failed, and the binary crawls a local HTTP server end to end returning HTTP 200 for both pages and generating an HTML report. 16 分钟。

适用于 10+ 平台的通用内容读取器 MCP 服务器

测试发现：x-reader installs in a fresh venv, all 10 tests pass, the CLI fetches real tweets/RSS/web content into the inbox, and the MCP server serves working stdio and SSE transports. 10 分钟。

高性能异步 Douyin(抖音) TikTok Xiaohongshu(小红书) Kuaishou(快手) Weibo(微博) Instagram YouTube(油管) Twitter(X) Captcha Solver(验证码解决器) Temp Mail(临时邮箱) API(接口)。

测试发现：SDK installs, all 110 tests pass, CLI prints version and help, type-check and lint are clean, and spec coverage confirms 1010/1010 endpoints are implemented. 1 分钟。

一个多线程爬虫框架，内置许多图片爬虫。

测试发现：icrawler installs in a virtualenv, all 5 built-in crawler tests pass against live services (Google, Bing, Baidu, BBC, Pinterest), and downloaded images are valid media files verifiable by Pillow. 21 分钟。

Scrapy spiders 的 HTTP API

测试发现：scrapyrt 0.18.1 is installed in a Python 3.12 venv with all dependencies. All 126 tests pass (including real end-to-end crawl tests with Flask site, Twisted reactor, and scrapyrt HTTP server). The CLI command scrapyrt --help prints usage correctly. 5 分钟。

开源 SEO 审计工具。

测试发现：SEOnaut installed from source (Go build OK, frontend CSS built, migrations applied to a local MariaDB) and ran on port 9000, serving sign-in (200), creating a user and project, completing a real crawl of 2 URLs, and rendering the dashboard. 9 分钟。

AI 驱动的网页抓取编辑器，带有可视化工作流构建器。使用自然语言构建、测试和部署网页抓取器。由 ScrapeGraphAI & LangGraph 提供支持。

测试发现：ScrapeCraft backend serves all API endpoints on uvicorn, WebSocket connections establish and respond, frontend builds successfully, chat uses real OpenRouter API key, JWT auth works, pipeline CRUD works, scraping endpoints respond. 14 分钟。

NetDiscovery 是一个基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

测试发现：The full multi-module Gradle build passes (BUILD SUCCESSFUL, 63 tasks), and the NetDiscovery spider engine example ran as a Java process answering HTTP 200 with JSON on port 8715 routes /netdiscovery/health/, /netdiscovery/spiders/, /netdiscovery/spider/tony1/detail, and /netdiscovery/metrics/, while the downloader... 69 分钟。

跨社交媒体获取用户数据。

测试发现：8/8 tests pass; GitHub and Pinterest CLI scrapers fetch real profile data; the 6 remaining scrapers return None gracefully given no browser binary or API access. 16 分钟。

FreeProxy：从互联网收集免费代理。（全球海量高质量免费代理，支持爬取数十个免费代理分享源，支持自定义规则代理筛选，爬虫与数据分析必备，每日更新海量免费代理。）

测试发现：pyfreeproxy v0.5.2 installed in /tmp/venv, all 48 proxy sources registered, Proxifly/ProxyScrape/Geonode scrape real proxies successfully, filter rules and ProxiedSessionClient work, JSON export round-trips correctly, free proxies collected but actual proxy requests time out as expected from this container network. 3 分钟。

Python 下的无检测网页抓取与无缝 HTML 解析

测试发现：stealth-requests 2.1.0 installed in a fresh venv, all 64 tests pass, and real requests to httpbin.org succeed with proper Chrome impersonation headers. 1 分钟。

开源 Web 基础设施，用于 AI。抓取、爬取和自动化网页，清理 Markdown，浏览器会话，为你的 agent 做好准备。

测试发现：Reader CLI builds, all 346 unit tests pass, the daemon starts with Playwright's bundled Chromium, and an end-to-end scrape of https://example.com via HTTP returns clean HTML and markdown content with a 200 status. 13 分钟。

将任何网站转化为专为 AI agents 设计的紧凑型 CLI。用数百个 tokens 浏览网页，而非数万个。

测试发现：only-cli v0.5.8 installs and runs on Node 20.11.1; 263 of 264 tests pass; the CLI fetches, distills, and renders public web pages with token-budgeted output and actionable numbered links. 3 分钟。

一个非常简单的新闻爬虫，名字很有趣。

测试发现：Fundus 0.5.7 installs cleanly in a venv, all 1131 unit tests pass, and the crawler successfully fetches live article data from The New Yorker via HTTP. 3 分钟。

自托管的多渠道机票价格监控：航班明细采集 → DingTalk 图文推送 → 达标时 four 层强提醒。国内航线与国际航线（five 洲目的地）共用同一套采集-判定-推送链路；数据留在本机，配置全在网页，开源免费。

测试发现：All dependencies installed in venv, Playwright chromium installed, test suite runs 3712 tests all passing, web UI demo server starts and responds with HTTP 200 on its state API endpoint, main.py --version reports 1.5.303. 44 分钟。

面向 AI agent 的反检测浏览器服务器：REST API 封装了 Camoufox 引擎并支持 OpenClaw 插件

测试发现：Camofox Browser Server builds, starts on port 9998, creates browser contexts via Camoufox Firefox headless, navigates to https://example.com, returns accessibility snapshots, and passes the 677-test full suite with Node.js 20 and rebuilt native addons. 34 分钟。

一个爬取工作站：查看、控制与爬取。Solr CrawlDB、Tika、Vue 3。

测试发现：Sparkler is installed with Python 3.12, Solr 10.0.0 on port 8983, Vue UI built and served on port 8180, APIs responding, all 32 unit tests passing. 7 分钟。

为超级个体和一个人公司打造一个人的大厂，Hydra九头龙构筑大规模AI调度、数据采集、情报系统、数据平台、分析决策、产品生产的'军事'工业基座。

测试发现：After fixing module-name casing, pinning Lombok 1.18.34, and adding the missing System.Subsystem config block, the Maven build succeeds and the ShadowBoot crawler app boots and runs the README's Void minimum-system demo to completion (exit 0, 'Pinecone Runtime : 136 /ms !'), while the full test suite passes with 48... 9 分钟。

用于 Google Trends 分析的 Python CLI，具有高级速率限制、Cookie 认证和美观的终端输出。基于 pytrends 构建，具备企业级功能。

测试发现：trends-checker installs and runs, hitting the real Google Trends API via pytrends (with urllib3 compat patch applied to pytrends request.py), returning real interest data verified at score 42.83 for 'AI agents' in US. 5 分钟。

韩国政府支持项目全面调查 agent 技能 [Claude Code·Codex·agy(Antigravity)·Cursor·Gemini CLI·Grok Build 支持] K-Startup·기업마당·NIPA·KOCCA·SMTECH 爬取 + 即时/路线图/变体 3 分类（仅限韩国）

测试发现：The project's 229 pytest tests pass, and every CLI crawler (K-Startup, bizinfo, NIPA, KOCCA, SMTECH list and detail, scope_plan, diff_surveys) ran successfully against live Korean government sites, producing real announcement records and detail text. 4 分钟。

用于快速开发 Web 爬虫和抓取器的库

测试发现：1278 of 1278 default unit tests pass, PHP 8.3.32 and Composer installed, Chrome for Testing available, integration server fixed with short_open_tag=Off and 500ms startup delay. 27 分钟。

Zyte Smart Proxy Manager（原名 Crawlera）的 Scrapy 中间件

测试发现：scrapy-zyte-smartproxy v2.4.1 installed in a Python 3.12 venv; all 82 existing unit tests pass with zero failures. 1 分钟。

一个极快的网页抓取器，能在眨眼间解析数兆字节的无效 HTML。PHP5.3+，无依赖项。

测试发现：hQuery.php library installs, autoloads, and parses HTML via fromHTML() and find()/text() selectors; all 27 unit tests pass on PHP 8.4.1 via FrankenPHP static binary. 7 分钟。

获取 Aliexpress 商品详情作为 JSON 响应，包括反馈、变体、配送信息、描述、图片等。

测试发现：Node 24.0.0 installed, pnpm installed, Chromium 145 downloaded, all 22 tests pass, ESLint clean. 1 分钟。

CrawlerDetect 的 Laravel 封装：一个网页爬虫检测库。

测试发现：All 9 package tests pass, detecting crawlers from real user agent lists via the Laravel facade and Request macros. 4 分钟。

提取公开的 Spotify 数据（曲目、专辑、艺术家、播放列表、播客和歌词），无需官方 API。同步 + 异步，类型化模型，单一依赖项。

测试发现：The SpotifyScraper package is installed and working: the CLI fetches real Spotify track/artist/playlist data over the network, the hermetic unit test suite passes completely (921/921), and the MCP server imports and launches correctly after pinning mcp to v1. 3 分钟。

开源 AI 可见性追踪器。追踪 ChatGPT、Perplexity、Gemini、Copilot 和 Google AI Mode 如何提及和引用你的品牌。自托管，MIT 许可证，自带密钥。

测试发现：Next.js dev server runs on port 3000 serving the GEO/AEO Tracker dashboard in demo mode, with all pages (/ and /demo) and API routes (/api/analyze, /api/audit, /api/scrape, etc.) responding correctly. 4 分钟。

一个 Python3 脚本，无需安装，可持续同时下载多个 4chan 帖子的所有图片和视频。

测试发现：Both scripts compile, install cleanly via venv, and successfully interact with live 4chan API endpoints: thread-watcher queries the catalog and writes URLs, inb4404 downloads thread media files with validated content. 4 分钟。

xvideos API 库

测试发现：npm install completes with warnings. Build succeeds. 5 unit test files (47 tests) pass on fixture HTML. Full test suite: 52 pass, 4 fail (all integration tests blocked by xvideos.com returning 404 on /verified/videos/). 5 分钟。

为 LLMs 优化的高性能网页爬虫 API。使用远程浏览器将任何搜索或网站转换为干净的 Markdown。Firecrawl 的替代方案。

测试发现：Teracrawl server starts on port 8085 with a local Chromium-backed mock Browser.cash pool and a mock SERP service. GET /health returns 200, POST /scrape converts a live URL to markdown via real headless Chromium, POST /serp/search returns mock search results, and POST /crawl combines both to scrape multiple URLs. 15 分钟。

免费、自托管的 MCP 服务器，抓取公开的 Facebook Ad Library（无需 API token），用于竞争性广告研究：广告主、广告文案、落地页、CTA、任何国家。

测试发现：MCP server loads, imports, starts over stdio, registers both tools, and all 8 offline parser tests pass. 7 分钟。

golang spider 电影爬虫

测试发现：GoMovies binary compiled and serves HTTP on port 8899 with Redis persistence; spider successfully fetches movie data from hhzyapi.com external API. 20 分钟。

一个用于直接从 oddsportal.com 抓取和处理体育博彩数据的 python 应用

测试发现：OddsHarvester v0.15.0 installed from source with uv and Playwright; all 1308 unit tests pass and 63 of 82 integration tests pass against HAR replay (19 live-only skipped, 1 xfail for known in-play replay limitation). 40 分钟。

Project Eyes On 是一个由 Y0oshi (@rde0) 开发的高速、多线程监控工具，用于定位全球开放的 IP 摄像头。它将 Google Dorking 和 Directory Scraping 统一到一个 OSINT 引擎中。

测试发现：Project Eyes On installed, imports correctly, Insecam scraping and multi-engine dorking both find real live IP cameras, and JSON/CSV/HTML reports all produce valid output. 2 分钟。

一个命令适用于所有接口：跨 API、浏览器、桌面、本地工具和 MCP 搜索、运行并检查真实软件。

测试发现：Uni-CLI v1.2.1 builds from source, runs 3735 passing unit tests, and executes live commands (hackernews top, search, doctor) against real remote APIs on Node 22.23.4 (npm v10.9.9). The npm global package v1.3.0 is also installed and functional. 20 分钟。

自托管的 TikTok、Douyin 与 Bilibili 数据采集器和无水印视频下载器，提供异步 REST API、面向 AI agent 的 MCP 服务器、CLI 和 Web 控制台，并支持对已下载视频进行离线语音转文字。具备自愈身份池和 PostgreSQL 归档，一条 docker compose up...

测试发现：All 2960 tests pass (2450 unit + 510 integration), the CLI outputs version 5.0.1 and lists 10 commands, the API server starts and serves healthz/readyz/swagger/console root with 200, the web console SPA is built, and both the PostgreSQL+TimescaleDB and Redis backends and the Go downloader sidecar run. 48 分钟。

用于 AI agent 的隐形无头浏览器，可绕过 Cloudflare、机器人检测和反爬虫。可直接替代 Puppeteer/Playwright。

测试发现：Camofox browser server installed and verified: Node 22.23.4 in user space, Camoufox 1.3GB binary downloaded, GeoIP 50MB database downloaded, server starts at localhost:9377 with Xvfb virtual display, creates browser tabs navigating to real URLs, returns accessibility snapshots with element refs, and closes cleanly on... 13 分钟。

最好的 Web 抓取框架，从单个请求到全规模爬取都能处理！

测试发现：Scrapling 0.4.15 is installed in a venv with all extras, Playwright Chromium browser available, all 985 tests pass, CLI and MCP server commands respond correctly, and HTTP fetches via StealthyFetcher return status 200 from real endpoints. 8 分钟。

用于网络爬虫的 Python ProxyPool

测试发现：The ProxyPool Flask API server runs on port 5010 backed by real Redis, responds to all endpoints (/get, /pop, /all, /delete, /count), and all 248 automated tests pass. 6 分钟。

用于 Web 自动化和数据抓取的 Chrome DevTools Protocol 驱动程序。

测试发现：Rod compiles with go build ./... passes all vet checks, and launches Chromium 128 to control a browser page via DevTools Protocol. The main rod test suite and lib/cdp, lib/launcher (except known flaky TestManaged), and lib/proto sub-packages all pass. 59 分钟。

可模拟浏览器的 HTTP 客户端

测试发现：The repo builds and runs: Rust workspace checks and per-crate test suites pass (1354 tests), the primp Python extension installs into crates/primp-python/.venv, 493 Python tests pass, and a real impersonated HTTPS request returns HTTP 200. 15 分钟。

高性能浏览器自动化桥接与多实例编排器，支持高级隐身注入和实时仪表盘。

测试发现：A source-built PinchTab server (binary reports pinchtab dev) is running on 127.0.0.1:9867 with a responsive always-on headless instance and answered authenticated health, navigate, snapshot, text, title, and screenshot API calls with HTTP 200, while the full Go unit suite (9166 passed, 18 skipped) and scripts/check.sh... 19 分钟。

基于 Puppeteer 的无头 Chrome/Chromium 驱动程序。通过生产就绪的 API 截取屏幕截图、生成 PDF、提取文本和 HTML。

测试发现：The browserless monorepo installs under Node.js 24 with puppeteer 25, all packages build, Chrome 154 launches headlessly, and 390+ tests pass across 10 packages (browserless, goto, screenshot, capture, function, devices, errors, screencast, lighthouse, ai). 32 分钟。

100% 免费且完全开源的 Firecrawl 边缘替代品，为 agents 提供更好的链接提取，你可以自行部署到 cloudflare 或 vercel。

测试发现：Dependencies installed, SDK tests pass (41/41), full monorepo build and typecheck pass, and the dashboard serves HTTP 200 on / and /docs in both dev and production modes; the V0 API worker served 200 on /, /docs and /openapi in local mode with D1 migrations applied. 33 分钟。

一个轻量级的网页爬虫框架（Java爬虫框架）

测试发现：XXL-CRAWLER builds from source, produces a deployable jar, and passes 6 tests (4 utility tests + 2 real website crawls extracting structured data from cnblogs.com and baidu.com). 13 分钟。

用于 Web 自动化的开源 AI agent。使用你真实且已登录的浏览器来自动化日常任务。

测试发现：Dependencies installed, project builds to dist/ (background.iife.js 1,848.81 kB, manifest.json), TypeScript type-check passes across all 15 packages, unit tests pass (14/14). 15 分钟。

官方 Firecrawl MCP 服务器，面向 AI agent 的上下文 API：从任何 MCP 客户端搜索、抓取和与 Web 交互，以及解析、爬取、映射、Agent 和监控。

测试发现：Firecrawl MCP Server v3.24.1 builds, installs, and starts successfully with 25 tools in stdio mode; 81 of 84 tests pass. 24 分钟。

Scrapy，一个快速高级的网络爬虫与数据抓取框架，用于 Python。

测试发现：Scrapy 2.19.0 installed from source, 4539 tests pass, scrapy bench crawls 1781 pages against a local HTTP server. 15 分钟。

使用 Go 编写的快速简单的视频下载库和 CLI 工具

测试发现：Lux builds and runs: the binary outputs version info and help, extracts real YouTube video metadata, and 14 of 48 test packages pass (all failures are network-dependent extractor tests or DNS-timeout content-type lookups; no code defects remain). 11 分钟。

news-please：用于新闻的集成 Web 爬虫与信息提取器，开箱即用

测试发现：news-please 1.6.13 installs and its library API extracts structured article data (title, authors, publication date, description, maintext, language) from HTML via both HTTP and raw HTML input; CLI entry points respond with help output. 5 分钟。

Playwright 测试和自动化库的 NodeJS 无检测版本。

测试发现：Repo dev deps install and both repo checks pass under Node 22; the npm-published patchright 1.63.0 with its installed patched Chromium launches headless and loads a page from a local HTTP server in this container. 3 分钟。

始终更新的前 100 个最常见浏览器 User-Agent 列表，适用于 HTTP 客户端。

测试发现：The repo installs cleanly; with the bundled prebuilt Node 22, npm test passes all 5 tests and require/consumer checks return 100 main, 82 desktop, and 18 mobile valid user-agents from the packed tarball; only the Redis-backed update scripts remain unrunnable in this container because no redis-server is available. 3 分钟。

为 AI agents 构建的浏览器自动化 CLI。突破反机器人墙，卡住时跨平台移交给人类。并行多任务执行，独立多会话操作，隔离多账号浏览。

测试发现：browser-act-cli 1.4.2 installed, skill handshake completed, and the chrome browser type (bundled Chromium) works end-to-end, browser create, open, navigate, state inspection, content extraction via get markdown, and session close all succeed; stealth features require a server-side-validated BrowserAct API key that... 5 分钟。

一个无关网络的爬虫，提供全面的对等节点信息和网络拓扑信息。

测试发现：Nebula builds with just build, runs dry-run and ClickHouse-backed IPFS crawls against real network peers, persists crawl data to ClickHouse, and passes all 92 tests across 7 packages with Postgres 16.4 and ClickHouse 26.11 running locally on their test ports. 16 分钟。

开源 Python 库，用于 AI 浏览器 agents，以及 Browser Use Cloud：隐身浏览器和托管 agent API。

测试发现：browser-use v0.13.10 installed in a Python 3.12 venv with uv, Chromium headless installed via playwright, and all 563 CI tests pass (529 passed, 34 skipped) after fixing 2 test identity comparison bugs in test_beta_agent.py that only manifest under xdist worker parallelism. 75 分钟。

离线隔离、注重隐私的开源 NotebookLM 替代方案。一个 AI agent，用于研究、转换和编辑你的文档。

测试发现：All Python trees (surfsense_backend, surfsense_mcp, surfsense_local/backend, plugins/remote/host core tests) install and pass tests; the web frontend fails to compile native deps on Node 18.x (requires 24.x); docs are clean. 20 分钟。

用于网站变更检测、网页监控和网站变更提醒的最佳且最简单的工具。非常适合跟踪内容变更、价格下降、补货提醒和网站篡改监控，全部免费，或使用我们的 SaaS 套餐。

测试发现：changedetection.io 0.60.7 installs into a venv, launches and serves HTTP 200, its full basic test suite passes (680 passed in the main group, exit 0, plus 508 unit tests), and a real watch added via the API detected a content change and exposed it in the UI. 76 分钟。

jsoup：Java HTML 解析器，专为 HTML 编辑、清理、抓取和 XSS 防护而构建。

测试发现：jsoup compiles from source with JDK 21 and Apache Maven, all 2,341 unit tests pass, and the built JAR correctly parses HTML and extracts DOM content. 2 分钟。

免费、开源的 Google Maps 抓取器，用于线索生成（使用 Python，无需 API 密钥，无需 Docker）。导出名称、电话、邮箱、网站、评分以及 Instagram/Facebook/LinkedIn/TikTok/YouTube/X 链接到 CSV。44 秒内获取 100 个含邮箱的地点；--grid...

测试发现：The scraper binary runs natively, its REST API on localhost:8080 creates/polls/downloads scraping jobs against real Google Maps data, and the included Python CLI successfully produces lead CSVs with real business listings. 7 分钟。

Elasticsearch 文件系统爬虫 (FS Crawler)

测试发现：FSCrawler 3.1-SNAPSHOT builds successfully via Maven, distributable ZIP is packaged, binary launches and shows help/version, and 687 unit tests pass when run with the `-Dtest="*Test"` workaround. 82 分钟。

针对 puppeteer 和 playwright 的补丁集合，用于避免自动化检测和泄露。有助于绕过 Cloudflare 和 DataDome 的 CAPTCHA 页面。易于打补丁/取消补丁，可按需启用或禁用。

测试发现：rebrowser-patches repo installs, builds 6 drop-in tarballs, and passes all 43 test gates (detection, scenarios, smoke) with patched Puppeteer-core 25.14.0 and Playwright-core 1.64.1 defeating Runtime.Enable detection. 8 分钟。

一个易用、注重隐私的 Rust HTTP 客户端

测试发现：wreq v0.16.1 builds and all 280+ integration tests pass on Rust 1.99.0 (Ubuntu 24.04) with BoringSSL via btls, after providing libclang (extracted from libclang1-20 + libclang-20-dev debs) and workaround for Ubuntu 24.04's missing /usr/include/stddef.h. 35 分钟。

Chrome 和 Firefox 的 Java API

测试发现：Jvppeteer 3.6.5 builds and installs via Maven, Chrome for Testing 149.0.7798.0 runs headlessly, and the library launches the browser, navigates pages, evaluates JavaScript, and retrieves page content successfully. 26 分钟。

面向 AI agent 的全内容网页抓取器：Chrome TLS 指纹识别、浏览器模拟、多策略文章提取和网页抓取。

测试发现：agent-fetch 0.1.6 builds from source, passes all 904 unit tests, and successfully fetches and extracts real web pages (Wikipedia, example.com) via CLI. 6 分钟。

官方 Crawl4AI Cloud：一个面向 AI agents 的托管网页抓取、提取和搜索 API。免费 API 密钥，每月 1,000 积分，无需信用卡。完全开源：这个 repo 就是驱动它的 Apache-2.0 爬虫，你可以自托管它。

测试发现：Crawl4AI 0.9.4 installed in venv, Playwright chromium browser downloaded, DB initialized, AsyncWebCrawler can crawl real URLs with markdown output, cache operations work, and 360 core tests pass with 11 skipped. 14 分钟。

用于 NodeJS 的 Web 爬虫/蜘蛛 + 服务端 jQuery;-)

测试发现：crawler v2.1.1 builds from TypeScript, all 51 tests pass under Node 22 with nock-mocked HTTP, and the module loads correctly to process crawl tasks. 4 分钟。

一个面向消息平台的智能体提取器。Noria 通过聊天接收消息和 URL；抓取 web 数据，根据自定义评分矩阵（例如：职位、奖学金）评估内容；并将带有计算出的匹配分数的结构化摘要直接发送到指派的聊天号码。

测试发现：npm install completes, all 232 unit tests pass across 17 suites with zero lint errors, the Node.js backend boots registering all domain/LLM/scraper/sender/listener plugins and loading the scholarships and jobs pipeline configs, and the Streamlit Control Center UI starts and serves on port 8501. 11 分钟。

Python 库和 CLI，用于 X/Twitter 抓取，支持多账号轮换并内置限流处理。

测试发现：All 217 tests pass with 81% coverage; twscrape CLI reports version 0.20.1 and SQLite 3.45.1. 1 分钟。

一个强大的 Model Context Protocol (MCP) 服务器，提供公共 Web 访问的一站式解决方案。

测试发现：The Bright Data MCP server installs and all 21 unit tests pass under Node.js v22.12.0; the server starts listening when API_TOKEN is set. 4 分钟。

将任意网站逆向工程为类型化的 API 客户端。支持 Claude Code、Codex、Cursor 和 OpenCode。

测试发现：Project installed via uv sync, all 1025 tests pass, and the reverse-api-engineer CLI displays its help output. 2 分钟。

Diskover Community Edition：由 Elasticsearch 提供支持的开源文件索引器、文件搜索引擎和数据管理与分析

测试发现：Python scanner installs successfully, compiles cleanly, and completes a directory crawl indexing 22 file/directory documents into a mock Elasticsearch instance. 18 分钟。

Finviz 分析 Python 库。

测试发现：All 128 offline tests pass and every package module (quote, news, insider, calendar, screener, group, earnings, forex, crypto, future, util) returns correct data from committed HTML fixtures. 2 分钟。

Boss直聘爬虫 / BOSS直聘职位数据抓取工具，基于 Chrome CDP 协议复用真实登录态，绕过字体反爬，输出明文薪资 JSON/CSV + 薪资技能分析。一个基于 Chrome CDP 的 BOSS 直聘职位爬虫/抓取工具。

测试发现：Virtual environment created with all dependencies, both Python scripts compile clean, and the full mock-based test suite (109 tests) passes. The CLI prints correct version (2.2.0), help output, and city listings. Real Chrome CDP scraping cannot be verified in this container, --check correctly reports no Chrome... 2 分钟。

用于 X（原 Twitter）的 Twitter 爬虫：可抓取推文、个人资料、关注者和正在关注，无需 API 密钥。Python 库，支持多账号池、代理和异步，或使用 Apify 上的托管版本。

测试发现：Scweet 5.8.0 installs cleanly in a Python 3.12 venv; 495 unit tests pass; the CLI builds and shows help; the library imports without network; material in tests/fixtures/ provides captured X responses for offline test coverage. 9 分钟。

Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy 持续在欧洲的 ImmoScout24、Immowelt、eBay Kleinanzeigen 等平台上搜索新的公寓、房屋和住宅，并通过 Slack、Telegram、Email、Discord 或 ntfy...

测试发现：Fredy backend serves HTTP 200 on port 9998 after DB migration and all 4260 offline tests pass with Node v22.23.3. 6 分钟。

自托管网络搜索，适用于AI agents和SEO。Google, Bing, Yandex, Baidu, DuckDuckGo & Ecosia，在真实浏览器中渲染：排序结果和干净的页面Markdown。API, CLI, MCP

测试发现：OpenSERP v0.8.13 builds, all 12 unit test suites pass, and the HTTP server starts, serving /health and /ready endpoints on 127.0.0.1:7000. 8 分钟。

WebAI2API: 基于 Camoufox 的网页 AI 转 OpenAI API 工具，支持 LMArena/Gemini 等，多窗口并发与账号隔离。

测试发现：The WebAI2API server starts on port 3000 with a Camoufox browser instance, serving OpenAI-compatible /v1/models, WebUI, and /v1/cookies endpoints, all returning HTTP 200, though chat completions require real AI website credentials via the browser adapter and cannot produce output without logging in. 7 分钟。

Go HTTP 客户端，具有与浏览器一致的 TLS/HTTP2 指纹识别，通过精确模仿 Chrome、Firefox 和 Safari 在加密层面的特征（JA3/JA4、Akamai 指纹、请求头顺序）来绕过机器人检测，并支持 HTTP/1.1、HTTP/2、HTTP/3、会话、Cookie 和代理。

测试发现：Go 1.26.0 installed, httpcloak project builds cleanly, all 1402 Go test cases pass across 7 sub-packages, shared C library built, Python bindings installed and all 12 Python alias tests pass. 8 分钟。

HTTP(S)/SOCKS5 轮换住宅代理 - 代码示例与一般信息。

测试发现：Mock proxy verified: shell/shell.sh (curl via proxy): returned 200 with IP JSON; python/python.py (requests via proxy): returned 200 with IP JSON; nodejs/node.js (axios via proxy): returned 200 with IP JSON. 9 分钟。

Golang 短视频去水印：抖音、皮皮虾、火山、微视、最右、快手、全民小视频、皮皮搞笑、西瓜视频、虎牙、梨视频、acfun、好看视频...

测试发现：with a user-local Go 1.24.0 toolchain the project builds, its 90 passing tests and race suite are green, the HTTP server starts and serves 200 on / and /api/v1/health with 26 platforms, and a real Sohu video was parsed, downloaded (22.6 MB), and fully decoded by ffmpeg to h264/aac. 20 分钟。

660+ muapi-hosted generative-media models 加上社区提交的第三方 API 工具（SEO、增强、社交、抓取），每个条目一个 YAML 文件，可按能力浏览。

测试发现：PyYAML 6.0.3 installed; catalog_validate.py validates all 665 YAML entries in providers/ and models/ successfully with zero errors. 3 分钟。

针对 X.com 的纯 Python 协议工具和研究笔记，附带 VibeLoft Twitter 账户数据集

测试发现：All 20 integration tests pass - uv sync installs cleanly, all 8 tool modules import and return valid data from the mock X API server, plus VibeLoft account collection via real API, tweet analysis, and status scripts verified. 6 分钟。

ChatGPT Team(Business) 促销码自动扫描工具, 批量发现/验证/价格收集，支持 17 国 34 个码，最高折扣 71% | ChatGPT Business promo code scanner, batch discovery, validation, price collection, 34...

测试发现：Repo installs cleanly and every scanning script (auto_scan, discover_codes, verify, open_stripe, partner_scan, and the us/mega/uk scanners) ran and produced valid output against the local mock of the ChatGPT API and Clash proxy; open_stripe.py works after switching its hardcoded proxy to the project config-driven... 43 分钟。

ChatWeb 可以爬取网页、读取 PDF、DOCX、TXT 文件，提取主要内容，然后根据内容回答你的问题，或总结关键要点。

测试发现：ChatWeb runs in all three modes (console, API, webui) behind a mock OpenAI API. The /upload_file, /summary, and /answer endpoints return HTTP 200 with correct JSON schemas; the console mode accepts file paths and /help commands; the webui Gradio interface binds on port 7860 and returns HTTP 200. 15 分钟。

Oxylabs 产品的官方 Agent 技能

测试发现：Five Oxylabs agent skills (headless-browser, proxies, video-data, web-scraper-api, web-unblocker) install via skills CLI. .env.example corrected to OXY_RES_USERNAME/OXY_RES_PASSWORD consistent with all docs. Mock server confirms the documented API endpoint patterns respond 200 OK. 7 分钟。

在粘贴网站上搜索泄露信息的爬虫（Bot）

测试发现：Dashboard serves HTTPS on localhost:8080 returning complete HTML page; analysis engine detects email:password combos, private keys, API tokens, and 18 configurable search terms; SQLite dedup DB tracks crawled paste IDs; all 6 module scripts pass syntax check; pastes.io and pastebin.com external endpoints are reachable. 5 分钟。

一个多功能的 Ruby 网络爬虫库，可以爬取单个网站、多个域名、特定链接或无限爬取。Spidr 设计为快速且易于使用。

测试发现：Spidr builds and its full 317-example RSpec suite passes, and a locally hosted mock site is spidered end to end (3 URLs visited, no failures) with no real third-party credentials. 20 分钟。

一个命令操作小红书、抖音、TikTok、B 站、快手、微博、闲鱼、淘宝、京东、X：统一的命令与 JSON 输出，供人和 AI Agent 使用。

测试发现：catbus CLI is installed, builds, and all 962 offline tests pass. The binary outputs valid JSON envelopes for version, help, platforms, doctor, and handles platform aliases, native name mapping, and auth-required errors correctly. 3 分钟。

一项面向 AI agent 的技能：使用 SearXNG 搜索网络，使用 Camofox 浏览网页，使用 CloakBrowser 绕过保护。设计上防幻觉。自托管、免费、无限制。

测试发现：npm install succeeds, CloakBrowser (with Node 20, not the system Node 18) fetches live pages, the SearXNG and Camofox deterministic scripts pass their full command flows against schema-correct mocks, unit suites pass 28/28 and 12/12, and setup.sh/audit.sh complete without failed checks; real SearXNG/Camofox Docker... 33 分钟。

用于创建半自动 web 内容提取器的框架

测试发现：Scrapple 0.3.0 installs, builds, and runs on Python 3.12; the CLI accepts all 4 commands (genconfig, run, generate, web), scrapes live websites via XPath and CSS selectors, and generates executable Python scripts, 13/17 tests pass with the remaining 4 failing only because their target websites changed HTML since 2018. 10 分钟。

适用于 Node.js 的 HTTP 客户端，支持浏览器 TLS 指纹模拟。

测试发现：wreq-js v3.2.0 is installed and working on x86_64 Ubuntu 24.04 with Node.js v22.23.4, the TypeScript build completes, the native Rust binding (prebuilt) loads correctly, and the test suite passes 195/200 tests against a local HTTP/HTTPS/WebSocket test server. 5 tests are skipped: 4 requiring full native setCookies... 28 分钟。

使用姓名和域名发现并验证专业电子邮件

测试发现：Project builds from source, 9 unit tests pass, and both email-sleuth/es binaries execute their full pipeline (config loading, SMTP connectivity testing, pattern generation, DNS resolution, JSON output) without crashing. 6 分钟。

轻量级 Google News 爬虫

测试发现：TypeScript compiles cleanly, rollup builds CJS+ESM bundles, both module formats load at runtime, and all 46 unit/integration tests pass on Node 18.19.1. 13 分钟。

使用 StormCrawler 进行新闻爬取，将内容存储为 WARC 格式

测试发现：The NewsCrawlBootstrap topology runs in Storm LocalCluster against local mocks, detects the RSS feed, acks status updates to OpenSearch, and writes a gzip-valid WARC containing fetched feed content; mvn clean package and all 10 unit tests pass. 69 分钟。

Graphlit 平台的 Model Context Protocol (MCP) 服务器

测试发现：Graphlit MCP Server builds, launches, and responds correctly to MCP protocol initialize, resources/list, and tools/list requests. 10 分钟。

用于轻松抓取 Twitter 前端的 Python 软件包

测试发现：Package installs in a virtualenv, all 9 tests pass, xquik search function verified end-to-end with a mock HTTP server. 2 分钟。

GPlay Scraper 是一个强大的 Python Google Play 爬虫库，用于从 Google Play Store 提取全面的应用数据。爬取 Google Play Store 应用以获取评分、安装次数、评论、ASO 指标、开发者信息以及 65+ 个数据字段。

测试发现：gplay-scraper installed in a venv with all dependencies; all 48 tests pass after fixing the tls_client client_identifier string from 'chrome112' to 'chrome_112'. 4 分钟。

MinerU-HTML: 一个基于 SLM 的 HTML 主要内容提取器，输出干净的 HTML 正文。非常适合 Deep Research Agents、RAG 应用和训练数据生成。

测试发现：MinerU-HTML v1.1.2 package installs and imports correctly in a Python 3.12 venv; the full extraction pipeline runs to completion with a mock LLM backend, correctly filtering navigation/auxiliary elements from main content HTML; all 20 internal modules import cleanly; config validation, fallback handlers... 16 分钟。

扫描你的 Laravel 应用路由以获取 SEO 改进建议。

测试发现：composer install works, the full Pest suite passes (171 passed, 3 skipped), PHPStan passes, and seo:scan-url returns real results (score 65, 28 passed, 15 failed) plus seo:scan persists the score to MariaDB, all against local mocks in this container. 28 分钟。

Hyperbrowser 使用示例

测试发现：After npm install, TypeScript compiles cleanly and the company-researcher CLI runs end-to-end against a schema-correct local Hyperbrowser mock, producing the full structured research output and exiting 0; real Hyperbrowser/OpenAI API keys were not exercised. 12 分钟。

从 Google Maps 抓取数据，提取每个地点的名称、地址、电话号码、网站 URL、评分、评论数、纬度和经度、评论、电子邮件等信息。

测试发现：All 17 Go packages with tests passed with race detection (no failures), the binary compiled and prints version/help correctly, and all 10 JS skill tests plus shell helper tests passed. 6 分钟。

使用 LLM 稳健地提取 Web 数据

测试发现：npm install builds successfully, 84 unit tests pass, 63 html-to-markdown integration tests pass with real ground-truth data, all LLM extraction integration tests pass with mocked providers, processedContent tests pass; Playwright browser tests fail at import on Node 18. 39 分钟。

基于 AI 的 Python 爬虫

测试发现：191 unit tests pass after fixing 13 categories of test failures covering models_tokens assertions, cleanup_html behaviors, logger capture, file paths, missing dependencies, mock patterns, and mock class completeness. 46 分钟。

通过 SERP API 获取 Google 搜索结果的 pip Python 包

测试发现：The SerpApi Python client library installs from source, runs 43 unit tests successfully against a locally hosted mock backend, and passes a basic smoke test (OOBT). 11 分钟。

住宅、数据中心、ISP 和移动代理，覆盖 200 多个国家，拥有 90M+ IP，以及 Scraper & SERP API。提供 Python、Node.js、PHP、Go、Java、C#、Ruby 和 curl 的代码示例。

测试发现：Python and Node.js quickstarts and curl HTTP-examples work through mock proxy to real ipinfo.io; PHP/Go/Ruby/Java/C# examples not tested due to missing runtimes; SOCKS5 not tested due to minimal mock. 5 分钟。

基于搜狗微信搜索的微信公众号爬虫接口

测试发现：wechatsogou 4.5.4 is installed in a Python 3.12 venv; pip install -e . succeeds; 24 offline tests pass (constants, tools, URL generation, HTML parsing); real-network API tests fail only because DNS is unavailable in the container. 13 分钟。

开源、自托管的企业级和站点搜索服务器，基于 OpenSearch 构建。可爬取 Web/文件/数据库/云来源，支持 20+ 种语言，提供 REST API，并具备 AI/RAG 及语义搜索功能。Apache-2.0。

测试发现：Fess 15.9.0-SNAPSHOT builds and packages successfully from source. All 7,634 unit tests pass. The embedded Tomcat binds port 8080 but the application fails at runtime because no OpenSearch backend is reachable on localhost:9200. 36 分钟。

AgentQL 是一套用于将你的 AI 连接到 Web 的工具套件。它提供查询语言和 Playwright 集成，用于快速、精确且大规模地交互元素和提取数据。包含 REST API、Python 和 JavaScript SDK，以及浏览器调试器。

测试发现：无法验证；log 显示了它停在哪里。

Krawl 是一个可定制的轻量级云原生 Web 诱骗服务器与反爬虫工具，通过逼真的随机生成诱饵数据和 AI 生成的 HTML 模板，创建带有明显漏洞的虚假 Web 应用。

测试发现：Krawl standalone mode builds and runs: after pip install -r requirements.txt, uvicorn app:app --app-dir src serves HTTP 200 on /dashf/healthz, answers the honeypot with 204 on /, records POST credentials from 198.51.100.77 into SQLite, and the dashboard API shows that IP categorized as attacker. 92 分钟。