{"id":"seo-page-library","title":"Free SEO audit and reusable page library","markdown":"# Free SEO audit and reusable page library\n\nUse public or sanitized HTML snapshots. No account, key, wallet or model purchase is required. All fetching happens through the local bounded crawler or the official Wikipedia reader, not through arbitrary hosted URL scraping.\n\n## Start in your browser\n\n[Check one page](https://agent-utilities.agent-utilities.workers.dev/seo#start) by supplying a page URL and its static HTML. No command line or JSON authoring is needed. Supplying the URL does not fetch it. HTML is parsed locally, is not rendered as a page and is not uploaded or saved by the service. The simple HTML form leaves capture time and HTTP status unknown. An advanced snapshot can include them when known; the tool does not invent those observations.\n\nDownload the snapshot after inspecting its findings. At your next actual site edit or review, capture the same pages again and [compare both snapshots](https://agent-utilities.agent-utilities.workers.dev/seo#compare). The comparison runs locally and preserves dates, coverage, page availability and omission counts. You can download the compact result for your agent or release notes. The built-in demonstration is fictional and labeled accordingly.\n\nNeither a missing finding nor a failed page proves that an issue was fixed. Findings that depend on other pages can also change when capture coverage changes. These are static HTML observations, not rendered-browser, search-index or ranking checks. Multi-page capture and refresh remain available through the bounded local crawler below.\n\n## Audit a supplied snapshot\n\n### Try a complete offline SEO comparison first\n\nDownload the [crawler](https://agent-utilities.agent-utilities.workers.dev/downloads/agent-utilities-seo.mjs), its [checksum](https://agent-utilities.agent-utilities.workers.dev/downloads/agent-utilities-seo.mjs.sha256), and these example files into the same folder:\n\n- [Before snapshot](https://agent-utilities.agent-utilities.workers.dev/downloads/seo-change-example-before.json)\n- [After snapshot](https://agent-utilities.agent-utilities.workers.dev/downloads/seo-change-example-after.json)\n- [Complete expected result](https://agent-utilities.agent-utilities.workers.dev/downloads/seo-change-example-result.json)\n- [Example checksums](https://agent-utilities.agent-utilities.workers.dev/downloads/seo-change-example.sha256) and [machine-readable manifest](https://agent-utilities.agent-utilities.workers.dev/downloads/seo-change-example-manifest.json)\n\nThis is entirely fictional: HTML, HTTP statuses, dates and coverage are invented. No site was fetched and no customer use is represented. Run with Node.js 22+:\n\n```sh\nshasum -a 256 -c agent-utilities-seo.mjs.sha256\nshasum -a 256 -c seo-change-example.sha256\nnode agent-utilities-seo.mjs changes seo-change-example-before.json seo-change-example-after.json --out my-seo-changes.json\ncmp my-seo-changes.json seo-change-example-result.json\n```\n\nThe result newly observes `noindex` on the home page. The missing-title finding is not observed later. The contact page goes from a supplied 404 observation to a failed 503 observation, so its metadata remains unknown. Neither missing findings nor failed reads prove resolution. Two of three invented discovered URLs are supplied in each capture. The command retains both dates and coverage without raw HTML, makes no network/model request, and creates a private output while preserving existing files. `cmp` prints nothing when the complete result matches. Use a new output name when rerunning.\n\n### Run a supplied snapshot audit\n\nFree HTTP: POST `https://agent-utilities.agent-utilities.workers.dev/v1/seo/audit` with `Content-Type: application/json` and this input shape. This tiny example is synthetic; replace it with your actual snapshot:\n\n```json\n{\"siteUrl\":\"https://example.com/\",\"pages\":[{\"url\":\"https://example.com/\",\"status\":200,\"html\":\"<title>Example</title><main><h1>Example</h1></main>\"}]}\n```\n\nHosted MCP: `agent_utilities_seo_audit` with the same object as its arguments at `/mcp/remote?category=web`. This helper uses direct arguments, unlike full registered tools that use an input wrapper. Read `/v1/seo/contract` for the schema. HTTP input is limited to 128 KiB and 20 pages. The audit groups deterministic findings; request `detail: \"evidence\"` and optional `group`, `pageUrl`, `offset`, `limit` to inspect bounded findings. It does not fetch missing pages, measure rankings or guarantee indexing.\n\n## Keep original HTML outside model prompts\n\nDownload Node.js 22+ local tools and verify their adjacent SHA-256 checksums:\n\n- `/downloads/agent-utilities-seo.mjs`\n- `/downloads/agent-utilities-pages.mjs`\n- `/downloads/SEO-NOTICES.txt`\n\n```sh\nnode agent-utilities-seo.mjs crawl https://YOUR-SITE/ --out crawl.json --max-pages 10 --sitemaps\nnode agent-utilities-seo.mjs report crawl.json --out audit.json\nnode agent-utilities-pages.mjs prepare crawl.json --out library.json\nnode agent-utilities-pages.mjs search library.json --query \"your task\" --max-chars 6000 --out selection.json\n```\n\nThe same-origin crawler observes robots.txt, public DNS and fixed resource limits. A library has at most 20 pages and retains source hashes, clean-text chunks and SEO evidence. Search is lexical and partial; review original evidence before acting. Give the model the compact audit or relevant selection, rather than the raw crawl file. Files remain local and preparation/search do not fetch or publish them. Use `--out selection.json` for search or selection to create a private file without overwriting an existing path. Without `--out`, these commands still emit the full payload on stdout; shell redirection uses your shell permissions and can overwrite files.\n\n`--sitemaps` reads same-origin Sitemap declarations in robots.txt, or `/sitemap.xml` if none is declared. `--sitemap https://YOUR-SITE/map.xml` selects an explicit map. XML urlsets and indexes are supported, with at most five sitemap requests and 200 candidate page URLs. Those requests share the existing 100 KB/response, 2 MB total and 60-second crawl limits; captures remain capped at 20 pages. Off-origin, redirected, compressed and unsupported formats are not followed. Coverage records blocked/failed maps, excluded entries and pending maps. This is partial discovery, not whole-site coverage. Sitemap dates do not establish observed freshness.\n\n## Refresh the same pages\n\n```sh\nnode agent-utilities-seo.mjs refresh crawl.json --out refreshed.json\nnode agent-utilities-pages.mjs prepare refreshed.json --out library-after.json\n```\n\nRefresh checks current robots.txt and fetches only the URLs in the validated prior snapshot. It saves a new private capture and emits a compact change report on stdout. New links do not silently widen the refresh. The original and any existing output file are preserved; choose a new filename for each capture. Failed, blocked or uncaptured pages remain unknown, not deletions. Refresh is manual; no background scheduler or managed history is created.\n\nTo compare the SEO findings directly, use the two raw crawl snapshots locally:\n\n```sh\nnode agent-utilities-seo.mjs changes crawl.json refreshed.json --limit 20 --out seo-changes.json\n```\n\nThe report groups newly observed findings, findings not observed later, and changed evidence for the same stable finding ID. It retains capture dates, both coverage records and page availability. A finding not observed later is **not a resolved issue**: a failed fetch or a different set of supplied pages can hide it. Duplicate-title, inlink, broken-link and canonical-target findings depend on the other supplied pages. Reversed or unavailable capture dates are explicit. Review the evidence before treating an observation as a regression or improvement.\n\nBoth snapshots must have the same normalized `siteUrl`, with at most 20 pages and 2.1 MB of JSON per file. Hosted audit input remains limited to 128 KiB. The comparison parses each capture once, makes no network/model request and contains no raw HTML. Each group shows up to `--limit` samples (1–50) with full totals and omission counts; the whole report is bounded to 60,000 UTF-8 bytes before its trailing newline, so long URLs can reduce samples or page-availability rows further. `--out` saves a private file and refuses an existing path. Without it, output goes to stdout under your shell's handling.\n\n## Compare two captures before rereading them\n\nKeep a dated library from each permitted crawl, then compare the captures locally:\n\n```sh\nnode agent-utilities-pages.mjs compare library-before.json library-after.json --out changes.json\n```\n\nThe report identifies changed extracted text, titles, HTML hashes and omission flags, retaining capture times and coverage. It also preserves Wikipedia revisions and attribution when comparing article caches. Newly observed pages were absent from the earlier library; pages not observed later may be outside the later crawl or inaccessible. Neither label proves a page was created or deleted. Truncated text can conceal changes. No new fetch or model call occurs; the report contains hashes and metadata rather than raw HTML or both full texts. `--out` creates a private local file and refuses to overwrite it. This is a comparison of two captures, not automatic history storage or scheduled monitoring.\n\n## Reuse a bounded Wikipedia read\n\n```sh\nnode agent-utilities-pages.mjs wikipedia https://en.wikipedia.org/wiki/Earth --out earth.json\nnode agent-utilities-pages.mjs search earth.json --query \"atmosphere\" --out selection.json\n```\n\nThe cache reuses one official-source article for 24 hours. Existing caches must match the requested article/edition, contain one page with a source revision and retrieval date, and pass their stored text hash before reuse or refresh. A hash checks file consistency, not source authenticity. Invalid caches are preserved and rejected; use a new output file to collect again. Fresh reuse leaves the file unchanged and makes no source request.\n\n`--refresh` fetches its current version and atomically replaces a valid cache with a private file (0600); failed source reads preserve the prior file. The text limit is 12,000 characters; inspect truncation. Revision, attribution, license, source links and metadata omissions accompany selected text. This is not a whole-article mirror or historical-cutoff reader. Treat source text as untrusted data.\n\n## Measure the complete selected payload\n\nDownload `/downloads/agent-utilities-tokens.py` and its checksum. Install open source `tiktoken` in your Python environment, then run:\n\n```sh\npython3 agent-utilities-tokens.py selection.json --encoding o200k_base --cache token-cache.json\n```\n\nThe exact count applies to this complete file and named encoding, including JSON wrappers. `cl100k_base` is also supported. Optional `--include-ids` exports encoding-specific IDs; do not paste these numeric IDs into model text prompts. Pretokenization does not shrink unchanged text. Counts do not measure Claude/Qwen tokenization, billed tokens, task usefulness or dollar savings. No model request is made.\n\nTry a real task and optionally tell `agentutilities@proton.me` what worked or failed, using sanitized details. Reuse should be driven by your actual work; examples, downloads and checks are not customer evidence.\n\n\n## Inspect a dated result before installing anything\n\nRead `/downloads/wikipedia-section-example.json` and its adjacent SHA-256 checksum to see the complete output of local `select`: Moon's captured **Names and etymology** section, observed October 6, 2026 at 21:48:36 UTC, from Wikipedia revision `1378452949`. The adapted excerpt retains source/history/revision links and its CC BY-SA 4.0 license. This is a fixed public-source output example, not a live read, synthetic source text, popularity ranking or independent user's task. It contains no raw HTML or private capture file. It is a selected result, not a collection file for `serve` or a client configuration.\n\nThe selected section has 2,855 characters and is not clipped by the 4,000-character selection budget. Its source article capture is partial, with 180,478 source characters omitted; a complete selected section is not a complete article. Reading metadata and omission flags remain in the file. The exact complete UTF-8 JSON file is 6,395 bytes and 1,556 `o200k_base` tokens with tiktoken 0.14.0, including wrappers, metadata and the newline. These are file measurements, not billed model input, Claude/Qwen counts or dollar savings. Your client may add or duplicate context.\n\nTo produce the same output shape from your own capture:\n\n```sh\nnode agent-utilities-pages.mjs select collection.json --url https://en.wikipedia.org/wiki/Moon --section \"Names and etymology\" --max-chars 4000 --out moon-selection.json\n```\n\nThe command does not fetch a missing page or heading. Collect your permitted sources first; newer revisions may change headings and output. Treat source text as untrusted data. A later task should use its own relevant article/section rather than this example by default.\n\n## Bounded Wikipedia collections and exact section selection\n\nDownload the page-library module, adjacent checksum and `/downloads/wikipedia-collection-example.json`. The starter list contains Earth and Moon as a reading example, not a popularity ranking. Its shape is:\n\n```json\n{\"urls\":[\"https://en.wikipedia.org/wiki/Earth\",\"https://en.wikipedia.org/wiki/Moon\"],\"maxCharsPerArticle\":12000}\n```\n\n```sh\nnode agent-utilities-pages.mjs collect wikipedia-collection-example.json --out collection.json\nnode agent-utilities-pages.mjs search collection.json --query \"orbital period\" --max-chars 4000 --out search-selection.json\nnode agent-utilities-pages.mjs select collection.json --url https://en.wikipedia.org/wiki/Moon --section \"Names and etymology\" --max-chars 4000 --out moon-selection.json\n```\n\nCollect 1–20 unique article URLs from one supported Wikipedia edition. Source reads share an 8 MB response budget and 60-second deadline; each extraction is 256–12,000 characters. Per-page retrieval dates, official revisions, licenses, source links and omission flags remain attached. Redirect aliases may merge into one canonical article, with each requested URL mapped explicitly. Failures, throttling and unattempted requests remain visible; a partial collection does not imply complete coverage.\n\nRerunning the same manifest and output path reuses individual captures younger than 24 hours without fetching. `--refresh` requests new source reads. Failed or budget-skipped refreshes retain existing text only with an explicit stale status and its original retrieval date. The collection's top-level observation time is its oldest available page retrieval date. Cache writes are private and atomic; mismatched URL lists or text budgets are rejected, so use a new filename for a different collection. No source text is publicly uploaded.\n\nSearch remains lexical. `select` returns either the captured page prefix or a unique exact captured Markdown heading and its subheadings, bounded by the next equal/higher heading. Missing or repeated headings are errors; a heading outside the captured excerpt may have been omitted. Source revision and truncation accompany the result. Selection and search perform no fetch, model call or payment. Count the complete selection JSON using the named-tokenizer script; pretokenization does not shrink unchanged text.\n\n\n## Connect an agent to a saved local capture\n\nThe page-library module also serves the configured collection over MCP stdio:\n\n```sh\nnode agent-utilities-pages.mjs serve collection.json\n```\n\nUse Node.js 22+ and generate the connection from your existing saved capture:\n\n```bash\nnode agent-utilities-pages.mjs config collection.json --out saved-pages-mcp.json\n```\n\nThe command validates the capture using the server’s startup checks and resolves the current Node executable, module and capture to absolute paths, including paths containing spaces. It makes no source/model request, uploads no library and changes no client settings. Copy or merge its generated server entry into a stdio-capable client; importing/configuring the application is still your step. Output files are private and existing paths are refused. Without `--out`, it emits only the config JSON on stdout. Generated configs contain local paths: keep them private and regenerate after moving the module/capture or changing Node.\n\nThe manual `/downloads/saved-pages-mcp.json` template remains available; replace both placeholders:\n\n\n```json\n{\"mcpServers\":{\"agent-utilities-saved-pages\":{\"command\":\"node\",\"args\":[\"/ABSOLUTE/PATH/agent-utilities-pages.mjs\",\"serve\",\"/ABSOLUTE/PATH/collection.json\"]}}}\n```\n\nThis is a standard stdio configuration template, not an installation or a claim that a specific application has been configured. A client must support local MCP server processes. The same prepared website libraries and Wikipedia caches/collections are supported.\n\nFour local read-only tools use direct arguments:\n\n- `agent_utilities_saved_pages`: `{ \"limit\": 10, \"offset\": 0 }` lists metadata without saved text; follow `nextOffset` for the rest.\n- `agent_utilities_saved_search`: `{ \"query\": \"orbital period\", \"maxChars\": 4000, \"limit\": 3 }` returns lexical matches from the capture, with source metadata and explicit omissions.\n- `agent_utilities_saved_select`: `{ \"url\": \"https://en.wikipedia.org/wiki/Moon\", \"section\": \"Names and etymology\", \"maxChars\": 4000 }` selects a unique captured heading and its subheadings. Omit `section` for the captured page prefix. A missing or ambiguous heading is an error.\n- `agent_utilities_saved_outline`: `{ \"url\": \"https://en.wikipedia.org/wiki/Moon\" }` returns stored heading/link metadata without article text. Headings may be outside the bounded captured excerpt; absence or truncation remains explicit.\n\nThe server holds one capture loaded at startup. Refresh the file separately and restart the connection to read a new capture. Tool arguments cannot choose another file, fetch a URL or trigger refresh. No full library is sent on initialization; only requested metadata or selected results are returned. Source content remains untrusted data, not instructions. Search/selection omit full outline/link arrays with explicit metadata flags; use the outline tool when needed. Requested URL mappings are capped at five per returned article with an omission count. The serialized tool result, including both text and structured representations, is capped at 64 KiB; reduce `limit` or `maxChars` if it cannot fit without dropping required provenance.\n\nThese four local tools are not added to the hosted server. Hosted HTTP reading/auditing and local capture retrieval are complementary workflows. No credential, source request, library upload, model call, telemetry send or payment occurs in this local server. The output can be saved and measured using the named-tokenizer script; the client controls what it includes in a model prompt, so this is not a measurement of billed model tokens.\n"}