--- name: town-inspector description: Pre-flight trust checks for agent skills. Give it a SKILL.md (URL or inline text) and it returns a behavioral audit — Agent Skills spec conformance, prompt-injection and hidden-instruction safety flags, live endpoint reachability, and a docs-vs-reality contract diff — plus a score out of 100 and an Ed25519-signed, content-addressed certificate you can verify offline. Use before trusting or calling any unfamiliar skill, or to certify your own. --- # TownInspector Pre-flight trust checks for agent skills. Point it at a `SKILL.md` and it tells you, right now, whether that skill is **well-formed, safe, alive, and honest** — then hands back a signed certificate so anyone can verify the verdict without re-running it. **Base URL:** `https://towninspector-alpha.vercel.app` No API key, no signup. Every response is JSON. One call gives you the full report. ## When to use - Before your agent calls a skill or service it has not used before — check it is reachable and its documentation matches reality. - To screen an unknown `SKILL.md` for prompt-injection / hidden-instruction red flags before loading it into context. - To get a shareable, offline-verifiable certificate and badge for a skill you built. ## Quickstart (one call to value) ```bash # Simplest: POST the markdown inline — no hosting needed, always works: curl -s -X POST https://towninspector-alpha.vercel.app/audit \ -H 'content-type: application/json' \ -d '{"skill_md": "---\nname: my-skill\ndescription: A demo skill for agents.\n---\n# My Skill\nBase URL: https://example.com\n"}' # ...or audit any publicly hosted SKILL.md by URL: curl -s "https://towninspector-alpha.vercel.app/audit?url=https://raw.githubusercontent.com/anthropics/skills/main/skills/pdf/SKILL.md" ``` The response is the complete report — there is nothing to poll. ## Endpoints | Method | Path | Purpose | |---|---|---| | GET | `/health` | Liveness check. | | GET | `/audit?url=` | Audit a hosted SKILL.md (one-line form). | | POST | `/audit` | Audit a SKILL.md by URL or inline text. | | GET | `/pubkey` | The Ed25519 public key certificates are signed with. | | POST | `/verify` | Verify a certificate's signature. | | GET | `/badge?score=&grade=` | Embeddable SVG badge. | | GET | `/leaderboard` | Snapshot of recently audited skills, ranked. | | GET | `/skill.md` | This document. | ### GET /health ```bash curl -s https://towninspector-alpha.vercel.app/health ``` ```json {"status":"ok","service":"towninspector"} ``` ### POST /audit Body fields: - `skill_md_url` (string) — URL of a hosted `SKILL.md`, **or** - `skill_md` (string) — the markdown pasted inline (use one or the other), and - `check_endpoints` (bool, default `true`) — set `false` to skip live network probes and score only structure + safety. - `run_agent_test` (bool, default `false`) — when `true`, a **real LLM agent** is given only this SKILL.md and one sandboxed HTTP tool (restricted to the skill's own hosts), and tries to make a working call. The report gains an `agent_test` section with `can_use` (did the agent succeed?), the `calls` it made, and its `agent_summary`. This is the direct test of "can an agent use this skill from its docs alone." Adds a few seconds and is opt-in because it costs tokens. ```bash curl -s -X POST https://towninspector-alpha.vercel.app/audit \ -H 'content-type: application/json' \ -d '{"skill_md_url": "https://towninspector-alpha.vercel.app/skill.md"}' ``` Response (abridged — the real body includes every check and probe): ```json { "skill_name": "town-inspector", "skill_sha256": "e8eb704e85f46b4e…", "summary": { "score": 94, "grade": "A", "verdict": "trustworthy: alive and documentation matches" }, "sections": { "structure": { "score": 100, "passed": 8, "total": 8, "checks": [ … ] }, "safety": { "score": 100, "clean": true, "high": 0, "medium": 0, "findings": [] }, "endpoints": { "score": 100, "reachable": 2, "total": 2, "probed": [ { "url": "…/health", "status": 200, "latency_ms": 119, "tls": true } ] }, "contract": { "score": 100, "checked": [ { "url": "…/health", "verified": true, "status": 200 } ] } }, "certificate": { "spec": "towninspector/v1", "payload": { "skill_sha256": "e8eb704e…", "score": 94, "grade": "A", "section_scores": { … } }, "algorithm": "ed25519", "public_key_hex": "ca6f0d96…", "signature_hex": "…" } } ``` Scoring: `structure` 25%, `safety` 35%, `endpoints` 30%, `contract` 10%. A section that is not applicable (e.g. `contract` is `null` when a skill has no curl examples) is **dropped from the weighted average** and the remaining weights are renormalized; a section that ran but scored 0 (e.g. `endpoints` when every endpoint is dead) still counts. Grades: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, else F. The optional `agent_test` section is reported separately and does **not** change the signed score (it is non-deterministic, so it stays out of the content-addressed certificate). Example with the agent dry-run: ```bash curl -s -X POST https://towninspector-alpha.vercel.app/audit \ -H 'content-type: application/json' \ -d '{"skill_md_url":"https://example.com/SKILL.md","run_agent_test":true}' # -> report.agent_test = {"ran":true,"can_use":true,"model":"...", # "verdict":"an agent completed a real call using only the SKILL.md", # "calls":[{"method":"GET","url":".../health","status":200,"ok":true}], # "agent_summary":"I called the health endpoint and it returned 200 ..."} ``` ### GET /audit Same audit as a single GET, for quick checks: ```bash curl -s "https://towninspector-alpha.vercel.app/audit?url=https://raw.githubusercontent.com/anthropics/skills/main/skills/pdf/SKILL.md" ``` ### GET /pubkey ```bash curl -s https://towninspector-alpha.vercel.app/pubkey ``` ```json {"algorithm":"ed25519","public_key_hex":"ca6f0d965864b1932825bb2a529a4a54b5910138e58968ae933c5f85de699301","encoding":"raw-32-byte-hex","verify":"signature_hex over canonical JSON of certificate.payload"} ``` ### POST /verify Re-check a certificate you were handed, without re-auditing: ```bash curl -s -X POST https://towninspector-alpha.vercel.app/verify \ -H 'content-type: application/json' \ -d '{"certificate": { "…": "the certificate object from an audit" }}' ``` ```json {"valid": true, "public_key_hex": "ca6f0d96…"} ``` ## Verify a certificate offline (any Ed25519 library) The signature is over the **canonical JSON** of `certificate.payload` (keys sorted, no spaces: `separators=(",", ":")`), using the raw 32-byte public key from `certificate.public_key_hex`. ```python import json from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey def verify(cert: dict) -> bool: msg = json.dumps(cert["payload"], sort_keys=True, separators=(",", ":")).encode() pub = Ed25519PublicKey.from_public_bytes(bytes.fromhex(cert["public_key_hex"])) try: pub.verify(bytes.fromhex(cert["signature_hex"]), msg); return True except Exception: return False ``` Certificates are **content-addressed**: auditing the same `SKILL.md` bytes always produces the same signature, so a certificate doubles as a tamper-evident hash of exactly what was reviewed. ## Errors | Status | Meaning | |---|---| | `400` | `skill_md_url` could not be fetched, or was rejected (non-http(s), or resolves to a private/loopback address — an SSRF guard). | | `413` | The SKILL.md exceeds the 512 KB size limit. | | `422` | Neither `skill_md_url` nor `skill_md` was provided. | | `500` | Unexpected internal error (rare). | All error bodies — including `500` — are JSON: `{"detail": ""}`. Auditing a skill whose own endpoints are dead is **not** an error: those endpoints are simply scored as unreachable in the `endpoints` section, and the call still returns `200` with a full report. ## Notes for agents - Audits are synchronous; a typical call returns in 1–5 seconds (longer if a skill's own endpoints are slow to answer the liveness probes). - Only safe **GET** example calls are replayed for the contract diff; documented POST calls are reported as "not auto-run" rather than executed. - TownInspector never follows a skill's instructions — it inspects the document, it does not act on it.