Scoring Methodology

mcpscore’s value rests on its score being trustworthy. This page explains exactly how the score is computed, what is and is not checked, and why — so you can verify every claim yourself, dispute a rule you disagree with, or propose a new one.

​

Principles

1. Deterministic. No LLM judging, no API keys, no sampling. The same server state always produces the same score, in seconds. This makes mcpscore usable as a CI gate and its results reproducible by anyone.
2. Spec-anchored. Every rule enforces something the MCP specification or its published best practices actually says. Readiness rules cite the SEP (Specification Enhancement Proposal) they enforce, and the citation is included in every result.
3. Fair across spec versions. MCP is a versioned protocol; servers on different revisions are scored against the revision they negotiated, never punished for a spec they don’t speak yet (see Multi-spec-version scoring).
4. Honest about limits. Anything mcpscore cannot verify from the outside is documented as untested, not silently skipped.

​

The score

Every rule has a severity, and each passing rule contributes its weight to the score. The final score is reported as earned/maximum. Only applicable rules count toward the maximum: a rule that cannot meaningfully run against your server is skipped and excluded from both the earned and the maximum score — never failed. Every skip appears in the JSON report (skipped_rules) with one of three reasons: For example, a successfully fetched empty tools, prompts, resources, or resource-template catalog gives per-item quality rules nothing to assess, so those rules contribute 0/0. If a declared catalog could not be fetched, or an incomplete listing returned no items, the same rules use insufficient-data because an item may exist but remain unobserved. A dedicated presence rule can still judge whether a complete catalog is empty.

​

How an audit runs

1. Connect — stdio (local servers in any language: .py/.js files directly, or an arbitrary command via --stdio), Streamable HTTP, or SSE, with automatic transport detection. mcpscore first attempts the legacy MCP handshake; if that lifecycle is absent, stateless probes can establish that the target is a modern-only server instead of treating it as unreachable.
2. Collect — protocol version, server info, capabilities, tools, resources, prompts, TLS details.
3. Probe — a set of sessionless requests observing behavior outside the negotiated session, e.g. whether the server also speaks the latest spec revision’s stateless lifecycle. Probes are read-only by design: an audit never invokes tools/call against your server, so it can never trigger tool side effects. (The one deliberate exception is the CLI’s opt-in smoke mode, which exists for developers to smoke-test their own server in CI by actually calling its tools; smoke results are reported separately and never enter the score. The web service has no such mode.) Several probes are deliberately malformed or security-relevant, because the spec defines how a server must answer them. Expect an audit to send a foreign Origin header, a request for an unknown RPC method, an unsupported protocol version, a bogus session ID, a request for a nonexistent resource, fabricated pagination cursors for declared list capabilities, and one truncated JSON body. These can look like probing in a WAF or access log; they read state and nothing more. Each one exists to check documented normative behavior — for example, the spec requires HTTP 403 for an invalid Origin, HTTP 404 with JSON-RPC -32601 for an unknown method, recommends JSON-RPC -32602 for an invalid cursor, and JSON-RPC requires malformed JSON to produce -32700 with a null ID. The modern lifecycle is stateless, so JSON-RPC behaviors that can be expressed through the SDK are probed over stdio too. A successful suite launches two short-lived sibling processes through the MCP SDK’s stdio transport and sends no initialize handshake: one runs the probe suite, while the other provides an independent connection for catalog-stability comparisons. If the gateway probes establish no modern support, the suite stops after the first process. The truncated-JSON probe currently runs only over HTTP because the SDK path cannot inject malformed wire input safely; stdio therefore reports insufficient evidence for that rule. Questions that only exist in HTTP—such as Origin, headers, status codes, sessions, and auth metadata—are recorded as not applicable for stdio rather than failed. Invalid-cursor probes use the established initialized SDK session when one is available. This matters for legacy and dual-era servers, where a sessionless request may be rejected before its cursor can be judged. Modern-only servers, which have no legacy SDK session, are assessed through the stateless probe path. For modern servers, mcpscore also follows declared paginated list surfaces to compare the actual wire-level cacheScope across pages. This traversal is read-only, bounded to 100 pages per surface and 30 seconds in total, and skips when no list returns multiple complete pages.
4. Evaluate — every registered rule runs against the collected data and probe observations. See the Rules Reference for the full list.

​

Multi-spec-version scoring

MCP publishes dated spec revisions (2024-11-05 … 2026-07-28, the current revision, published final on 2026-07-28), and the 2026 revision replaces the stateful initialize lifecycle with a stateless one. The spec itself describes servers as Legacy (stateful), Modern (stateless), or Dual-era (both). mcpscore treats the spec version as a property of your server that it detects, not a mode of the tool:

• Your server is scored against the spec revision it negotiated. Rules declare the version range they apply to; outside it they are skipped, keeping scores comparable across servers on different revisions.
• The report states the era your server was observed to support (legacy, modern, or dual-era), following the spec’s own detection guidance.
• Modern-only HTTP and stdio servers are fully supported: if the legacy handshake fails but the server answers stateless requests, mcpscore audits it through its probe layer instead of reporting a connection failure.

​

The readiness score (separate axis)

mcpscore answers a second question: “is this server ready for the 2026-07-28 spec?” Results always appear as their own axis (readiness.score / readiness.max_score in the report). For legacy-only servers that axis stays informative — guidance, not punishment — so a fully compliant 2025-11-25 server keeps a clean main score. For servers already on the modern lifecycle, readiness points are also counted in the main score (see Promotion below). Readiness rules for the 2026-07-28 revision, each anchored to normative spec language (MUST/MUST NOT unless noted): Two things worth knowing about how these run:

• The session-ID and removed-method rules are legacy-leakage checks: they only run when your server shows modern support. Missing protocol-version rejection and GET-stream removal are narrower still: they skip dual-era endpoints, because the spec explicitly permits compatibility with older Streamable HTTP revisions.
• Promotion (since 1.1.0): a server that negotiates the modern lifecycle (era modern or dual-era) has its readiness points counted in the main score — the report’s readiness.counted_in_main flag says which mode a given audit used, and the readiness block stays populated as the transparent breakdown of that share. Legacy-only servers keep readiness informative (guidance, not punishment), and partial audits are never promoted — a partial score is already not comparable to a full audit’s. The rule_ids never change, so integrations keyed on rule IDs are unaffected — but score-based consumers (e.g. a CI min-score threshold) should note that a modern-lifecycle server’s score/max_score now include the readiness points; check readiness.counted_in_main to know which mode scored a given audit.

The human-readable readiness line includes how many checks were assessed whenever any were skipped. JSON consumers get the same coverage explicitly as readiness.assessed, readiness.skipped, and readiness.total; a perfect severity-weighted fraction never implies that every readiness check ran.

​

What mcpscore deliberately does not test

Honesty about limits is part of the methodology. The following are currently out of scope, either because they are not externally observable or because testing them safely from the outside is not yet possible:

• True statelessness (no hidden cross-request state) — only probabilistically testable; the spec itself concedes this.
• requestState integrity enforcement (MRTR) — only tamper-rejection is observable, and active tampering probes are deferred.
• subscriptions/listen behavior and SSE stream semantics.
• Tasks and MCP Apps extensions — they version independently of the core spec.
• Client-side OAuth obligations (issuer validation, credential binding, scope accumulation) — not observable from a server audit.
• Tool behavior — the audit never calls your tools, so the score cannot judge what they do, only how they are described and typed. The CLI’s opt-in smoke mode exists for exactly this gap on your own server: it invokes tools (only readOnlyHint: true ones unless --call-all) and verifies declared output schemas, invalid-argument rejection, and unknown-tool rejection — reported outside the score, with its own exit code (4).

​

Stability contract

• rule_id is stable. Display names and messages may be reworded; the rule_id in the JSON report is a machine contract and never changes meaning.
• The JSON report schema is versioned (schema_version), bumped only on backwards-incompatible changes. New fields are added additively.
• Rule additions and severity changes are documented in the changelog.

​

Why did my score change?

The score is deterministic for the same server state and the same mcpscore version — when it moves, one of these is what moved:

1. A different mcpscore version ran. The ruleset grows in most releases, which changes both score and max_score — scores are only comparable within one version (see the stability contract). The report’s mcpscore_version field says which version scored it.
2. The audit type differs. A partial audit (auth-gated server, no credential) covers far fewer checks than a full one, and a modern-lifecycle server’s main score includes its readiness points (readiness.counted_in_main) while a legacy server’s does not.
3. The server itself answered differently. Real fleets do this more often than you’d expect: a rolling deploy or mixed replica versions behind a load balancer can make byte-identical probes get different answers across runs, and a server that rate-limits the audit’s probes turns some rules into insufficient-data skips — which changes the maximum, not just the earned points. If repeated audits of one URL disagree, diff the two JSON reports rule by rule: the flipping rule_id is the finding, and it usually points at the server’s consistency, not the auditor’s.

​

Disagree with a rule?

The methodology is open by design. If a rule seems wrong for your server, open an issue with the rule_id and the spec text you believe contradicts it — rules are required to be defensible by citation, and we treat a rule that can’t be defended as a bug. To propose a new rule, see CONTRIBUTING.