> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mcpscore.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Smoke mode

> Check on every change that your MCP server still connects, lists its tools, and runs them, with one command in CI.

A refactor, a dependency bump or a new tool can break your MCP server without
failing a single unit test. The server still starts. An agent that connects
to it gets a tool whose result no longer matches the declared schema, or a
handler that accepts arguments it should reject. Smoke mode catches that in
CI: `--smoke` runs the normal audit, then calls your server's tools over the
same connection and checks the answers.

```bash theme={null}
# One CI step: does my server still work?
mcpscore path/to/your/server.py --smoke

# Any-language servers and deployed servers work the same way
mcpscore --smoke --stdio ./my-go-server
mcpscore https://your-server.example/mcp --smoke --token $TOKEN

# Explicit consent to call every tool, not only read-only ones
mcpscore path/to/your/server.py --smoke --call-all
```

```text theme={null}
Running smoke checks — invoking tools/call, as requested with --smoke...

Audit finished. Final score: 75/105
Spec: 2025-11-25 negotiated (latest: 2026-07-28) · era: dual-era

Smoke checks (tools/call): 4 passed, 2 failed, 3 skipped — only readOnlyHint: true tools called (see --call-all). Never counted in the score.
  Tools called: 3 of 4 (a tool counts as called once the server answered a tools/call for it)
  pass smoke_structured_content [honest]: structuredContent conforms to the declared outputSchema
       returned 1 content block(s) [text]; structuredContent present; first text "{\"result\": \"ok\"}"
  FAIL smoke_structured_content [dishonest]: declared outputSchema is not honored: Invalid structured content returned by tool dishonest: 'result' is a required property
  skip smoke_structured_content [sloppy]: no outputSchema declared — the tool was not called for this check; declare an outputSchema to have its result verified
  skip smoke_structured_content [writer]: not called under the safety default (readOnlyHint is not true); pass --call-all to include it
  pass smoke_invalid_arguments [honest]: rejected with JSON-RPC error -32602
  pass smoke_invalid_arguments [dishonest]: rejected with JSON-RPC error -32602
  FAIL smoke_invalid_arguments [sloppy]: accepted schema-invalid arguments (returned a success result)
       returned 1 content block(s) [text]; first text "{}"
  skip smoke_invalid_arguments [writer]: not called under the safety default (readOnlyHint is not true); pass --call-all to include it
  pass smoke_unknown_tool: rejected with JSON-RPC error -32602
Gate failed — --smoke: 2 smoke check(s) failed
```

One run gives you the audit score and, under it, one line per smoke check.
Each line reads `verdict check_id [tool]: reason`. This server has four tools:
`honest` works, `dishonest` breaks its own output schema, `sloppy` accepts
anything, and `writer` was never called because it is not annotated
read-only.

> [!WARNING]
>
> `--smoke` invokes `tools/call` on the target. Run it against servers you
> operate. The ordinary audit never calls tools, so it is the right choice
> for anyone else's server.

## Smoke-test your own server

The steps below take a minimal server built with the MCP Python SDK from no
coverage to a CI gate. Each step changes one thing and shows the run that
proves it. The server has one tool:

```python server.py theme={null}
from mcp.server import MCPServer

mcp = MCPServer("Basic Server")


@mcp.tool()
def add(a: int, b: int) -> int:
    """Add two numbers."""
    return a + b


if __name__ == "__main__":
    mcp.run()
```

### Step 1: Run it and read the skips

Point mcpscore at the server and add `--smoke`. A server in its own project
environment runs through `uv run`, the same way you start it yourself:

```bash theme={null}
mcpscore --smoke --stdio uv run server.py
```

```text theme={null}
Smoke checks (tools/call): 0 passed, 1 failed, 2 skipped — only readOnlyHint: true tools called (see --call-all). Never counted in the score.
  Tools called: 0 of 1 (a tool counts as called once the server answered a tools/call for it)
  skip smoke_structured_content [add]: not called under the safety default (readOnlyHint is not true); pass --call-all to include it
  skip smoke_invalid_arguments [add]: not called under the safety default (readOnlyHint is not true); pass --call-all to include it
  FAIL smoke_unknown_tool: reported a tool execution error (isError) for a nonexistent tool — the spec's unknown-tool example is a protocol error, and an isError result claims the tool exists and ran
Gate failed — --smoke: 1 smoke check(s) failed
```

`Tools called: 0 of 1`. mcpscore calls only tools annotated
`readOnlyHint: true`, and `add` has no annotations. The spec
[defaults a missing `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-11-25/schema#toolannotations-readonlyhint)
to `false`, so `add` might write something, and smoke mode leaves
it alone. Every skip says why, so the gap in coverage is visible.

The failure is about the server, not about `add`. Step 3 explains it.

### Step 2: Annotate the tool

Tell clients that `add` has no side effects. This is the one-line change:

```python server.py highlight={2,7} theme={null}
from mcp.server import MCPServer
from mcp.types import ToolAnnotations

mcp = MCPServer("Basic Server")


@mcp.tool(annotations=ToolAnnotations(readOnlyHint=True))
def add(a: int, b: int) -> int:
    """Add two numbers."""
    return a + b
```

Run the same command again:

```text theme={null}
Audit finished. Final score: 121/141
...
Smoke checks (tools/call): 2 passed, 1 failed, 0 skipped — only readOnlyHint: true tools called (see --call-all). Never counted in the score.
  Tools called: 1 of 1 (a tool counts as called once the server answered a tools/call for it)
  pass smoke_structured_content [add]: structuredContent conforms to the declared outputSchema
       returned 1 content block(s) [text]; structuredContent present; first text "0"
  pass smoke_invalid_arguments [add]: rejected with an isError tool result
       returned 1 content block(s) [text]; isError: true; first text "Error executing tool add: 1 validation error for addArguments\na\n  Input should b" [truncated]
  FAIL smoke_unknown_tool: reported a tool execution error (isError) for a nonexistent tool — the spec's unknown-tool example is a protocol error, and an isError result claims the tool exists and ran
```

`add` is now called twice. With valid arguments it returned a result that
matches the output schema the SDK derived from `-> int`. With a string where
an integer belongs it refused, which is the rejection the check wants. A
tool that answers gets a `returned` line: the number and types of content
blocks, whether `structuredContent` was present, whether the result was
flagged `isError`, and the first 80 characters of text.

The annotation also raised the score from 118/140 to 121/141, because
`tools_annotations_present` now passes. The same annotation tells every
client that `add` does not modify its environment, and your smoke run now
covers it.

### Step 3: Read a failure

`smoke_unknown_tool` calls a tool name that cannot exist on your server and
expects a JSON-RPC error, as
[Tools §Error Handling](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#error-handling) requires for
unknown tools. This server answers with a tool result flagged
`isError`, which tells the client the tool exists and ran. The MCP Python SDK
answers this way by default, in both `FastMCP` (1.x) and `MCPServer` (2.x),
so a server built on it fails this check until the SDK changes or the server
rejects unknown names itself.

A failure in your own tool reads the same way. The first example on this
page shows the two most common ones:

```text theme={null}
  FAIL smoke_structured_content [dishonest]: declared outputSchema is not honored: Invalid structured content returned by tool dishonest: 'result' is a required property
  FAIL smoke_invalid_arguments [sloppy]: accepted schema-invalid arguments (returned a success result)
       returned 1 content block(s) [text]; first text "{}"
```

`dishonest` declares an output schema with a required `result` field and
returns an object without it. `sloppy` returned a success result for
arguments its input schema forbids. Both would reach an agent in production.

### Step 4: Gate CI on it

Any smoke failure exits with code `4`. Add the score gate the same way, and
CI can tell a low score apart from broken tools:

```bash theme={null}
# Fail the build on a low score or on broken tools, distinguishably
mcpscore --fail-under 80 --smoke --stdio uv run server.py
```

| Exit code | Meaning |
| - | - |
| `0` | Score at or above 80, and every smoke check passed or skipped |
| `3` | Score below 80. Takes precedence when both gates fail |
| `4` | Score fine, but a smoke check failed |

Raise the threshold to 90 against the 84% server from Step 1 and both gates
fail. The score gate wins:

```text theme={null}
Audit finished. Final score: 118/140
Smoke checks (tools/call): 0 passed, 1 failed, 2 skipped — only readOnlyHint: true tools called (see --call-all). Never counted in the score.
Gate failed — --fail-under 90: score 118/140 (84%) is below the required 90%
```

Skips never gate. A server with no `readOnlyHint: true` tools and a correct
unknown-tool rejection exits `0` and reports what it skipped and why. The
full exit-code table is in the [stability contract](https://docs.mcpscore.dev/stability.md#cli-interface).

## What each check catches

Each check reports `pass`, `fail` or `skip`. Skip means the check could not
run. It never means the server failed it.

| Check (`check_id`) | The regression it catches |
| - | - |
| `smoke_structured_content` | A handler no longer matches its declared [`outputSchema`](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#output-schema). Every callable tool that declares one is invoked, and the result must carry `structuredContent` that conforms to it, a spec MUST. An explicitly empty `{}` schema still requires the field to be present. |
| `smoke_invalid_arguments` | Input validation stopped working ([Tools §Error Handling](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#error-handling)). Each callable tool is invoked with arguments that violate its schema and must reject them. A tool result flagged `isError` and a JSON-RPC error both count as rejection. Accepting the arguments, hanging, or crashing fails the check. |
| `smoke_filesystem_boundary` | A file tool reads or writes outside the directory it was given ([Tools](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#security-considerations) and [Roots](https://modelcontextprotocol.io/specification/2025-11-25/client/roots#security-considerations) §Security Considerations). Runs only when you configure it; see [Check that file tools stay in their root](#check-that-file-tools-stay-in-their-root). |
| `smoke_network_boundary` | A URL-fetching tool reaches a destination its policy denies ([Tools §Security Considerations](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#security-considerations); the [SSRF mitigations](https://modelcontextprotocol.io/specification/2025-11-25/basic/security_best_practices#server-side-request-forgery-ssrf) in Security Best Practices). Runs only when you configure it; see [Check that fetch tools reach only what they should](#check-that-fetch-tools-reach-only-what-they-should). |
| `smoke_unknown_tool` | Routing broke. A `tools/call` for a name that cannot exist on your server must be rejected with a [JSON-RPC protocol error](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#error-handling). A result, an `isError` result, and an internal error all fail. |

For the two rejection checks, any JSON-RPC error code counts as a rejection
except three: a request timeout, a closed connection, and `-32603` internal
error. Those three mean the server tried to run the call and failed. Each
call has 30 seconds to answer.

## Which tools get called

* Only tools annotated `readOnlyHint: true` are called by default. An
  unannotated tool is skipped, and every skip is reported with its reason.

* `--call-all` lifts that default. No other flag implies it, and it has no
  environment-variable form. On the unannotated server from Step 1, it calls
  `add` without the annotation:

  ```bash theme={null}
  mcpscore --smoke --call-all --stdio uv run server.py
  ```

  ```text theme={null}
  Smoke checks (tools/call): 2 passed, 1 failed, 0 skipped — every tool called (--call-all). Never counted in the score.
  ```

* The unknown-tool check uses a name derived from your server's own catalog,
  so it cannot match a real tool. If the tool listing failed or is
  incomplete, the check skips instead of guessing a name.

* Arguments come from each tool's input schema, in this order: `default`,
  `const`, the first entry of `examples`, the first `enum` value, then a zero
  value for the type. No language model is involved, so the same server
  receives the same calls on every run.

> [accordion] Technical details: errors from upstream skip rather than fail
>
> A tool that returns an error result for valid synthesized arguments reports
> as `skip` in `smoke_structured_content`. It may be rejecting the sample
> value, or a service behind it may be down, and an outage elsewhere should
> not fail your build. A JSON-RPC error to valid arguments skips for the same
> reason. Only behavior your server owns can fail a smoke check.
>
> Calls run one at a time, in catalog order: every structured-content call
> first, then every invalid-arguments call, then the unknown-tool call. A
> malformed schema fragment is tolerated rather than aborting the run.

## Check that file tools stay in their root

If your server reads or writes files, smoke mode can check that a path never
escapes the directory the server was given. mcpscore builds a throwaway
directory tree, starts your server with its `allowed` directory as the root,
and asks your tools for files outside it.

Name the tools in `mcpscore.toml` (or `[tool.mcpscore]` in `pyproject.toml`),
then put `{mcpscore_fs_root}` where your `--stdio` command names the directory:

```toml mcpscore.toml theme={null}
[smoke.filesystem]
read_tool = "read_text_file"
write_tool = "write_file"
```

```bash theme={null}
mcpscore --smoke --stdio npx -y @modelcontextprotocol/server-filesystem '{mcpscore_fs_root}'
```

```text theme={null}
Filesystem fixture: {mcpscore_fs_root} is /private/var/folders/…/mcpscore-fs-_qxxpo7y/allowed
...
  pass smoke_filesystem_boundary [read_text_file: parent_traversal]: rejected with an isError tool result
  pass smoke_filesystem_boundary [read_text_file: absolute_path]: rejected with an isError tool result
  pass smoke_filesystem_boundary [read_text_file: sibling_prefix]: rejected with an isError tool result
  pass smoke_filesystem_boundary [read_text_file: symlink_directory]: rejected with an isError tool result
  pass smoke_filesystem_boundary [read_text_file: symlink_file]: rejected with an isError tool result
  skip smoke_filesystem_boundary [read_text_file: windows_forward_slash_traversal]: Windows path form; runs on Windows only
  ...
```

That is the official reference filesystem server: every escape is refused.
Each line is one attempt:

| Vector | The path it sends |
| - | - |
| `parent_traversal` | `<root>/../outside/…` |
| `absolute_path` | The absolute path of a file outside the root |
| `sibling_prefix` | A file in `allowed-sibling`, which starts with the root's name. A plain string-prefix check lets it through |
| `symlink_directory` | A file through a symlink inside the root that points to a directory outside |
| `symlink_file` | A symlink inside the root that points to a file outside |
| `windows_forward_slash_traversal`, `windows_extended_length_path` | `C:/…/../outside/…` and `\\?\C:\…\outside\…`, on Windows only |

A read fails the check when the response contains the content of the file
outside. A write fails it when a file appears or changes outside the root.
What the tool says about its own call doesn't count either way.

* **Naming a tool is consent to call it**, even a write tool without
  `readOnlyHint: true`; `--call-all` isn't needed, and the summary line names
  these tools. Writes only target the throwaway directory, which is removed
  after the run.
* **Each tool must work inside the root first.** mcpscore reads a file it put
  in the root, or writes one there. If that fails, every attempt skips with
  the reason, because a misconfigured tool would otherwise refuse everything
  and look safe.
* **The argument names default to `path` and `content`.** Set
  `read_path_argument`, `write_path_argument` or `write_content_argument` when
  your tools use others.
* **It needs a `--stdio` command with the placeholder.** Against a URL, or
  without `{mcpscore_fs_root}`, the check reports one skip saying so.

A pass means these paths were refused, not that the server is secure.

## Check that fetch tools reach only what they should

If a tool fetches URLs, smoke mode can check that it can't be steered to a
destination it should refuse: server-side request forgery. mcpscore starts
two HTTP listeners on this machine, one your server may fetch and one it must
not, and asks your tool for the second one in a dozen ways. The verdict comes
from the denied listener's request log: a request that arrives there fails
the attempt, whatever the tool answered.

Name the tool and the policy it enforces. With `policy = "allowlist"`, put
`{mcpscore_allowed_origin}` where your `--stdio` command, or an `--env` value,
takes the origins the server may fetch:

```toml mcpscore.toml theme={null}
[smoke.network]
tool = "fetch"
policy = "allowlist"
```

```bash theme={null}
mcpscore --smoke --stdio python my_server.py --allow-origin '{mcpscore_allowed_origin}'
```

```text theme={null}
Network fixture: {mcpscore_allowed_origin} is http://127.0.0.1:49452
...
  pass smoke_network_boundary [fetch: direct]: rejected with an isError tool result; no request reached the denied destination
  pass smoke_network_boundary [fetch: localhost_name]: rejected with an isError tool result; no request reached the denied destination
  ...
  pass smoke_network_boundary [fetch: userinfo_confusion]: rejected with an isError tool result; no request reached the denied destination
  FAIL smoke_network_boundary [fetch: redirect]: a request reached a destination the policy denies (GET /v/redirect)
  skip smoke_network_boundary [fetch: dns_rebinding]: needs control of the server's DNS resolver to change an answer between check and use; not assessed
```

This server compares the URL's host and port with its allowlist, but then
follows redirects without checking them again: a URL on the allowed origin
answers `302` to the denied listener, and the server goes there. Each line is
one attempt:

| Vector | What it sends |
| - | - |
| `direct` | `http://127.0.0.1:<denied-port>/…` |
| `localhost_name`, `ipv6_loopback`, `unspecified_address` | The same listener as `localhost`, `[::1]` and `0.0.0.0` |
| `ipv4_mapped_ipv6`, `decimal_ipv4`, `hex_ipv4`, `octal_ipv4`, `short_ipv4` | `127.0.0.1` spelled `[::ffff:127.0.0.1]`, `2130706433`, `0x7f000001`, `017700000001` and `127.1`. A deny list of host strings misses them |
| `userinfo_confusion` | `http://127.0.0.1:<allowed-port>@127.0.0.1:<denied-port>/`. A string-prefix check sees the allowed origin |
| `backslash_confusion`, `fragment_confusion` | The allowed origin after `\@` or `#@`. Parsers disagree about which host such a URL names |
| `redirect` | A URL on the allowed origin that redirects to the denied listener |
| `dns_rebinding` | Always skipped: it needs control of the server's DNS resolver |

* **`policy = "deny-private"`** is for a server that refuses loopback and
  private addresses. It can't fetch the local allowed listener either, so
  there is no positive control: a request that reaches either listener, both
  loopback, fails, and a refusal is reported as not assessed, never as a pass.
* **With `allowlist`, the tool must reach the allowed origin first.**
  mcpscore fetches `{mcpscore_allowed_origin}/control`. If that request never
  arrives, every attempt skips with the reason, because a misconfigured tool
  would otherwise refuse everything and look safe. A request the control
  sends to the denied listener still fails, as `positive_control`. The
  allowlist must match the origin's port: both listeners are on `127.0.0.1`.
* **A fetch that arrives after its call returned still counts.** mcpscore
  waits a second after the last attempt; a request that reaches a denied
  listener by then is reported as `late_request`, because the tool fetches in
  the background and its answers don't show what it reached.
* **Naming the tool is consent to call it**, as for file tools. The argument
  name defaults to `url`; set `url_argument` when your tool uses another.
* **Nothing outside this machine is contacted.** No cloud metadata endpoint,
  internal service or callback service: every destination is one of the two
  listeners, and the report shows their ports as `<allowed-port>` and
  `<denied-port>`.
* **It needs a `--stdio` server without a proxy.** Against a URL, with
  `allowlist` and no placeholder, or when `--env` gives the server
  `HTTP_PROXY`, `HTTPS_PROXY` or `ALL_PROXY`, the check reports one skip
  saying so: through a proxy the probe URLs could leave this machine.

A pass means these requests didn't arrive, not that the server is safe from
request forgery.

## Results stay out of the score

The score is deterministic: the same server gets the same score. Tool calls
depend on upstream services and on the environment, so smoke results live in
their own `smoke` section of the [JSON report](https://docs.mcpscore.dev/stability.md#report-schema) and
never change `score` or `max_score`:

```json theme={null}
"smoke": {
  "executed": true,
  "reason": null,
  "call_all": false,
  "summary": { "passed": 4, "failed": 2, "skipped": 3 },
  "coverage": { "tools_called": 3, "tools_total": 4 },
  "checks": [
    {
      "check_id": "smoke_structured_content",
      "tool_name": "dishonest",
      "verdict": "fail",
      "message": "declared outputSchema is not honored: Invalid structured content returned by tool dishonest: 'result' is a required property",
      "details": {
        "basis": "MCP 2025-11-25 Tools §Output Schema (structured results MUST conform to the declared schema)",
        "called": true
      }
    },
    ...
  ]
}
```

When `--smoke` is requested but there is no session to call tools on, the
section reports `"executed": false` with the reason instead of disappearing.
That happens on a [partial audit](https://docs.mcpscore.dev/authenticated-servers.md) of an auth-gated
server, on a modern-only probe audit, and when the tools catalog had to be
recovered from the stateless lifecycle because the negotiated session could
not list it.

## Where it does not run

* mcpscore.dev never runs smoke checks. Invoking tools on third-party URLs
  from our infrastructure is outside the website's safety model, so smoke
  mode is CLI-only.
* The [GitHub Action](https://docs.mcpscore.dev/github-action.md) accepts `--smoke` in `args`. A smoke
  failure fails the job after the report and PR comment are published. The
  comment does not list the smoke checks yet, and there is no dedicated
  `smoke` input.
* Your test suite still has a job. Smoke checks prove the MCP contract:
  honest schemas, correct rejections, tools that answer. They do not test
  your business logic.

## Recap

* Run `mcpscore --smoke` against your own server and read the skip reasons.
* Annotate read-only tools with `readOnlyHint: true` so smoke mode calls them.
* Read each `FAIL` line as check, tool, and reason.
* Gate CI with `--fail-under` and `--smoke`: exit `3` is the score, exit `4`
  is broken tools.
* Add `[smoke.filesystem]` and `{mcpscore_fs_root}` to check that file tools
  stay in their root.
* Add `[smoke.network]` and `{mcpscore_allowed_origin}` to check that fetch
  tools reach only what their policy allows.

## What's next

> [card] [GitHub Action](https://docs.mcpscore.dev/github-action.md)
>
> Run the same gate on every pull request

> [card] [CI beyond GitHub](https://docs.mcpscore.dev/ci-other-platforms.md)
>
> Exit codes and `--json` in any CI system

> [card] [Stability Contract](https://docs.mcpscore.dev/stability.md)
>
> Exit codes and the `smoke` report section

> [card] [Authenticated Servers](https://docs.mcpscore.dev/authenticated-servers.md)
>
> Pass credentials so smoke can reach behind the gate


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.