GitHub Action

A refactor that drops a tool description or breaks a schema should fail the pull request, not surface later inside someone’s agent. One step does that. It audits the server, fails the job below your threshold, and posts the report as a comment that updates in place on every push.

.github/workflows/mcpscore.yml

name: MCP quality
on: [pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write   # lets the action post its comment
    steps:
      - uses: mcp-box/mcpscore-action@v1
        with:
          target: https://your-server.example/mcp
          min-score: 80

Commit that file and open a pull request. The job log ends with the score:

mcpscore: 91/108 (84%), era legacy

The same report lands in the PR comment and in the job summary. The job summary needs no token at all. The report looks like this:

## 🔦 mcpscore — 91/108 (84%)

**Target:** `https://mcp.deepwiki.com/mcp`  ·  **Spec:** 2025-11-25 (legacy)

| | CRITICAL | HIGH | MEDIUM | LOW |
|---|---|---|---|---|
| ✅ passed | 9 | 10 | 4 | 8 |
| ❌ failed | 0 | 1 | 3 | 8 |

**Readiness for MCP 2026-07-28:** 3/13 (23%) — informative, not counted in the main score.

<details><summary>12 failed check(s)</summary>

- `protocol_version_latest` (MEDIUM)
- `security_origin_validation` (HIGH)
...

Every failed check is listed by rule_id, so a fix is one lookup in the rules reference away.

Set it up step by step

The workflow above has three parts that matter. Each step below adds one.

Step 1: Audit on every pull request

.github/workflows/mcpscore.yml

name: MCP quality
on: [pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: mcp-box/mcpscore-action@v1
        with:
          target: https://your-server.example/mcp

target is the server to audit: a URL, or a .py or .js file in the repository. The action installs mcpscore, runs the audit, writes mcpscore-report.json, and publishes the report to the job summary. With no threshold the job passes whatever the score, which is a safe way to see your number first.

Step 2: Give it permission to comment

.github/workflows/mcpscore.yml

name: MCP quality
on: [pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write   # lets the action post its comment
    steps:
      - uses: mcp-box/mcpscore-action@v1
        with:
          target: https://your-server.example/mcp

This is the same file plus one block. The comment is created on the first run and edited on every push after it, so a pull request carries one comment, not one per commit. Without the permission the action skips the comment and still passes or fails on the score. Commenting never fails the job.

Step 3: Fail below a threshold

.github/workflows/mcpscore.yml

name: MCP quality
on: [pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write   # lets the action post its comment
    steps:
      - uses: mcp-box/mcpscore-action@v1
        with:
          target: https://your-server.example/mcp
          min-score: 80

min-score is a percentage from 0 to 100. A server at 84% passes a threshold of 80. Set it to 90 and the same server fails the job:

mcpscore: 91/108 (84%), era legacy
::error::score 91/108 (84%) is below the required 90%

The report and comment are still published, so the author sees which rules cost the points. Set the threshold at or a little under today’s score, and every regression from here fails the pull request that caused it.

Variations

Each variation below changes the Step 3 workflow in one place.

A server that lives in the repo

Check out the code and point target at the entry file. mcpscore starts it with python or node from the runner, so that runtime and the server’s dependencies must be installed first. This job runs the server code from the pull request, so give it no write scope: checkout needs contents: read and nothing else, and the report is in the job summary. Keep the commenting job, if you want one, on a remote target that the pull request cannot change.

jobs:
  audit:
    runs-on: ubuntu-latest
    permissions:
      contents: read         # for actions/checkout; no write scope in a job that runs PR code
    steps:
      - uses: actions/checkout@v7
      - uses: mcp-box/mcpscore-action@v1
        with:
          target: ./server.py
          min-score: 85
          comment: false     # read the job summary instead

An auth-gated server

Store the credential as an Actions secret and export it as MCPSCORE_TOKEN. The CLI sends it as a bearer token, and the value never reaches the log or the report.

- uses: mcp-box/mcpscore-action@v1
  with:
    target: https://your-server.example/mcp
    min-score: 80
  env:
    MCPSCORE_TOKEN: ${{ secrets.MCPSCORE_TOKEN }}

API-key servers pass a header through args. The action splits args on whitespace, so write the header as one token with no space after the colon:

- uses: mcp-box/mcpscore-action@v1
  with:
    target: https://your-server.example/mcp
    args: --header=X-Api-Key:${{ secrets.MCP_API_KEY }}

Without a credential an auth-gated server gets a partial audit, and min-score is judged on that partial score, which covers only the auth, TLS, and transport surface. Pass a credential before trusting the gate. See authenticated servers.

Turn rules off for this repository

A mcpscore.toml in the checkout is picked up the same way the CLI picks it up, so the action needs an actions/checkout step before it. Rules you have turned off or re-ranked shape the score the action gates on and comments with. The badge never changes. See configure rules.

mcpscore.toml

[rules]
server_websiteurl_present = "off"

[gate]
fail_on = "high"

A [gate] failure fails the job with the rule named:

::error::mcpscore [gate] fail_on = HIGH: failed rule(s) security_origin_validation

Also require readiness for MCP 2026-07-28

- uses: mcp-box/mcpscore-action@v1
  with:
    target: https://your-server.example/mcp
    min-score: 80
    min-readiness: 50   # fail below 50% ready for MCP 2026-07-28

mcpscore: 91/108 (84%), era legacy
::error::readiness 3/13 (23%) is below the required 50%

The gate is skipped only when no readiness check could be assessed at all, which the report shows as a zero readiness.max_score. A legacy server still has its gateway checks assessed, so it is gated on the few points those carry, like the 13 above. Check the readiness line of a run before choosing the threshold.

Send findings to code scanning

Set sarif-path and upload the file. Every failed rule becomes an alert in the repository’s Security tab, next to the findings of your other scanners. The upload needs security-events: write, and in a private repository contents: read as well.

jobs:
  audit:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      security-events: write   # lets upload-sarif create the alerts
      contents: read           # upload-sarif needs it in a private repository
    steps:
      - uses: mcp-box/mcpscore-action@v1
        with:
          target: https://your-server.example/mcp
          min-score: 80
          sarif-path: mcpscore.sarif
      - uses: github/codeql-action/upload-sarif@v4
        # A failed gate still wrote the file; a connection failure did not.
        if: ${{ !cancelled() && hashFiles('mcpscore.sarif') != '' }}
        with:
          sarif_file: mcpscore.sarif

The file is written before the score gate runs, so a failing build still carries its findings. Alerts are keyed on the rule and the target, so a re-run updates them in place. For a URL target the findings are about the running server, not a line of source, so they appear in the Security tab rather than as annotations in the diff. For a local server file they sit at line 1 of that file, so GitHub annotates the diff when a pull request changes that line. min-score is what fails the pull request. upload-sarif allows one upload per job for a given tool and category. To audit two servers in one job, give each upload step its own category: input, or the second upload is rejected with “only one run of the codeql/upload-sarif action is allowed per job per tool/category”. Severity mapping and what the file shows of the target are on the CLI reference.

Smoke-test the tools

For a server you operate, add --smoke through args. After the audit, mcpscore calls the tools annotated readOnlyHint: true and checks their answers. A failed check fails the job after the report and comment are published:

- uses: actions/checkout@v7
- uses: mcp-box/mcpscore-action@v1
  with:
    target: ./server.py
    min-score: 80
    args: --smoke
    comment: false

::error::mcpscore --smoke: failed check(s) smoke_structured_content

The comment does not list smoke checks, so read them in the job log. See smoke mode for what each check catches.

Use the numbers in later steps

Give the step an id and read its outputs:

- id: mcp
  uses: mcp-box/mcpscore-action@v1
  with:
    target: https://your-server.example/mcp
- run: echo "Scored ${{ steps.mcp.outputs.percentage }}% (era ${{ steps.mcp.outputs.era }})"

Scored 84% (era legacy)

Pin everything

@v1 follows the latest v1 release, and version defaults to the latest mcpscore on PyPI. For reproducible runs, pin the action to a commit and version to an mcpscore release:

- uses: mcp-box/mcpscore-action@<commit-sha>  # v1.x.y
  with:
    target: https://your-server.example/mcp
    version: "1.20.0"
    min-score: 80

Inputs

Input	Default	What it does
`target`	required	Server URL, or a local `.py` or `.js` path
`min-score`	no gate	Fail the job when the main score percentage (0–100) is below this. With a `mcpscore.toml` in the checkout, that is the configured score
`min-readiness`	no gate	Fail the job when the readiness percentage (0–100) is below this. Skipped when readiness was not assessed
`comment`	`true`	Post or update the report comment on the pull request
`args`	none	Extra arguments passed to the mcpscore CLI, split on whitespace. Write each as one token, for example `--header=X-Api-Key:$KEY`
`version`	latest	mcpscore release to run, for example `1.20.0`
`report-path`	`mcpscore-report.json`	Where the JSON report is written
`sarif-path`	off	Also write the failed rules as SARIF 2.1.0 to this path, for `upload-sarif`. Needs mcpscore 1.14.0 or later
`github-token`	`github.token`	Token for the comment. Needs `pull-requests: write`

Outputs

Output	Meaning
`score`, `max-score`, `percentage`	Main score: points earned, points available, integer percentage
`readiness-score`, `readiness-max`, `readiness-percentage`	The same three for the readiness axis
`era`	`legacy`, `modern`, or `dual-era`
`negotiated-version`	Spec revision the server negotiated

When it fails

::error::score 91/108 (84%) is below the required 90%

Cause: the server scored under the threshold. This is the action doing its job.
Fix: open the comment or job summary, take the failed rule_ids to the rules reference or improve your score, and fix the server.

The job passes but no comment appears on the PR

Cause: the workflow token is read-only. GitHub does this on pull requests from forks, and when the job lacks permissions: pull-requests: write.
Fix: add the permission. For fork PRs, read the report in the job summary. Do not switch the workflow to pull_request_target to get the comment. That event runs with a write token in the base repository’s context, and running the fork’s server there hands the token to untrusted code. The GitHub Security Lab write-up explains the trap.

The job passes, but the comment says the audit was partial

Cause: the server needs a credential and the workflow passed none, so min-score was judged on the observable surface only.
Fix: export MCPSCORE_TOKEN from a secret, as above, so the gate judges the full audit.

::error::mcpscore could not audit ./server.py (exit 2)

Cause: the server did not start or did not answer. The CLI’s own error is printed with it. Server script not found: ./server.py means no actions/checkout step, or a path relative to the wrong directory.
Fix: check out the repo first, use a path relative to its root, and install the server’s dependencies before the action runs. Troubleshooting covers the other connection failures.

::error::'sarif-path' and a `--sarif` in `args` name two files; keep one (prefer the input)

Cause: the SARIF file was requested twice.
Fix: remove --sarif from args and keep sarif-path.

Recap

Add a workflow that runs mcp-box/mcpscore-action@v1 on pull_request with a target.
Grant pull-requests: write so the report lands as a comment.
Set min-score to fail pull requests that lower the score.

What’s next

Smoke mode

Call your server’s tools in CI, not only list them

CI beyond GitHub

The same gate in GitLab or any CI with exit codes

Score badge

A README badge that follows your server’s score

Action source

mcp-box/mcpscore-action on GitHub

Get started

Guides

Use in CI

Scoring

Reference

Set it up step by step

Step 1: Audit on every pull request

Step 2: Give it permission to comment

Step 3: Fail below a threshold

Variations

A server that lives in the repo

An auth-gated server

Turn rules off for this repository

Also require readiness for MCP 2026-07-28

Send findings to code scanning

Smoke-test the tools

Use the numbers in later steps

Pin everything

Inputs

Outputs

When it fails

Recap

What’s next

Smoke mode

CI beyond GitHub

Score badge

Action source

​Set it up step by step

​Step 1: Audit on every pull request

​Step 2: Give it permission to comment

​Step 3: Fail below a threshold

​Variations

​A server that lives in the repo

​An auth-gated server

​Turn rules off for this repository

​Also require readiness for MCP 2026-07-28

​Send findings to code scanning

​Smoke-test the tools

​Use the numbers in later steps

​Pin everything

​Inputs

​Outputs

​When it fails

​Recap

​What’s next

Smoke mode

CI beyond GitHub

Score badge

Action source

Set it up step by step

Step 1: Audit on every pull request

Step 2: Give it permission to comment

Step 3: Fail below a threshold

Variations

A server that lives in the repo

An auth-gated server

Turn rules off for this repository

Also require readiness for MCP 2026-07-28

Send findings to code scanning

Smoke-test the tools

Use the numbers in later steps

Pin everything

Inputs

Outputs

When it fails

Recap

What’s next