# Spanly

> Spanly is observability for MCP (Model Context Protocol) servers. It captures
> every tool call, resource read, and prompt as a first-class signal, so
> engineering teams running MCP in production can debug failures, watch
> performance per tool, and reconcile MCP traffic with the rest of their stack.

Spanly is built to run alongside an existing APM (Datadog, Sentry, New Relic),
not to replace it. The SDK cross-links every span to the host APM via W3C
trace context, so an MCP-scoped error in Spanly opens straight into the
parent trace in the APM the team already runs.

The product is targeted at B2B SaaS companies (Series A to C, 20 to 500 engineers)
that have shipped or are shipping an MCP server. The SDK and CLI are
open source (Apache 2.0); the hosted backend is paid with a free tier.

## Core concepts

- **MCP (Model Context Protocol)**: a JSON-RPC based protocol used by AI
  clients (Claude, Cursor, Windsurf, custom agents) to call tools, read
  resources, and fetch prompts from a server. Standard APMs see the
  outer HTTP request; Spanly sees the protocol-level operation.
- **Tool call**: an individual `tools/call` operation. Spanly captures the
  tool name, arguments, response, latency, and error if any.
- **Resource read**: a `resources/read` operation. Spanly captures the
  resource URI, size, and cache outcome.
- **Session**: a single client-server MCP connection. Spanly groups all
  requests under a session so an incident can be traced end to end.
- **Client identity**: the MCP client that initiated traffic (e.g. Claude
  Desktop, Cursor, Windsurf, an in-house agent). Spanly breaks down
  metrics per client.

## Differences from general APMs

| Signal      | General APM                     | Spanly                                                 |
| ----------- | ------------------------------- | ------------------------------------------------------ |
| Protocol    | HTTP POST /mcp                  | tools/call, resources/read, prompts/get                |
| Errors      | 500 + stack trace               | Tool-call rejection with the full prompt and arguments |
| Performance | p95 endpoint latency            | p95 per tool, per server name, per client              |
| Payload     | Bytes in / bytes out            | Tokens in / tokens out, resource read sizes            |
| Clients     | One client identity per session | Claude, Cursor, Windsurf, custom                       |

## Pages

- [Home](https://spanly.com/): product overview, APM comparison, dashboard preview.
- [MCP observability](https://spanly.com/mcp-observability/): what MCP-native observability covers and why an APM alone misses it.
- [Pricing](https://spanly.com/pricing/): tiers and ingest pricing.
- [Security](https://spanly.com/security/): data residency, encryption, compliance posture.
- [Founder program](https://spanly.com/founding/): early-access design partner offer.
- [About](https://spanly.com/about/): company description.
- [Careers](https://spanly.com/careers/): open roles.
- [Privacy policy](https://spanly.com/privacy/)
- [Terms of service](https://spanly.com/terms/)

## Open source

- GitHub: https://github.com/spanlyhq/spanly (Apache 2.0)
- TypeScript SDK: `npm install @spanly/sdk`
- Python SDK: `pip install spanly`
- CLI: `npx -y @spanly/spanly` or `brew install spanlyhq/tap/spanly`
- Docker: `docker pull spanly/spanly:latest`

## Data handling

- Regions: US and EU.
- All ingest TLS-encrypted; storage encrypted at rest.
- GDPR compliant; data does not cross regions.
- SDK overhead: under 1 ms per traced operation in benchmarks.

## Contact

- Website: https://spanly.com
- Status: https://spanly.com/#status

# Documentation

> The full text of every page at https://spanly.com/docs/.

---

## Quickstart

Source: https://spanly.com/docs/

Spanly is the observability platform for
[Model Context Protocol](https://modelcontextprotocol.io) servers and
AI agents. Drop in an SDK or run the CLI in front of your server, and
every JSON-RPC packet (tools, prompts, resources, errors) streams to
a dashboard you can search, slice, and alert on.

## 1. Create a project and grab an API key

1. Sign up at [spanly.com](https://spanly.com/).
2. Create a project (or use the default one).
3. Go to **Settings → API keys** and copy the key. It starts with
   `spanly_us_` or `spanly_eu_`. The region is encoded in the
   prefix, so you do not have to configure it separately.

Set it in your shell:

```bash
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

Always read the key from the environment. Never hard-code it.

## 2. Pick an integration path

All paths produce the same data; pick whichever fits your stack.

| Path                              | Language        | Code changes                                      |
| --------------------------------- | --------------- | ------------------------------------------------- |
| [CLI `spanly run`](https://spanly.com/docs/cli/run/)      | Any             | None. Wraps your server command.                  |
| [CLI `spanly proxy`](https://spanly.com/docs/cli/proxy/)  | Any             | None. Reverse proxy in front of a running server. |
| [TypeScript SDK](https://spanly.com/docs/typescript-sdk/) | Node, Bun, Deno | One line: mount HTTP/ASGI middleware.             |
| [Python SDK](https://spanly.com/docs/python-sdk/)         | Python 3.10+    | One line: mount HTTP/ASGI middleware.             |
| [Docker sidecar](https://spanly.com/docs/docker/sidecar/) | Any             | None. Container next to your server.              |

### Fastest path: wrap your server with the CLI

No code changes, works for any language, stdio or HTTP:

```bash
# 1. Set your API key (region auto-detected from the prefix)
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxx

# 2. Wrap your MCP server. stdio:
npx -y @spanly/spanly run -- node ./server.js

# Or HTTP. The wrapper takes your port; the child gets a random one:
npx -y @spanly/spanly run --port 3000 -- node ./server.js
```

That's the whole integration. Skip to step 3.

### Prefer in-process middleware?

The SDKs are HTTP (TypeScript) and ASGI (Python) middleware: mount them
in front of your existing MCP server and identify end users from your
own auth state with the `identity` option.

**TypeScript**

```bash
npm install @spanly/sdk
```

```typescript
import express from "express";
import { spanly } from "@spanly/sdk";

const app = express();
app.use(spanly({ apiKey: process.env.SPANLY_API_KEY }));

// app.use("/mcp", mcpRouter);
```

**Python**

```bash
pip install spanly
# or: uv add spanly
```

```python
import os

from spanly import SpanlyMiddleware

app.add_middleware(SpanlyMiddleware, api_key=os.environ["SPANLY_API_KEY"])
```

## 3. Run your server normally

The SDK and the CLI both run the server unchanged. No new ports, no
extra processes from the user's perspective. Talk to it from your MCP
client (Claude Desktop, Cursor, Windsurf, a Python script) as you
would normally.

## 4. See the request in the dashboard

Open [spanly.com](https://spanly.com/), pick your project, and head to
the **Requests** view. Within a few seconds of the first MCP call, you
will see a row appear with the method, server, client, duration, and
a link to the full JSON-RPC payload.

## What gets captured

For every MCP request, Spanly records:

- The raw JSON-RPC request and response.
- Tool / prompt / resource name and arguments.
- Duration (request received to response sent).
- Transport metadata. For HTTP: method, path, and headers.
- Server and client `name` + `version` from the MCP `initialize` handshake.
- Errors, including stack traces when available.

Credential-bearing headers (`Authorization`, `Cookie`, `Set-Cookie`,
`Proxy-Authorization`, `X-Api-Key`, `X-Auth-Token`,
`X-Amz-Security-Token`, `X-Forwarded-Authorization`) are replaced with
`[REDACTED]` before the telemetry packet leaves your process, so
secrets never reach Spanly. You can extend the list with the
`redactHeaders` SDK option or the CLI's `--redact-header` flag.

Nothing else leaves your process. No request body or response body is
modified. The SDK and CLI both forward the original bytes verbatim,
including the original headers; only the telemetry copy is redacted.

## Data handling

- **Redaction**: credential-bearing headers are redacted automatically
  (see above); attribute traffic to an end user instead of dropping it
  with the SDK's [`identity`](https://spanly.com/docs/typescript-sdk/api-reference/#identity)
  option (TypeScript) or the
  [Python equivalent](https://spanly.com/docs/python-sdk/api-reference/#identity). With the
  CLI, `--inspect-prefix` limits capture to specific paths; the SDKs'
  `paths` option does the same.
- **Retention**: captured requests are kept for 30 days on Free, 90
  days on Pro, and 365 days on Business. See
  [pricing](https://spanly.com/pricing).
- **Residency**: data is stored in the region your API key belongs to
  (US or EU) and never leaves it. See the
  [security overview](https://spanly.com/security) and
  [privacy policy](https://spanly.com/privacy).

## What's next

- The [TypeScript SDK reference](https://spanly.com/docs/typescript-sdk/api-reference/) covers
  the full `spanly()` and `wrapFetchHandler()` middleware, including
  the `identity` option for end-user attribution.
- The [Python SDK reference](https://spanly.com/docs/python-sdk/api-reference/) is the
  equivalent for Python.
- The [CLI reference](https://spanly.com/docs/cli/flags/) lists every flag for `spanly run`
  and `spanly proxy`.
- If nothing shows up in the dashboard, check the
  [troubleshooting guide](https://spanly.com/docs/troubleshooting/) first.

---

## TypeScript SDK

Source: https://spanly.com/docs/typescript-sdk/

The TypeScript SDK is HTTP middleware: mount `spanly()` in front of any
MCP server that speaks HTTP and it captures every tool call, prompt,
resource access, and JSON-RPC packet, with no changes to your server
code. It runs on Node, Bun, and Deno.

```bash
npm install @spanly/sdk
```

```typescript
import express from "express";
import { spanly } from "@spanly/sdk";

const app = express();
app.use(spanly({ apiKey: process.env.SPANLY_API_KEY }));

// app.use("/mcp", mcpRouter);
```

## In this section

- [Installation](https://spanly.com/docs/typescript-sdk/installation/) covers package install and
  the `SPANLY_API_KEY` environment variable.
- [Quickstart](https://spanly.com/docs/typescript-sdk/quickstart/) walks through mounting the
  middleware in a typical server end to end, including the Fetch/Hono
  binding and serverless delivery.
- [API reference](https://spanly.com/docs/typescript-sdk/api-reference/) documents `spanly()`,
  `wrapFetchHandler()`, and every option they accept.
- [Examples](https://spanly.com/docs/typescript-sdk/examples/) shows end-user attribution, a
  local test harness, and what the middleware does not do.

---

## Installation

Source: https://spanly.com/docs/typescript-sdk/installation/

The TypeScript SDK ships as `@spanly/sdk` on npm. It works under Node,
Bun, and Deno (with `npm:` specifiers). No native dependencies.

## Install

```bash
npm install @spanly/sdk
# or
pnpm add @spanly/sdk
# or
yarn add @spanly/sdk
# or
bun add @spanly/sdk
```

## Supported runtimes

| Runtime | Versions                       | Status    |
| ------- | ------------------------------ | --------- |
| Node.js | 20 LTS, 22 LTS                 | Supported |
| Bun     | ≥ 1.1                          | Supported |
| Deno    | ≥ 1.40 (via `npm:@spanly/sdk`) | Supported |

The package is ESM-only and ships type definitions. Under CommonJS,
load it with a dynamic `import()`.

## Configure the API key

The middleware reads the API key from the `apiKey` option, falling
back to the `SPANLY_API_KEY` environment variable. Always source it
from the environment, never hard-code it:

```ts
import express from 'express';
import { spanly } from '@spanly/sdk';

const app = express();
app.use(spanly({ apiKey: process.env.SPANLY_API_KEY }));
```

Get a key by signing in at [spanly.com](https://spanly.com/), opening
your project, and going to **Settings → API keys**. The region (`us` /
`eu`) is encoded in the key prefix, so no extra config is needed.

## When to use the SDK vs the CLI

Mount `spanly()` (or `wrapFetchHandler()`) in front of your MCP server
when it runs over HTTP on Node, Bun, or Deno. It needs no extra
process, no sidecar, and no separate binary.

Reach for the [Spanly CLI](https://spanly.com/docs/cli/) (`@spanly/spanly`) instead when:

- Your MCP server speaks stdio, not HTTP.
- Your server is not Node or Python (Go, Rust, Java, and so on).
- You are on AWS Lambda, where this SDK's serverless delivery
  guarantees do not hold. See
  [Serverless delivery](https://spanly.com/docs/typescript-sdk/quickstart/#serverless-delivery).

```bash
npx -y @spanly/spanly run -- node ./server.js
```

---

## Quickstart

Source: https://spanly.com/docs/typescript-sdk/quickstart/

This walks through mounting the Spanly middleware in front of a
TypeScript MCP server that speaks HTTP. `spanly()` inspects requests
under `/mcp` and `/sse` by default, tees the request and response
bytes, and never sends a response, mutates a header, or delays the
app.

## 1. Install

```bash
npm install @spanly/sdk
```

## 2. Set the API key

```bash
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

Get a key by signing in at [spanly.com](https://spanly.com/), opening
your project, and going to **Settings → API keys**. The region is
encoded in the key prefix (`spanly_us_` / `spanly_eu_`) and
auto-detected, so there is nothing else to configure.

## 3. Mount the middleware

```typescript
import express from "express";
import { spanly } from "@spanly/sdk";

const app = express();
app.use(spanly({ apiKey: process.env.SPANLY_API_KEY }));

// app.use("/mcp", mcpRouter);
```

That's it. Traffic on `/mcp` and `/sse` is now reported to your Spanly
project. `spanly()` works whether it is mounted before or after a JSON
body parser like `express.json()`, and also works with Koa or a bare
`http.createServer` handler.

## Full example: Express

```ts
import express from 'express';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { spanly } from '@spanly/sdk';

const mcpServer = new McpServer({
  name: 'demo-http-server',
  version: '1.0.0',
});

mcpServer.registerTool('ping', {}, async () => ({
  content: [{ type: 'text', text: 'pong' }],
}));

// Stateless mode: one shared transport serves every request.
const transport = new StreamableHTTPServerTransport({
  sessionIdGenerator: undefined,
});
await mcpServer.connect(transport);

const app = express();
app.use(spanly({ apiKey: process.env.SPANLY_API_KEY }));
app.use(express.json());
app.post('/mcp', (req, res) => {
  transport.handleRequest(req, res, req.body);
});
app.listen(3000);
```

## Fetch and Hono

If your MCP server runs behind a Fetch-shaped handler instead of
Node's `http` module, wrap it with `wrapFetchHandler` instead:

```ts
import { Hono } from 'hono';
import { wrapFetchHandler } from '@spanly/sdk';

const app = new Hono();
// … register /mcp routes on app …

export default {
  fetch: wrapFetchHandler(app.fetch.bind(app), {
    apiKey: process.env.SPANLY_API_KEY,
  }),
};
```

`wrapFetchHandler` wraps any Fetch-shaped handler `(req, ctx?) =>
Response`: Hono, Next.js route handlers, Deno, or a Cloudflare Workers
`fetch` export. The response body is teed with `ReadableStream.tee()`,
so the caller always gets byte-identical status, headers, and body.

### Serverless delivery

Capture packets are posted after the response is already built, so a
short-lived runtime needs to keep the process alive long enough for
that post to land:

- **Cloudflare Workers**: a function-valued `ctx.waitUntil` on the
  second handler argument is detected automatically, no extra option
  needed.
- **Vercel**: there is no `ctx.waitUntil` equivalent, so pass one
  explicitly:

  ```ts
  import { waitUntil } from '@vercel/functions';

  export const GET = wrapFetchHandler(handleRequest, {
    apiKey: process.env.SPANLY_API_KEY,
    waitUntil,
  });
  ```

- **No `waitUntil` available**: delivery falls back to a keepalive
  `fetch`, best effort only.
- **AWS Lambda** (function URLs, Lambda Web Adapter): there is no
  `waitUntil` equivalent, and a keepalive fetch can be frozen
  mid-flight once the invocation ends. Until a Spanly Lambda extension
  exists, put the [Spanly CLI](https://spanly.com/docs/typescript-sdk/installation/#when-to-use-the-cli-instead)
  in front of a Lambda-hosted MCP server instead.

## Next steps

- The [API reference](https://spanly.com/docs/typescript-sdk/api-reference/) covers every
  option `spanly()` and `wrapFetchHandler()` accept.
- [Examples](https://spanly.com/docs/typescript-sdk/examples/) show end-user attribution and a
  local test harness.
- If you'd rather not change code, the [CLI](https://spanly.com/docs/cli/run/) wraps the same
  server with the same capture behavior.
- Nothing showing up in the dashboard? See the
  [troubleshooting guide](https://spanly.com/docs/troubleshooting/).

---

## API reference

Source: https://spanly.com/docs/typescript-sdk/api-reference/

The TypeScript SDK exposes two bindings, `spanly()` for Node HTTP
frameworks and `wrapFetchHandler()` for Fetch-shaped handlers, plus the
shared option types and the wire-level packet schemas both bindings
send.

## `spanly(options)`

```ts
import express from 'express';
import { spanly } from '@spanly/sdk';

const app = express();
app.use(spanly({ apiKey: process.env.SPANLY_API_KEY }));
```

Returns Node HTTP middleware `(req, res, next?) => void`. Works with
Express, Koa, or a bare `http.createServer` handler, anything that
calls `(req, res, next?)`.

Mount this before `compression()`: once a response carries a
`Content-Encoding` header, the bytes reaching the middleware are
already compressed, so capture falls back to status and headers only,
with no request or response body.

Works whether it is mounted before or after a JSON body parser like
`express.json()`. Mounted before it, capture tees the raw request
stream without consuming it, so the parser downstream still sees every
byte. Mounted after it, capture reads the already-parsed `req.body`
instead of the drained stream.

### `SpanlyMiddlewareOptions`

Extends [`CaptureEngineOptions`](#captureengineoptions) with:

| Option  | Type       | Required | Description                                                                                                                                                          |
| ------- | ---------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `paths` | `string[]` | no       | Path prefixes to capture, matched against the URL pathname. Requests outside every prefix pass through with zero engine involvement. Defaults to `['/mcp', '/sse']`. |

## `wrapFetchHandler(handler, options)`

```ts
import { Hono } from 'hono';
import { wrapFetchHandler } from '@spanly/sdk';

const app = new Hono();

export default {
  fetch: wrapFetchHandler(app.fetch.bind(app), {
    apiKey: process.env.SPANLY_API_KEY,
  }),
};
```

Wraps a Fetch-shaped handler `(req: Request, ctx?) => Response |
Promise<Response>`: Hono, Next.js route handlers, Deno, or a
Cloudflare Workers `fetch` export. The response body is teed with
`ReadableStream.tee()`, so the caller always gets byte-identical
status, headers, and body.

### `WrapFetchHandlerOptions`

Extends [`CaptureEngineOptions`](#captureengineoptions) with:

| Option      | Type                                  | Required | Description                                                                                                                                                                                                                                                                      |
| ----------- | ------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `paths`     | `string[]`                            | no       | Path prefixes to inspect, matched against the request pathname. Requests outside these prefixes bypass the engine entirely. Defaults to `['/mcp', '/sse']`.                                                                                                                      |
| `waitUntil` | `(promise: Promise<unknown>) => void` | no       | Registers a promise to keep running after the response is returned. When omitted, a function-valued `ctx.waitUntil` on the second handler argument is used automatically (Cloudflare Workers style). When neither is available, capture falls back to a keepalive fetch instead. |

See [Serverless delivery](https://spanly.com/docs/typescript-sdk/quickstart/#serverless-delivery)
for the Vercel, Cloudflare Workers, and AWS Lambda specifics.

## `CaptureEngineOptions`

Shared by both bindings.

| Option                 | Type                                              | Required | Description                                                                                                                                                                                                                              |
| ---------------------- | ------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `apiKey`               | `string`                                          | no       | Your Spanly API key. Falls back to the `SPANLY_API_KEY` environment variable. Region (`us` / `eu`) is auto-detected from the prefix.                                                                                                     |
| `ingestUrl`            | `string \| ((region: 'us' \| 'eu') => string)`    | no       | Override the ingest endpoint. Useful for tests (point at a local capture server) or custom routing. Defaults to `https://ingest.us.spanly.com` / `https://ingest.eu.spanly.com` by region.                                               |
| `maxCollectAttempts`   | `number`                                          | no       | Retry budget for `/collect` posts answered with 503. Defaults to the `SPANLY_COLLECT_MAX_ATTEMPTS` environment variable, then 4.                                                                                                         |
| `redactHeaders`        | `string[]`                                        | no       | Additional header names to redact from captured transport context, on top of `DEFAULT_REDACTED_HEADERS`. Case-insensitive.                                                                                                               |
| `onError`              | `(error: Error) => void`                          | no       | Called when capture itself fails (malformed header, a body that will not stringify, a failed ingest post). Never affects the request or response your app serves.                                                                        |
| `onWarning`            | `(warnings: CollectWarning[]) => void`            | no       | Called with warnings the ingest endpoint returns for an accepted packet.                                                                                                                                                                 |
| `fetchInit`            | `RequestInit`                                     | no       | Merged into every `/collect` fetch call. The seam a binding uses to add transport-specific options (for example `wrapFetchHandler` sets `keepalive: true` when no `waitUntil` is available).                                             |
| `identity`             | `IdentityOptions`                                 | no       | End-user attribution: a `resolve` callback, JWT claim decoding, a hosted resolver, and/or the bearer-token fingerprint. See [Identity](#identity) below.                                                                                 |
| `sessionIdleTimeoutMs` | `number`                                          | no       | Idle gap after which a synthetic session (see [Session tracking](https://spanly.com/docs/session-tracking/)) is considered ended and a new one starts. Defaults to 30 minutes. Only relevant when a transaction carries no real `Mcp-Session-Id` request header. |
| `contextHeaders`       | `Record<string, 'projectId' \| 'organisationId'>` | no       | Multi-tenant context tagging: maps request header names (case-insensitive) onto packet context fields, the same mechanism as the CLI's `--context-header` flag.                                                                          |
| `instanceName`         | `string`                                          | no       | Stable producer name stamped on every packet, the CLI's `--instance-name`. Defaults to the `SPANLY_INSTANCE_NAME` environment variable, then (Node binding only) the machine hostname.                                                   |
| `reviews`              | `ReviewsOptions`                                  | no       | Experimental agent self-reviews, configured from the dashboard. The only local option is `{ disabled: true }` to force the loop off. See [Reviews](#reviews) below.                                                                      |

The constructor throws if no API key is available, or if the key does
not start with `spanly_us_` or `spanly_eu_`. A throwing engine never
propagates into either binding: the failure is routed to `onError`
instead.

## Reviews

Agent self-reviews are configured from the dashboard (Settings,
Reviews) and delivered to running middleware over the Pulse channel,
like Mend fixes and identity settings. When enabled there, the
middleware injects an `mcp_review` tool (rating 0 to 5, feedback,
model) into every `tools/list` response and, once a session has made
the configured minimum of tool calls, answers one `tools/call` itself
with a request to submit a review before retrying, denying up to the
configured retries until one arrives. The `mcp_review` call is also
answered by the middleware and never reaches your server; submissions
appear on the dashboard's Reviews page. Interception requires the
request body to be available when the middleware runs (mount a JSON
body parser such as `express.json()` before it).

The SDK exposes a single local option:

| Option     | Type      | Required | Description                                                                                            |
| ---------- | --------- | -------- | ------------------------------------------------------------------------------------------------------ |
| `disabled` | `boolean` | no       | Forces the review loop off on this producer regardless of the dashboard configuration. Off by default. |

## Identity

Attribute captured traffic to an end user with one (or more) of:

### `resolve` callback

```ts
app.use(
  spanly({
    apiKey: process.env.SPANLY_API_KEY,
    identity: {
      resolve: ({ headers }) => lookupUserFromSessionCookie(headers['cookie']),
    },
  }),
);
```

### JWT claims

```ts
app.use(
  spanly({
    apiKey: process.env.SPANLY_API_KEY,
    identity: { jwtClaims: true },
  }),
);
```

`jwtClaims: true` decodes the bearer token with the default claim
mapping (`sub` for id, `email`, `name`). Pass an object to select
custom dot-path claims: `{ jwtClaims: { id: 'user.id', email:
'user.email' } }`. The token is decoded only, never
signature-verified: your server has already authenticated the request.

### Hosted resolver

```ts
app.use(
  spanly({
    apiKey: process.env.SPANLY_API_KEY,
    identity: {
      resolver: {
        url: 'https://internal.example.com/spanly/resolve',
        secret: process.env.SPANLY_RESOLVER_SECRET,
      },
    },
  }),
);
```

Spanly POSTs `{ token, mcpSessionId }` to your resolver and caches
results per token (15 minutes on a hit, 60 seconds on a miss).

### `IdentityOptions`

| Option              | Type                           | Required | Description                                                                                                                                                         |
| ------------------- | ------------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `resolve`           | `ResolveCallback`              | no       | Exclusive when set: `jwtClaims` and `resolver` are ignored entirely. Receives `{ headers, mcpSessionId? }` (raw, pre-redaction headers).                            |
| `jwtClaims`         | `true \| JwtClaimMapping`      | no       | Decode the end user straight from a bearer JWT's claims, no network call. Tried before `resolver`, but only wins when it actually yields a user.                    |
| `resolver`          | `{ url, secret?, timeoutMs? }` | no       | POSTs the bearer token to a customer-hosted identity resolver. Header `X-Spanly-Identity-Secret` is sent when `secret` is set. Default timeout 3000 ms, no retries. |
| `attachFingerprint` | `boolean`                      | no       | Attach the truncated sha256 fingerprint of the bearer token to every packet of a transaction. Defaults to `true`, independent of the other options.                 |

Precedence: `resolve` (when set) is exclusive; otherwise a `jwtClaims`
decode that actually yields a user wins; otherwise the `resolver`
result. The bearer token fingerprint is attached independently of all
three, so traffic under the same credential can still be correlated
even when no user resolves. See
[Multi-tenant attribution](https://spanly.com/docs/typescript-sdk/examples/#multi-tenant-attribution)
for a worked example using `accountId`.

Identity can also be configured with no code at all, from the dashboard's
**Settings → Identity** tab — see
[Identify users](https://spanly.com/docs/identify-users/#the-easiest-path-configure-it-in-the-dashboard).
Any `identity` option set in code makes the middleware ignore the
dashboard configuration wholesale, matching the CLI's `--identity-*`
flags.

## Environment variables

| Variable                      | Description                                                            |
| ----------------------------- | ---------------------------------------------------------------------- |
| `SPANLY_API_KEY`              | Used when no `apiKey` option is passed.                                |
| `SPANLY_COLLECT_MAX_ATTEMPTS` | Max delivery attempts per packet when ingest responds 503 (default 4). |
| `SPANLY_INSTANCE_NAME`        | Used when no `instanceName` option is passed.                          |

## Types and the wire schema

The package re-exports the capture engine, the packet schemas, and
their inferred types:

```ts
import {
  CaptureEngine,
  MAX_INSPECT_BYTES,
  SYNTHETIC_SESSION_ID_PREFIX,
  DEFAULT_REDACTED_HEADERS,
  spanlyPacketSchema,
  spanlyPacketContextSchema,
  spanlyPacketTransportContextSchema,
  spanlyUserSchema,
} from '@spanly/sdk';
import type {
  CaptureEngineOptions,
  CaptureTransaction,
  BeginTransactionRequest,
  IdentityOptions,
  CollectWarning,
  SpanlyRegion,
  SpanlyPacket,
  SpanlyPacketContext,
  SpanlyPacketTransportContext,
  SpanlyPacketTransportContextHttp,
  SpanlyPacketTransportContextStdio,
  SpanlyPacketOversized,
  SpanlyUser,
  McpPacket,
} from '@spanly/sdk';
```

- `CaptureEngine`: the framework-free capture engine both bindings
  wrap. Exported so test code can drive it directly; see
  [Tests: capture locally with `ingestUrl`](https://spanly.com/docs/typescript-sdk/examples/#tests-capture-locally-with-ingesturl).
- `SpanlyPacket`: the envelope sent to ingest, including `context`,
  `transportContext`, `mcpPacket`, and the optional `user` and
  `authTokenFingerprint` fields.
- `SpanlyPacketContext`: `spanlyClientId` and `spanlyMonitorId`
  identify the process and the transaction; `projectId`,
  `projectId`, and `organisationId` are set by ingest, not by SDK
  options.
- `SpanlyPacketTransportContext`: transport metadata. For HTTP:
  method, path, headers, remote address and port, status code, and
  the dual-era fields (`mcpProtocolVersion`, `mcpMethod`, `mcpName`,
  `mcpParamHeaders`). For stdio: just the transport type (the TS SDK
  itself never runs over stdio; this variant exists for wire
  compatibility with the CLI).
- `SpanlyUser`: the shape `identity` resolves to (`id`, plus optional
  `email`, `name`, `accountId`, `accountName`).
- `McpPacket`: the captured JSON-RPC request, response, or
  notification. Only the `jsonrpc` field is typed statically;
  everything else passes through as the raw JSON-RPC payload.
- `MAX_INSPECT_BYTES`: 16 MiB. Bodies larger than this are forwarded
  untouched; only their first `MAX_INSPECT_BYTES` are buffered for
  inspection, and the packet's `oversized.originalSize` field carries
  the true wire size.
- `SYNTHETIC_SESSION_ID_PREFIX`: `'spanly-'`, the prefix on synthetic
  session ids the sessionizer mints. See [Session tracking](https://spanly.com/docs/session-tracking/).

## Trace context propagation

If your infrastructure propagates a W3C `traceparent` value, it
survives untouched in the captured packet: HTTP headers are forwarded
as captured except for the credential headers on the redact list, and
`traceparent` is not one of them, so it comes through in
`transportContext.headers` verbatim. The same holds for a
`traceparent` a client embeds in `params._meta` on the JSON-RPC
message itself, since `mcpPacket` is captured as-is. Pick your APM
provider in the dashboard (Settings, Integrations) and the request
detail view renders a cross-link to the matching trace in Datadog,
Sentry, or New Relic.

There is nothing to configure in the SDK for this: it is a consequence
of not stripping any header beyond the redact list, not a dedicated
tracing feature. If you don't already propagate `traceparent` through
your stack, the cross-link is simply omitted.

## Mend fixes on the wire

When your project has an applied fix (shim) or a running canary, the
middleware serves it exactly like the CLI: matching `tools/list`
responses are rewritten per the delivery state (JSON and SSE responses
alike), `tools/call` requests for a renamed tool are mapped back to the
name your server declares, and the exposure is recorded on the captured
packet (`mendServed`). Sessions are sticky — the state and canary arm
are pinned at a session's first `tools/list` — and arm assignment is
deterministic and identical across the SDK, the CLI, and the backend.

Delivery state syncs in the background: every collector response carries
the current state digest in the `Spanly-Pulse` header, and the document
is fetched content-addressed with the same API key. There is nothing to
configure and no traffic is ever blocked waiting for state: with no
state loaded (or on any fetch, validation, or rewrite failure) every
byte passes through untouched. The dashboard kill switch stops all
rewriting instantly. See [Patches](https://spanly.com/docs/mend/) for the product side.

## What the SDK does _not_ do

- Outside Mend (above), it never sends a response, mutates a header, or
  otherwise alters the bytes your server exchanges with its caller.
  Unlike the Python SDK and the CLI, the TypeScript middleware never
  injects a synthetic `Mcp-Session-Id` on the wire: it groups sessionless
  traffic for telemetry only (see [Session tracking](https://spanly.com/docs/session-tracking/)),
  and there is no `injectSessionId` option because there is nothing
  for it to toggle.
- It does not offer a hook to drop or rewrite an individual packet
  before it is sent. Every packet the engine parses is delivered;
  scope what is captured at all with the `paths` option, or attribute
  it to a user with `identity` instead.
- It does not block your server's request handling. Delivery to
  ingest happens asynchronously; failures surface through `onError`,
  never as exceptions in your request path.
- It does not intercept traffic outside the configured `paths`. If
  your MCP server is not reachable under `/mcp` or `/sse`, pass a
  matching `paths` list.

---

## Examples

Source: https://spanly.com/docs/typescript-sdk/examples/

## Claude Desktop / Cursor / Windsurf

These hosts launch your MCP server as a child process over stdio. The
TypeScript SDK is HTTP middleware, so it does not apply to a stdio
server; wrap it with the [Spanly CLI](https://spanly.com/docs/cli/run/) instead:

```bash
npx -y @spanly/spanly run -- node ./dist/server.js
```

If your server also exposes an HTTP transport for other clients, mount
`spanly()` on that HTTP surface as usual; the two integration methods
are independent.

## Multi-tenant attribution

When the same MCP server handles requests from many tenants, attach
the tenant id to captured traffic with `identity.resolve`. The
resolved user's `accountId` is exactly the tenant-scoped field the
dashboard's filter bar picks up automatically:

```ts
import express from 'express';
import { spanly } from '@spanly/sdk';

const app = express();
app.use(
  spanly({
    apiKey: process.env.SPANLY_API_KEY,
    identity: {
      resolve: ({ headers }) => {
        const tenant = extractTenantFromAuth(headers['authorization']);
        return tenant ? { id: tenant.userId, accountId: tenant.orgId } : null;
      },
    },
  }),
);
```

`resolve` runs concurrently with request handling and never blocks it;
a packet emitted before resolution settles simply goes out without a
`user`, and the batcher backfills the rest of that session downstream.

## What the middleware does not filter

There is no hook to drop or rewrite an individual captured packet: the
middleware is a byte-copy tee, not an interception point for your
JSON-RPC payloads. If you need to keep certain tools or paths out of
Spanly entirely, scope capture at the transport level instead:

```ts
app.use(
  spanly({
    apiKey: process.env.SPANLY_API_KEY,
    paths: ['/mcp/public'], // /mcp/internal is never inspected
  }),
);
```

This is coarser than per-packet filtering: it excludes a whole path
prefix, not a specific tool call. Credential-bearing headers are
always redacted automatically (see
[the privacy model](https://spanly.com/docs/typescript-sdk/api-reference/#captureengineoptions));
there is nothing to configure there.

## Tests: capture locally with `ingestUrl`

There is no client-side queue to flush on a fixed schedule; each
packet is posted to ingest as soon as its request or response leg
finishes. For deterministic tests, construct `CaptureEngine` directly
(bypassing the HTTP binding) and point `ingestUrl` at a local server
that records the payloads:

```ts
import { createServer } from 'node:http';
import { CaptureEngine } from '@spanly/sdk';

test('captures a tool call', async () => {
  const received: unknown[] = [];
  const capture = createServer((req, res) => {
    let body = '';
    req.on('data', (chunk) => (body += chunk));
    req.on('end', () => {
      received.push(JSON.parse(body));
      res.writeHead(200).end(JSON.stringify({ success: true }));
    });
  }).listen(0);
  const { port } = capture.address() as { port: number };

  const engine = new CaptureEngine({
    apiKey: 'spanly_us_test',
    ingestUrl: () => `http://127.0.0.1:${port}`,
  });

  const txn = engine.beginTransaction({
    method: 'POST',
    path: '/mcp',
    headers: { 'content-type': 'application/json' },
  });
  txn.onRequestBody(
    JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/call' }),
  );
  txn.onResponseStart(200, { 'content-type': 'application/json' });
  txn.onResponseBody(JSON.stringify({ jsonrpc: '2.0', id: 1, result: {} }));
  txn.onTransactionEnd();

  await engine.flush();
  expect(received.length).toBeGreaterThan(0);

  capture.close();
});
```

Use `onError` in tests to fail loudly when delivery breaks instead of
silently losing packets.

## Correlate with your APM (Datadog, Sentry, …)

If your infrastructure propagates a W3C `traceparent` value, on an
HTTP header or embedded in `params._meta.traceparent` on the JSON-RPC
message, it survives untouched in the captured packet: neither is
stripped or rewritten before the packet leaves your process. Pick your
APM provider in Settings, Integrations and the request detail view
links straight to the corresponding trace in your APM. See
[Trace context propagation](./api-reference#trace-context-propagation).

---

## Python SDK

Source: https://spanly.com/docs/python-sdk/

The Python SDK is ASGI middleware: mount `SpanlyMiddleware` in front of
any MCP server that speaks ASGI and it captures every tool call,
prompt, resource access, and JSON-RPC packet, with no changes to your
server code. Requires Python 3.10+.

```bash
pip install spanly
# or: uv add spanly
```

```python
import os

from spanly import SpanlyMiddleware

app.add_middleware(SpanlyMiddleware, api_key=os.environ["SPANLY_API_KEY"])
```

## In this section

- [Installation](https://spanly.com/docs/python-sdk/installation/) covers package install and
  the `SPANLY_API_KEY` environment variable.
- [Quickstart](https://spanly.com/docs/python-sdk/quickstart/) walks through mounting the
  middleware in a typical server end to end.
- [API reference](https://spanly.com/docs/python-sdk/api-reference/) documents
  `SpanlyMiddleware` and every option it accepts.
- [Examples](https://spanly.com/docs/python-sdk/examples/) shows end-user attribution, error
  reporting via Sentry, and a local test harness.

---

## Installation

Source: https://spanly.com/docs/python-sdk/installation/

The Python SDK ships as `spanly` on [PyPI](https://pypi.org/project/spanly/).
It is pure ASGI3 middleware: it works with FastAPI, Starlette, or any
other ASGI app, including the app `mcp`'s `streamable_http_app()`
returns.

## Install

```bash
pip install spanly
# or
uv add spanly
# or
poetry add spanly
```

## Supported Python versions

- Python 3.10+
- Pure ASGI3: works under any ASGI server (uvicorn, hypercorn,
  daphne).

## Configure the API key

`SpanlyMiddleware` reads the API key from the `api_key` argument,
falling back to the `SPANLY_API_KEY` environment variable. Always
source it from the environment, never hard-code it:

```python
import os

from spanly import SpanlyMiddleware

app.add_middleware(SpanlyMiddleware, api_key=os.environ["SPANLY_API_KEY"])
```

Never hard-code the key. Get one by signing in at
[spanly.com](https://spanly.com/), opening your project, and going to
**Settings → API keys**. The region (`us` / `eu`) is encoded in the key
prefix and auto-detected.

## When to use the SDK vs the CLI

Mount `SpanlyMiddleware` in front of your MCP server when it runs over
ASGI. It needs no extra process, no sidecar, and no separate binary.

Reach for the [Spanly CLI](https://spanly.com/docs/cli/) (`@spanly/spanly`) instead when:

- Your MCP server speaks stdio, not HTTP.
- Your server is not Node or Python (Go, Rust, Java, and so on).
- You are running a Mend shim or canary (Mend rewriting lives in the
  CLI only, not in this SDK).
- You are on AWS Lambda, where this SDK's delivery guarantees do not
  hold.

```bash
npx -y @spanly/spanly run -- python -m my_mcp
```

---

## Quickstart

Source: https://spanly.com/docs/python-sdk/quickstart/

This walks through mounting `SpanlyMiddleware` in front of a Python MCP
server that speaks ASGI (FastAPI, Starlette, or the ASGI app the `mcp`
package's `streamable_http_app()` returns). `SpanlyMiddleware` inspects
requests under `/mcp` and `/sse` by default, tees the request and
response bytes, and never mutates a body or delays a message.

## 1. Install

```bash
pip install spanly
```

## 2. Set the API key

```bash
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

Get a key by signing in at [spanly.com](https://spanly.com/), opening
your project, and going to **Settings → API keys**. The region is
encoded in the key prefix (`spanly_us_` / `spanly_eu_`) and
auto-detected, so there is nothing else to configure.

## 3. Mount the middleware

```python
import os

from spanly import SpanlyMiddleware

app.add_middleware(SpanlyMiddleware, api_key=os.environ["SPANLY_API_KEY"])
```

`add_middleware` defers construction until the app builds its
middleware stack, so `SpanlyMiddleware` receives your ASGI app as its
first positional argument automatically. If your framework does not
expose `add_middleware` (a bare ASGI app, for instance), wrap it
directly instead:

```python
import os

from spanly import SpanlyMiddleware

app = SpanlyMiddleware(app, api_key=os.environ["SPANLY_API_KEY"])
```

Either way, traffic on `/mcp` and `/sse` is now reported to your
Spanly project.

## Full example: FastMCP over Streamable HTTP

```python
import os
from mcp.server.fastmcp import FastMCP
from spanly import SpanlyMiddleware

mcp = FastMCP("demo-http-server")


@mcp.tool()
async def echo(input: str) -> str:
    return input


app = mcp.streamable_http_app()
app = SpanlyMiddleware(app, api_key=os.environ["SPANLY_API_KEY"])

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="0.0.0.0", port=3000)
```

## Full example: FastAPI

```python
import os
from fastapi import FastAPI
from spanly import SpanlyMiddleware

app = FastAPI()
# … mount your MCP server's ASGI app at /mcp …

app.add_middleware(SpanlyMiddleware, api_key=os.environ["SPANLY_API_KEY"])
```

## Sessionless servers

When your server runs stateless (no `Mcp-Session-Id` on its
`initialize` response), `SpanlyMiddleware` injects a synthetic one
(prefixed `spanly-`) on the response so requests still group into
sessions. This is a real, on-the-wire header, unlike the TypeScript
SDK's telemetry-only grouping. See
[Session tracking](https://spanly.com/docs/session-tracking/) for the full behavior, and
opt out with `inject_session_id=False` if you'd rather Spanly never
touch a response header.

## Next steps

- The [API reference](https://spanly.com/docs/python-sdk/api-reference/) covers every option
  `SpanlyMiddleware` accepts.
- [Examples](https://spanly.com/docs/python-sdk/examples/) cover end-user attribution, error
  reporting, and a local test harness.
- If you'd rather not change code, the [CLI](https://spanly.com/docs/cli/run/) wraps the same
  server with the same capture behavior.
- Nothing showing up in the dashboard? See the
  [troubleshooting guide](https://spanly.com/docs/troubleshooting/).

---

## API reference

Source: https://spanly.com/docs/python-sdk/api-reference/

The Python SDK exposes a single ASGI3 middleware class,
`SpanlyMiddleware`, plus the supporting dataclasses for identity and
the wire-level packet types.

## `SpanlyMiddleware`

```python
import os

from spanly import SpanlyMiddleware

app.add_middleware(SpanlyMiddleware, api_key=os.environ["SPANLY_API_KEY"])
```

`add_middleware` defers construction until the app builds its
middleware stack, so `SpanlyMiddleware` receives your ASGI app as its
first positional argument automatically. Wrap a bare ASGI app directly
instead when your framework has no `add_middleware`:

```python
app = SpanlyMiddleware(app, api_key=os.environ["SPANLY_API_KEY"])
```

### `SpanlyMiddleware(app, **options)`

| Option                         | Type                                             | Default            | Description                                                                                                                                                                                                                                              |
| ------------------------------ | ------------------------------------------------ | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `api_key`                      | `str \| None`                                    | `None`             | Your Spanly API key. Falls back to `SPANLY_API_KEY`. Region (`us` / `eu`) is auto-detected from the prefix.                                                                                                                                              |
| `ingest_url`                   | `str \| Callable[[SpanlyRegion], str] \| None`   | `None`             | Override the ingest endpoint. Useful for tests (point at a local capture server) or custom routing.                                                                                                                                                      |
| `paths`                        | `Sequence[str]`                                  | `("/mcp", "/sse")` | Path prefixes to capture, matched against the request path. Requests outside every prefix pass through with zero engine involvement.                                                                                                                     |
| `redact_headers`               | `Sequence[str] \| None`                          | `None`             | Additional header names to redact from captured transport context, on top of `DEFAULT_REDACTED_HEADERS`. Case-insensitive.                                                                                                                               |
| `on_error`                     | `Callable[[Exception], None] \| None`            | `None`             | Called when capture itself fails. When omitted, failures are logged through the `spanly` logger instead. Never affects the request or response your app serves.                                                                                          |
| `on_warning`                   | `Callable[[list[CollectWarning]], None] \| None` | `None`             | Called with warnings the ingest endpoint returns for an accepted packet.                                                                                                                                                                                 |
| `inject_session_id`            | `bool`                                           | `True`             | Injects a synthetic `Mcp-Session-Id` response header (prefixed `spanly-`) on sessionless initialize responses, and owns the DELETE handshake that terminates a synthetic session. A real, on-the-wire header. See [Session tracking](https://spanly.com/docs/session-tracking/). |
| `max_collect_attempts`         | `int \| None`                                    | `None`             | Retry budget for `/collect` posts answered with 503. Defaults to the `SPANLY_COLLECT_MAX_ATTEMPTS` environment variable, then 4.                                                                                                                         |
| `identity`                     | `IdentityOptions \| None`                        | `None`             | End-user attribution: a `resolve` callable, JWT claim decoding, a hosted resolver, and/or the bearer-token fingerprint. See [Identity](#identity) below.                                                                                                 |
| `session_idle_timeout_seconds` | `float`                                          | `1800.0`           | Idle gap after which a synthetic session (see [Session tracking](https://spanly.com/docs/session-tracking/)) is considered ended and a new one starts. Only relevant when a transaction carries no real `Mcp-Session-Id` request header.                                         |
| `context_headers`              | `Mapping[str, str] \| None`                      | `None`             | Multi-tenant context tagging: maps request header names (case-insensitive) onto the packet context fields `project_id` or `organisation_id`, the same mechanism as the CLI's `--context-header` flag.                                                    |

## Identity

Attribute captured traffic to an end user with one (or more) of:

### `resolve` callback

```python
from spanly import IdentityOptions, ResolveContext, SpanlyMiddleware


def resolve(ctx: ResolveContext):
    return lookup_user_from_session_cookie(ctx.headers.get("cookie"))


app.add_middleware(
    SpanlyMiddleware,
    api_key=os.environ["SPANLY_API_KEY"],
    identity=IdentityOptions(resolve=resolve),
)
```

### JWT claims

```python
from spanly import IdentityOptions, SpanlyMiddleware

app.add_middleware(
    SpanlyMiddleware,
    api_key=os.environ["SPANLY_API_KEY"],
    identity=IdentityOptions(jwt_claims=True),
)
```

`jwt_claims=True` decodes the bearer token with the default claim
mapping (`sub` for id, `email`, `name`). Pass a dict to select custom
dot-path claims: `IdentityOptions(jwt_claims={"id": "user.id", "email":
"user.email"})`. The token is decoded only, never signature-verified:
your server has already authenticated the request.

### Hosted resolver

```python
from spanly import IdentityOptions, ResolverOptions, SpanlyMiddleware

app.add_middleware(
    SpanlyMiddleware,
    api_key=os.environ["SPANLY_API_KEY"],
    identity=IdentityOptions(
        resolver=ResolverOptions(
            url="https://internal.example.com/spanly/resolve",
            secret=os.environ["SPANLY_RESOLVER_SECRET"],
        ),
    ),
)
```

Spanly POSTs `{token, mcpSessionId}` to your resolver and caches
results per token (15 minutes on a hit, 60 seconds on a miss).

### `IdentityOptions`

| Field         | Type                              | Default | Description                                                                                                                                              |
| ------------- | --------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `resolve`     | `ResolveCallback \| None`         | `None`  | Exclusive when set: `jwt_claims` and `resolver` are ignored entirely. Receives a `ResolveContext(headers, mcp_session_id)` (raw, pre-redaction headers). |
| `jwt_claims`  | `bool \| JwtClaimMapping \| None` | `None`  | Decode the end user straight from a bearer JWT's claims, no network call. Tried before `resolver`, but only wins when it actually yields a user.         |
| `resolver`    | `ResolverOptions \| None`         | `None`  | POSTs the bearer token to a customer-hosted identity resolver.                                                                                           |
| `fingerprint` | `bool`                            | `True`  | Attach the truncated sha256 fingerprint of the bearer token to every packet of a transaction, independent of the other options.                          |

### `ResolverOptions`

| Field             | Type          | Default  | Description                                    |
| ----------------- | ------------- | -------- | ---------------------------------------------- |
| `url`             | `str`         | required | The resolver endpoint Spanly POSTs to.         |
| `secret`          | `str \| None` | `None`   | Sent as the `X-Spanly-Identity-Secret` header. |
| `timeout_seconds` | `float`       | `3.0`    | Request timeout. No retries.                   |

Precedence: `resolve` (when set) is exclusive; otherwise a `jwt_claims`
decode that actually yields a user wins; otherwise the `resolver`
result. The bearer token fingerprint is attached independently of all
three. See
[Multi-tenant attribution](https://spanly.com/docs/python-sdk/examples/#multi-tenant-attribution)
for a worked example using `account_id`.

## Environment variables

| Variable                      | Description                                                            |
| ----------------------------- | ---------------------------------------------------------------------- |
| `SPANLY_API_KEY`              | Used when no `api_key` argument is passed.                             |
| `SPANLY_COLLECT_MAX_ATTEMPTS` | Max delivery attempts per packet when ingest responds 503 (default 4). |

## Types

The package re-exports the types you'd want for typed hooks:

```python
from spanly import (
    SpanlyMiddleware,
    SpanlyPacket,
    SpanlyPacketContext,
    HttpTransportContext,
    StdioTransportContext,
    TransportContext,
    DEFAULT_REDACTED_HEADERS,
    SESSION_TERMINATED_METHOD,
    SYNTHETIC_SESSION_ID_PREFIX,
    IdentityOptions,
    ResolverOptions,
    ResolveContext,
    SpanlyUser,
)
```

- `SpanlyPacket`: the envelope sent to ingest.
- `SpanlyPacketContext`: `spanly_client_id` and `spanly_monitor_id`
  identify the process and the transaction; `project_id`,
  `project_id`, and `organisation_id` are set by ingest, not by
  middleware options.
- `HttpTransportContext` / `StdioTransportContext`: transport
  metadata. HTTP carries method, path, headers, remote address and
  port, status code, and the dual-era fields. `StdioTransportContext`
  exists for wire compatibility with the CLI; the Python SDK itself
  only runs over ASGI.
- `SpanlyUser`: the shape `identity` resolves to (`id`, plus optional
  `email`, `name`, `accountId`, `accountName`).
- `SYNTHETIC_SESSION_ID_PREFIX`: `'spanly-'`, the prefix on synthetic
  session ids. See [Session tracking](https://spanly.com/docs/session-tracking/).

## Trace context propagation

If your infrastructure propagates a W3C `traceparent` value, it
survives untouched in the captured packet: HTTP headers are forwarded
as captured except for the credential headers on the redact list, and
`traceparent` is not one of them, so it comes through in the
transport context's headers verbatim. The same holds for a
`traceparent` a client embeds in `params._meta` on the JSON-RPC
message itself, since the packet is captured as-is. Pick your APM
provider in the dashboard (Settings, Integrations) and the request
detail view renders a cross-link to the matching trace in Datadog,
Sentry, or New Relic.

There is nothing to configure in the SDK for this: it is a consequence
of not stripping any header beyond the redact list, not a dedicated
tracing feature. If you don't already propagate `traceparent` through
your stack, the cross-link is simply omitted.

## What the SDK does _not_ do

- It does not offer a hook to drop or rewrite an individual packet
  before it is sent. Every packet the middleware parses is delivered;
  scope what is captured at all with `paths`, or attribute it to a
  user with `identity` instead.
- It does not block your server on its network call to ingest. Each
  packet is delivered on a background task; delivery failures surface
  through `on_error`, never as exceptions in your request path.
- Aside from the `inject_session_id` response header on sessionless
  initialize responses (and answering a synthetic session's DELETE
  itself), it does not modify the bodies or headers your server
  returns.

---

## Examples

Source: https://spanly.com/docs/python-sdk/examples/

## Claude Desktop / Cursor / Windsurf

These hosts spawn your MCP server as a child process over stdio. The
Python SDK is ASGI middleware, so it does not apply to a stdio server;
wrap it with the [Spanly CLI](https://spanly.com/docs/cli/run/) instead:

```bash
npx -y @spanly/spanly run -- python -m my_mcp
```

If your server also exposes an HTTP transport for other clients, mount
`SpanlyMiddleware` on that ASGI app as usual; the two integration
methods are independent.

## Multi-tenant attribution

When the same MCP server handles requests from many tenants, attach
the tenant id to captured traffic with `identity.resolve`. The
resolved user's `account_id` is exactly the tenant-scoped field the
dashboard's filter bar picks up automatically:

```python
import os

from spanly import IdentityOptions, ResolveContext, SpanlyMiddleware


def resolve(ctx: ResolveContext):
    tenant = extract_tenant_from_auth(ctx.headers.get("authorization"))
    if tenant is None:
        return None
    return {"id": tenant.user_id, "accountId": tenant.org_id}


app.add_middleware(
    SpanlyMiddleware,
    api_key=os.environ["SPANLY_API_KEY"],
    identity=IdentityOptions(resolve=resolve),
)
```

`resolve` runs as a background task started at the transaction, and
never blocks serving; a packet emitted before resolution settles
simply goes out without a `user`, and the batcher backfills the rest
of that session downstream.

## What the middleware does not filter

There is no hook to drop or rewrite an individual captured packet: the
middleware is a byte-copy tee, not an interception point for your
JSON-RPC payloads. If you need to keep certain tools or paths out of
Spanly entirely, scope capture at the transport level instead:

```python
app.add_middleware(
    SpanlyMiddleware,
    api_key=os.environ["SPANLY_API_KEY"],
    paths=["/mcp/public"],  # /mcp/internal is never inspected
)
```

This is coarser than per-packet filtering: it excludes a whole path
prefix, not a specific tool call. Credential-bearing headers are
always redacted automatically; there is nothing to configure there.

## Error reporting via Sentry

```python
import os

import sentry_sdk
from spanly import SpanlyMiddleware

sentry_sdk.init(dsn=os.environ["SENTRY_DSN"])


def on_error(exc: Exception) -> None:
    sentry_sdk.capture_exception(exc)


app.add_middleware(
    SpanlyMiddleware,
    api_key=os.environ["SPANLY_API_KEY"],
    on_error=on_error,
)
```

The middleware never raises into your app, so capturing through
`on_error` is the only way to surface its internal failures (it falls
back to logging through the `spanly` logger when `on_error` is
omitted).

## Tests: capture locally with `ingest_url`

There is no client-side queue to flush on a fixed schedule; each
packet is delivered on a background task as soon as its request or
response leg finishes. For a local test, point `ingest_url` at a
server that records the payloads and drive the app through an ASGI
test client:

```python
import asyncio
import json
import os
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer

import httpx
import pytest
from spanly import SpanlyMiddleware


class _CaptureHandler(BaseHTTPRequestHandler):
    received: list[dict] = []

    def do_POST(self):
        length = int(self.headers["Content-Length"])
        self._CaptureHandler.received.append(json.loads(self.rfile.read(length)))
        body = json.dumps({"success": True}).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass  # keep test output quiet


@pytest.mark.asyncio
async def test_captures_a_tool_call(my_asgi_app):
    server = HTTPServer(("127.0.0.1", 0), _CaptureHandler)
    threading.Thread(target=server.serve_forever, daemon=True).start()
    port = server.server_address[1]

    app = SpanlyMiddleware(
        my_asgi_app,
        api_key="spanly_us_test",
        ingest_url=lambda region: f"http://127.0.0.1:{port}",
    )

    async with httpx.AsyncClient(
        transport=httpx.ASGITransport(app=app), base_url="http://test"
    ) as client:
        await client.post(
            "/mcp", json={"jsonrpc": "2.0", "id": 1, "method": "tools/call"}
        )

    await asyncio.sleep(0.1)  # let the background delivery task run
    assert _CaptureHandler.received

    server.shutdown()
```

Use `on_error` in tests to fail loudly when delivery breaks instead of
silently losing packets.

## Correlate with your APM (Datadog, Sentry, …)

If your infrastructure propagates a W3C `traceparent` value, on an
HTTP header or embedded in `params._meta.traceparent` on the JSON-RPC
message, it survives untouched in the captured packet: neither is
stripped or rewritten before the packet leaves your process. Pick your
APM provider in Settings, Integrations and the request detail view
links straight to the corresponding trace in your APM. See
[Trace context propagation](./api-reference#trace-context-propagation).

---

## CLI

Source: https://spanly.com/docs/cli/

The Spanly CLI instruments MCP servers written in any language. Run your
server under `spanly run`, or sit `spanly proxy` in front of a remote
server, and every JSON-RPC packet streams to your dashboard. Nothing in
your server code changes.

```bash
npx -y @spanly/spanly run -- <your-mcp-command>
```

```bash
# 1. Set your API key (region auto-detected from the prefix)
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxx

# 2. Wrap your MCP server. stdio:
npx -y @spanly/spanly run -- node ./server.js

# Or HTTP. The wrapper takes your port; the child gets a random one:
npx -y @spanly/spanly run --port 3000 -- node ./server.js
```

## In this section

- [Installation](https://spanly.com/docs/cli/installation/) covers npm, Homebrew, and the
  `curl | sh` installer.
- [`spanly run`](https://spanly.com/docs/cli/run/) wraps a server you launch yourself.
- [`spanly proxy`](https://spanly.com/docs/cli/proxy/) sits in front of a server you connect to.
- [Flags](https://spanly.com/docs/cli/flags/) lists every option for both commands.
- [Production](https://spanly.com/docs/cli/production/) covers running the CLI in a deployed
  project.

---

## Installation

Source: https://spanly.com/docs/cli/installation/

The Spanly CLI is a small Go binary that captures MCP traffic from any
server in any language, with no code changes.

## One-command setup (macOS, Linux)

```bash
curl -fsSL https://spanly.com/setup.sh | sh
```

Run it from your project root. It installs the CLI, asks for your API key
and writes it to `.env`, and connects the hosted
[Spanly MCP server](https://spanly.com/docs/mcp/overview/) to Claude Code and Codex when they are
installed. Pass the key inline to skip the prompt (the dashboard setup
page pre-fills this for you):

```bash
curl -fsSL https://spanly.com/setup.sh | sh -s -- --key spanly_us_xxxxxxxxxxxx
```

`--no-install` and `--no-mcp` skip the respective steps. If you only want
the binary, use the plain installer below.

## Quick install (macOS, Linux)

```bash
curl -fsSL https://spanly.com/install.sh | sh
```

The script detects your OS and architecture, downloads the matching
binary from GitHub Releases, verifies its checksum, and installs it onto
your `PATH`. Pin a version with `SPANLY_VERSION` or pick a directory with
`SPANLY_INSTALL_DIR`:

```bash
SPANLY_VERSION=0.1.0 SPANLY_INSTALL_DIR="$HOME/.local/bin" \
  sh -c "$(curl -fsSL https://spanly.com/install.sh)"
```

## npm (recommended for MCP client configs)

```bash
npx -y @spanly/spanly run -- node ./server.js
```

`npx` fetches the right binary for your OS on first run and caches it.
This is the recommended path for embedding the CLI directly in Claude
Desktop, Cursor, Windsurf, and other MCP client configs. No install
step required.

## Homebrew (macOS, Linux)

```bash
brew install spanlyhq/tap/spanly
spanly run -- node ./server.js
```

## Direct download

Grab the latest binary for your platform from the
[GitHub Releases page](https://github.com/spanlyhq/spanly/releases?q=cli-v).
Drop it on your `PATH`:

```bash
sudo install spanly /usr/local/bin/
spanly version
```

## From source (Go)

```bash
go install github.com/spanlyhq/spanly/cli@latest
```

Installs into `$(go env GOPATH)/bin` as `cli` (the module path's last
element), so rename it: `mv $(go env GOPATH)/bin/cli $(go env GOPATH)/bin/spanly`.
Builds this way report `dev` for `spanly version`, since the version is
only stamped on released binaries.

## Docker

```bash
docker pull spanly/spanly:latest
```

See the [Docker section](https://spanly.com/docs/docker/installation/) for sidecar and compose patterns.

## Configure the API key

The CLI reads `SPANLY_API_KEY` from the environment. The region is
encoded in the prefix (`spanly_us_…` / `spanly_eu_…`) and auto-detected:

```bash
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

`SPANLY_INGEST_URL` overrides the ingest endpoint; you only need it for
local development against a non-production Spanly stack.

## Verify

```bash
spanly version
# 0.1.0
```

## Platforms

| Platform | Architectures | Status                            |
| -------- | ------------- | --------------------------------- |
| macOS    | x86_64, arm64 | Supported                         |
| Linux    | x86_64, arm64 | Supported                         |
| Windows  | x86_64        | Best-effort, untested; prefer WSL |

## Two modes

The CLI has two subcommands:

- [`spanly run`](https://spanly.com/docs/cli/run/): wraps your MCP server as a child process.
  Works for stdio and HTTP transports. This is the default path for
  most users.
- [`spanly proxy`](https://spanly.com/docs/cli/proxy/): a standalone HTTP/SSE reverse proxy.
  Use when you can't wrap the child (third-party services, declarative
  k8s sidecars, network-level interception).

Both subcommands share the same flag surface for buffering, retry, and
admin endpoints. See the [flag reference](https://spanly.com/docs/cli/flags/).

---

## spanly run

Source: https://spanly.com/docs/cli/run/

`spanly run` is the default path for most users. It launches your MCP
server as a child process and captures every JSON-RPC frame on its
transport. Works identically for stdio servers, HTTP servers, and SSE
servers.

## 30-second demo

```bash
# 1. Set your API key (region auto-detected from the prefix)
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxx

# 2. Wrap your MCP server. stdio:
npx -y @spanly/spanly run -- node ./server.js

# Or HTTP. The wrapper takes your port; the child gets a random one:
npx -y @spanly/spanly run --port 3000 -- node ./server.js
```

That's it. Run your MCP client (Claude Desktop, Cursor, …) as usual.
Telemetry shows up in the Spanly dashboard.

## stdio mode (default)

When `--port` is not set, the CLI runs in stdio mode. It spawns your
server, hooks stdin/stdout, and forwards JSON-RPC frames in both
directions, capturing them in the process.

```bash
spanly run -- node ./server.js
spanly run -- python -m my_mcp
spanly run -- ./my-go-binary
```

Your MCP client sees exactly the same stdio behavior as if it had
launched the server directly. There is no protocol change.

## HTTP mode

Set `--port` to switch to HTTP mode:

```bash
spanly run --port 3000 -- node ./server.js
```

What happens:

- The CLI binds to port `3000` (the port your MCP client connects to).
- The child server gets a random free port via the `PORT` environment
  variable (rename with `--child-port-env`).
- Every HTTP request to `/mcp` / `/sse` (configurable via
  `--inspect-prefix`) is captured. Everything else is forwarded
  untouched.

**Your MCP client URL doesn't change.** It still points at port
`3000`. The fact that there's a wrapper in between is transparent.

### Pinning the child port

If your server can't pick its own port:

```bash
spanly run --port 3000 --child-port 3001 -- ./server
```

## Multi-tenant tagging via headers

Map inbound HTTP headers to context fields with `--context-header`:

```bash
spanly run --port 3000 \
  --context-header=X-Tenant=projectId \
  --context-header=X-Org=organisationId \
  -- ./server
```

In the dashboard the captured requests can then be sliced by
`projectId` or `organisationId`. No code change in the server
required.

## Examples

```bash
# Node MCP, stdio
spanly run -- node server.js

# Python MCP, HTTP on port 3000
spanly run --port 3000 -- python -m my_mcp

# Go MCP, HTTP with admin metrics
spanly run --port 3000 --admin-addr=:9090 -- ./my-mcp-server

# Multi-tenant tagging from request header
spanly run --port 3000 \
  --context-header=X-Tenant=projectId \
  -- ./srv
```

## Embedding in MCP client configs

### Claude Desktop / Cursor / Windsurf

```json
{
  "mcpServers": {
    "my-server": {
      "command": "npx",
      "args": ["-y", "@spanly/spanly", "run", "--", "node", "./server.js"],
      "env": {
        "SPANLY_API_KEY": "spanly_us_xxxxxxxxxxxxxxxxxxxxxx"
      }
    }
  }
}
```

Client configs store the key as a literal value, so treat the config
file as a secret.

The user-facing behavior is identical to running `node ./server.js`
directly, with the same prompts and tools, but every interaction is now
visible in Spanly.

## Flag reference

See the [full flag reference](https://spanly.com/docs/cli/flags/) for the complete list of
flags shared by `run` and `proxy` (buffering, retry, admin endpoints).

## Running in production

Deploying behind nginx, on Kubernetes, or as a sidecar? See the
[production guide](https://spanly.com/docs/cli/production/) for Helm, Kustomize, SSE
pass-through, and health checks.

---

## spanly proxy

Source: https://spanly.com/docs/cli/proxy/

`spanly proxy` is for the cases where you can't wrap the MCP server as
a child process. Typically:

- A third-party MCP service you don't control.
- A server already deployed behind k8s, where introducing a wrapping
  CLI is impractical.
- Network-level interception where the proxy sits between many clients
  and one server.

The proxy is HTTP-only (no stdio). Inbound traffic on `<bind>` is
forwarded to `<upstream>`; every JSON-RPC frame on inspected paths is
captured.

## Quickstart

```bash
# 1. Set your API key
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxx

# 2. Run the standalone proxy in front of an existing MCP server.
#    upstream  -> the MCP server you can't / don't want to wrap
#    bind      -> the address your MCP client should connect to instead
npx -y @spanly/spanly proxy localhost:3000 localhost:3001
```

Then point your MCP client at the bind address (`localhost:3001`)
instead of the upstream.

## Anatomy

```
┌─────────────┐      ┌──────────────────┐      ┌────────────────┐
│  MCP client │ ───▶ │  spanly proxy    │ ───▶ │  upstream MCP  │
└─────────────┘      │  <bind>          │      │  <upstream>    │
                     └──────────────────┘      └────────────────┘
                            │
                            ▼
                       Spanly ingest
```

The proxy is transparent at the HTTP layer:

- Status codes, headers, and bodies are passed through unchanged.
- SSE (`text/event-stream`) streams flow through with response
  buffering disabled, so they stay live.
- Non-MCP paths (anything not matching `--inspect-prefix`, default
  `/mcp,/sse`) are forwarded without parsing.

## Examples

```bash
# Loopback: monitor a server running on localhost:3000
spanly proxy localhost:3000 :3001

# In a Kubernetes Pod: front a sidecar MCP server
spanly proxy mcp:3000 0.0.0.0:3001

# Behind your own ingress / load balancer
spanly proxy upstream.svc.cluster.local:8080 :3001

# Multi-tenant tagging
spanly proxy --context-header=X-Tenant=projectId \
  localhost:3000 :3001
```

## SSE pass-through

MCP often uses `text/event-stream` for streaming notifications. The
proxy:

- Holds the upstream connection open for the duration of the SSE
  stream.
- Parses each `data:` frame as a JSON-RPC packet and emits one
  telemetry event per frame.
- Flushes immediately to the downstream client. There is no buffering
  between you and the upstream.

If you later put another reverse proxy (nginx, Caddy, Envoy) in front
of `spanly proxy`, configure it for SSE pass-through. See
[Production](https://spanly.com/docs/cli/production/) for sample configs.

## Per-request headers

Inbound headers recognized by the proxy:

| Header                                  | Effect                                                                                                   |
| --------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `X-Spanly-Monitor-Id`                   | Override `spanlyMonitorId` for this request. Useful for joining many requests under one logical session. |
| Any header named via `--context-header` | Maps to the matching context field (`projectId`, `organisationId`).                                      |

## Comparison: `run` vs `proxy`

|                                        | `spanly run` | `spanly proxy`      |
| -------------------------------------- | ------------ | ------------------- |
| stdio support                          | yes          | no                  |
| HTTP support                           | yes          | yes                 |
| Wraps child process                    | yes          | no                  |
| Affects MCP client URL                 | no           | yes (point at bind) |
| Easiest to embed in MCP client config  | yes          | no                  |
| Best for third-party / unowned servers | no           | yes                 |

If you can wrap, prefer `run`. Use `proxy` when you can't.

## Flag reference

See the [full flag reference](https://spanly.com/docs/cli/flags/) for the complete list of
flags. `proxy` accepts the same flags as `run` minus `--port` and the
`--child-*` family.

---

## Flag reference

Source: https://spanly.com/docs/cli/flags/

This page is the authoritative flag reference. `spanly run` and
`spanly proxy` share most flags; differences are called out below.

## Both subcommands

### Buffer & retry

| Flag                     | Default | Description                                                                    |
| ------------------------ | ------- | ------------------------------------------------------------------------------ |
| `--buffer-size`          | `10000` | Max packets buffered when ingest is unreachable. Oldest are dropped when full. |
| `--collect-max-attempts` | `4`     | Max POST attempts per packet.                                                  |
| `--retry-backoff`        | `1s`    | Initial retry backoff (exponential).                                           |
| `--retry-max-backoff`    | `30s`   | Cap on retry backoff.                                                          |
| `--shutdown-grace`       | `10s`   | Time to flush in-flight telemetry on shutdown (SIGINT/SIGTERM).                |

### Multi-tenant tagging

| Flag               | Default | Description                                                                |
| ------------------ | ------- | -------------------------------------------------------------------------- |
| `--context-header` | _none_  | `HEADER=field` mapping. Repeatable. Fields: `projectId`, `organisationId`. |

Example:

```bash
spanly run --port 3000 \
  --context-header=X-Tenant=projectId \
  --context-header=X-Org=organisationId \
  -- ./srv
```

### Header redaction (HTTP / proxy only)

| Flag              | Default | Description                                                      |
| ----------------- | ------- | ---------------------------------------------------------------- |
| `--redact-header` | _none_  | Additional header to redact from captured telemetry. Repeatable. |

`Authorization`, `Cookie`, `Set-Cookie`, `Proxy-Authorization`,
`X-Api-Key`, `X-Auth-Token`, `X-Amz-Security-Token` and
`X-Forwarded-Authorization` are always redacted: their values are replaced with
`[REDACTED]` in the captured packet. The proxied request and response
keep their original headers.

Example:

```bash
spanly run --port 3000 --redact-header=X-Custom-Token -- ./srv
```

### Inspection scope (HTTP / proxy only)

| Flag               | Default     | Description                                                          |
| ------------------ | ----------- | -------------------------------------------------------------------- |
| `--inspect-prefix` | `/mcp,/sse` | Comma-separated path prefixes to inspect. Empty = inspect all paths. |

### End-user identity (HTTP / proxy only)

| Flag                         | Default | Description                                                                                                                                           |
| ---------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--identity-jwt`             | off     | Decode the bearer JWT with the default claim mapping: id from `sub`, email from `email`, name from `name`. No signature verification.                 |
| `--identity-jwt-claim`       | _none_  | `field=claim` mapping override, repeatable. Fields: `id`, `email`, `name`, `accountId`, `accountName`. Dot-paths supported. Implies `--identity-jwt`. |
| `--identity-resolver-url`    | _none_  | URL of your token-to-user resolver endpoint.                                                                                                          |
| `--identity-resolver-secret` | _none_  | Shared secret sent as `X-Spanly-Identity-Secret`. Also via `SPANLY_IDENTITY_RESOLVER_SECRET` (the flag wins).                                         |
| `--no-auth-fingerprint`      | off     | Disable the anonymous bearer-token fingerprint attached to telemetry in HTTP mode.                                                                    |

Attributes captured sessions to your end users. See
[Identify your users](https://spanly.com/docs/identify-users/) for the setup guide and the
[resolver protocol](https://spanly.com/docs/identity-resolver/) for the endpoint contract. In
stdio mode these flags are ignored with a warning (no auth headers to
read).

Example:

```bash
spanly run --port 3000 \
  --identity-jwt \
  --identity-jwt-claim=accountId=org_id \
  -- ./srv
```

### Session ID injection (HTTP / proxy only)

| Flag                  | Default | Description                                                                                     |
| --------------------- | ------- | ----------------------------------------------------------------------------------------------- |
| `--inject-session-id` | `true`  | Assign a synthetic `Mcp-Session-Id` on initialize responses when the upstream does not set one. |

When the upstream server runs sessionless, its initialize responses
carry no `Mcp-Session-Id`, so Spanly cannot group requests into
sessions. With this flag enabled (the default), the proxy assigns a
synthetic session ID (prefixed `spanly-`) on initialize responses that
don't already have one. The client echoes it on subsequent requests and
the proxy strips it before forwarding upstream, so the upstream never
sees a header it didn't create. Servers that assign their own session
IDs are untouched. See [Session tracking](https://spanly.com/docs/session-tracking/).

Disable with:

```bash
spanly run --port 3000 --inject-session-id=false -- ./srv
```

### Admin endpoints

| Flag           | Default    | Description                                                       |
| -------------- | ---------- | ----------------------------------------------------------------- |
| `--admin-addr` | _disabled_ | If set (e.g. `:9090`), exposes `/healthz`, `/readyz`, `/metrics`. |

Endpoints when enabled:

- `GET /healthz`: 200 if the listener is up.
- `GET /readyz`: 200 if the upstream is reachable (1s cache).
- `GET /metrics`: Prometheus text format. Counters: packets
  collected / sent / dropped / failed, retry attempts, buffer depth,
  request counts by inspection class.

## `spanly run` only

| Flag                      | Default | Description                                                                                       |
| ------------------------- | ------- | ------------------------------------------------------------------------------------------------- |
| `--port`                  | `0`     | If set, run in HTTP mode (wrapper takes this port; child gets a random one). Default `0` = stdio. |
| `--child-port`            | `0`     | Port the child binds in HTTP mode. `0` = pick random unused port.                                 |
| `--child-port-env`        | `PORT`  | Env var passed to child with the chosen port.                                                     |
| `--child-startup-timeout` | `30s`   | Max wait for child to listen.                                                                     |

`proxy` does not accept these. The bind address is positional.

## Environment variables

| Variable            | Required | Description                                                                  |
| ------------------- | -------- | ---------------------------------------------------------------------------- |
| `SPANLY_API_KEY`    | yes      | Region detected from prefix (`spanly_us_…` / `spanly_eu_…`).                 |
| `SPANLY_INGEST_URL` | no       | Override ingest base URL (local development against a non-production stack). |

## Per-request headers

The CLI recognizes the following inbound headers on every captured
request:

| Header                                 | Effect                                                                                                   |
| -------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `X-Spanly-Monitor-Id`                  | Override `spanlyMonitorId` for this request. Useful for joining many requests under one logical session. |
| Any header named in `--context-header` | Maps to the matching context field on the captured packet.                                               |

## `spanly version`

```bash
spanly version
```

Prints the CLI version.

---

## Production deploy

Source: https://spanly.com/docs/cli/production/

This page collects the patterns we recommend for running Spanly in
production environments.

## Deployment shapes

| Shape                           | When to use                                                                              |
| ------------------------------- | ---------------------------------------------------------------------------------------- |
| `spanly run` as a child wrapper | The MCP server is yours and starts as part of your container/process. Most common.       |
| `spanly proxy` as a sidecar     | The MCP server is its own process or container, and you sit Spanly next to it.           |
| `spanly proxy` as a standalone  | Front a third-party MCP service from your own ingress.                                   |
| Docker container                | Compose-style deployments or simple Kubernetes Pods. See [Docker](https://spanly.com/docs/docker/installation/). |
| Helm chart                      | Standalone Pod + Service in front of an internal MCP.                                    |
| Kustomize component             | Co-locate Spanly as a sidecar in your existing Pod.                                      |

## Helm

A maintained chart is at
[`charts/spanly`](https://github.com/spanlyhq/spanly/tree/main/charts/spanly).
Install it from a clone of the repo. The chart reads the API key from a
Kubernetes Secret, never from a chart value:

```bash
git clone https://github.com/spanlyhq/spanly.git
kubectl create secret generic spanly --from-literal=api-key=$SPANLY_API_KEY

helm install spanly ./spanly/charts/spanly \
  --set proxy.upstream=http://mcp.default.svc.cluster.local:3000
```

The chart creates a Deployment running `spanly proxy`, a Service in
front of it (port `3001` by default), and an optional Prometheus
`ServiceMonitor`. See the chart's
[values reference](https://github.com/spanlyhq/spanly/blob/main/charts/spanly/README.md)
for context headers, admin endpoints, and resource overrides.

## Kustomize sidecar

A maintained component is at
[`kustomize/spanly-sidecar`](https://github.com/spanlyhq/spanly/tree/main/kustomize/spanly-sidecar).

```yaml
# kustomization.yaml
resources:
  - my-deployment.yaml
components:
  - https://github.com/spanlyhq/spanly//kustomize/spanly-sidecar?ref=main
```

It injects the Spanly container into Deployments labelled
`spanly-sidecar=true`, exposes a new port (default `3001`), and reads
the API key from a Secret.

## Putting Spanly behind nginx / Caddy / Envoy

If you front Spanly with another reverse proxy, SSE responses can
stall in the front proxy's response buffer. Disable buffering on the
relevant routes.

### nginx

```nginx
location /mcp {
    proxy_pass http://spanly:3001;
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header X-Accel-Buffering no;
    proxy_read_timeout 1h;
}
```

### Caddy

```text
reverse_proxy spanly:3001 {
    flush_interval -1
}
```

### Envoy

- Disable response buffering on the relevant route.
- Set `auto_host_rewrite: true` if Spanly is selected by name.

## Admin endpoints

For health checks and Prometheus scraping, enable the admin listener:

```bash
spanly proxy --admin-addr=:9090 mcp:3000 0.0.0.0:3001
```

- `GET /healthz`: 200 if the listener is up.
- `GET /readyz`: 200 if the upstream is reachable (1s cache).
- `GET /metrics`: Prometheus text format.

Wire `/readyz` into your orchestrator's readiness probe to avoid sending
traffic to a Spanly proxy whose upstream is down.

## OpenTelemetry

The CLI does not export OTel spans. It ships telemetry to Spanly only.
The inbound `traceparent` header (when present) is preserved verbatim
on each captured packet. Pick your APM provider in the Spanly dashboard
(Settings, Integrations) and every request with trace context links
straight to the matching trace in Datadog, Sentry or New Relic. Nothing
to configure on the CLI side.

## Capacity & sizing

The CLI is single-binary and stateless. Indicative figures for a
typical sidecar, measured on our own deployments:

- Around 15 MB resident memory at idle.
- 25 to 40 MB at sustained 1k packets/s.
- Single-core CPU bound on TLS at high throughput.

For load above a few thousand packets/s on a single instance, consider
sharding clients across multiple Spanly proxies (DNS round-robin or
service mesh).

## What's not yet supported

- **Windows**: best-effort, not regression-tested. Use WSL.
- **WebSockets**: only HTTP, SSE, and stdio.
- **TLS termination** on the bind side: front Spanly with your own
  reverse proxy.

---

## Docker

Source: https://spanly.com/docs/docker/

The Spanly CLI ships as a container image, so you can monitor an MCP
server without installing anything on the host. Run it directly or as a
sidecar next to your server.

```bash
docker pull spanly/spanly:latest
```

## In this section

- [Installation](https://spanly.com/docs/docker/installation/) covers pulling the image and the
  available tags.
- [Sidecar](https://spanly.com/docs/docker/sidecar/) shows how to run Spanly next to your server
  in Docker Compose or Kubernetes.

---

## Installation

Source: https://spanly.com/docs/docker/installation/

The CLI ships as a Docker image at `spanly/spanly:latest` (Docker Hub)
and `ghcr.io/spanlyhq/spanly:latest` (GHCR). It is the same Go binary
that npm and Homebrew install, so all subcommands and flags work
identically.

## Pull

```bash
docker pull spanly/spanly:latest
```

## Run

```bash
# Proxy an MCP server reachable from the container.
# Point your MCP client at localhost:3001.
docker run --rm \
  -e SPANLY_API_KEY="$SPANLY_API_KEY" \
  -p 3001:3001 \
  spanly/spanly:latest proxy host.docker.internal:3000 0.0.0.0:3001
```

For most users the `proxy` form is the more useful shape inside a
container, since the wrapped child (`run -- …`) would itself need to
be inside the same image. If your MCP server is already a container,
run Spanly alongside it and proxy. See [Sidecar patterns](https://spanly.com/docs/docker/sidecar/)
for compose and Kubernetes examples.

## Image tags

| Tag                       | When to use                                            |
| ------------------------- | ------------------------------------------------------ |
| `latest`                  | Track the latest stable release.                       |
| `<semver>` (e.g. `0.1.0`) | Pin to a specific version, recommended for production. |

## Image contents

- A single static Go binary at `/usr/local/bin/spanly`.
- The default `ENTRYPOINT` is `spanly` so you only pass subcommand +
  flags as `command:` / `args:`.
- Built for `linux/amd64` and `linux/arm64`.

## Local development

To run against your local dev Spanly stack, override the ingest URL:

```bash
docker run --rm \
  -e SPANLY_API_KEY=spanly_us_localdev \
  -e SPANLY_INGEST_URL=http://host.docker.internal:3010 \
  spanly/spanly:latest proxy host.docker.internal:3000 0.0.0.0:3001
```

---

## Sidecar patterns

Source: https://spanly.com/docs/docker/sidecar/

The recommended deployment shape for the Spanly container is as a
sidecar that proxies traffic to your MCP server. The MCP server stays
internal, the sidecar takes the public port.

## docker-compose

A typical setup: your MCP server container plus a Spanly sidecar in
the same compose service group.

```yaml
services:
  mcp:
    image: my-org/my-mcp:1.0.0
    expose:
      - '3000' # reachable from the sidecar only, not the host
    environment:
      MCP_PORT: 3000

  spanly:
    image: spanly/spanly:latest
    command: ['proxy', 'mcp:3000', '0.0.0.0:3001']
    ports:
      - '3001:3001' # this is what your MCP client connects to
    environment:
      SPANLY_API_KEY: ${SPANLY_API_KEY}
    depends_on:
      - mcp
```

Point your MCP client at `http://localhost:3001`. The MCP server
itself is no longer exposed externally.

## Kubernetes

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-mcp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-mcp
  template:
    metadata:
      labels:
        app: my-mcp
    spec:
      containers:
        - name: mcp
          image: my-org/my-mcp:1.0.0
          ports:
            - containerPort: 3000

        - name: spanly
          image: spanly/spanly:latest
          args: ['proxy', 'localhost:3000', '0.0.0.0:3001']
          ports:
            - containerPort: 3001
          env:
            - name: SPANLY_API_KEY
              valueFrom:
                secretKeyRef:
                  name: spanly
                  key: api-key
```

The Service exposes port `3001` (the Spanly proxy), not `3000`.

## Session grouping

If your MCP server runs sessionless, the sidecar assigns a synthetic
`Mcp-Session-Id` on initialize responses so Spanly can still group
requests into sessions. The ID is stripped before requests are
forwarded to your server. Add `--inject-session-id=false` to the
`proxy` arguments to turn this off. See
[Session tracking](https://spanly.com/docs/session-tracking/).

A maintained Kustomize component is at
[`kustomize/spanly-sidecar`](https://github.com/spanlyhq/spanly/tree/main/kustomize/spanly-sidecar).
For a standalone Pod + Service pattern, see the
[Helm chart](https://github.com/spanlyhq/spanly/tree/main/charts/spanly).

---

## Overview

Source: https://spanly.com/docs/mcp/overview/

The **Spanly MCP server** is an MCP server _we run_, not one you
instrument. Point any MCP-compatible agent (Claude Desktop, Cursor,
custom) at it and the agent can:

- Search and aggregate requests across your projects.
- Pull out errors and group by code.
- Read dashboards: top servers, top tools, slowest operations.
- Manage alert rules and channels.
- Share links to specific dashboards or requests.

This is how you ask "what changed in the last hour?" or "which tool is
slowest on the new server version?" without leaving your chat / IDE.

## Endpoint

The Spanly MCP server is hosted at:

```
https://mcp.spanly.com
```

It speaks HTTP MCP with OAuth sign-in: the first time a client
connects, it opens a browser window where you authenticate with your
Spanly account. No API key involved.

## Authentication & scope

- Authentication: OAuth. The MCP client handles the token exchange;
  you just sign in once.
- Scope: the agent acts as you. It sees the organisations and projects
  your account can see in the dashboard.
- Read vs write: most tools are read-only. Tools that change state
  (alert rules, channels, shared links) act with your account's
  permissions, the same ones you'd need in the dashboard.

## What's the difference between Spanly MCP and instrumenting?

|           | Instrumenting (SDK / CLI)         | Spanly MCP                                  |
| --------- | --------------------------------- | ------------------------------------------- |
| You run   | Your MCP server, wrapped          | Just an MCP client                          |
| Direction | You send data _to_ Spanly         | An agent reads data _from_ Spanly           |
| Use case  | "Capture what my MCP server does" | "Let an agent inspect what Spanly captured" |

The two work together: instrument your MCP server so Spanly has data,
then connect an agent via Spanly MCP so it can query that data.

## See also

- [Connecting](https://spanly.com/docs/mcp/connecting/): setup steps for Claude, Claude Code,
  Cursor, Windsurf, and curl.
- [Tools](https://spanly.com/docs/mcp/tools/): the full surface of MCP tools the server
  exposes.

---

## Connecting

Source: https://spanly.com/docs/mcp/connecting/

The Spanly MCP server speaks HTTP MCP at `https://mcp.spanly.com`.
Authentication uses OAuth: the first time a client connects, it opens a
browser window where you sign in with your Spanly account. There is no
API key to copy. The agent gets the same access you have.

## Claude (Desktop and claude.ai)

Add Spanly as a custom connector:

1. Open **Settings → Connectors → Add custom connector**.
2. Enter `https://mcp.spanly.com` as the URL.
3. Complete the sign-in flow in the browser window that opens.

## Claude Code

```bash
claude mcp add --transport http spanly https://mcp.spanly.com
```

Then run `/mcp` inside Claude Code and pick **spanly** to complete the
sign-in flow.

## Cursor

In `.cursor/mcp.json` (per project) or `~/.cursor/mcp.json` (global):

```json
{
  "mcpServers": {
    "spanly": {
      "url": "https://mcp.spanly.com"
    }
  }
}
```

Cursor detects that the server requires OAuth and prompts you to sign
in on first use.

## Windsurf

Same shape as Cursor: add the server URL to Windsurf's MCP settings and
complete the sign-in prompt.

## curl smoke test

The server requires an OAuth access token, so a bare request is
rejected. That rejection is itself a useful connectivity check:

```bash
curl -i -X POST https://mcp.spanly.com/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"curl-test","version":"1.0.0"}}}'
```

A healthy server answers `401` with a `WWW-Authenticate` header that
points at the OAuth metadata:

```bash
curl -s https://mcp.spanly.com/.well-known/oauth-protected-resource
```

If both respond, the server is reachable and any remaining issue is in
your client's sign-in flow.

## Authentication & scope

- Sign in with your Spanly account when the client prompts you.
- The agent's access mirrors yours: it sees the organisations and
  projects you can see in the dashboard.
- Tokens are issued and refreshed by the client. Revoke access from
  your account settings at any time.

## Troubleshooting

- **401 Unauthorized**: the client has no valid token. Re-run the
  client's sign-in flow (for Claude Code, `/mcp` → spanly →
  authenticate).
- **Sign-in window never opens**: some clients only support
  OAuth-protected MCP servers in recent versions. Update the client
  first.
- **406 Not Acceptable on curl**: include
  `-H "Accept: application/json, text/event-stream"`. The server
  streams responses as SSE frames.

If a tool call returns an error with a specific message, that's the
backend speaking. Open the request in
[your dashboard](https://spanly.com/) to see the same error in human
form.

---

## Tools

Source: https://spanly.com/docs/mcp/tools/

The Spanly MCP server exposes the tools below. All of them resolve
against projects your signed-in account can access. Time-range arguments
accept the presets `5m`, `30m`, `1h`, `3h`, `1d`, `7d`, `30d`, `90d`,
`180d`, `365d`. Anywhere a tool takes a `projectId`, you can list valid
values with `list_projects`.

## Project discovery

| Tool            | Purpose                                                 |
| --------------- | ------------------------------------------------------- |
| `list_projects` | Projects accessible to your account, with their region. |

## Requests

The leaf entity: one JSON-RPC request/response pair captured from your
MCP server.

| Tool                           | Purpose                                                            |
| ------------------------------ | ------------------------------------------------------------------ |
| `list_requests`                | Filter requests by server, client, method, status, and time range. |
| `get_request`                  | Fetch a single request by id, including raw payload.               |
| `aggregate_requests_by_method` | Group request counts by `method`.                                  |

## Servers & clients

| Tool                     | Purpose                                                                                   |
| ------------------------ | ----------------------------------------------------------------------------------------- |
| `list_servers`           | Servers seen in the time range, with request counts and error rates.                      |
| `aggregate_servers`      | Roll up traffic by server name/version.                                                   |
| `client_calls_over_time` | Time series of calls by client name/version. Useful for spotting client-side regressions. |

## Errors

| Tool                       | Purpose                                          |
| -------------------------- | ------------------------------------------------ |
| `list_errors`              | Errored requests in the time range.              |
| `aggregate_errors_by_code` | Counts grouped by error code.                    |
| `get_errors_by_code`       | Pull the underlying requests for one error code. |

## Issues

The findings shown on the [Issues](https://spanly.com/docs/mend/) tab: problems detected in live
traffic, plus static tool-manifest issues.

| Tool                      | Purpose                                                                     |
| ------------------------- | --------------------------------------------------------------------------- |
| `list_issues`             | Runtime issues detected in live traffic (open plus recently resolved).      |
| `list_static_scan_issues` | Issues found by statically scanning a server's advertised tool manifest.    |
| `list_issue_checks`       | Per-check summary: open and all-time counts, last activity, and fixability. |

## End users & identity

| Tool                    | Purpose                                                                       |
| ----------------------- | ----------------------------------------------------------------------------- |
| `list_users`            | End users seen calling your servers, with session, request, and error counts. |
| `get_user`              | Aggregated activity for a single end user by `endUserId`.                     |
| `get_identity_settings` | The project's end-user identity configuration (secrets omitted).              |

## Dashboards

The same views that power the dashboard UI.

| Tool                           | Purpose                                                    |
| ------------------------------ | ---------------------------------------------------------- |
| `dashboard_stats`              | Headline numbers: total requests, error rate, p50/p95/p99. |
| `dashboard_top_servers`        | Top servers by traffic.                                    |
| `dashboard_top_tools`          | Top tools (MCP `tools/call`) by traffic.                   |
| `dashboard_slowest_operations` | The slowest operations in the window.                      |

## Integration help

| Tool                           | Purpose                                                                                                                                 |
| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `get_integration_instructions` | Returns copy-paste integration instructions for the TS SDK, Python SDK, CLI, or Docker. The same content as the in-app onboarding flow. |

## Notification channels

Manage the channels that receive a notification when a new [Issue](https://spanly.com/docs/mend/)
opens. Each channel opts in with `notifyOnNewIssues` and gates delivery by
`minIssueSeverity`.

| Tool                   | Purpose                                                                   |
| ---------------------- | ------------------------------------------------------------------------- |
| `list_alert_channels`  | List notification channels (`EMAIL`, `SLACK`, `WEBHOOK`, `IN_APP`).       |
| `create_alert_channel` | Create a new channel, optionally with new-Issue notification settings.    |
| `update_alert_channel` | Update a channel's destination, auth, or new-Issue notification settings. |
| `delete_alert_channel` | Delete a channel.                                                         |

## Shared links

| Tool                 | Purpose                                                    |
| -------------------- | ---------------------------------------------------------- |
| `list_shared_links`  | Existing share links for the project.                      |
| `create_shared_link` | Create a read-only link to a dashboard, request, or chart. |
| `revoke_shared_link` | Revoke an existing link.                                   |

## Gateway

Read the configuration and health of the hosted [Gateway](https://spanly.com/docs/gateway/)
reverse proxy that fronts your MCP server.

| Tool                       | Purpose                                                                   |
| -------------------------- | ------------------------------------------------------------------------- |
| `get_gateway`              | The project's gateway config (upstream, capture/auth modes, rules).       |
| `get_gateway_health`       | Current upstream health (up / down / unknown, last error).                |
| `list_gateway_domains`     | Custom domains attached to the gateway, with verification and TLS status. |
| `list_gateway_client_keys` | Client auth keys issued for the gateway (key values omitted).             |

## Patches (Mend)

Inspect and drive [Mend](https://spanly.com/docs/mend/): manifest patches, A/B canaries, and the
shims (applied patches) they promote to.

| Tool                       | Purpose                                                                    |
| -------------------------- | -------------------------------------------------------------------------- |
| `mend_server_manifests`    | Live tool manifests per server (author custom patches against real tools). |
| `mend_list_patches`        | Manifest patches for the project, optionally filtered by status.           |
| `mend_create_custom_patch` | Author a hand-written manifest patch (rename/hide tools, rewrite copy).    |
| `mend_apply_patch`         | Apply a patch to every session immediately, skipping the canary.           |
| `mend_launch_canary`       | Launch an A/B canary for a proposed patch.                                 |
| `mend_list_canaries`       | Canaries for the project (running and concluded).                          |
| `mend_get_canary`          | Full detail for one canary: patch, per-arm metrics, gate board.            |
| `mend_stop_canary`         | Stop a running canary and roll back to the original manifest.              |
| `mend_promote_canary`      | Promote a winning canary so its patch applies to every session.            |
| `mend_list_shims`          | Applied patches (shims), including those awaiting an upstream patch.       |
| `mend_activity`            | Recent Mend activity: proposals, canary launches, shim installs.           |

## Conventions

- **Read-only by default**: every tool documents whether it mutates
  state. Write tools act with your account's permissions.
- **No pagination cursor surprises**: listing tools take a plain
  `limit` rather than opaque cursors, so an agent can plan multi-step
  scans deterministically.
- **Explicit ids**: `create_*` tools return the new entity's id;
  `get_*`, `update_*`, and `delete_*` tools take it as an argument.

---

## Session tracking

Source: https://spanly.com/docs/session-tracking/

MCP's Streamable HTTP transport has an optional session mechanism: the
server assigns an `Mcp-Session-Id` header on the initialize response,
and the client echoes it on every subsequent request. When that header
is present, Spanly groups requests into sessions, so you can follow a
single client's conversation (initialize, tool calls, prompts) as one
thread instead of a flat list of requests.

## Servers with sessions

If your server assigns session IDs (for example a stateful
`StreamableHTTPServerTransport` with a `sessionIdGenerator`), there is
nothing to configure. Spanly picks the ID up from the captured headers
and session grouping works out of the box.

## Sessionless servers: synthetic session IDs

Many production MCP servers run stateless: a fresh server instance per
request, no session ID assigned, every request self-contained. That is
a perfectly valid deployment shape, but without a session ID Spanly
cannot tell which requests belong to the same client conversation.

To close that gap, every Spanly instrumentation method assigns a
synthetic session ID (prefixed `spanly-`) when the server doesn't. How
that ID reaches the wire differs by surface:

- **TypeScript SDK**: the middleware never sends a response or mutates
  a header. It mints the synthetic ID internally (a "sessionizer" keyed
  on the request's bearer token or remote address) and stamps it only
  into the transport context it sends to Spanly for telemetry. Your
  server's actual response to the client is untouched, and the client
  never sees or echoes this ID. Grouping still works, because the
  same key produces the same synthetic ID for the life of the idle
  timeout.
- **Python SDK and CLI (and the Docker sidecar, which wraps the CLI)**:
  these inject a real `Mcp-Session-Id` response header on an
  initialize response that doesn't already carry one. Per the MCP
  spec, the client then echoes that header on its subsequent requests,
  which is what groups them. This is a real, on-the-wire header. The
  CLI and Docker sidecar strip it back out of requests before
  forwarding them upstream, so the server itself never sees a header
  it didn't create; Python's ASGI middleware also owns the DELETE
  handshake that terminates a synthetic session, answering it directly
  instead of forwarding it to a server that never issued that ID.

Both approaches are invisible to your server: it serves each request
as usual, and a synthetic session is a grouping label only. It does
not make the server stateful and does not enable server-to-client
notifications or resumability.

Stdio transports are unaffected: a stdio connection is a single
conversation already, and there are no HTTP headers to carry a session
ID.

## Turning it off

The Python SDK, the CLI, and the Docker sidecar mutate a response
header to do this, so each offers a toggle. The TypeScript SDK never
mutates a response in the first place, so there is nothing to turn
off: its synthetic grouping is telemetry-only and always on.

| Surface                             | Toggle                                                     |
| ----------------------------------- | ---------------------------------------------------------- |
| TypeScript SDK                      | Not applicable. Synthetic grouping never touches the wire. |
| Python SDK                          | `SpanlyMiddleware(app, inject_session_id=False)`           |
| CLI (`spanly run` / `spanly proxy`) | `--inject-session-id=false`                                |
| Docker sidecar                      | add `--inject-session-id=false` to the `proxy` args        |

With injection off (Python, CLI, Docker), requests to sessionless
servers are still captured and attributed; they just aren't grouped
into sessions.

## Notes and edge cases

- Synthetic IDs injected by the Python SDK or the CLI are visible to
  MCP clients (that is how they get echoed back). The `spanly-` prefix
  makes them easy to identify in client logs. The TypeScript SDK's
  synthetic IDs never leave your process, so this does not apply to
  it.
- Clients that don't implement the session part of the Streamable
  HTTP spec won't echo a real, injected header, and those requests
  stay ungrouped on the surfaces that rely on the echo (Python, CLI,
  Docker). All mainstream MCP clients echo it. The TypeScript SDK's
  telemetry-only grouping does not depend on the client echoing
  anything.
- Load-balanced, multi-replica servers work fine on every surface: the
  Python/CLI/Docker session lives in the client's echo, not in server
  state; the TypeScript SDK's sessionizer keys on the request itself
  (bearer token or remote address), so it doesn't matter which replica
  serves each request within the idle timeout.
- In the Python SDK, injection works on any ASGI app, including the
  one `streamable_http_app()` (FastMCP) returns; see the
  [Python SDK reference](https://spanly.com/docs/python-sdk/api-reference/#spanlymiddlewareapp-options).

---

## Identify your users

Source: https://spanly.com/docs/identify-users/

Spanly can tie every captured session to the end user (and account) behind
it. Once configured you get:

- Sessions attributed to users: the sessions list and session detail show
  who drove each conversation.
- A Users page listing everyone who used your MCP server, with sessions,
  requests, errors, and last-seen per user, plus a per-user detail view.
- Filtering: scope any session view to a single user.

{/* Screenshot placeholder: Users page with a populated list. */}

## The easiest path: configure it in the dashboard

Open your project's **Settings → Identity** tab and configure identity
there — JWT claim mapping, the resolver endpoint URL and secret, or both.
Running CLI proxies and TypeScript SDK middleware pick the settings up
automatically, usually within a minute, with no restart, no flags, and no
code change.

The dashboard needs your org-admin role to save changes, and the resolver
secret is write-only: after saving it is never shown again, and it is
delivered only to your own producers over the same authenticated channel
they already use to ship telemetry.

Local configuration still works and always wins: a CLI proxy started with
any `--identity-*` flag, or SDK middleware constructed with any `identity`
option, ignores the dashboard settings on that producer.

## JWT claim mapping via CLI flags

If your MCP server authenticates with JWT bearer tokens and you prefer
flags over the dashboard, identity is one flag on the Spanly CLI proxy:

```bash
spanly run --port 3000 --identity-jwt -- node server.js
```

The proxy decodes the `Authorization: Bearer` JWT payload and maps
`sub` to the user id, `email` to email, and `name` to name. Override any
mapping with `--identity-jwt-claim` (repeatable, dot-paths supported):

```bash
spanly run --port 3000 \
  --identity-jwt \
  --identity-jwt-claim accountId=org_id \
  --identity-jwt-claim id=user.id \
  -- node server.js
```

The token is decoded only, never signature-verified: your server behind
the proxy has already authenticated the request, so the claims are only
used to label traffic it accepted.

Using the TypeScript SDK middleware instead of the CLI? The same tiers are
available in-process: `spanly({ identity: { jwtClaims: true } })`, an
`identity.resolve` callback, or `identity.resolver`. See the
[SDK API reference](https://spanly.com/docs/typescript-sdk/api-reference/#identity) for the SDK
shapes; the rest of this page uses the CLI flags.

## Opaque tokens: the resolver endpoint

If your tokens are opaque (API keys, session tokens), host a small HTTP
endpoint that maps a token to a user, and point the proxy at it:

```bash
spanly run --port 3000 \
  --identity-resolver-url=https://api.example.com/spanly-identity \
  --identity-resolver-secret=$SPANLY_IDENTITY_SECRET \
  -- node server.js
```

The endpoint can be one route on the MCP server itself; any URL works. The
full request/response contract, caching behavior, and a copy-paste Express
handler are in the [resolver protocol reference](https://spanly.com/docs/identity-resolver/).

When both tiers are configured, a JWT that decodes to a user id wins and
the resolver is never called; opaque tokens fall through to the resolver.

## Deployment model

Identity is resolved by the producer running inside your infrastructure:
the Spanly CLI proxy (`spanly run --port` or `spanly proxy`) in front of
your HTTP MCP server, or the TypeScript SDK middleware in-process. stdio
mode has no auth headers, so identity does not apply there.

## Privacy model

Your users' tokens never leave your infrastructure. The JWT decode happens
inside the proxy process, and the resolver endpoint is yours: Spanly's
backend never sees the token and never calls your resolver. What Spanly
receives is only the resolved `id`, `email`, `name`, `accountId`, and
`accountName` you choose to send, plus a one-way token fingerprint (the
first 32 hex characters of the SHA-256 of the bearer credential) that you
can disable with `--no-auth-fingerprint`.

If you configure the resolver in the dashboard, Spanly stores the shared
secret encrypted at rest and delivers it only to your producers, over the
same authenticated channel they ship telemetry on. Prefer to keep the
secret entirely on your side? Use the CLI flags — they take precedence and
nothing is stored with Spanly.

## Token rotation

Attribution keys on your stable user id, so rotating tokens do not
fragment users: a new token re-resolves to the same id and the sessions
line up under one user. The fingerprint-only fallback (no identity
configured) counts distinct tokens, so it over-counts users under
rotation. That is why the Users page shows "distinct auth tokens" rather
than "users" until identity is configured.

## What to return

| Field         | Required | Use                                                     |
| ------------- | -------- | ------------------------------------------------------- |
| `id`          | yes      | Your internal user id. Stable across logins and tokens. |
| `email`       | no       | Display in the dashboard.                               |
| `name`        | no       | Display in the dashboard.                               |
| `accountId`   | no       | Company or workspace level attribution.                 |
| `accountName` | no       | Display for the account.                                |

Do not use the raw token or a rotating claim (`jti`, `exp`-derived values)
as `id`: every rotation would then look like a new user, which defeats the
point. An email works for display but makes a poor id, since users change
emails.

## CLI flag reference

| Flag                         | Default | Description                                                                                                                            |
| ---------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `--identity-jwt`             | off     | Decode the bearer JWT with the default mapping: id from `sub`, email from `email`, name from `name`.                                   |
| `--identity-jwt-claim`       | none    | `field=claim` override, repeatable. Fields: `id`, `email`, `name`, `accountId`, `accountName`. Dot-paths OK. Implies `--identity-jwt`. |
| `--identity-resolver-url`    | none    | URL of your resolver endpoint (see the [protocol reference](https://spanly.com/docs/identity-resolver/)).                                                      |
| `--identity-resolver-secret` | none    | Shared secret sent as `X-Spanly-Identity-Secret`. Also via `SPANLY_IDENTITY_RESOLVER_SECRET`; the flag wins.                           |
| `--no-auth-fingerprint`      | off     | Disable the anonymous token fingerprint (on by default in HTTP mode).                                                                  |

## Behavior reference

| Behavior           | Value                                                             |
| ------------------ | ----------------------------------------------------------------- |
| Positive cache     | 15 minutes per token                                              |
| Negative cache     | 60 seconds per token (unknown tokens and resolver failures)       |
| Resolver timeout   | 3 seconds, no retries (natural retry after negative cache expiry) |
| Concurrency        | One in-flight resolution per token                                |
| Failure mode       | Fail-open: resolution failure never affects MCP serving           |
| Fingerprint scheme | `Bearer` only; `Basic` and other schemes are never fingerprinted  |

---

## Identity resolver protocol

Source: https://spanly.com/docs/identity-resolver/

For opaque tokens, or when you want custom logic, [identity
resolution](https://spanly.com/docs/docs/identify-users/) calls an HTTP endpoint you host that maps
a token to a user. This page specifies that protocol (v1).

The caller is always a producer running inside your infrastructure: the
Spanly CLI proxy (`spanly run --port` / `spanly proxy`), the TypeScript SDK
middleware, and on the roadmap the hosted Spanly Gateway, which will speak
this same protocol. Your users' raw tokens never leave your perimeter, and
Spanly's backend never calls this endpoint.

## Request

The producer sends one POST per unknown token:

```
POST <configured URL>
Content-Type: application/json
X-Spanly-Identity-Secret: <shared secret>

{"token": "<raw bearer credential>", "mcpSessionId": "<session id>"}
```

- `token` is the credential from the `Authorization: Bearer` header,
  verbatim.
- `mcpSessionId` is optional context. Under the sessionless MCP spec
  (2026-07-28) it may be omitted entirely, or carry a Spanly-synthetic
  value prefixed `spanly-`. Resolvers must not depend on it.

The URL is whatever you configure — in the dashboard (project
Settings → Identity) or via `--identity-resolver-url`; any route works,
and the endpoint can live on the same server as the MCP itself. Note that
producers configured through the dashboard call the endpoint from your
infrastructure exactly the same way; only the configuration travels
through Spanly, never the tokens.

## Response

| Case          | Response                         |
| ------------- | -------------------------------- |
| Known token   | `200` with a user object (below) |
| Unknown token | `200` with body `null`, or `404` |
| Anything else | Treated as a transient failure   |

User object, only `id` is required:

```json
{
  "id": "usr_123",
  "email": "ada@example.com",
  "name": "Ada Lovelace",
  "accountId": "acct_42",
  "accountName": "Acme Corp"
}
```

`id` must be your stable internal user id, not the token and not a
rotating claim: it is the attribution key, so rotating tokens keep
pointing at the same user. `accountId` and `accountName` enable
company-level attribution.

## Caller behavior (normative)

The producer:

- times out after 3 seconds and does not retry; a failed token is retried
  naturally when its negative cache entry expires (about 60 seconds),
- makes a single in-flight request per token, so concurrent traffic with
  the same token piggybacks on the pending resolution,
- caches a resolved identity for about 15 minutes per token, so token
  rotation and producer restarts do not hammer the endpoint,
- fails open: resolution failure never affects MCP serving; the sessions
  simply stay unattributed until a later retry succeeds.

## Your obligations

- Verify `X-Spanly-Identity-Secret` with a constant-time comparison.
- Treat `token` as the sensitive credential it is: do not log it.
- Serve HTTPS in production (`http://` is fine for localhost development).

## Example (Express)

```js
const express = require('express');
const { timingSafeEqual } = require('node:crypto');

const app = express();
app.use(express.json());

const SECRET = process.env.SPANLY_IDENTITY_SECRET;

app.post('/spanly-identity', async (req, res) => {
  const given = Buffer.from(req.get('X-Spanly-Identity-Secret') ?? '');
  const expected = Buffer.from(SECRET);
  if (given.length !== expected.length || !timingSafeEqual(given, expected)) {
    return res.status(401).end();
  }
  const session = await sessions.findByApiToken(req.body.token);
  if (!session) return res.status(404).end();
  res.json({
    id: session.userId,
    email: session.userEmail,
    name: session.userName,
    accountId: session.orgId,
    accountName: session.orgName,
  });
});

app.listen(4000);
```

Point the CLI at it:

```bash
spanly run --port 3000 \
  --identity-resolver-url=http://localhost:4000/spanly-identity \
  --identity-resolver-secret=$SPANLY_IDENTITY_SECRET \
  -- node server.js
```

---

## Overview

Source: https://spanly.com/docs/gateway/

The gateway is a hosted reverse proxy you put in front of your MCP server.
Your end users connect to a Spanly endpoint, the gateway proxies every request
to your server, and you get full observability with no SDK and no code
changes.

## When to use the gateway

|               | Gateway                                           | SDK                  | CLI                     |
| ------------- | ------------------------------------------------- | -------------------- | ----------------------- |
| Code changes  | none                                              | wrap your server     | none                    |
| Distribution  | you point users at a Spanly URL                   | ships in your app    | runs beside your server |
| Transport     | streamable HTTP                                   | any the SDK supports | HTTP                    |
| Edge features | rate limit, IP rules, client keys, custom domains | no                   | no                      |

Choose the gateway when you want observability without touching your server,
or when you need edge controls (auth, rate limiting, IP rules) in front of a
server that has none.

## How it works

```
end user / MCP client
        │  HTTPS to <slug>.gateway.us.spanly.com
        ▼
Spanly regional gateway  ──►  your MCP server
        │
        └──►  capture (side path) ──► your Spanly dashboard
```

The gateway runs one service per region (US in Oregon, EU in Frankfurt) and
is pinned to your project's region. Capture is a side path: if any part
of Spanly is degraded, your traffic keeps flowing and only the observability
copy is affected.

---

## Overview

Source: https://spanly.com/docs/gateway/

The gateway is a hosted reverse proxy you put in front of your MCP server.
Your end users connect to a Spanly endpoint, the gateway proxies every request
to your server, and you get full observability with no SDK and no code
changes.

## When to use the gateway

|               | Gateway                                           | SDK                  | CLI                     |
| ------------- | ------------------------------------------------- | -------------------- | ----------------------- |
| Code changes  | none                                              | wrap your server     | none                    |
| Distribution  | you point users at a Spanly URL                   | ships in your app    | runs beside your server |
| Transport     | streamable HTTP                                   | any the SDK supports | HTTP                    |
| Edge features | rate limit, IP rules, client keys, custom domains | no                   | no                      |

Choose the gateway when you want observability without touching your server,
or when you need edge controls (auth, rate limiting, IP rules) in front of a
server that has none.

## How it works

```
end user / MCP client
        │  HTTPS to <slug>.gateway.us.spanly.com
        ▼
Spanly regional gateway  ──►  your MCP server
        │
        └──►  capture (side path) ──► your Spanly dashboard
```

The gateway runs one service per region (US in Oregon, EU in Frankfurt) and
is pinned to your project's region. Capture is a side path: if any part
of Spanly is degraded, your traffic keeps flowing and only the observability
copy is affected.

---

## Quickstart

Source: https://spanly.com/docs/gateway/quickstart/

You need an MCP server reachable over HTTPS. No code changes are required.

## 1. Create the gateway

In the dashboard, open the **Gateway** tab (under the Spanly group in the
sidebar) for the project you want. Enter your MCP server URL, for example
`https://mcp.your-company.com/mcp`, and select **Create gateway**.

You get a dedicated endpoint immediately:

```
https://your-project.gateway.us.spanly.com/mcp
```

(`eu` instead of `us` for EU projects.)

## 2. Test the connection

Select **Test connection**. Spanly runs a minimal MCP handshake from the same
network path real traffic takes and reports the server name, version, tool
count, and duration. If it fails, the error names the stage (DNS, connect,
TLS, HTTP, initialize, or tools list) so you know exactly what to fix.

## 3. Point a client at the endpoint

Configure any MCP client with the gateway URL. For example, with the
TypeScript SDK client:

```ts
const transport = new StreamableHTTPClientTransport(
  new URL('https://your-project.gateway.us.spanly.com/mcp'),
);
await client.connect(transport);
```

## 4. See requests in the dashboard

Make a few tool calls, then open the **Requests** tab. Each request shows the
method, tool, duration, and status. Requests that went through the gateway
also show the gateway overhead added versus your upstream's own time.

That is the whole loop. Everything else on the Gateway tab (custom domains,
edge protection, client keys, capture modes) is optional.

---

## Custom domains

Source: https://spanly.com/docs/gateway/custom-domains/

Custom domains are available on the Business plan. They let your users
connect to `mcp.your-company.com` instead of a Spanly subdomain. TLS
certificates are issued and renewed for you.

## Add a domain

On the Gateway tab, in the Custom domain section, enter your domain and select
**Add domain**. Spanly shows two DNS records you must create at your DNS
provider:

| Type  | Name                                  | Value                                |
| ----- | ------------------------------------- | ------------------------------------ |
| TXT   | `_spanly-verify.mcp.your-company.com` | the verification token shown         |
| CNAME | `mcp.your-company.com`                | `your-project.gateway.us.spanly.com` |

The TXT record proves you own the domain. The CNAME routes traffic to your
gateway. For an apex domain that cannot carry a CNAME, use an ALIAS or ANAME
record pointing at the same target.

## Verification and TLS

Once both records resolve, the status moves from Waiting for DNS to Issuing
certificate to Active. DNS propagation can take anywhere from a few minutes to
a few hours depending on your provider. The dashboard polls and updates the
status automatically. If verification fails, the reason is shown and you can
retry after fixing the records.

## Remove a domain

Removing a domain stops it resolving immediately, so any client using that
hostname loses connectivity. Your gateway's default `.gateway.<region>.spanly.com`
endpoint keeps working.

---

## Auth behind the gateway

Source: https://spanly.com/docs/gateway/auth/

## OAuth passthrough

If your MCP server uses OAuth 2.1, it keeps working behind the gateway. The
gateway forwards `Authorization` verbatim and rewrites your server's protected
resource metadata so the client sees the gateway origin as the resource
server.

One thing to check on your side: your server must accept the gateway origin as
a valid resource and audience value. Set your server's canonical resource to
the gateway URL. Audience checks must tolerate the trailing-slash variant of
the resource URL, because clients send it both with and without the trailing
slash.

## Static header injection

If your upstream requires an internal secret (a shared key, a service token),
add it under static headers on the Gateway tab. The gateway sets these headers
on every request to your server, overriding anything the client sent, and they
are never visible to your end users or stored in your captured telemetry.

## Gateway-managed client keys

If your MCP server has no auth of its own, the gateway can add one. Switch the
access mode to require a gateway client key, then create keys for the clients
or teams that should have access. Each key is shown once.

Clients send the key as `Authorization: Bearer spanlygw_...`. The gateway
validates it at the edge and strips it before the request reaches your server,
so your server never sees the key. Revoking a key takes effect within about 30
seconds.

---

## Edge protection

Source: https://spanly.com/docs/gateway/edge-protection/

All edge controls are optional and off (or wide open) by default. Configure
them on the Gateway tab.

## Rate limiting

Turn on rate limiting to protect your upstream. You set requests per minute
and a burst allowance, applied per client. The client is identified by its
gateway client key if you use one, otherwise by a hash of its Authorization
header, otherwise by IP.

When a client exceeds the limit the gateway returns HTTP 429 with a
`Retry-After` header and a JSON-RPC error body. These rejections are captured,
so you can see rate-limit pressure in your dashboard.

## Request size cap

The gateway rejects request bodies larger than the size cap (4 MiB by default,
configurable from 64 KiB to 64 MiB) with HTTP 413 before they reach your
server. This is separate from what Spanly captures; it only bounds what your
upstream receives.

Bodies that are not JSON at all are rejected with HTTP 400. Valid JSON that
omits the `jsonrpc` field is forwarded unchanged, so the gateway never blocks
a legitimate MCP client.

## IP rules

Restrict access to specific IP ranges, or block a source, with CIDR lists.
Enter one CIDR per line. An empty allow list allows every source. Blocks
always win over the allow list. Denied requests get HTTP 403. Behind the
gateway, the client IP is taken from the trusted forwarding hop, so inbound
`X-Forwarded-For` spoofing cannot bypass the rules.

---

## Security and data handling

Source: https://spanly.com/docs/gateway/security/

The gateway puts Spanly in the request path in front of your MCP server, so
this page answers the questions a security review asks. Everything here
describes shipped behavior.

## What data does the gateway see?

Every request and response that flows between your end users and your MCP
server, as JSON-RPC over HTTP. The gateway terminates TLS, applies your edge
rules, proxies the request to your server, and captures a copy for
observability on a side path.

## What is stored, and where?

Captured traffic stays in the region you chose (US in Oregon, EU in
Frankfurt) for its whole life. What gets stored depends on the capture mode
you set per gateway:

- **Full**: request and response bodies are stored (gzipped) in regional
  object storage, alongside metadata rows.
- **Redacted**: bodies are stored, but secret and personal-data spans are
  replaced with `[REDACTED:<kind>]` before anything durable is written. Raw
  bodies exist only in memory and in the in-region queue during transit.
- **Metadata only**: bodies never leave the gateway process. Only metadata
  (method, tool name, sizes, timing, status) is stored.

Metadata rows always exclude bodies; they hold the method, tool name, byte
sizes, durations, error and status codes, and identities.

## How long is data kept?

Data is retained for your plan's retention window. You can set a shorter
per-project override, which can only shorten retention, never extend it.
A daily job physically deletes data past the effective retention.

## Region guarantees

A gateway is pinned to its project's region. The routing layer refuses to
resolve or capture across regions, so a US gateway never routes or stores
into EU infrastructure, and the reverse.

## Is traffic encrypted?

Yes, end to end: client to gateway, gateway to your server, and gateway to
Spanly ingest all use TLS. Per-gateway secrets (the ingest key and any static
headers you inject) are encrypted at rest with AES-256-GCM. Client keys are
stored only as hashes.

## What happens if Spanly is degraded?

The gateway is fail-open by design. If capture, ingest, or the config service
is degraded, your traffic keeps flowing to your server; only the observability
copy is affected. A quota cap never blocks traffic either: at the cap the
gateway keeps proxying at full fidelity and only reduces capture sampling.

## Access controls at the edge

Per gateway you can require a Spanly-issued client key (for servers with no
auth of their own), restrict access to IP ranges, cap request sizes, and rate
limit per client. Client keys are validated at the edge and stripped before
the request reaches your server.

## Subprocessors

Hosting is on Render in the region you select. Object storage and the
analytics database are the same regional stores used by the rest of Spanly.
Ask support for the current subprocessor list and a DPA.

---

## Limits and troubleshooting

Source: https://spanly.com/docs/gateway/limits/

## Quota never blocks traffic

Reaching your plan's monthly request limit does not stop the gateway from
proxying. At the limit, every request still reaches your server and returns
the correct response. Only monitoring fidelity is reduced: capture is sampled
rather than fully stored. There is no setting that blocks proxying for billing
reasons.

## Fail-open

The gateway is fail-open by design. If Spanly capture, ingest, or the config
service is degraded, your traffic keeps flowing to your server. You may see a
gap in captured telemetry during the incident, but your users are not
affected.

## Common responses

| Status                   | Meaning                                     | What to do                                                                    |
| ------------------------ | ------------------------------------------- | ----------------------------------------------------------------------------- |
| 404 unknown gateway host | The hostname does not map to a gateway      | Check the endpoint URL and that the gateway exists                            |
| 503                      | The gateway is disabled                     | Re-enable it on the Gateway tab                                               |
| 502                      | The gateway could not reach your server     | Check your server is up; the upstream status card shows this too              |
| 504                      | Your server did not respond in time         | Raise the upstream timeout, or check your server's latency                    |
| 429                      | Rate limited by the gateway                 | The client exceeded your configured rate limit; it should honor `Retry-After` |
| 413                      | Request body too large                      | The body exceeded your size cap                                               |
| 403                      | Blocked by IP rules                         | The client's IP is outside your allow list or in your block list              |
| 401                      | A gateway client key is required or invalid | Send a valid `Authorization: Bearer spanlygw_...`                             |

For data handling, retention, and region guarantees, see the
[security and data handling](https://spanly.com/docs/docs/gateway/security/) page.

---

## Notifications

Source: https://spanly.com/docs/notifications/

Spanly's live scans continuously watch your MCP traffic and open an
[Issue](https://spanly.com/docs/mend/) when a check fires: a poisoned tool description, a schema
violation, a tool that has never succeeded, an oversized result, and so on.
Notification channels push those new Issues to your team the moment they open.

## Notification channels

- **Email**: one or more addresses per channel.
- **Slack**: paste a Slack incoming-webhook URL.
- **Webhook**: POST a signed JSON payload to any HTTPS endpoint for
  routing into PagerDuty, Opsgenie, or your own incident system. The
  signing secret is generated when you create the channel.
- **In-app**: surface the Issue in the Spanly dashboard.

Notification channels require the Pro plan or above.

## Notifying on new Issues

Each channel opts in to new-Issue notifications independently:

- **Notify on new Issues**: turn delivery on for the channel.
- **Minimum severity**: only Issues at or above this severity notify the
  channel. Severities are `info`, `low`, `medium`, `high`, and `critical`.

When a scan sweep opens new Issues in a project, every channel in that
project's organization with notifications enabled receives a single
digest of the Issues that meet its severity threshold. Each Issue notifies
once, when it first opens.

## Webhook payload

```json
{
  "kind": "issues_opened",
  "project": { "id": "...", "name": "Production" },
  "issueCount": 2,
  "highestSeverity": "high",
  "issues": [
    {
      "checkKey": "unhelpful_errors",
      "title": "Unhelpful errors",
      "subject": "search_docs",
      "severity": "high"
    }
  ],
  "appUrl": "https://app.spanly.com/projects/.../issues"
}
```

If the channel has a signing secret, requests carry an
`X-Spanly-Signature: sha256=<hmac>` header over the raw body.

## Managing channels

Manage channels from your organization settings, or programmatically through
the [Spanly MCP server](https://spanly.com/docs/mcp/tools/#notification-channels). If you need a
channel type that isn't there yet, email
[support@spanly.com](mailto:support@spanly.com).

---

## Concepts

Source: https://spanly.com/docs/mend/

Mend is the part of Spanly that fixes MCP manifest problems. Spanly
already sits in-line on your JSON-RPC stream through the SDK or CLI you
run in front of your server. Mend uses that position to rewrite
`tools/list` responses: when a scanner finds a manifest-fixable fault, it
drafts a fix, serves it to a slice of sessions as a live A/B test, checks
that nothing regressed, and promotes the winner. The promoted fix rides
on the wire until you ship the same change in your own source.

Everything Mend does on its own is deliberately narrow. A generated fix
only ever clarifies or tightens what a client is told about your tools —
it never renames a tool, removes one, or loosens a schema. Renaming and
disabling exist only in [custom canaries](https://spanly.com/docs/mend/canaries/#custom-canaries),
experiments you author yourself. If anything about Mend fails, your
original manifest is served untouched. Those guarantees are the whole
point, and they are spelled out on the [Trust and safety](https://spanly.com/docs/mend/trust/)
page.

## The loop

Mend runs one loop, from a scanner finding to a fix you own:

```
probes -> faults -> patches -> canaries -> shims -> upstream
```

### Probes

Probes are the scanner checks, static and live, that watch your server.
They are the same checks that power the Faults view. A probe describes a
manifest problem worth fixing, for example a parameter with no
description, a missing `outputSchema`, or a schema looser than the
arguments your tools actually receive.

Probes only observe. They open and close faults. They never change what
is served.

### Faults

A fault is a single scanner result: one problem on one tool, on one
version of your manifest. (Faults were previously called Findings in the
product; the concept is unchanged.) A fault carries a severity and,
where Mend can help, a suggested patch.

A fault records what is wrong. It does not change anything on its own.

### Patches

A patch is a candidate fix: a small list of typed overlay operations
against exactly one version of your manifest. A patch is authored against
a specific manifest content hash and is only ever applied to that version
(see manifest-hash anchoring on the [Trust and safety](https://spanly.com/docs/mend/trust/)
page). Every operation is client-facing only, so a patch can set a
description, add an annotation hint, tighten a parameter schema, mark a
parameter required, or set an output schema, and nothing else.

A patch is validated by re-scanning against it before it goes anywhere.
A patch that does not clear validation is rejected and never served.

### Canaries

A canary is a live A/B test of one patch. Sessions are split into a
Baseline arm (your original manifest) and a Candidate arm (the patch
applied), bucketed deterministically by session. Mend measures the
fault's own metric plus a set of always-on guardrail gates, and reaches
a verdict as early as the data allows.

A canary measures. It does not decide policy: a passing verdict makes a
patch eligible to promote, but whether promotion is automatic depends on
your project mode.

### Shims

A shim is a promoted patch being carried on the wire. Once a canary
concludes in favor of the candidate and the patch is promoted, it serves
to every session at 100 percent. A shim is a stopgap: it makes clients
see the better manifest while you still run the old one.

A shim changes only the `tools/list` response. It does not change your
server, and it stays anchored to the manifest version it was written for.

### Upstream

Upstreaming is you shipping the same fix in your own source. When your
server starts serving the improved manifest itself, Mend detects it on
the next scan, retires the shim, and resolves the fault as
`fixed_at_source`. The wire goes back to pure pass-through for that fix,
because your server now does the right thing on its own. The
[Upstreaming guide](https://spanly.com/docs/mend/upstreaming/) walks through reading the
diff and applying it.

## Glossary

These terms are used consistently across the product and these docs.

| Term               | Meaning                                                                                       |
| ------------------ | --------------------------------------------------------------------------------------------- |
| **Probe**          | A scanner check, static or live. The watchers.                                                |
| **Fault**          | A scanner result (called Finding in older copy).                                              |
| **Patch**          | A candidate fix: a list of typed overlay operations against one manifest version.             |
| **Canary**         | A live A/B of a patch: Baseline arm against Candidate arm, split by session.                  |
| **Gates**          | Always-on guardrail metrics a canary must not degrade.                                        |
| **Rollback**       | Automatic, immediate revert when a gate is breached.                                          |
| **Promoted**       | Canary verdict accepted; the patch serves at 100 percent.                                     |
| **Shim**           | A promoted patch carried on the wire, awaiting upstream.                                      |
| **Upstream**       | You ship the fix in your source; the shim retires.                                            |
| **Pulse**          | The `Spanly-Pulse` response header that keeps collectors in sync with delivery state.         |
| **Delivery state** | The per-project document (active shims, running canaries, kill switch) the SDK and CLI fetch. |

## Modes

Every project has a Mend mode, and the default is **suggest**.

- **suggest** (default): Mend opens faults and drafts patches, and stops
  there. Nothing is served until a human applies a patch as a shim or
  launches a canary. This is where every existing and new project
  starts.
- **auto-canary**: Mend may launch canaries within your policy, but
  promotion to a shim still needs a human.
- **autopilot**: Mend may launch canaries and promote winners within
  your policy, hands-off.

Modes, policy, and the kill switch are covered on the
[Trust and safety](https://spanly.com/docs/mend/trust/) page.

---

## How canaries work

Source: https://spanly.com/docs/mend/canaries/

A canary is a live A/B test of one [patch](https://spanly.com/docs/mend/#patches). It runs
on your real traffic, splits sessions into two arms, measures the fault's
own metric plus a set of always-on guardrails, and reaches a verdict. The
design goals are that it concludes as early as the data allows, that it
never degrades performance without rolling back immediately, and that
every session sees a stable manifest for its whole lifetime.

## Session bucketing and arms

A canary has two arms:

- **Baseline**: your original manifest, served untouched.
- **Candidate**: the patch applied to the `tools/list` response.

Each session is assigned to an arm deterministically, with no coordination
between the collector and the backend. The assignment is a hash of the
session id and the canary id:

```
bucket = bigEndianUint64(first 8 bytes of sha256(sessionId + ":" + canaryId)) mod 10000
arm    = candidate  when  bucket < round(candidateShare * 10000)
```

A `candidateShare` of 0.5 sends buckets 0 through 4999 to the candidate
arm. The same function runs in the TypeScript SDK, the Python SDK, the Go
CLI, and the backend analysis, so every side agrees on which arm a session
was in without exchanging anything.

The arm is computed once, at a session's first `tools/list`, and pinned
for the session's lifetime along with the delivery-state snapshot it was
computed from. A session never sees the manifest change mid-conversation.

## Primary metric per fault type

Each fault type carries the one metric a canary judges the patch on. The
metric's direction rides with the measurement, so the test never guesses
whether higher or lower is better.

| Primary metric            | Direction       | Example fault                                                                 |
| ------------------------- | --------------- | ----------------------------------------------------------------------------- |
| `subject_tool_error_rate` | lower is better | Error-prone parameter (schema-valid but rejected)                             |
| `schema_violation_rate`   | lower is better | Underspecified parameters, missing `outputSchema`, schema looser than reality |
| `switch_pair_rate`        | lower is better | Tool-selection confusion between sibling tools                                |
| `retry_rate`              | lower is better | Retry storm (stuck-agent loop)                                                |

The primary metric decides whether the candidate wins. The gates decide
whether it is allowed to keep running at all.

## Gates

Gates are the always-on performance promise. They are separate from the
primary-metric verdict: the sequential test decides win or lose on the
fault's own metric, while gates are instant guardrails against collateral
damage. A breach on any gate rolls the canary back immediately, whatever
the primary metric is doing.

There are seven gates. Each compares the candidate arm against the
baseline arm and fires only when the candidate is credibly worse, so small
samples never breach on noise.

| Gate                  | What it checks                                                    | Default threshold                   |
| --------------------- | ----------------------------------------------------------------- | ----------------------------------- |
| `duration`            | Candidate subject-tool p95 duration may not exceed baseline       | 20%                                 |
| `token_spend`         | Candidate tokens per session may not exceed baseline              | 15%                                 |
| `calls_per_session`   | Candidate calls per session may not exceed baseline               | 25%                                 |
| `error_rate`          | Candidate session error rate may not exceed baseline              | 0% (any credible increase breaches) |
| `no_new_faults`       | No fault may open only under the candidate arm                    | any candidate-only fault breaches   |
| `client_family_floor` | No client family above the traffic floor may be credibly degraded | traffic floor 20%, no slack         |
| `exposure_integrity`  | Candidate sessions served without the overlay applied stay rare   | 10%                                 |

Gates need a minimum sample before they judge: at least 30 subject-tool
calls per arm for the duration gate, and at least 30 sessions per arm for
the session-level gates. Below that a gate reports insufficient data
rather than a false pass. Every credible-difference test uses a one-sided
95 percent level.

Thresholds are the defaults. A project policy can tighten them; a
malformed override is dropped rather than applied, so a bad policy row can
never widen a gate open by accident.

## Sequential testing, in plain language

The primary metric is judged by a sequential test (an mSPRT, a mixture
sequential probability ratio test). In plain terms:

- The canary is looked at once per sweep, on the cumulative data so far.
- It concludes as early as the data allows. A clear win or a clear loss is
  called quickly; a marginal effect runs longer.
- It is anytime-valid. The false-positive guarantee holds no matter how
  many times the data is peeked at, which is what lets Mend check hourly
  without inflating error.
- It is two-sided. A credible degradation triggers a revert with the same
  guarantee that a credible improvement triggers a promote.
- Any gate breach rolls back immediately, independent of the sequential
  verdict.

The defaults are a 5 percent false-positive rate and a 20 percent minimum
relative effect (the smallest change worth acting on, which the test tunes
itself to detect fastest). A look needs at least 30 sessions in each arm
to count.

Before a canary launches, a feasibility check walks the expected
trajectory and estimates how long it would take to conclude. If that lands
within about four weeks the canary runs as a standard A/B. If it would
take longer, or if there is too little traffic, Mend falls back to
before/after mode.

## Before/after mode, and why it is weaker

When traffic is too thin for a randomized split to conclude in a
reasonable time, Mend runs the patch at 100 percent and compares a window
before the patch applied against a window after. It uses Bayesian
posteriors on each window and fires when the credible interval excludes
zero.

Before/after mode carries less weight on purpose. There is no
randomization, so anything that changed with time (a traffic mix shift, a
model upgrade on the client side, a seasonal pattern) confounds the
comparison. Every verdict from this path is labeled weak evidence, and
downstream policy (autopilot, digest, badges in the UI) treats it
accordingly. A randomized canary is labeled A/B evidence.

## Custom canaries

Not every experiment starts from a scanner fault. A custom canary is a
patch you author yourself — from the **New canary** button on the
Canaries page, or over the Spanly MCP (`mend_create_custom_patch`) — and
it can do things a generated fix never will:

- **Rename a tool** (`rename_tool`). The candidate arm sees the tool
  under the new name, and calls to that name are routed back to the
  original before they reach your server, so the tool keeps working. Use
  it to test whether a clearer name changes how agents pick tools.
- **Disable a tool** (`remove_tool`). The candidate arm's `tools/list`
  omits the tool entirely. Calls are not blocked — a client that already
  knows the name still gets through — but agents that pick tools from the
  manifest stop seeing it.
- Anything a generated patch can do: rewrite tool and parameter
  descriptions, add annotation hints, constraints, or an output schema.

Your ops are checked against the server's live manifest when you create
the patch — a typo'd tool name or a rename into a name that is already
taken is refused on the spot, not silently skipped on the wire. The
static validation loop still runs, but for hand-authored patches it is
advisory: warnings are attached for review instead of auto-rejecting your
experiment.

From there it is an ordinary canary. It defaults to measuring the subject
tools' error rate, the same guardrail gates watch it, a manifest change
suspends it, and you promote or roll it back exactly like a drafted fix.
Custom patches are never auto-launched, whatever the project mode:
you created it, you launch it.

## Local runbook

For running Mend against a local server, the canary sweep is the
thing that advances every running canary one step: take a metrics
snapshot, evaluate the gates, run the sequential verdict, and act on it.

### Inspect a canary

Read a canary's current state, gate board, and latest verdict through the
Canaries view in the dashboard, or through the region tRPC procedures
`mend.canaryDetail` and `mend.canariesByProject`.

### Force-stop a stuck canary

Stop a canary immediately with the `mend.stopCanary` mutation. It takes
the `canaryId`, a short human `reason`, and an optional target status of
`rolled_back`, `concluded_revert`, or `inconclusive`. This is the manual
override when a canary is stuck or you want it gone now; it writes to the
audit log like any other Mend state change.

### Re-run a sweep

The sweep runs hourly on a cron in each region. To trigger it by hand
against a local or test region:

```bash
# Advance every running canary once and wait for the summary.
curl -X POST 'http://localhost:<api-region-port>/internal/canary-sweep?wait=true' \
  -H 'Authorization: Bearer <internal-auth>'
```

Without `?wait=true` the endpoint schedules the sweep and returns `202`
immediately. In code, `runCanarySweep({ projectId })` scopes a sweep
to one project; called with no options it sweeps every running canary
in the region.

---

## Trust and safety

Source: https://spanly.com/docs/mend/trust/

This page is written for a security reviewer. It describes the exact
boundary of what Mend can do to your MCP server, and every claim here is a
property of the shipped code, not an aspiration. Where a guarantee comes
from a specific mechanism, that mechanism is named.

The short version: Mend can only clarify or tighten what a client is told
about your tools, only on the manifest version a fix was written for, only
when you have allowed it, and never at the cost of blocking or delaying
your traffic. Everything below is how that is enforced.

## The operation allowlist

A patch is a list of typed overlay operations. There are exactly six, and
the type of a patch is a discriminated union with no other members, so a
parsed patch cannot instruct the wire layer to do anything outside this
list:

| Operation               | Effect                                                                                            |
| ----------------------- | ------------------------------------------------------------------------------------------------- |
| `set_description`       | Set a tool's description text.                                                                    |
| `set_param_description` | Set the description of one input parameter.                                                       |
| `add_annotation`        | Set one annotation hint: `readOnlyHint`, `destructiveHint`, `idempotentHint`, or `openWorldHint`. |
| `add_param_constraint`  | Set one JSON Schema keyword on a parameter: `enum`, `pattern`, `minimum`, `maximum`, or `format`. |
| `add_required`          | Mark an already-declared parameter as required.                                                   |
| `set_output_schema`     | Set a tool's `outputSchema`.                                                                      |

Two further operations exist only for patches **you** author (custom
canaries, created from the Canaries page or over the Spanly MCP). Spanly
never generates them, and the pipeline refuses a generated patch that
carries one:

| Operation     | Effect                                                                                      |
| ------------- | ------------------------------------------------------------------------------------------- |
| `rename_tool` | Serve the tool under a different name. Calls to the served name are routed to the original. |
| `remove_tool` | Hide the tool from `tools/list`. Calls from clients that already know the name still pass.  |

Every operation is a plain "set", so applying the same patch twice is a
no-op. Operations address tools by name and parameters by top-level name.

## What is deliberately impossible

For everything Spanly generates on its own — templates and the LLM
drafter — these cannot happen by construction:

- **No renames.** No generated patch changes a tool name or a parameter
  name. The `rename_tool` operation exists only for experiments you write
  yourself.
- **No removals.** No generated patch deletes a tool, a parameter, or a
  schema constraint. `remove_tool` is likewise customer-authored only.
- **No loosening.** There is no operation, generated or hand-authored,
  that widens an `enum`, relaxes a `pattern`, or makes a required
  parameter optional.
- **No touching `tools/call`.** Mend rewrites `tools/list` responses, and
  it never blocks, delays, or drops a `tools/call`. The single exception
  exists to keep your own experiments working: while a session is served a
  `rename_tool` you authored, a call to the served name has its `name`
  field mapped back to the original before it is forwarded. Arguments,
  ids, and everything else pass through byte-identical, and sessions
  without a served rename are never touched.

Because the manifest tells the client what the server can do, and the
server behind it is untouched, the worst a generated patch can do is
describe your tools more precisely than your manifest already does. A
custom canary can go further — that is its point — but only because you
wrote it, and it rolls back like any other canary.

## Fail-open guarantees

Mend never blocks or delays your MCP traffic and never depends on Spanly
being reachable. Any Mend error results in your original manifest being
served untouched. Concretely:

- **No delivery state means pass-through.** Until the collector has
  fetched a delivery-state document, and any time it has none, manifests
  are served exactly as your server produced them.
- **All fetches are background.** The collector reads only an in-memory
  snapshot synchronously on the hot path. Fetching delivery state is
  fire-and-forget and never delays a flush or an MCP message.
- **Every fetch failure keeps the last known-good state.** A missing
  endpoint, a network error, a 10-second timeout, malformed JSON, a `404`
  for a stale digest, or a digest that does not match the document it
  addresses all leave the current state in place. A digest that just
  failed is not retried for a cool-down window, so a broken digest cannot
  turn every flush into a failing request.
- **The rewrite itself cannot throw.** Rewriting a `tools/list` response
  is synchronous local JSON work. Any unexpected shape or internal error
  returns the original message object unchanged.
- **A patch that no longer parses is simply not served.** On the backend,
  composing the delivery-state document skips any patch whose stored
  operations no longer validate, rather than failing the whole document.
- **A missing Pulse header is never an error.** When the sync header is
  absent, the collector keeps its current state and re-reads the digest on
  a later flush.
- **The Pulse header costs one cache read.** On the ingest hot path,
  attaching the header is at most a single Redis read; a miss or error
  skips the header and the client keeps its state.

The [Patch delivery](https://spanly.com/docs/pulse/) page documents the sync mechanism and its
fail-open rules in full.

## Manifest-hash anchoring

Every patch is authored against exactly one version of your manifest,
identified by its content hash (the same `toolListContentHash` the scanner
already stores). When the collector is about to apply a patch, it hashes
the tool list your server actually returned and compares:

- **Hash matches**: the patch is applied to a private deep copy of the
  response.
- **Hash does not match**: the patch is not applied, and the response is
  served as your server produced it.

This is the drift-safety invariant. If you change your manifest, a patch
written against the old version stops applying on the wire instantly, and
server-side the affected shim is auto-suspended and reclassified on the
next scan (see the [Upstreaming guide](https://spanly.com/docs/mend/upstreaming/)). A patch
can never land on a manifest version it was not written for.

## Session stickiness

A session never sees the manifest change mid-conversation. The
delivery-state snapshot and the canary arm are pinned at the session's
first `tools/list` and reused for the session's entire lifetime,
regardless of how the project's state moves afterward. This holds even
for a session that started before the collector finished its first fetch:
it stays pass-through for its whole life.

## Exposure logging

Spanly records what was actually served, not what was assigned. Each
rewritten `tools/list` response carries a record of the shims applied, the
canary and arm if any, the delivery-state digest in effect, and whether
the overlay actually applied. When a patch does not apply (for example a
hash mismatch), that is recorded as such, and those sessions are excluded
from candidate metrics. Canary verdicts and gates read from this
served-truth log.

## Kill switch

Every project has a kill switch, off by default. When it is on, the
delivery-state document carries no shims and no canaries, and the
collector serves every manifest untouched.

- **Who can flip it.** The kill switch and all Mend settings are gated to
  your organization's admins in the Spanly app. This is your app admin,
  not Spanly staff.
- **How fast it takes effect.** Flipping the kill switch refreshes the
  delivery state through the same single choke point every Mend state
  write uses, so the digest changes at once. Every collector picks up the
  new digest on its next flush and fetches the empty state. The
  propagation bound is one Pulse cycle, which is one flush. New sessions
  stop receiving any overlay within that bound; sessions already in flight
  keep the manifest they were pinned to, by session stickiness, until they
  end.

## Modes and defaults

Nothing changes on your wire without an action you took. Every project
starts in **suggest** mode, with the kill switch off and no policy. In
suggest mode Mend opens faults and drafts patches and stops there: a shim
or a canary only exists because a person applied a patch or launched a
canary. Moving to **auto-canary** or **autopilot** is an explicit,
audited choice that widens what Mend may do within a policy you set
(severity ceiling, maximum concurrent canaries, which operation kinds are
allowed, and gate thresholds).

## The audit log

Every Mend state write flows through a single choke point that records an
audit event and refreshes the delivery-state cache in the same step, so a
change can never be recorded by one and forgotten by the other. That
includes a shim activated or suspended, a canary started, stopped, or
rolled back, the kill switch toggled, and settings changed. Each event
records the actor, the action, the subject, an optional detail payload,
and a timestamp, scoped to the project. You can read it from the Mend
audit view in the dashboard.

## What Spanly can and cannot see or change

**Can see.** Spanly sits in-line on your JSON-RPC stream through the SDK or
CLI you run, so it observes your MCP traffic. That is the observability
product, and it is unchanged by Mend.

**Can change.** Only `tools/list` responses, only through the operations
above, only when the manifest hash matches, and only when your mode and
your actions have allowed a shim or canary to exist. Tool names and tool
presence change only through a patch you authored yourself.

**Cannot change.** Server behavior, parameter names, the looseness of a
schema, or the outcome of any `tools/call` (a hand-authored rename maps
the called name back to the original; nothing else is touched). It cannot
apply a patch to a manifest version it was not written for, and it cannot
serve anything while the kill switch is on.

---

## SDK and CLI reference

Source: https://spanly.com/docs/mend/reference/

Mend needs no configuration in your collector. The TypeScript SDK, the
Python SDK, and the Go CLI all sync Mend delivery state automatically over
the same connection they already use to ship telemetry, using the same API
key and base URL. There is nothing to install and nothing to turn on.

This page covers the wire mechanism, how to see what is actually being
served, and how to disable Mend.

## The Pulse header and delivery state

Every collector keeps an in-memory copy of its project's
delivery-state document: the active shims, the running canaries, and the
kill switch. It learns when that document changes from the `Spanly-Pulse`
response header on ordinary ingest responses, and fetches the new document
in the background.

The mechanism, the endpoints, and the fail-open rules are documented once
on the [Patch delivery](https://spanly.com/docs/pulse/) page. In short:

- Every authenticated ingest response carries `Spanly-Pulse: <digest>`,
  the content hash of the current delivery state.
- `GET /mend/state/current` returns `{ "digest": "<digest>" }`.
- `GET /mend/state/:digest` returns the delivery-state document, content
  addressed and immutable.

Both endpoints live on the same base URL as `/collect` and use the same
`Authorization: Bearer <api key>` header, so they add no new auth surface.

Fetching runs in the background and never blocks or delays a flush or an
MCP message. Any failure keeps the last known-good state, and with no
state at all the collector serves your manifest untouched. See
[Trust and safety](https://spanly.com/docs/mend/trust/) for the full list of fail-open
guarantees.

## Verifying what is served

Mend is designed so you never have to trust it blindly. Two views tell you
exactly what is on your wire:

- **Delivery state, from the source.** Call `GET /mend/state/current` to
  get the current digest, then `GET /mend/state/:digest` with your API key
  to read the exact document your collectors are consuming: every active
  shim, every running canary, and the kill-switch flag. This is the same
  bytes the SDK and CLI fetch.

  ```bash
  DIGEST=$(curl -s https://ingest.us.spanly.com/mend/state/current \
    -H "Authorization: Bearer $SPANLY_API_KEY" | jq -r .digest)
  curl -s "https://ingest.us.spanly.com/mend/state/$DIGEST" \
    -H "Authorization: Bearer $SPANLY_API_KEY" | jq
  ```

  Use `ingest.eu.spanly.com` for an EU-region key.

- **Exposure log, from the dashboard.** Every rewritten `tools/list`
  response is logged with the shims applied, the canary and arm if any,
  and whether the overlay actually applied. The Canaries view renders
  from that served-truth record, so you can see what real sessions
  received rather than what was intended.

## Disabling Mend

The way to disable Mend for a project is the **kill switch**. Flip it
from the Mend settings in your project (organization-admin only). While it
is on, the delivery-state document your collectors fetch carries no shims
and no canaries, so every manifest is served untouched. The change reaches
your collectors within one Pulse cycle, which is one flush; see the
kill-switch section of [Trust and safety](https://spanly.com/docs/mend/trust/) for the
propagation bound and how in-flight sessions are handled. A project
left in the default **suggest** mode also serves nothing automatically,
because no shim or canary exists until a human creates one.

**Note.** There is currently no environment-variable opt-out to disable
Mend directly in the SDK or CLI process. Control is server-side: the kill
switch (and suggest mode) fully disable serving for a project. A
dedicated in-process opt-out is a planned follow-up.

---

## Upstreaming guide

Source: https://spanly.com/docs/mend/upstreaming/

A [shim](https://spanly.com/docs/mend/#shims) is a stopgap. It makes clients see a better
manifest while your server still produces the old one. The end state for
every shim is that you ship the same change in your own source, so your
server serves the improved manifest directly and the shim is no longer
needed. This page is how you get there.

Once you upstream a fix, Mend detects it on the next scan, retires the
shim, and resolves the underlying fault as `fixed_at_source`. Your wire
goes back to pure pass-through for that fix.

## Read the diff

Each shim carries the patch it applies: the tools it targets and the exact
list of operations. Open the shim in the Canaries view to see, per tool, what
the overlay changes relative to your live manifest. Every operation maps
to a concrete edit in your tool definition:

| Operation               | What to change in source                                                                                       |
| ----------------------- | -------------------------------------------------------------------------------------------------------------- |
| `set_description`       | Set the tool's description.                                                                                    |
| `set_param_description` | Set the parameter's description in its input schema.                                                           |
| `add_annotation`        | Set the annotation hint (`readOnlyHint`, `destructiveHint`, `idempotentHint`, or `openWorldHint`) on the tool. |
| `add_param_constraint`  | Add the JSON Schema keyword (`enum`, `pattern`, `minimum`, `maximum`, or `format`) to the parameter.           |
| `add_required`          | Add the parameter to the tool's required list.                                                                 |
| `set_output_schema`     | Declare the tool's `outputSchema`.                                                                             |

None of these change what your tool does. They change what your manifest
tells a client about it, which is exactly the change a shim is already
serving.

## Apply it in your MCP server

The edit lives wherever you declare your tools. The examples below use the
official MCP SDKs.

### TypeScript SDK

If you register tools with `server.registerTool`, the description,
annotations, and schemas all live in the registration call. For example,
applying a `set_description` and an `add_annotation` (`readOnlyHint`):

```ts
server.registerTool(
  'search_orders',
  {
    description: 'Search orders by customer, date range, or status. Read only.',
    annotations: { readOnlyHint: true },
    inputSchema: {
      customerId: z.string().describe('The customer id to search within.'),
      status: z.enum(['open', 'shipped', 'cancelled']),
    },
  },
  handler,
);
```

An `add_param_constraint` becomes a tighter Zod (or JSON Schema) type on
the parameter (`z.enum([...])`, `.regex(...)`, `.min(...)`, `.max(...)`).
An `add_required` means the parameter is no longer optional. A
`set_output_schema` becomes an `outputSchema` on the registration.

### Python SDK

With `FastMCP`, the description comes from the docstring or the `@tool`
argument, and parameter constraints come from your type hints and
`Field(...)` declarations:

```python
@mcp.tool(annotations={"readOnlyHint": True})
def search_orders(
    customer_id: str = Field(description="The customer id to search within."),
    status: Literal["open", "shipped", "cancelled"] = "open",
) -> OrderPage:
    """Search orders by customer, date range, or status. Read only."""
    ...
```

A `Literal` or an `Enum` applies an `enum` constraint, `Field(pattern=...,
ge=..., le=...)` applies `pattern`, `minimum`, and `maximum`, a
non-defaulted parameter is required, and a typed return value plus
`output_schema` applies the output schema.

## What happens after you ship

When your server starts serving the improved manifest, the tool-list
content hash changes. On the next scan Mend sees the new manifest and does
two things:

- **The shim retires.** Because a shim is anchored to the manifest version
  it was written for, a changed manifest suspends it immediately on the
  wire (the drift-safety invariant on the
  [Trust and safety](https://spanly.com/docs/mend/trust/) page). Mend then classifies each
  suspended shim: if your new manifest already contains the fix, the shim
  is marked upstreamed; if the fix still applies cleanly to an unrelated
  change, it is re-anchored to the new version; if the tool it patched is
  gone, it is retired.

- **The fault resolves.** The fault the shim was carrying is resolved with
  reason `fixed_at_source`. It leaves your open faults because your server
  now does the right thing on its own, not because Mend is masking it.

You do not need to tell Mend that you upstreamed a change. Detection is
automatic on the next scan. If you want to record the handoff explicitly
(for example the moment you merge the change), you can mark a shim as
upstreamed from the Canaries view, but shipping the manifest is what actually
retires it.

---

## Patch delivery

Source: https://spanly.com/docs/pulse/

Pulse is the sync mechanism between the Spanly ingest API and the
collectors (TypeScript SDK, Python SDK, and CLI). It tells a running
collector when the Mend delivery state for its project has
changed, without adding any request, latency, or failure mode to data
ingestion.

## The Spanly-Pulse header

Every ingest API response for an authenticated project carries:

```
Spanly-Pulse: <digest>
```

The digest is a content hash of the project's current delivery
state (the active patches being served). It changes if and only if the
delivery state changes.

Collectors read the header on every flush and compare it to the digest
of the state they currently hold:

- **Same digest**: nothing to do.
- **Different digest**: fetch the new state in the background with
  `GET /mend/state/:digest`.
- **Header absent**: keep the current state. The server omits the
  header when the digest is not cached yet or the cache is
  unreachable; it recomputes in the background and the header returns
  on a later response.

The header is set on all responses where the API key resolved to an
project, including error responses such as `400` and `503`, so a
failed flush can still deliver a fresh digest. Responses rejected
before authentication (`401`, or `503` while key validation is
unavailable) carry no header.

## Fetching delivery state

Both endpoints live on the same base URL as `/collect` and use the
same `Authorization: Bearer <api key>` header.

### GET /mend/state/:digest

Returns the delivery-state document when `:digest` is the
project's current digest. The response is content-addressed and
immutable (`Cache-Control: immutable, max-age=31536000`), so it can be
cached forever.

If the digest is no longer current, the endpoint returns `404` with
`{ "error": "unknown_digest" }`. The collector keeps its current state
and picks up the fresh digest from the `Spanly-Pulse` header on its
next flush, or resyncs through `/mend/state/current`.

### GET /mend/state/current

Returns `{ "digest": "<digest>" }`, the project's current digest.
Use it to resync after a `404`, or at startup before any flush has
observed a Pulse header.

## Digest computation

The digest is the lowercase hex SHA-256 of the delivery-state document
serialized as canonical JSON: object keys sorted ascending by UTF-16
code units, no whitespace, arrays kept in order, members with
undefined values dropped. Every producer and consumer derives the same
digest from the same document, which is what makes the fetch URL
content-addressed.

## Fail-open rules

Pulse never interferes with monitoring or with your MCP traffic:

- A missing header means keep the current state; it is never an error.
- State fetches run in the background and never delay a flush.
- A failed state fetch leaves the current state in place; the
  collector retries on a later flush.
- If the collector cannot obtain any delivery state, it serves your
  server's manifest untouched.

---

## Shared dashboards

Source: https://spanly.com/docs/shared-dashboards/

A shared dashboard exposes a project's dashboard at a stable,
read-only URL that you can hand to people without a Spanly account.

## How it works

1. Open **Settings → Share links** in your project.
2. Create a link, optionally with a label and an expiry date.
3. Send the `/share/<token>` URL to your viewers.

The token is unguessable; anyone with the URL can view. You can revoke
or expire a link at any time, and revoked links stop working
immediately.

## What viewers see

Viewers get the project dashboard: request, error, and duration
charts, plus the per-server and per-tool breakdowns. They cannot:

- See other projects.
- Change filters that would widen the view.
- Access raw request payloads or settings.

## Availability

Share links are live today on the Pro plan and above, and can also be
managed through the [Spanly MCP server](https://spanly.com/docs/mcp/tools/#shared-links).
Password protection and viewer allowlists are not available yet; if
you need them, email
[support@spanly.com](mailto:support@spanly.com).

---

## Organisations

Source: https://spanly.com/docs/organisations/

A Spanly **organisation** holds your billing relationship and one or
more **projects**. Each project has its own members list and role
assignments, so production data can stay separate from a sandbox
project while sharing the same billing.

## Roles

| Role   | View dashboards | Change settings | Manage members | Manage billing |
| ------ | --------------- | --------------- | -------------- | -------------- |
| Owner  | yes             | yes             | yes            | yes            |
| Admin  | yes             | yes             | yes            | no             |
| Member | yes             | no              | no             | no             |

The user who created the organisation is the initial owner. There
must always be at least one owner. Promote someone else first if you
want to leave.

## Inviting members

1. Open **Settings → Members** in your project.
2. Click **Invite** and enter the email address.
3. Pick a role. The invitee gets an email with a magic link to accept.

Invites that have not been accepted appear under **Pending** and can
be revoked at any time.

## Per-project membership

A user added to one project does not automatically gain access to
other projects in the same organisation. Add them to each project
they need to see. This keeps blast radius small for contractors,
support staff, or external collaborators.

## Availability

The three-role model, invites, and per-project membership are live
today. If you need finer-grained permissions, for example read-only
access to billing or per-server scoping, email
[support@spanly.com](mailto:support@spanly.com).

---

## Troubleshooting

Source: https://spanly.com/docs/troubleshooting/

If you've followed the [quickstart](https://spanly.com/docs/) and nothing is showing
up in the dashboard, walk through these checks in order.

## Nothing appears in the dashboard

### 1. Is the API key set?

```bash
echo $SPANLY_API_KEY
```

If empty, set it:

```bash
export SPANLY_API_KEY=spanly_us_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

A missing key is loud in all three integrations: the TypeScript and
Python middleware throw at construction, and the CLI exits with
`SPANLY_API_KEY environment variable is required`. If your process
starts cleanly, the key was set; keep walking the checklist.

### 2. Is the prefix correct?

The region is encoded in the prefix:

- `spanly_us_…` → `https://ingest.us.spanly.com`
- `spanly_eu_…` → `https://ingest.eu.spanly.com`

Any other prefix raises `Invalid API key format` at construction in
both SDKs. Generate a fresh key in the dashboard if unsure.

### 3. Does the request path match `paths`?

`spanly()` and `SpanlyMiddleware` only inspect requests whose path
starts with one of `paths` (default `/mcp`, `/sse`). If your MCP
server is mounted somewhere else, pass a matching list:

```ts
app.use(spanly({ apiKey: process.env.SPANLY_API_KEY, paths: ['/api/mcp'] }));
```

```python
app.add_middleware(
    SpanlyMiddleware,
    api_key=os.environ["SPANLY_API_KEY"],
    paths=["/api/mcp"],
)
```

Requests outside every prefix pass through with zero engine
involvement, so this fails silently rather than throwing.

### 4. Is anything actually being called?

Run any MCP client request: `tools/list`, a `tools/call`, a `prompts/get`.
If nothing is exercised, nothing shows up. The dashboard updates within
a few seconds of the first request.

### 5. Is the network egress allowed?

The SDK / CLI POST to `https://ingest.<region>.spanly.com`. If your
host is behind a strict egress firewall:

- Allow outbound HTTPS to `ingest.us.spanly.com` and / or
  `ingest.eu.spanly.com`.
- The SDK never opens inbound connections.

For diagnostic visibility, enable the error hook:

```ts
app.use(
  spanly({
    apiKey: process.env.SPANLY_API_KEY,
    onError: (err) => console.error('spanly:', err),
  }),
);
```

```python
app.add_middleware(
    SpanlyMiddleware,
    api_key=os.environ["SPANLY_API_KEY"],
    on_error=lambda e: print("spanly:", e),
)
```

### 6. Is a CLI wrapper actually proxying?

When using `spanly run --port 3000`, the wrapper takes port 3000 and
the child gets a random port. If you accidentally point the MCP client
directly at the child port, the wrapper is bypassed and nothing is
captured.

Confirm the child's port via the CLI logs (it prints the assigned port
at startup), then verify your MCP client is on the wrapper port.

## Requests appear but the body is missing

The middleware falls back to status and headers only, with no request
or response body, whenever a response carries a `Content-Encoding`
header: the bytes reaching the middleware are already compressed, and
parsing them as JSON-RPC would just fail. Mount `spanly()` (or add
`SpanlyMiddleware`) before `compression()` (or your framework's
equivalent) to capture bodies too.

Body parser order does not have this effect either way: `spanly()`
tees the raw request stream when mounted before a JSON body parser
like `express.json()`, and reads the already-parsed body when mounted
after it.

## Duration looks wrong (always 0 ms, or huge)

Spanly pairs each response with its request by JSON-RPC `id` at ingest
and computes duration from the captured timestamps. If you see 0:

- The request/response pair may not have been matched by JSON-RPC
  `id`. Notifications don't have a response and report no duration.
- For SSE streams, each `data:` frame is its own packet, so what you
  see is per-frame time-to-first-byte, not the lifetime of the stream.

## Dropped packets under ingest pressure

The CLI has a bounded in-memory buffer (default 10,000 packets,
`--buffer-size`) for periods when ingest is unreachable; when it fills,
the oldest packets are dropped and the `/metrics` endpoint counts them.
The SDKs don't buffer: each packet is delivered asynchronously with
retries, and failures surface through the `onError` / `on_error` hook.

- Check `onError` (TS) / `on_error` (Python) or the CLI logs for the
  underlying network error.
- If ingest is healthy and the CLI still drops packets, raise
  `--buffer-size` or shard traffic across more instances.

## CLI: HTTP mode doesn't work behind nginx/Caddy/Envoy

SSE responses stall in the front proxy's response buffer. Disable
buffering on the relevant route. See
[Production deploy → SSE pass-through](https://spanly.com/docs/cli/production/#putting-spanly-behind-nginx--caddy--envoy)
for sample configs.

## Spanly MCP returns 401

The [Spanly MCP server](https://spanly.com/docs/mcp/overview/) authenticates with OAuth, not
your API key.

- Re-run your client's sign-in flow (in Claude Code: `/mcp` → spanly →
  authenticate).
- If the sign-in window never opens, update the MCP client. OAuth
  support for HTTP servers is recent in some clients.

## Still stuck?

- Open an issue at
  [github.com/spanlyhq/spanly](https://github.com/spanlyhq/spanly/issues).
- Join the [Discord](https://discord.gg/r4U3hPhM).
- Email support@spanly.com with the SDK or CLI version, the failing
  command, and any `onError` output.

---

## Migrating to sessionless MCP (2026-07-28)

Source: https://spanly.com/docs/sessionless-migration/

The MCP `2026-07-28` spec release makes the protocol stateless at the
transport layer. It is a clean break rather than a deprecation: existing
servers keep working on the older protocol versions through version
negotiation, and the ecosystem supports the `2025-11-25` generation for at
least a year. But new servers, and servers that adopt the v2 SDKs, work
differently enough that a migration is real engineering work. This guide
covers what changed, a checklist for server teams, and where Spanly fits.

Every claim below traces to a linked SEP or the official release post.
Read those before making changes; this page is an orientation, not a
substitute for the spec.

## What changed and why

- **Sessions are gone** ([SEP-2567](https://modelcontextprotocol.io/seps/2567-sessionless-mcp)).
  The `Mcp-Session-Id` header is removed. New servers do not mint or echo
  it, and `GET`/`DELETE` on the MCP endpoint return `405`. Cross-call state
  moves to "explicit state handles": a tool result carries an opaque id, and
  the model threads it back into later tool calls. Handles are invisible to
  the protocol, so a gateway or middleware cannot see them.
- **The initialize handshake is gone** ([SEP-2575](https://modelcontextprotocol.io/seps/2575-stateless-mcp.md)).
  Instead, every request carries `_meta` keys
  `io.modelcontextprotocol/protocolVersion`,
  `io.modelcontextprotocol/clientInfo`, and
  `io.modelcontextprotocol/clientCapabilities` (all required; a missing set
  is a `400`). A `server/discover` RPC returns what the old initialize
  response carried. The long-lived `GET` SSE stream is replaced by a
  `subscriptions/listen` POST whose response is an event stream.
  `Last-Event-ID` resumability, `ping`, and `logging/setLevel` are removed;
  the log level is now a per-request `_meta` opt-in and servers must not
  emit `notifications/message` without it.
- **Header standardization** ([SEP-2243](https://modelcontextprotocol.io/seps/2243-http-standardization)).
  `MCP-Protocol-Version` (every POST), `Mcp-Method` (every request), and
  `Mcp-Name` (`tools/call`, `resources/read`, `prompts/get`) are mandatory
  and mirrored from the body, so gateways route and authorize without
  parsing the body. Servers must reject a header/body mismatch with
  `HeaderMismatchError` (`-32020`). Non-ASCII header values use a
  `=?base64?...?=` sentinel.
- **Server-initiated requests are multi round-trip**
  ([SEP-2322](https://modelcontextprotocol.io/seps/2322-MRTR)). A tool can no
  longer `await` an elicitation mid-execution. It returns
  `resultType: "input_required"` with an opaque `requestState`, and the
  client retries the same method with the answer plus the echoed
  `requestState` under a new request id.
- **Sampling, Roots, and Logging are deprecated**
  ([SEP-2577](https://modelcontextprotocol.io/specification/draft/deprecated)),
  with at least a 12-month window.
- **List results must declare caching**
  ([SEP-2549](https://modelcontextprotocol.io/seps/2549)). `ttlMs` and
  `cacheScope` (`"public"` or `"private"`) are required on `tools/list`,
  `prompts/list`, `resources/list`, `resources/read`, and
  `resources/templates/list`, and a list result must not vary per
  connection. Variance by authenticated principal is allowed; deterministic
  `tools/list` ordering is recommended.
- **Full JSON Schema 2020-12** ([SEP-2106](https://modelcontextprotocol.io/seps/2106))
  in `inputSchema`/`outputSchema`: composition (`oneOf`/`anyOf`/`allOf`),
  conditionals, and internal `$ref` are all legal.
- **W3C trace context** ([SEP-414](https://modelcontextprotocol.io/seps/414))
  reserves `traceparent`, `tracestate`, and `baggage` in `_meta`.

Source: the [2026-07-28 release candidate post](https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/)
and the [SDK v2 beta post](https://blog.modelcontextprotocol.io/posts/sdk-betas-2026-07-28/).

## Migration checklist for server teams

1. **Upgrade to the v2 SDK.** The maintainers stated that "existing code
   keeps compiling" is a non-goal, so plan for source changes rather than a
   drop-in bump.
2. **Move session state to explicit state handles.** Anywhere you relied on
   `Mcp-Session-Id` to key server-side state, mint an opaque handle in the
   tool result and read it back from the next call's arguments:

   ```ts
   // Before: state keyed on the session id.
   // After: the tool returns a handle the model threads forward.
   async function openDataset(args) {
     const handle = createHandle(); // e.g. "ds_a1b2c3"
     store.set(handle, await loadDataset(args));
     return { content: [{ type: 'text', text: `Opened as ${handle}` }] };
   }

   async function queryDataset(args) {
     const dataset = store.get(args.handle); // handle came from the model
     return { content: [{ type: 'text', text: run(dataset, args.query) }] };
   }
   ```

   Treat a handle as model-visible: it can appear in transcripts and be
   replayed, so scope it to the authenticated principal and expire it.

3. **Stop varying `tools/list` per connection.** A toolset that changes as a
   side effect of other calls (for example a `connect_database()` that
   reveals more tools) is forbidden. Serve identical list results per
   authenticated principal, and set `cacheScope: "private"` when the list is
   principal-specific.
4. **Replace Sampling with direct LLM calls.** Call your model provider's
   API from the server instead of asking the client to sample.
5. **Refactor mid-call elicitation** to the `input_required` +
   `requestState` shape, or gate the feature on a client capability during
   the transition. Against a dual-era server, old clients keep working and
   simply lose elicitation until they upgrade.
6. **Emit and validate the new headers.** Mirror `Mcp-Method`/`Mcp-Name`
   from the body and reject a mismatch with `-32020`.
7. **Add cache directives** (`ttlMs`, `cacheScope`) to every list result,
   and sort list results deterministically.
8. **Move server logs off the MCP channel.** Emit to stderr (stdio) or your
   APM/OTel pipeline, and only send `notifications/message` when a request
   opted in via the per-request log level.

Timeline facts: the spec is final on `2026-07-28`; the `2025-11-25`
generation is supported for at least a year; new clients fall back to
`initialize` against old servers, but old clients cannot talk to a
modern-only server, and a modern-only client cannot talk to a legacy
server. Expect mixed-era fleets for the whole window.

## How Spanly helps

Spanly's producers capture both protocol eras with no configuration
change. The CLI, the Docker sidecar, and the TypeScript and Python
middleware read raw request and response frames, so they capture legacy
`initialize` traffic and modern per-request `_meta` identity the same way.
On the product side:

- **Protocol version per server and session.** The server dashboard shows
  which protocol versions your fleet is on, and for a mixed fleet it breaks
  the traffic down by client family, which is the go/no-go signal for
  dropping legacy support. Session detail shows each session's version.
- **Migration-readiness scan checks.** The static and live scans flag the
  concrete blockers: a session-based protocol version with the migration
  ahead of it, Sampling/Roots/Logging usage that SEP-2577 deprecates,
  `tools/list` results that vary per connection, missing cache directives,
  mid-call elicitation that SEP-2322 reshapes, clients leaning on the
  removed SSE resumability, and version-negotiation failures from a client
  family that cannot reach your server.
- **Sessionless traffic stays grouped.** For servers with no session id,
  Spanly synthesizes a telemetry-only session key from the request's
  credential and idle gap, so sessions, dashboards, and session-scoped
  checks keep working. See [Session tracking](https://spanly.com/docs/session-tracking/).

None of this requires you to migrate first. Spanly captures the old and
new protocols side by side, so you can watch your fleet's era mix move and
verify each migration step against real traffic.

# Blog

> Every post from https://spanly.com/blog/, newest first.

---

## Agentjacking: when your telemetry becomes a prompt-injection vector

Source: https://spanly.com/blog/agentjacking-telemetry-prompt-injection/

Published 2026-06-26 · By Tim Quinteiro

A developer asks their coding agent to debug a production error. The agent pulls the error from Sentry through an MCP server, reads the suggested fix, and runs it. The fix was written by an attacker, and the command shipped the developer's AWS keys out the door. No exploit and no malware, just a string in a log the agent trusted.

That is agentjacking. The attack is narrow. The class of problem behind it is not.

## The Sentry attack

In June 2026, Tenet Security demonstrated agentjacking via Sentry DSN injection. A Sentry DSN is a write-only credential meant to be public; it ships in frontend code so browsers can report errors. Tenet found 2,388 organizations with exposed DSNs, which is DSNs working as designed.

Using that public write access, they POSTed a fake error event with shell commands hidden in its Resolution field, formatted to read like Sentry's own remediation advice. When a developer later asked their agent to debug, the agent fetched the error through an MCP server, treated it as authoritative, and ran the command under the developer's credentials. Across more than 100 agent instances against Claude Code, Cursor, and Codex, this worked about 85 percent of the time, reaching environment variables, AWS keys, GitHub tokens, and repository URLs. The agent runs as the developer, so EDR, WAF, and IAM see a trusted user running a trusted tool.

## Bigger than one tool

Sentry shipped a filter for the demonstrated payload. The risk does not live in any one product, though. It lives in the agent's assumption that tool output carries the same authority as its operator's instructions. Sentry was only the first carrier demonstrated. The same shape works through any tool that relays third-party-writable content: a support desk whose tickets customers write, an issue tracker with outside comments, an analytics tool surfacing user request strings, a web-fetch tool relaying the open internet.

The next carrier will be a different tool and the next payload a rephrase of the last. A defense tuned per tool and per payload always trails the attack. It has to sit where every tool's output reaches the agent, which is the MCP protocol.

## How Spanly surfaces it

Agentjacking is hard to stop at the model, but it leaves an obvious trace. The signature is a pivot: a read or fetch tool returns external content, and the same session immediately makes an exec, shell, or write call built from it. Untrusted content in, privileged action out, two calls apart.

That sequence is what Spanly records. It captures MCP traffic at the transport layer: every tool call, its arguments, the response, the client, and the session. The read-to-exec pivot is not a metric you build, it is the raw sequence of calls in a session, captured the moment you put Spanly in front of your server. Endpoint and network tools see an ordinary command. A manifest scan sees ordinary tools. The pivot is visible only at the protocol layer.

Some of it shows up before runtime. Spanly's [MCP scanner](https://spanly.com/scan/) already flags tool poisoning, deceptive naming, shadowing, embedded secrets, and weak auth. Agentjacking adds one question to ask of any tool: does it relay third-party-writable content as output? A Sentry-style errors tool does, and that is a property of the manifest, not of any single request.

![Spanly MCP scan results, with the indirect prompt-injection surface check expanded to show two tools that relay third-party content to the model](./agentjacking-scan.png)

_A scan of a support-desk MCP server. The prompt-injection surface check flags `fetch_issue` and `get_error`, the tools that hand outside content straight to the model._

## What to do

There is no single fix. Treat tool output as data and never as instructions, which is a prompt and agent-design problem; Tenet published hardening for Cursor and Claude Code. Scope credentials and gate privileged actions so a successful injection has a small blast radius. And watch the protocol, because some injections will land and the read-to-exec pivot needs to be on record when they do.

Agentjacking is a new name for an old problem: injection, a tool that relays attacker-writable content, and an agent willing to act on it. You cannot filter your way out. Treat every tool's output as untrusted, and keep a protocol-level record of what your agents do with it.

## Keep reading

- [MCP observability vs APM](https://spanly.com/blog/mcp-observability-vs-apm/): why HTTP telemetry cannot see this and the protocol layer can.
- [Scan an MCP server](https://spanly.com/scan/): the free manifest-level view, no sign-up.
- [Live demo dashboard](https://app.spanly.com/share/demo): session-level tool-call sequences
  on real data.

Tim

---

## MCP OpenTelemetry tracing vs Spanly: what each one actually captures

Source: https://spanly.com/blog/mcp-opentelemetry-tracing-vs-spanly/

Published 2026-06-11 · By Tim Quinteiro

In June 2026, the MCP Python SDK shipped built-in OpenTelemetry tracing, following [SEP-414](https://modelcontextprotocol.io/seps/414-request-meta). If your team already runs OTel, the obvious question is whether that covers MCP observability now. It covers something real, but something narrow, and it is worth being precise about which question each tool answers.

## TL;DR

They answer different questions. The SDK's OpenTelemetry tracing creates lightweight spans around each MCP request and propagates W3C trace context, so the MCP hop joins your application's existing trace in Datadog, Jaeger, or any OTel backend. It captures almost no MCP-specific data, and it only produces output if you run an OTel stack on both sides.

Spanly records the wire itself: every JSON-RPC packet with full payloads, paired into requests, grouped into sessions, with no OTel setup and no code in your handlers. The two link up through `traceparent`, which Spanly preserves on every captured packet so you can deep-link into your APM.

## Side by side

| Dimension                     | MCP SDK OTel tracing                 | Spanly                                                             |
| ----------------------------- | ------------------------------------ | ------------------------------------------------------------------ |
| Language coverage             | Python SDK only (TypeScript: none)   | TS SDK, Python SDK, CLI for any language                           |
| Integration                   | OTel provider, exporter, and backend | One-line `.monitor()` or CLI wrap, nothing in handlers             |
| Payloads                      | None (method name and request id)    | Full params, results, and errors, up to 16 MiB                     |
| Notifications and progress    | Not modeled (request spans only)     | Captured, with a dedicated notifications view                      |
| Sessions and client analytics | No (one trace per request)           | Yes (sessions, client identity, per-client views)                  |
| Cross-service continuity      | Yes, joins the app's whole trace     | No, observes the MCP hop only                                      |
| Inside-the-handler detail     | Yes, via app-added child spans       | No, the handler is a black box                                     |
| Server crash mid-request      | Span likely lost (in-process)        | Request packet already captured, missing response flagged `-32002` |
| Backend                       | Generic APM                          | Purpose-built MCP dashboard                                        |

## How the SDK's OTel tracing works

The Python SDK ships tracing in-process. On the client side, each outgoing request is wrapped in a CLIENT span, and the SDK injects `traceparent` and `tracestate` into `params._meta`. That is the core of SEP-414: because the carrier is a JSON-RPC field rather than an HTTP header, trace context survives stdio pipes the same way it crosses service meshes. On the server side, an OTel middleware extracts that context and opens a child SERVER span around your handler.

The spans are deliberately minimal. Each one carries exactly two attributes, `mcp.method.name` and `jsonrpc.request.id`, plus error status. No params, no results, no payloads. That is by design, and it matches the [OpenTelemetry semantic conventions for MCP](https://opentelemetry.io/docs/specs/semconv/gen-ai/mcp/), which keep request content out of spans.

Export is the host application's job. The SDK calls the OTel API, and without a configured provider and exporter, those calls are a no-op. You bring the collector, the backend, and the sampling configuration.

The TypeScript SDK has none of this today. It passes `_meta` through untouched, so a TypeScript server team that wants spans must hand-roll the instrumentation.

## How Spanly works

Spanly's instrumentation is observation, not span creation. The TypeScript SDK taps the transport streams, the Python SDK wraps the read and write streams, and the CLI sits outside the process entirely, as a stdio wrapper or an HTTP reverse proxy. Nothing runs inside your handlers, and no OTel provider is involved.

Raw packets ship to Spanly's ingest, and the heavy lifting happens server-side: requests and responses are paired by JSON-RPC id, durations are computed from packet timestamps, sessions are reconstructed, and client identity is read from the `initialize` handshake. The result is a queryable record of every exchange: tool calls with their arguments and results, notifications, errors with their payloads, and per-client analytics.

## Where OTel tracing wins

The SDK's tracing is genuinely better at two things, and Spanly does not attempt either.

**Trace continuity.** If the MCP client also runs an OTel-enabled stack, the server span parents onto the client span, and the server's own downstream calls, database queries, HTTP requests, all parent onto the MCP span. You get one trace across the whole system, from the host application through the MCP hop and into your infrastructure.

**Inside-the-handler detail.** Because it composes with everything else your team already instruments, "why was this tool call slow" can resolve to a specific query in your APM's waterfall view. Spanly sees the handler as a black box between request and response.

Two practical caveats temper this. The big one: real-world MCP clients, including Claude Desktop, Cursor, and ChatGPT, do not inject `traceparent` today. In production, the "distributed" part usually degrades to standalone server-side spans, and only if your server happens to be Python with OTel wired up. The second: because spans deliberately exclude payloads, an OTel trace tells you a `tools/call` errored in 240ms, but not what arguments triggered the error or what came back.

## Where Spanly wins

Spanly wins on everything protocol-shaped: what actually crossed the wire. That includes full request and response payloads, malformed traffic, notifications and progress updates, server-initiated requests like sampling and elicitation, and orphaned requests that never got a response, which are flagged with error code `-32002`. Because capture is independent of your handler completing, a server crash mid-request still leaves the request packet in your dashboard, with the missing response called out.

It also wins on adoption cost. There is no OTel stack to deploy, the CLI covers any language without code changes, and the experience is uniform across an SDK ecosystem where tracing support is anything but. We go deeper on the protocol-vs-platform split in [MCP observability vs APM](https://spanly.com/blog/mcp-observability-vs-apm/).

## Use both: traceparent is the join key

Spanly intentionally does not emit OTel spans. Instead, it preserves the inbound `traceparent` verbatim on every captured packet, and that powers the APM integrations: from any request in Spanly, you can jump to the matching trace in Datadog, Sentry, or New Relic.

For a SaaS team with an existing APM, the coherent setup is both layers. Your APM, optionally fed by the SDK's OTel tracing if you are on Python, explains what happened inside your code. Spanly explains what happened on the MCP surface: payloads, sessions, clients, protocol errors. The `traceparent` is the join key between the two, so neither layer is a silo.

## Where this is heading

SEP-414 is new, and it will likely spread beyond the Python SDK over time. If MCP clients start injecting trace context, the linking gets more valuable, because more requests arrive with a join key to your APM. What does not change is the data each layer holds: spans will still carry method names and ids, not payloads, sessions, or client analytics, and in-process tracing will still not cover the languages and servers you cannot instrument. Pick the tracing layer for continuity, pick the packet layer for the protocol, and let `traceparent` connect them.

## Keep reading

- [MCP observability vs APM](https://spanly.com/blog/mcp-observability-vs-apm/): the same question one level up, including what APM vendors' own MCP support covers.
- [How to monitor your MCP server in production](https://spanly.com/blog/monitor-mcp-server-in-production/): the practical setup guide.
- [What is MCP observability?](https://spanly.com/mcp-observability/): the category, defined.
- [Live demo dashboard](https://app.spanly.com/share/demo): see the packet-level view on real
  data.

Tim

---

## EU AI Act traceability for MCP tool calls

Source: https://spanly.com/blog/eu-ai-act-mcp-traceability/

Published 2026-06-04 · By Tim Quinteiro

If your MCP server sits inside a product that European customers use, the EU AI Act is now part of your engineering reality, not just your legal team's. The Act phases in obligations over several years, and a recurring theme across them is traceability: the ability to reconstruct what an AI system did and why. For an MCP server, the things that need to be traceable are tool calls. This is a practical look at what that means and how to get ahead of it.

A note before we start: this is engineering guidance, not legal advice. Whether and how the Act applies to your specific product is a question for your counsel. What we can speak to is the technical capability the Act keeps asking for, and how MCP monitoring provides it.

## What the Act keeps asking for

Across the high-risk provisions, the same capability appears in different words: record-keeping and logging that let you reconstruct the operation of the system over its lifecycle. The Act expects automatic logging of events, retention of those logs for an appropriate period, and enough detail to support oversight and post-incident analysis.

Strip out the legal phrasing and you are left with a set of engineering requirements that should feel familiar:

- Events are logged automatically, not on a developer's good intentions.
- Logs are detailed enough to reconstruct what happened.
- Logs are retained for a defined period.
- Access to those logs is controlled and auditable.
- Data handling respects where it is allowed to live.

For an MCP server, "events" means tool calls, resource reads, and the sessions that tie them together. Those are the actions an AI system takes through your server, and they are exactly what needs to be reconstructable.

## Mapping it to MCP tool calls

Tool calls are where the AI system does things. A traceability story for an MCP server has to cover them concretely:

**What was called, and with what.** The tool name and the arguments. A log that says "a tool was called" is not reconstructable. One that preserves the tool and its inputs is.

**What came back.** The response, or the error. Whether the call succeeded at the protocol level, not just the HTTP level.

**Who and when.** Which client, which session, and the timestamp. Client and session identity are what let you reconstruct a sequence of actions rather than isolated events.

**Linked context.** The ability to connect an MCP tool call back to the wider request in your own systems, so the trace is not an island.

If you can answer those for any tool call after the fact, you have the substance of what the traceability provisions are reaching for.

## Retention, residency, and access

Three operational details turn "we log things" into "we can stand behind our logs."

**Retention.** Logs need to survive long enough to be useful for oversight and investigation. Pick a retention period deliberately and make sure your tooling actually holds data that long. Spanly retains telemetry by plan: 30 days on Free, 90 days on Pro, and 12 months on Business, with custom retention for Enterprise.

**Residency.** For EU customers, where the data lives matters. Spanly runs two independent regions, US and EU, with separate storage and separate API endpoints. EU workspaces stay on EU-resident infrastructure, and data does not cross regions. The API key carries the region (`spanly_eu_...`), so the SDK routes correctly without extra configuration. This is the default architecture, not a bolt-on.

**Access control.** Logs that anyone can read or quietly alter do not support an audit. Workspace access is gated by SSO where configured, and every administrative action against a workspace is recorded in an audit log the owner can export. That export is often the artifact a reviewer actually wants to see.

## A practical checklist

If you are getting ahead of this, here is the engineering checklist:

1. **Every tool call is logged automatically**, with tool name, arguments, response or error, client, session, and timestamp. No manual instrumentation that can drift.
2. **Logs are retained for a defined, documented period** that matches your obligations.
3. **EU customer data stays in the EU**, with residency you can point to rather than promise.
4. **Access to the logs is gated and audited**, with an exportable audit trail.
5. **MCP traces link back to your wider systems** via trace context, so a reviewer can follow an action end to end.
6. **You can produce a record on demand** for a given session or time window without an engineering project.

A drop-in SDK plus protocol-native monitoring gets you most of this without building a logging pipeline from scratch:

```ts
import { SpanlyClient } from '@spanly/sdk';

const spanly = new SpanlyClient({
  apiKey: process.env.SPANLY_API_KEY,
});

spanly.monitor(mcpServer);
```

Tool calls, sessions, and client identity are captured automatically from that point, in the region your key is scoped to.

## Why this is worth doing now

The Act's obligations land on a timeline, and the practical work of building reconstructable logging takes longer than the compliance memo suggests. Teams that wait until a requirement is immediate end up retrofitting traceability under deadline pressure, which is the worst time to discover that your logs do not actually contain what you need.

Putting protocol-level monitoring on your MCP server now means the traceability capability is already there when you need to demonstrate it, and in the meantime you get the operational benefit of actually being able to debug your server. The compliance posture is a byproduct of good observability, not a separate project.

## Related reading

- [How to monitor your MCP server in production](https://spanly.com/blog/monitor-mcp-server-in-production/)
- [MCP observability vs APM](https://spanly.com/blog/mcp-observability-vs-apm/)
- [Security at Spanly](https://spanly.com/security/): data residency, encryption, and retention in detail.
- [Live demo dashboard](https://app.spanly.com/share/demo)

Tim

---

## How to monitor your MCP server in production

Source: https://spanly.com/blog/monitor-mcp-server-in-production/

Published 2026-06-02 · By Tim Quinteiro

You shipped an MCP server. Real agents are calling it now. So the question stops being "does it work" and becomes "how do I know when it doesn't, and how fast can I find out why." This is a guide to monitoring an MCP server in production: what to instrument, which numbers actually tell you something, and how to wire up alerts that fire on the things you care about.

It is written for engineering teams who already run a general-purpose APM (Datadog, Sentry, New Relic) and have now added an MCP surface that the APM does not understand. If that is you, start here.

## Why an MCP server needs its own monitoring

An MCP server is not a normal HTTP service. From the outside it looks like one request in, one response out. Inside, every exchange is a JSON-RPC message: a tool call, a prompt fetch, a resource read, an initialization handshake. Those are the units of work that matter, and they are invisible to anything that only sees the HTTP envelope.

When something breaks, the questions are MCP-shaped:

- Which tool was called, and with what arguments?
- Which client made the call: Claude Desktop, Cursor, Codex, or an agent you have never heard of?
- Did the response actually come back, or did the model get an error it quietly swallowed?
- Is this the same session that failed yesterday?
- Why is one customer seeing a 30% error rate when everyone else is fine?

You can answer some of these with logs and a generic APM, but you will spend more time mapping MCP concepts onto HTTP spans than debugging. The point of MCP monitoring is to work in the units your protocol actually uses.

## What to instrument

Five things cover almost every production question.

**Tool calls.** The center of everything. Capture the tool name, the arguments, the response (or the error), and the duration. Most incidents are a single tool misbehaving for a single class of input, and you cannot see that without the arguments next to the timing.

**Resource reads.** Resources are reads against your data. They fail differently from tools: permissions, missing URIs, oversized payloads. Track them separately so a spike in resource errors does not hide inside your tool numbers.

**Prompts.** If your server exposes prompts, capture which ones are fetched and how often. A prompt that suddenly stops being requested usually means a client changed behavior.

**Sessions.** A session is the thread that ties a sequence of calls to one client connection. Session-level grouping is what lets you reconstruct "what was this agent actually trying to do" instead of staring at isolated calls.

**Client identity.** Which client, which version. This is the single most useful breakdown dimension in practice, because regressions almost always correlate with a client release, not with your own deploy.

## The metrics that actually matter

Volume and a single global error rate are where most teams stop. They are not enough. The metrics that change how you operate are the ones broken down by dimension.

**Error rate by client and by tool.** A flat 2% error rate is meaningless. A 2% rate that is actually 40% on one tool for one client version is an incident. Always look at error rate sliced by tool and by client.

**Duration percentiles per tool (p50 / p95).** Averages hide the calls that matter. Track p50 and p95 per tool. The p95 on your slowest tool is usually what an agent experiences as "the server is slow."

**Session error concentration.** Are errors spread evenly, or concentrated in a handful of sessions? Concentrated errors point at a specific client or a specific workflow; spread-out errors point at your server.

**Throughput by tool over time.** A tool whose call volume drops to zero is often a more urgent signal than an error, because it means a client stopped calling it entirely, silently.

## Setting it up

There are two ways to get this data into Spanly. Pick whichever fits how you ship.

The SDK is a few lines in your server, for TypeScript and Python:

```ts
import { SpanlyClient } from '@spanly/sdk';

const spanly = new SpanlyClient({
  apiKey: process.env.SPANLY_API_KEY,
});

spanly.monitor(mcpServer);
```

If you cannot or do not want to touch the server code (third-party servers, containerized stacks, anything you do not own), the CLI wraps or proxies it with no code change:

```bash
export SPANLY_API_KEY="spanly_us_..."

# wrap your MCP server (stdio or HTTP)
npx -y @spanly/spanly run -- node ./server.js
```

Either way, tool calls, sessions, and client identity start showing up within seconds. The [TypeScript quickstart](https://spanly.com/docs/typescript-sdk/quickstart/) and [Python quickstart](https://spanly.com/docs/python-sdk/quickstart/) walk through the full setup.

## Alerting: fire on what you care about

Monitoring you have to remember to look at is monitoring you will not look at. Set alerts so the system tells you. A few that earn their keep:

- **Error rate over a threshold in a short window**, for example error rate above 10% over the last 5 minutes. This catches a bad client release or a downstream dependency failing.
- **p95 latency on a critical tool above a ceiling.** Slow is the failure mode agents notice first.
- **Throughput collapse.** A tool that drops to zero calls when it normally runs steadily.

Each rule can fan out to email, Slack, and signed webhooks, so the alert lands wherever your on-call already lives. Start with two or three rules and tune the thresholds against a week of real traffic rather than guessing up front.

## Keep your APM, add the MCP layer

None of this replaces your existing observability stack. Your APM still owns HTTP, infrastructure, and the rest of your service. MCP monitoring sits alongside it and adds the protocol layer your APM cannot see. Because Spanly propagates the W3C trace context on inbound MCP requests, every view links straight back to the matching trace in Datadog, Sentry, or New Relic. We go deeper on that split in [MCP observability vs APM](https://spanly.com/blog/mcp-observability-vs-apm/), and on the category itself in [what MCP observability is](https://spanly.com/mcp-observability/).

## Try it

- [Sign up](https://app.spanly.com/sign-in): free tier, no card.
- [Live demo dashboard](https://app.spanly.com/share/demo): real data, public.
- [Docs](https://spanly.com/docs/): install in two minutes.

Tim

---

## MCP observability vs APM: what still falls through the gap

Source: https://spanly.com/blog/mcp-observability-vs-apm/

Published 2026-05-28 · By Tim Quinteiro

If you already run Datadog, Sentry, or New Relic, you might reasonably ask why your MCP server needs anything else. You instrument everything else with your APM. Why not this?

The short answer: out of the box, your APM sees HTTP and infrastructure, not the MCP protocol. The vendors know it, and some now ship MCP instrumentation in their SDKs. That support is real and worth understanding. This post is about what it covers, what still falls through the gap, and why the answer is "both," not "either."

## What your APM sees

A general-purpose APM is excellent at what it was built for. It sees:

- HTTP requests and responses, status codes, and route-level latency.
- Infrastructure: CPU, memory, container health, database queries.
- Stack traces when your process throws.
- Distributed traces across your services, stitched by trace context.

For an MCP server, that means your APM can tell you the `POST /mcp` endpoint returned 200 in 180ms. Without protocol-aware instrumentation, it cannot tell you what happened inside.

## What it misses

An MCP exchange is a JSON-RPC message: a tool call, a prompt fetch, a resource read. One HTTP request can carry a tool call that failed at the protocol level while the HTTP layer reports a clean 200. That is the core of the problem. Here is what your APM cannot answer from the HTTP layer alone:

**Which tool was called, with what arguments.** To the APM it is an opaque request body. To you, the tool name and arguments are the whole story.

**Whether the tool call actually succeeded.** MCP errors live inside the JSON-RPC response. A tool can return an error object inside a 200 OK. Your APM counts that as a success. Your customer's agent counts it as a failure.

**Which client made the call.** Claude Desktop, Cursor, Codex, Windsurf, or some agent you have never seen. Client identity is the dimension that explains most regressions, and it is buried in a header your APM does not break out.

**Session continuity.** MCP work happens across a session. APM traces are per-request. Reconstructing "what was this agent trying to do" from per-request HTTP spans is painful.

**Per-tool performance.** Your APM gives you route-level latency. But `POST /mcp` is one route carrying twenty different tools with wildly different performance profiles. The average is a lie.

## A worked example

A customer reports that "the agent keeps failing." You open your APM. The `/mcp` endpoint shows a 99.8% success rate and a healthy p95. Nothing looks wrong. You close the ticket as "cannot reproduce."

What actually happened: one tool, `search_orders`, returns a JSON-RPC error for any query containing a date range, because of a parsing bug. That is a 200 OK at the HTTP layer every single time. The error is in the response body. To your APM it is invisible. To the agent calling it, every date-range search fails.

With protocol-level monitoring, this is a thirty-second find: filter to `search_orders`, sort by error, see that every failure carries a date-range argument. Same data, completely different debugging experience, because the unit of observation is the tool call, not the HTTP request.

## Why not just add custom spans?

You can. You can manually instrument your MCP handlers with custom spans and attributes in your APM. Teams do it. Two things tend to happen.

First, it is a lot of bespoke work, and it drifts. Every new tool needs new instrumentation, and the moment someone forgets, you have a blind spot exactly where you will eventually need to look.

Second, the MCP model does not map cleanly onto a span tree. An MCP request/response is one JSON-RPC exchange: one node, not a tree of spans. When you force it into a span hierarchy, the mismatch leaks into every query you write. We learned this building Spanly, and it is why the product models MCP natively instead of dressing it up as something it is not.

## What about the APMs' own MCP support?

The vendors have started closing the gap themselves, and to be fair, it works:

- **Sentry** ships MCP server monitoring in its Node and Python SDKs. One line wraps the official MCP server, and tool calls, resource reads, and prompt fetches show up as spans, with dashboards broken down by tool, client, and transport.
- **New Relic** added MCP support to its AI Monitoring, instrumenting the MCP invocation lifecycle with waterfall views. Python agent only, for now.
- **Datadog** traces MCP in LLM Observability, but from the client side: it instruments the MCP Python client your agent uses. If you operate the server and your clients are other people's Claude Desktop and Cursor installs, it does not see your traffic.

If your server is a Node or Python process you own, and span-level data inside your existing tracing quota is what you need, these are good options. Their structural limits are the reason Spanly exists:

- **Language coverage.** The instrumentation lives in the vendor's SDK. A Go, Rust, Java, or C# MCP server gets nothing until that vendor ships an agent for it.
- **In-process only.** You have to add their SDK to the server's code. A third-party server, a vendored binary, or a sidecar you do not own cannot be instrumented.
- **Spans, not messages.** You get span attributes, subject to your tracing quota and attribute limits, not the full JSON-RPC request and response.

Spanly sits at the transport layer instead. The CLI wraps any MCP server process, stdio or HTTP, in any language, with zero code changes, and captures complete packets. A proxy mode covers servers you cannot wrap at all.

## The answer is both

This is not a replacement pitch. Keep your APM. It owns HTTP, infrastructure, and your wider service, and it does that well. Add a layer that understands the MCP protocol on top.

Spanly is additive by design. Continue sending HTTP and infrastructure telemetry to Datadog, Sentry, or New Relic, and send MCP-shaped traffic to Spanly. Captured packets preserve the W3C trace context verbatim, the `traceparent` header on HTTP transports and `params._meta.traceparent` on stdio, so the same exchange can be correlated across both systems.

The two run side by side. The APM answers "is the service healthy." Spanly answers "is the protocol healthy," and those are genuinely different questions.

## Keep reading

- [MCP OpenTelemetry tracing vs Spanly](https://spanly.com/blog/mcp-opentelemetry-tracing-vs-spanly/): the same comparison against the MCP SDK's own SEP-414 spans.
- [How to monitor your MCP server in production](https://spanly.com/blog/monitor-mcp-server-in-production/): the practical setup guide.
- [What is MCP observability?](https://spanly.com/mcp-observability/): the category, defined.
- [Live demo dashboard](https://app.spanly.com/share/demo): see the protocol-level view on real
  data.

Tim

---

## Spanly architecture

Source: https://spanly.com/blog/spanly-architecture/

Published 2026-05-22 · By Tim Quinteiro

## What is Spanly?

Spanly tells you what's happening inside your MCP server: which clients are connecting, which tools are being called, where errors happen, and how long things take. Drop in our SDK or wrap your server with our CLI, and the dashboard does the rest.

MCP observability is new enough that there isn't a canonical architecture to
point at. Most of what we ended up with came from trial and error.

## Mindset and reasoning

Our initial goal is to capture the topology of our customers' MCP servers. Things like:

- servers' name and version
- clients' name and version
- tool, resource, or prompt calls
- sessions
- notifications
- all metadata

The [MCP protocol](https://modelcontextprotocol.io/) evolves fast. For instance, MCP applications were recently added to the spec (we are working on supporting them). The common ground is that it's a [JSON-RPC](https://www.jsonrpc.org/) based protocol.

So our intuition is that the best way to capture the topology of an MCP server is to capture the JSON-RPC packets that flow through it. And then process them to build the topology.

## Architecture

### Ingestion

First, customers install the [`@spanly/sdk`](https://www.npmjs.com/package/@spanly/sdk) in their MCP server. They can also use the [`@spanly/spanly`](https://www.npmjs.com/package/@spanly/spanly) CLI to wrap their MCP server, which is useful for servers that are not in their control, such as third-party services or K8s sidecar containers.

From here, all MCP traffic is captured and sent to our ingest service. Thanks to prefixed API keys, we can route the traffic to the correct region. Routing is handled by the SDK/CLI. An API key looks like `spanly_us_...` or `spanly_eu_...`.

The ingest service writes the raw packet to Cloudflare R2 (immutable, every event, no exceptions), then looks up a Redis key of the form `event:{monitorId}:{mcpRequestId}`. If the matching half hasn't arrived yet, the current event is stashed there. If it has, the two are merged and one row lands in ClickHouse.

A sweeper walks the `event:*` keys every few minutes and flushes anything older than five minutes as an "incomplete" MCP Request, so a dropped response doesn't leak into next week.

This part will likely be revisited as we scale. Redis might have trouble keeping up with the number of events. It may be a better bet to store each event in ClickHouse and reconstruct the request/response pairs on the fly.

### Dashboard

The dashboard is a static [Vite](https://vitejs.dev) build using [React](https://react.dev) and [ShadCN](https://ui.shadcn.com), served from Render. It talks to the global API in Ohio and the regional API in the customer's region. [Next.js](https://nextjs.org) was ruled out because server-side rendering was not needed and we wanted to keep the stack simple.

Auth is handled by [better-auth](https://www.better-auth.com/), and [PostHog](https://posthog.com) is used for analytics.

When viewing a project, the dashboard reads the project ID from the URL and uses it to decide which regional API to call. A project ID looks like `us_8f2a...` or `eu_d31c...`.

Account-level things (sign-in, billing, the project list) always go to the global API in Ohio. Telemetry queries go to the regional API that matches the project prefix.

### Usage based billing

A cron job runs every hour and reports usage to Stripe. Usage is tracked by the number of requests. Since requests live in regions, we need to fetch the usage from each region and sum it up. There are also some subtleties to make sure requests are not double counted.

### Alerting

Another cron job runs every minute and evaluates alerts. Alerts are configured by the user and are based on the telemetry data. For instance, an alert can be triggered if the error rate is greater than 10% over the last 5 minutes. It's very standard alerting logic.

### Spanly MCP

We eat our own dogfood. Spanly MCP helps your agent debug your MCP server. Of course, we use Spanly to monitor our own MCP server.

### Live demo

Having a [live demo](https://app.spanly.com/share/demo) is great marketing, but it's also great for debugging. We host a private demo MCP server that is monitored by Spanly. An MCP client then queries this demo server to produce telemetry that is displayed in the live demo dashboard.

The need for a live demo is what drove the implementation of the "public dashboard" feature. The live demo is just a public instance of the dashboard. This feature is accessible to paid plans. No special case, we kinda eat our own dog food again.

## Engineering

Keep it simple, stupid. Ship fast. Watch out for agents introducing unnecessary complexity.

A single [Nx](https://nx.dev) monorepo so that agents have full visibility of the stack. This also helps with the development experience. We can run the entire stack locally and test (unit and e2e) everything in one go.

TypeScript everywhere. Each commit forces a lint, typecheck and full test run.

A very limited AGENT.md maintained by hand. It grows organically: every time a coding agent makes a recurring mistake, a line is added to correct it. Non-exhaustive list:

```
- Only comment if code is not self-documenting
- Make use of ShadCN components whenever possible, or add them using `npx shadcn@latest add <component>`
- Refer to `chart-colors.ts` for color definitions and usage
```

Misc tools: [Linear](https://linear.app), [Slack](https://slack.com), [Cursor](https://cursor.com), [Claude Code](https://claude.com/claude-code) and [Codex](https://openai.com/codex/), Google workspace.

The next section mentions all the external services we use. If an MCP exists for this service, we use it.

## What we use

**[Render](https://render.com)** runs everything, except ClickHouse. The stack is deployed via a blueprint. Each push to `main` triggers a deployment.

**PostgreSQL** holds anything transactional and bounded. Users, organisations,
projects, API keys, Stripe customer IDs. We use [Prisma](https://www.prisma.io)
for the schema and the typed client.

**[ClickHouse](https://clickhouse.com)** holds telemetry. Every Request and
every Notification is a row, with `serverName`, `serverVersion`, `clientName`,
`clientVersion` stored as `LowCardinality(String)` columns directly on the
table, and of course more data.

**Redis** is the reconciliation buffer and cache.

**[Cloudflare R2](https://www.cloudflare.com/r2/)** stores raw event payloads.

**[Stripe](https://stripe.com)** is billing. A cron rolls usage up every hour
and reports it to Stripe.

**[Sentry](https://sentry.io)** is used for error tracking.

**[Resend](https://resend.com)** sends transactional email. Welcome messages
on sign-up. Alert notifications when a customer's MCP server starts
misbehaving.

**[better-auth](https://www.better-auth.com/)** handles authentication.

**[BetterStack](https://betterstack.com)** is used for status and health monitoring.

## The things we got wrong

A few mistakes worth naming, because they're the kind of thing you don't find
in a postmortem otherwise.

We started by calling the leaf entity a "trace". That was wrong. A trace, in
the OpenTelemetry sense, is a tree of spans. An MCP request/response pair is
one node. This came with a mindset shift: drop the general telemetry angle and focus more on MCP observability.

Server/Client name and version can be any string. The initial implementation stored a hash in ClickHouse and a mapping table in PostgreSQL. However, this complex setup had tricky edge cases. We later revisited it to use `LowCardinality(String)` columns directly in the `requests` table in ClickHouse.

We started on Clerk for auth. It got us moving fast, but seat-based pricing and limited control over the org/membership model didn't fit where we were heading. We migrated to better-auth, which gave us the flexibility we needed and a cleaner ownership story over our own user data.

## What's next

Faster ingestion at scale. Deeper search.

We don't just want to monitor. We want to help you refine your MCP server so it works better for AIs. Spanly wants to be the companion of your MCP server.

We're working on analyzing a sample of packets to find inconsistencies in MCP server implementations, then come back to you with concrete improvements.

## Try it

- [Sign up](https://app.spanly.com/sign-in): free tier, no card.
- [Live demo dashboard](https://app.spanly.com/share/demo): real data, public.
- [Docs](https://spanly.com/#docs): install in two minutes.
- [Introducing Spanly](https://spanly.com/blog/launching-spanly/): the launch post.
- [GitHub](https://github.com/spanlyhq): SDK and CLI source.
- [Discord](https://discord.gg/r4U3hPhM): come say hi.

Tim

---

## Introducing Spanly: observability built for MCP

Source: https://spanly.com/blog/launching-spanly/

Published 2026-05-22 · By Tim Quinteiro

You shipped an MCP server. Your customers' agents are calling it. Tools are being invoked, sessions are being opened, errors are happening, and you have almost no idea what's going on.

That's the gap Spanly fills.

**Spanly is observability for MCP servers.** Built for MCP from the first commit, rather than general telemetry adapted to fit it.

It's live today, and there's a free tier you can actually use.

## The problem

MCP servers are production infrastructure now. They're behind agents that customers pay for, internal tools your team relies on, and increasingly, public APIs that anyone can wire into Claude or ChatGPT. But the tooling around them is still in the "console.log and pray" era.

When something goes wrong, the questions you want to answer are MCP-shaped:

- Which tool got called? With what arguments?
- Which client called it? Claude Desktop, Cursor, or some agent you've never heard of?
- How long did it take? Did the response actually come back?
- Is this session the same session that errored on us yesterday?
- Why is one customer seeing 30% error rates when the rest are fine?

You can answer some of these with generic APM, but you'll spend more time mapping MCP concepts onto span trees than actually debugging. An MCP request/response is one JSON-RPC exchange: one node, not a tree of spans. We learned that the hard way (we initially called the leaf entity a "trace", which is wrong in OpenTelemetry's sense, and the mismatch leaks into the whole product if you let it).

So we don't try to dress MCP up as something it isn't. Spanly captures the JSON-RPC packets that actually flow between client and server, and gives you the view that matches.

## Two minutes to telemetry

We have two ways in, pick whichever fits your stack:

**SDK**: a few lines in your MCP server, available for TypeScript and Python.

```ts
import { SpanlyClient } from '@spanly/sdk';

const spanly = new SpanlyClient({
  apiKey: process.env.SPANLY_API_KEY,
});

spanly.monitor(mcpServer);
```

**CLI**: `@spanly/spanly`, a single binary that wraps or proxies your MCP server. Zero code change, no language constraint. Useful when you can't (or don't want to) instrument the server directly: third-party servers, containerised stacks, or anything you don't own.

```bash
export SPANLY_API_KEY="spanly_us_..."

# wrap your MCP server (works with stdio or HTTP)
npx -y @spanly/spanly run -- node ./server.js

# or, if you can't wrap it, run as a standalone proxy
npx -y @spanly/spanly proxy localhost:3000 localhost:3001
```

Either way, telemetry shows up in your dashboard within seconds.

## What you actually get

- **Every request, searchable.** Tool name, client, session, arguments, response, latency. Filter, drill in, follow a session end-to-end.
- **Per-tool, per-client, per-session breakdowns.** See which tools are hot, which clients are misbehaving, which sessions are blowing your error budget.
- **Alerts that fire on things you care about.** Error rate over 10% in the last 5 minutes. P99 latency above a threshold. Email, webhook, more on the way.
- **Public dashboards.** Want to show a status page or a live demo? One toggle. (We use this ourselves; [our live demo](https://app.spanly.com/share/demo) is a public dashboard.)
- **Spanly MCP.** Your agent can debug your MCP server through Spanly. We use this every day on our own infrastructure.

## EU and US, separately

If you have customers in Europe, you already know the conversation. Spanly runs two independent regions (Oregon and Frankfurt) with separate storage and separate API endpoints. EU data stays in the EU. API keys carry the region (`spanly_eu_...`, `spanly_us_...`) so the SDK routes correctly without you configuring anything.

That's the default architecture, not a compliance afterthought.

## Pricing

- **Free**: real free tier, enough requests to evaluate and run small projects.
- **Pro ($49/month)**: for individual builders and small teams.
- **Business ($249/month)**: for teams running real production traffic, with longer retention and higher limits.

Usage-based overage on paid plans, no surprise invoices. Annual plans get two months free.

[Full pricing here.](https://spanly.com/pricing)

## Why we built it

I've been shipping MCP servers in production for the last year, both as part of contracting work and on agent platforms. Every single one of them eventually hit the same wall: something's off, customers are complaining, and the only debugging surface is logs scattered across containers.

I built Spanly because I wanted it for myself. It turned out other people wanted it too.

We're early. The roadmap is long and shaped by what paying customers actually need: faster ingest, deeper search, and analysis that surfaces issues in your MCP server before you go looking for them. If you're running MCP in production and any of this resonates, the fastest way to influence what we build is to be a customer.

## Try it

- [Sign up](https://app.spanly.com/sign-in): free tier, no card.
- [Live demo dashboard](https://app.spanly.com/share/demo): real data, public.
- [Docs](https://spanly.com/#docs): install in two minutes.
- [How we built it](https://spanly.com/blog/spanly-architecture/): the technical post.
- [Discord](https://discord.gg/r4U3hPhM): come say hi.

Tim

---

## Hello, Spanly

Source: https://spanly.com/blog/hello-spanly/

Published 2026-05-21 · By Tim Quinteiro

Welcome to the Spanly blog. This is where we'll share what we're building, what we've
learned, and how we think about observability for MCP servers and AI agents.

## What you'll find here

We're going to write about the things we wish someone had written down when we started:

- **Engineering deep-dives** on how we collect, store, and query traces from MCP servers
  at scale: what we picked, what we broke, and what we'd do differently.
- **Product notes** on how teams are actually using Spanly to debug agents in production.
- **The MCP ecosystem** more broadly: what's changing, what's emerging, and what we're
  excited about.

## Why a blog?

Most of what we've learned building Spanly hasn't fit in a README or a docs page. A blog
is the right shape for the longer-form things: the rationale behind a design, the story
of an incident, a benchmark we ran to settle an argument.

If you have questions, want to push back on something, or just want to chat about MCP
observability, come find us on [Discord](https://discord.gg/r4U3hPhM) or
[GitHub](https://github.com/spanlyhq).

The Spanly team