> ## Documentation Index
> Fetch the complete documentation index at: https://mcpjam-claude-peaceful-lamport-qktsqz.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Author eval cases from a document

> The sibling of `cases/generate`: generation invents cases from the suite's tools, import reads them out of something a person already wrote — a test plan, a spreadsheet of scenarios, a JSON export. MCPJam's own model turns the document into full cases (ordered steps, tool calls, assertions), so the caller does not have to guess the case shape. No format is declared: the model reads the document's shape itself, so Markdown, JSON, CSV and plain notes all work.

**Spends the organization's credits**, unlike generation, which is on MCPJam. Pass an idempotency key so a retry replays the same job instead of authoring — and billing — the document twice.

Like `cases/generate`, this discovers the suite's tools over a live MCP connection first, so the authored cases are grounded in tools the suite can actually call.

Answers `200` when the job finishes inside the wait window and `202` with a `jobId` when it does not — poll the job, then commit it. A case the commit could not write comes back under `skipped`, with a `reviewUrl` that opens the app on exactly those drafts.



## OpenAPI

````yaml /reference/openapi.json post /projects/{projectId}/eval-suites/{suiteId}/cases/import
openapi: 3.1.0
info:
  title: MCPJam API
  version: 1.0.0-preview
  description: >-
    Programmatic access to MCP servers saved in your MCPJam projects — live
    diagnostics (validate, inspect, export) and operations: call tools, render
    prompts, run eval suites asynchronously and poll their results, and import
    OAuth tokens.


    **The API is in preview**: the surface may change while we finish the
    design. Error `code` values are stable; error `message` strings are not.
    Write clients that ignore unknown response fields.
  contact:
    name: MCPJam
    url: https://github.com/MCPJam/inspector/issues
servers:
  - url: https://app.mcpjam.com/api/v1
    description: Hosted MCPJam
security:
  - bearerAuth: []
tags:
  - name: Agent browsers
    description: >-
      Isolated cloud browser sessions for coding agents. Requires authenticated
      project membership and a configured desktop runtime.
  - name: Clients
    description: >-
      Clients — the named, reusable configurations that define how MCPJam
      connects to and talks to your MCP servers. The original `/hosts` paths
      remain as deprecated, ID-only compatibility aliases with their original
      DTOs and their original (tokenless) write contracts; every alias response
      carries `Deprecation: true`. New integrations should use `/clients`.
  - name: Environments
    description: >-
      Project environments: named, live-editable execution bundles (one host, an
      optional standalone server group, optionally pinned skills and plugin
      versions) that eval suites and goals run against. Distinct from Sandbox
      images, which are Computer base images. Reads require project membership;
      every write requires project admin.
  - name: Plugins
    description: >-
      Agent Plugins imported into a project — read-only inventory and version
      detail.
  - name: Skills
    description: >-
      Cloud Skills: authored SKILL.md files stored in a project. Read-only here.
      Environments pin skills by id (`skillSelection.skillIds`) and eval runs
      pin them with `--compose-skill`, so this surface exists to give an
      unattended caller those ids; authoring is an app flow behind a beta gate.
  - name: Sandbox images
    description: >-
      Custom Computer images: a digest-pinned Dockerfile built into an immutable
      image your project's computers boot from.
  - name: Server diagnostics
    description: Connect-level health checks against a saved MCP server.
  - name: Primitives
    description: 'The server''s MCP primitives: tools, prompts, and resources.'
  - name: Export
    description: Full-server snapshots for diffing and CI.
  - name: Execution
    description: 'Run the server''s primitives: call tools, render prompts.'
  - name: Eval runs
    description: >-
      Asynchronous eval suite runs: create with 202, poll status, iterations,
      and traces.
  - name: Conformance runs
    description: >-
      Ingest MCP spec-conformance results from the SDK/CLI into project-owned
      history. Distinct from Eval runs (authored LLM cases) and from directory
      readiness.
  - name: Server connections
    description: >-
      Connect an MCP server URL to a project, authorizing in a browser when the
      server requires it.
  - name: OAuth
    description: 'Bring-your-own OAuth: import externally obtained tokens for a server.'
  - name: Studies
    description: >-
      Publish a project environment as a **study** for people outside the
      project to talk to, read what they did, and control who can reach it.
      Several of these NARROW access and take effect immediately.


      The pre-rename paths — `/projects/{projectId}/scenarios`,
      `/projects/{projectId}/scenarios/{scenarioId}`,
      `/projects/{projectId}/environments/{environmentId}/scenario` and
      `/projects/{projectId}/user-testing/scenarios/{scenarioId}/…` — keep
      working and are deliberately not documented here. They return their
      original bodies, which spell the owning id `scenarioId` and whose detail
      read carries no execution settings, and every response answers with
      `Deprecation: true` and a `Link` header naming its successor. They are
      removed at general availability; a new integration should use the paths on
      this page.
  - name: Catalog
    description: >-
      Discover the resources the other routes operate on: your account,
      projects, servers, eval suites, and chat sessions.
  - name: Tunnels
    description: >-
      Relay tunnels that expose local MCP servers through a public URL,
      registered as first-class project servers (the `mcpjam cloud tunnel` CLI
      flow).
  - name: Agent
    description: >-
      Headless agent turns over the public API: send a message history, the
      server runs one assistant turn with project-scoped workspace tools (eval
      reads + suite creation) on a pinned hosted model, and returns the reply
      plus created-resource references.
  - name: Swarms
    description: >-
      Personas, journeys and swarm containers — the authoring half of Swarms —
      plus the model-backed generation that drafts them.
  - name: Swarm runs
    description: >-
      Launching journeys and reading what they produced. Launching SPENDS — see
      the per-operation notes.
  - name: Swarm insights
    description: >-
      What a swarm run revealed. The scorecard and findings are deterministic
      and free; requesting wave insights runs models and draws on your shared
      daily ledger.
  - name: Directory readiness
    description: >-
      Grade a saved server against a publisher's listing requirements:
      Anthropic's connector directory or OpenAI's plugin directory. Reported as
      lane status and coverage, never as a numeric score, and excluded from
      `pooledConformanceScore`. Deterministic grading is free; model-backed
      experience observations are an explicit opt-in whose model cost is on
      MCPJam — no credits are consumed — and they can never decide a verdict.
  - name: Registry
    description: >-
      Search the scraped MCP directories (Claude, ChatGPT, and any future
      source), list curated/org registry cards, and install them into a project.
      Install writes a `servers` row and provenance — it does not open a live
      session. There is no catalog-uninstall route: delete the project server
      instead. Directory reads require a bearer (including minted guest tokens)
      but do not materialize a user. Card/connection reads and all writes are
      authed-non-guest.
paths:
  /projects/{projectId}/eval-suites/{suiteId}/cases/import:
    post:
      tags:
        - Eval runs
      summary: Author eval cases from a document
      description: >-
        The sibling of `cases/generate`: generation invents cases from the
        suite's tools, import reads them out of something a person already wrote
        — a test plan, a spreadsheet of scenarios, a JSON export. MCPJam's own
        model turns the document into full cases (ordered steps, tool calls,
        assertions), so the caller does not have to guess the case shape. No
        format is declared: the model reads the document's shape itself, so
        Markdown, JSON, CSV and plain notes all work.


        **Spends the organization's credits**, unlike generation, which is on
        MCPJam. Pass an idempotency key so a retry replays the same job instead
        of authoring — and billing — the document twice.


        Like `cases/generate`, this discovers the suite's tools over a live MCP
        connection first, so the authored cases are grounded in tools the suite
        can actually call.


        Answers `200` when the job finishes inside the wait window and `202`
        with a `jobId` when it does not — poll the job, then commit it. A case
        the commit could not write comes back under `skipped`, with a
        `reviewUrl` that opens the app on exactly those drafts.
      operationId: importEvalCases
      parameters:
        - $ref: '#/components/parameters/projectId'
        - $ref: '#/components/parameters/suiteId'
        - name: x-mcpjam-idempotency-key
          in: header
          required: false
          schema:
            type: string
          description: >-
            Makes a retry replay the recorded job instead of re-authoring — and
            re-billing — the document.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/EvalCaseImportRequest'
      responses:
        '200':
          description: >-
            The cases the job authored and committed, or the job's own status
            when it finished without committing (failed, cancelled, or still
            pending at the end of the wait).
          content:
            application/json:
              schema:
                oneOf:
                  - $ref: '#/components/schemas/EvalCaseImported'
                  - $ref: '#/components/schemas/EvalAuthoringJobAccepted'
        '202':
          $ref: '#/components/responses/EvalAuthoringJobAccepted'
        '400':
          $ref: '#/components/responses/ValidationError'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '403':
          $ref: '#/components/responses/Forbidden'
        '404':
          $ref: '#/components/responses/NotFound'
        '409':
          $ref: '#/components/responses/Conflict'
        '429':
          $ref: '#/components/responses/RateLimited'
        '500':
          $ref: '#/components/responses/InternalError'
        '502':
          $ref: '#/components/responses/ServerUnreachable'
        '504':
          $ref: '#/components/responses/Timeout'
components:
  parameters:
    projectId:
      name: projectId
      in: path
      required: true
      description: ID of the hosted project that contains the server.
      schema:
        type: string
    suiteId:
      name: suiteId
      in: path
      required: true
      description: Eval suite ID, as returned by `POST /eval-runs`.
      schema:
        type: string
  schemas:
    EvalCaseImportRequest:
      type: object
      required:
        - content
      properties:
        content:
          type: string
          maxLength: 102400
          description: >-
            The document, as UTF-8 text. Markdown, JSON, CSV or plain notes — no
            format is declared, because the authoring model reads the document's
            shape itself. At most 100 KiB.
        fileName:
          type: string
          minLength: 1
          maxLength: 255
          description: >-
            Recorded on each case's provenance so a reviewer can find the
            source. Optional: a pasted document has no file behind it.
        servers:
          type: array
          items:
            type: string
          description: >-
            Server ids or names to discover tools from. Mutually exclusive with
            `environmentId`.
        environmentId:
          type: string
          description: >-
            Author against this environment's closed server set. Mutually
            exclusive with `servers`.
        caseModels:
          type: array
          items:
            type: object
            properties:
              model:
                type: string
              provider:
                type: string
        duplicatePolicy:
          type: string
          enum:
            - block
            - warn
            - create_anyway
          description: >-
            Applied when the cases are written, not when the job starts.
            Anything but `block` requires `overrideReason`.
        overrideReason:
          type: string
          minLength: 1
        idempotencyKey:
          type: string
          minLength: 1
          maxLength: 256
          description: >-
            Makes a retry replay the recorded job instead of authoring — and
            billing — the document a second time.
    EvalCaseImported:
      type: object
      required:
        - generationModel
        - created
        - counts
      properties:
        generationModel:
          type: string
        created:
          type: array
          items:
            $ref: '#/components/schemas/EvalCase'
        counts:
          type: object
          properties:
            normal:
              type: integer
            negative:
              type: integer
        skipped:
          type: array
          description: >-
            Cases the commit could not write, each with its reason. They stay on
            the job rather than being lost.
          items:
            type: object
        reviewUrl:
          type: string
          format: uri
          description: >-
            Opens the app on exactly the skipped drafts. Present only when
            something was skipped — finishing them there costs nothing, whereas
            re-sending the document re-authors and re-bills every case in it.
    EvalAuthoringJobAccepted:
      type: object
      required:
        - jobId
        - status
      properties:
        jobId:
          type: string
          description: Poll this job, then commit it.
        status:
          type: string
          enum:
            - pending
            - completed
            - failed
            - cancelled
        error:
          type: string
          description: Why the job stopped. Present only on a failed job.
        generationModel:
          type: string
        created:
          type: array
          description: Always empty here — nothing is written until the job is committed.
          items:
            $ref: '#/components/schemas/EvalCase'
        counts:
          type: object
          properties:
            normal:
              type: integer
            negative:
              type: integer
    EvalCase:
      type: object
      required:
        - id
        - title
        - steps
        - iterations
        - isNegative
        - models
      description: >-
        A persisted eval case, in the public steps-first shape. Note this is NOT
        `EvalTestCase`, which is the INLINE authoring shape accepted by suite
        creation.
      properties:
        judge:
          type: object
          additionalProperties: false
          properties:
            enabled:
              type: boolean
          description: >-
            Per-case judge override. enabled: false skips goal-completion
            grading for this case.
        id:
          type: string
        declaredId:
          type: string
          description: >-
            The case's effective declared id. Absent on cases authored before
            declared identity existed.
        title:
          type: string
        steps:
          type: array
          minItems: 1
          items:
            $ref: '#/components/schemas/EvalTestStep'
          description: >-
            Ordered test steps. A `prompt` step is a model turn; a single
            model-free `toolCall` step is a render-check; `assert` steps hold
            the expectations.
        expectedOutput:
          type: string
        iterations:
          type: integer
          minimum: 1
          maximum: 10
        repetitions:
          type: integer
          minimum: 1
          description: >-
            Trials this case runs under per-case grading, overriding the suite
            default. Absent means the case inherits it. NOT a second spelling of
            `iterations`: that one is read as a FLOOR (`max(iterations,
            suite.minimumIterations)`) by a suite decided on suite-wide
            accuracy, and a per-case-graded case still reports it for
            compatibility. This one REPLACES the count rather than raising it —
            a case at 7 runs 7 times under a floor of 3 and 3 times under a
            default of 3.
        passThreshold:
          type: number
          minimum: 0
          maximum: 1
          description: >-
            Fraction of this case's trials that must pass, overriding the suite
            default. Absent means the case inherits it. Never derived from the
            suite's `minimumAccuracy`, which is a PERCENT under a different
            resolver.
        isNegative:
          type: boolean
          description: When true, the case passes if NO tools are called.
        scenario:
          type: string
        intent:
          type: string
          minLength: 1
          maxLength: 64
          pattern: ^\S(?:[\s\S]*\S)?$
          description: >-
            Optional authored analytics grouping label. Must be already trimmed;
            absent means unlabelled.
        kind:
          type: string
          enum:
            - capability
            - regression
          description: >-
            Authored case kind for the simple editor. Absent means the editor
            derives it from matchOptions.
        models:
          type: array
          items:
            type: object
            required:
              - model
            properties:
              model:
                type: string
              provider:
                type: string
        matchOptions:
          type: object
          description: >-
            Absent when the case sets none — omitted from the response rather
            than sent as `null`.
        checks:
          type: object
          description: >-
            Absent when the case sets none — omitted from the response rather
            than sent as `null`.
          properties:
            mode:
              type: string
              enum:
                - inherit
                - replace
                - extend
            list:
              type: array
              items:
                $ref: '#/components/schemas/EvalAssertion'
        import:
          $ref: '#/components/schemas/EvalCaseImportClaim'
        source:
          $ref: '#/components/schemas/EvalCaseSource'
        createdAt:
          type:
            - number
            - 'null'
        updatedAt:
          type:
            - number
            - 'null'
        suppressedSuiteStandardCheckIds:
          $ref: '#/components/schemas/SuppressedSuiteStandardCheckIds'
          type: array
          items:
            type: string
    Error:
      type: object
      required:
        - code
        - message
      properties:
        code:
          type: string
          description: >-
            Stable, machine-readable error code. New codes may be added over
            time; treat unknown codes as non-retryable failures unless the HTTP
            status says otherwise.
          enum:
            - UNAUTHORIZED
            - FORBIDDEN
            - NOT_FOUND
            - CONFLICT
            - VALIDATION_ERROR
            - RATE_LIMITED
            - FEATURE_NOT_SUPPORTED
            - SERVER_UNREACHABLE
            - TIMEOUT
            - OAUTH_REQUIRED
            - INTERNAL_ERROR
        message:
          type: string
          description: >-
            Human-readable description. May change between releases — don't
            match on it.
        details:
          type: object
          description: Optional, unstructured context bag.
          additionalProperties: true
    EvalTestStep:
      type: object
      description: >-
        One authored test step (the unified test model). `kind` discriminates:
        `prompt` is a user message (model turn); `toolCall` is a deterministic,
        model-free tool call; `interact` is one pure widget action; `assert` is
        an assertion (a `Predicate` like `toolCalledWith` / `widgetRendered`, or
        a DOM `WidgetAssertion`).
      required:
        - id
        - kind
      properties:
        id:
          type: string
          minLength: 1
        kind:
          type: string
          enum:
            - prompt
            - toolCall
            - interact
            - assert
        prompt:
          type: string
          description: 'User message (`kind: prompt`).'
        serverName:
          type: string
          description: 'Server that owns the tool (`kind: toolCall`).'
        toolName:
          type: string
          description: 'Tool name (`kind: toolCall` / `interact`).'
        arguments:
          type: object
          description: 'Tool-call arguments (`kind: toolCall`).'
          additionalProperties: true
        action:
          type: object
          description: 'Widget action (`kind: interact`).'
          additionalProperties: true
        assertion:
          type: object
          description: >-
            The rule an `assert` step checks (`kind: assert`). NOT a narrower
            spelling of a case's `checks`: this is `WidgetAssertion |
            Predicate`, and that union is why the field is called an assertion
            rather than a check. A `Predicate` is evaluated against the
            PERSISTED transcript, so a stored run can be re-graded against it
            months later; a `WidgetAssertion` is evaluated against a LIVE DOM
            and can never be replayed. Calling this a check would promise the
            replayability only one half of it has.
          additionalProperties: true
      additionalProperties: true
    EvalAssertion:
      oneOf:
        - type: object
          properties:
            type:
              type: string
              const: toolDescriptionsPresent
            minLength:
              type: integer
              exclusiveMinimum: 0
              maximum: 9007199254740991
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: toolAnnotationsPresent
            require:
              maxItems: 2
              type: array
              items:
                type: string
                enum:
                  - readOnlyHint
                  - destructiveHint
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: toolNamesUnique
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: noDeprecatedToolExposed
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: toolInputSchemasWellFormed
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: toolOutputSchemasPresent
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: toolCalledWith
            toolName:
              type: string
              minLength: 1
            args:
              type: object
              properties:
                args:
                  type: object
                  propertyNames:
                    type: string
                  additionalProperties: {}
                argumentMatching:
                  type: string
                  enum:
                    - exact
                    - partial
                    - ignore
              required:
                - args
            minCount:
              type: integer
              exclusiveMinimum: 0
              maximum: 9007199254740991
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - toolName
            - args
        - type: object
          properties:
            type:
              type: string
              const: toolCalledAtLeastOnce
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - toolName
        - type: object
          properties:
            type:
              type: string
              const: toolNeverCalled
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - toolName
        - type: object
          properties:
            type:
              type: string
              const: onlyToolsCalled
            toolNames:
              type: array
              items:
                type: string
                minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - toolNames
        - type: object
          properties:
            type:
              type: string
              const: firstToolWas
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - toolName
        - type: object
          properties:
            type:
              type: string
              const: responseContains
            needle:
              type: string
              minLength: 1
            caseSensitive:
              type: boolean
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - needle
        - type: object
          properties:
            type:
              type: string
              const: responseMatches
            pattern:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - pattern
        - type: object
          properties:
            type:
              type: string
              const: noToolErrors
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: finalAssistantMessageNonEmpty
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: tokenBudgetUnder
            tokens:
              type: integer
              exclusiveMinimum: 0
              maximum: 9007199254740991
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - tokens
        - type: object
          properties:
            type:
              type: string
              const: widgetRendered
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: widgetRenderLatencyUnder
            ms:
              type: integer
              exclusiveMinimum: 0
              maximum: 9007199254740991
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - ms
        - type: object
          properties:
            type:
              type: string
              const: widgetNoConsoleErrors
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: turnCountUnder
            turns:
              type: integer
              exclusiveMinimum: 0
              maximum: 9007199254740991
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - turns
        - type: object
          properties:
            type:
              type: string
              const: noEndingQuestion
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: toolLatencyUnder
            ms:
              type: integer
              exclusiveMinimum: 0
              maximum: 9007199254740991
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - ms
        - type: object
          properties:
            type:
              type: string
              const: toolResultContains
            needle:
              type: string
              minLength: 1
              maxLength: 1000
            caseSensitive:
              type: boolean
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - needle
        - type: object
          properties:
            type:
              type: string
              const: toolResultMatchesSchema
            schema: {}
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - schema
        - type: object
          properties:
            type:
              type: string
              const: toolResultSizeUnder
            maxBytes:
              type: integer
              exclusiveMinimum: 0
              maximum: 9007199254740991
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - maxBytes
        - type: object
          properties:
            type:
              type: string
              const: argumentsMatchToolSchema
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: noRepeatedIdenticalCall
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: toolCallCountUnder
            count:
              type: integer
              exclusiveMinimum: 0
              maximum: 9007199254740991
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - count
        - type: object
          properties:
            type:
              type: string
              const: toolCalledBefore
            toolName:
              type: string
              minLength: 1
            beforeToolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
            - toolName
            - beforeToolName
        - type: object
          properties:
            type:
              type: string
              const: noDeprecatedToolCalled
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: noDestructiveToolCalled
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: toolErrorNamesInput
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
        - type: object
          properties:
            type:
              type: string
              const: fullPageHasContinuation
            toolName:
              type: string
              minLength: 1
            role:
              type: string
              enum:
                - gating
                - advisory
                - required
            severity:
              type: string
              const: warn
          required:
            - type
      description: >-
        An SDK assertion reported through UVC. Discovery assertions require
        complete raw tool declarations; missing evidence is an evaluator error.
        Observation kinds, including noDeprecatedToolExposed, require role
        advisory. Structural schema; runtime also validates policy and unique
        required annotation keys.
    EvalCaseImportClaim:
      type: object
      description: >-
        What a converter CLAIMED about one imported case. `exact` is
        CONVERTER-CLAIMED exact — the converter says it applied a structural
        mapping rule, cited in `note`. MCPJam has NOT verified semantic
        equivalence, so user-facing copy must say "claimed exact", never
        "verified" or "accepted". Claim-only: who approved an approximation,
        when, and why is a PER-RUN decision frozen on the run
        (`ImportEligibility.approvedApproximationReceipts`), never stored on the
        case — an approval that lived on a case would outlive the run it was
        granted for and the edit that invalidated it. Approval and internal keys
        are rejected with 400, never stripped.
      required:
        - status
      additionalProperties: false
      properties:
        status:
          type: string
          enum:
            - exact
            - approximated
            - unsupported
            - unresolved
          description: >-
            `exact`: the converter claims a 1:1 structural mapping, and must
            cite it in `note`. `approximated`: behaviour was intentionally
            approximated; a human must approve it for EVERY run. `unsupported`:
            the source behaviour cannot currently be represented. `unresolved`:
            a deterministic reference does not resolve against the live target.
            A selected `unsupported` or `unresolved` case cannot run.
        sourceCaseKey:
          type: string
          minLength: 1
          maxLength: 512
          description: The case's identity in the source system, when it had one.
        note:
          type: string
          minLength: 1
          maxLength: 2000
          description: >-
            Why the status is what it is — the mapping rule cited, or what was
            lost. REQUIRED when `status` is `exact`.
    EvalCaseSource:
      type: object
      description: >-
        Where an AI-assisted Markdown case was authored from. Provenance only:
        the file, its hash, and the line range the extractor read, so a reader
        can trace a case back to its source document. It is not an import claim
        (see `EvalCaseImportClaim`) and carries no approval semantics.
      required:
        - format
        - method
        - fileName
        - fileHash
        - excerpt
        - startLine
        - endLine
        - extractorVersion
      additionalProperties: false
      properties:
        format:
          type: string
          enum:
            - markdown
        method:
          type: string
          enum:
            - ai
          description: >-
            How the case was derived from the document. `ai`: an extractor model
            proposed it and a person reviewed it before it was saved.
        fileName:
          type: string
          minLength: 1
          maxLength: 255
        fileHash:
          type: string
          pattern: ^[a-f0-9]{64}$
          description: SHA-256 of the source document, lowercase hex.
        excerpt:
          type: string
          minLength: 1
          maxLength: 102400
          description: The passage the case was extracted from.
        startLine:
          type: integer
          minimum: 1
        endLine:
          type: integer
          minimum: 1
          description: Inclusive; never less than `startLine`.
        extractorVersion:
          type: string
          minLength: 1
          maxLength: 100
    SuppressedSuiteStandardCheckIds:
      maxItems: 64
      type: array
      items:
        type: string
        enum:
          - discovery.description
          - discovery.annotations
          - discovery.collisions
          - discovery.deprecated
          - call.schema
          - response.schema
          - selection.hops
          - call.parameters
          - call.repeated
          - response.performance
          - response.size
          - response.errors
          - response.recovery
          - response.pagination
          - userValue.turns
      description: >-
        Stable standard assertion family IDs suppressed from suite defaults.
        Applies before inherit/extend/replace resolution; explicit case and step
        assertions are preserved. Omitted updates preserve; [] clears. At most
        64 IDs, normalized to unique sorted values.
  responses:
    EvalAuthoringJobAccepted:
      description: >-
        The authoring job is still running when the wait window closes. Nothing
        has been written to the suite yet: poll the job, then commit it.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/EvalAuthoringJobAccepted'
    ValidationError:
      description: Malformed body or parameters.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            code: VALIDATION_ERROR
            message: Invalid JSON body
    Unauthorized:
      description: >-
        Missing, invalid, revoked, or orphaned key (`UNAUTHORIZED`) — or the
        **target MCP server** needs an OAuth grant (`OAUTH_REQUIRED`), which is
        a property of the server, not your key.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          examples:
            badKey:
              summary: Invalid or revoked key
              value:
                code: UNAUTHORIZED
                message: Invalid API key
            oauthRequired:
              summary: Target server needs an OAuth grant
              value:
                code: OAUTH_REQUIRED
                message: Server requires OAuth authorization
    Forbidden:
      description: Key is valid but not allowed to do this.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            code: FORBIDDEN
            message: You do not have access to this project
    NotFound:
      description: Unknown project, server, or resource.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            code: NOT_FOUND
            message: Server not found
    Conflict:
      description: >-
        The resource is not in a state that accepts this write — a stale
        `expectedRevision`, a duplicate name, or an environment that cannot
        currently be launched. The request was well-formed; re-read the resource
        and retry.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            code: CONFLICT
            message: >-
              Environment changed since you loaded it (expected revision 3,
              current 5). Reload and retry.
    RateLimited:
      description: >-
        Per-key rate limit exceeded (60 requests/minute sustained, bursts up to
        10). Honor `Retry-After` and back off with jitter.
      headers:
        Retry-After:
          description: Seconds to wait before retrying.
          schema:
            type: integer
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            code: RATE_LIMITED
            message: API key rate limit exceeded. Slow down and retry.
    InternalError:
      description: Something failed on MCPJam's side.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            code: INTERNAL_ERROR
            message: Unexpected internal error
    ServerUnreachable:
      description: Could not connect to the target MCP server.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            code: SERVER_UNREACHABLE
            message: Failed to connect to server
    Timeout:
      description: The target MCP server connected but didn't respond in time.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            code: TIMEOUT
            message: Request to server timed out
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: >-
        MCPJam API key (`sk_…`). Create one at [Settings → API
        keys](https://app.mcpjam.com/settings/api-keys). Guest sessions cannot
        use the API, and API keys cannot manage other API keys.

````