Skip to content

Assurance and release

A security claim needs a defined object and supporting evidence: which specification edition, implementation version, configuration, runtime and use case were assessed, and which controls the evidence actually establishes.

The requirements below are targets for evaluation and release. They do not claim that these processes or controls are already operating across Realworld's systems.

Current evidence and its limits

Current work includes internal review and local testing of reference and demonstration code. Parts of that review and testing were produced by AI agents and remain subject to human security review. The work is not an independent security audit, penetration test, certification or assessment of a production deployment.

Testing has identified failures and incomplete coverage as well as successful checks. Some evaluation material is not yet independently reproducible. A corrected document, a passing example or a large number of test cases does not close those limitations.

An implementation that only checks the consistency of a verification-result report must not be presented as a credential-signature verifier. Likewise, a test of a reference checker does not establish the behaviour of a hosted demonstration or another codebase. See the Verification Result Contract.

Evidence required for a control claim

  • Scope: Name the implementation, version, supported profiles, runtime, configuration and interfaces. Distinguish local testing from the deployed system.
  • Test definition: Record the requirement, preconditions, synthetic input, expected result, permitted side effects and reason for that expectation.
  • Positive control: Show that valid evidence meeting the requirement is accepted by the relevant check. A checker that rejects every input is not sufficient.
  • Negative control: Change or omit the relevant property and demonstrate the required refusal or incomplete result. Rejection for an unrelated reason does not prove the intended control works.
  • Sensitivity check: Demonstrate that the test detects a deliberately disabled or incorrect control where practicable.
  • Coverage and results: Preserve failures, unsupported cases and tests not run alongside passes. Identify limits rather than converting them into successful results.
  • Reproduction: Provide permitted reviewers with pinned inputs, dependencies, commands and expected outputs. Protect sensitive material while making the claim reviewable.
  • Review: Identify who reviewed the method and evidence, what was independent and what remains unreviewed.

The suite should cover signature and key selection, issuer authority, recipient context, time and status, disclosure, linked evidence, replay where relevant, malformed input, resource limits, access control and information leakage. Each interface also needs tests for its own authentication, authorisation and data-handling boundary.

If a required input is missing or stale, a dependency is unavailable or a check is unsupported, an implementation must preserve that outcome and withhold a successful verdict that depends on it. A scanner that fails to run must not produce a clean result. Optional checks should be labelled according to the applicable profile rather than silently promoted to requirements.

Separate release decisions

  • Specification text: Clear scope, versioned requirements, review of ambiguities and security boundaries, and an explicit maturity label.
  • Reference example or demonstration: Synthetic data, visible limitations and a clear statement that it is not approved for production reliance.
  • Software package: Identified source and build, dependency inventory, integrity and provenance evidence, supported configuration, test results and maintenance responsibilities.
  • Hosted service: The deployed version and operator, access and data controls, monitoring, incident ownership, recovery and evidence for the service's stated use.

Approval of text does not approve a software release. Approval of software does not establish that a particular deployment is configured or operated correctly. Release notes must state the permitted use and any material limitations; a later version needs evidence appropriate to its changes.

An unresolved risk remains unresolved until a responsible, authorised person records its treatment. Risk acceptance should identify the owner, scope, compensating controls, expiry and review conditions. Silence and the passage of time are not acceptance.

Publication and evidence handling

Public material should explain requirements, responsibility boundaries, supported behaviour and material limitations. It may include synthetic examples, public verification keys, version identifiers and sanitised test summaries. The key-security page sets out the distinction between public verification material and signing secrets.

Before publication, the proposed process requires:

  1. An explicit list of approved files and assets, with separate review of the generated output. Search indexes, downloadable Markdown, attachments and archives are part of that output.
  2. Checks for private keys, access tokens, personal data, restricted reports and internal operational details. Scanner errors and incomplete coverage must block a clean-scan claim.
  3. Synthetic fixtures and tests that demonstrate the publication checks detect prohibited material. Test output must not reproduce the material it detected.
  4. Review of each security claim against its supporting evidence and version. Dependency checks and a source-code scan alone do not establish the security of a deployed service.
  5. Recorded publication approval and a way to identify, correct or withdraw an affected edition.

Detailed vulnerability reproductions, incident records and sensitive evidence need restricted handling. Withholding exploitation detail must not become a claim that a known material limitation is absent. Public corrections should identify the affected versions, the practical limitation and any verified mitigation, without exposing secrets or giving instructions for exploiting an unresolved issue.

Security overview · Verifier behaviour · Integration profiles · Reporting security issues