Skip to main content

Conformance Requirements (§8)

GMP §8 makes the conformance suite part of the protocol. Three principles:

  1. Two-sided directional testing. Safety-critical capabilities are tested in both directions — for HARD_DELETE, both leakage (a deleted fact resurfaces) and over-restriction (a survivor is purged); for MULTI_TENANT, both cross-tenant leakage and withholding a tenant's own facts.
  2. Honest omission. A backend MUST NOT silently accept an operation for an undeclared capability; it MUST raise CapabilityNotSupported. Declaring less is honest; declaring more and failing is the error the suite exists to catch.
  3. Pass/fail via confidence intervals. A direction PASSES iff its bootstrap CI excludes the failing outcome (§8.2): leakage gate = CI upper bound ≤ EPS; recall gate = CI lower bound ≥ REC_FLOOR. With deterministic per-seed outcomes the CI is degenerate (lo = hi = mean).

Source: docs/gmp-spec-v0.2.md §8, src/aml/eval/conformance.py