Skip to main content

Evidence Mass and Uncertainty

Evidence mass is the quantity of resolved evidence underlying a reputation estimate. It prevents a high score based on a few outcomes from being presented as equivalent to the same score supported by a long track record.

Score and mass travel together​

In CGR, reputation for an agent in a domain is a Beta(α, β) posterior. Two numbers are always reported together:

  • Score — the posterior mean E = α / (α + β).
  • Evidence mass — n = α + β: the count of resolved observations plus the neutral prior's two pseudo-counts (so 22 resolved observations report a confidence of 24).

A score without its mass is ambiguous. 0.90 from five resolved outcomes and 0.90 from four hundred are not the same claim — one is a provisional reading, the other an established track record. CGR never lets you read the first as if it were the second, because the mass rides alongside the score at all times.

The evidence-gated ceiling​

Reporting the mass is not enough on its own — a high thin-evidence score still invites over-reading. So CGR also caps a reported score until enough outcomes have resolved:

E ← min(E, cap + ε + (1 − cap − ε)·s)

Here s rises with the number of verified outcomes and saturates once a lift threshold is reached. For a neutral agent the ceiling is:

ceiling(n) = 0.52 + 0.48·(n / 20)

So with zero verified outcomes the ceiling is 0.52; at ten it is 0.76; by twenty it releases to 1.0. A strong agent's raw posterior mean is held at the ceiling until roughly twenty resolved outcomes have accrued, then released toward its true value. Over-claiming and review-farm inflation are killed at the source, and accumulated ground truth dominates thereafter.

The benchmark (B1)​

CGR cold-start benchmark: a strong agent's reported score is held at the thin-evidence ceiling and released as resolved outcomes accrue; two agents with the same raw mean but different evidence mass report different scores.

Plot from cgr-bench (track B1), reproducible with make reproduce and asserted in CI.

The B1 track makes both effects concrete and deterministic:

  • The ceiling binds early and releases late. A strong agent is held near the thin-evidence ceiling after its first outcome, and only rises well above it once evidence accrues (ceiling 0.52 / 0.76 / 1.00 at n = 0 / 10 / 20).
  • Identical means, different mass, different report. Two agents constructed with the same raw posterior mean of 0.90 report 0.64 with five resolved outcomes versus 0.90 with forty — direct proof that score and evidence mass are inseparable.

Why this matters for trust decisions​

A relying party consuming a CGR score can act on it and on its uncertainty. A thin record signals "promising, not yet proven"; a thick record signals "dependable." Because both readings come from the same pair of numbers — and because the ceiling enforces the distinction rather than relying on the reader to remember it — a newcomer cannot present a lucky early result as an established track record.

References​

Scope: a validated method on public data — not a live outcome network.