Skip to content

Confidence and Uncertainty Model

Confidence without performance

Confidence is useful only when it carries its reasons and its limits. This model turns uncertainty into something a reader can inspect, update, and communicate without false precision. It is a safeguard against the persuasive answer that has no reliable foundation beneath it.

The next chapter follows confidence back to its origins in the Source Provenance Standard.

Purpose

Defines how Stygia expresses support, uncertainty, calibration, and decision limits without turning a numerical score into authority or certainty.

Normative clauses

  • KNOW-R031: A confidence statement shall name the claim, evidence set, assessment method, assessor or process, assessment time, applicable scope, and material limitations.
  • KNOW-R032: Confidence shall remain distinct from evidence class, authority, consequence, urgency, preference, and permission. A high confidence estimate does not authorise an action.
  • KNOW-R033: Uncertainty shall identify its source where practicable, including incomplete evidence, measurement error, model uncertainty, disagreement, distribution shift, stale data, or unknown applicability.
  • KNOW-R034: A numerical confidence value shall state its interpretation, reference population or calibration basis, units or scale, and decision threshold. An uncalibrated score shall not be presented as a probability.
  • KNOW-R035: Material unknowns, unavailable evidence, contested claims, and excluded alternatives shall remain visible in the confidence record and shall bound any dependent conclusion.
  • KNOW-R036: Confidence shall be reassessed when evidence changes, a source is reclassified, a contradiction appears, the claim's scope changes, or the consequence of error materially increases.
  • KNOW-R037: Aggregation shall preserve the confidence and limitations of each material input. Correlated evidence shall not be treated as independent merely because it appears in separate records.
  • KNOW-R038: A confidence statement shall not be used to conceal a missing authority decision, human approval requirement, privacy restriction, or unresolved safety concern.
  • KNOW-R039: Model-generated confidence shall be labelled as model output, preserve minimised input and instruction lineage, and receive independent calibration or review before consequential use.
  • KNOW-R040: A decision record shall state how uncertainty affected the selected course, alternatives, safeguards, reversibility, and review trigger.
  • KNOW-R045: Confidence SHALL record claim, evidence set, assessment method, assessor, time, scope, calibration, limitations, and decision consequence.
  • KNOW-R046: Aggregated confidence SHALL preserve input correlation, missingness, disagreement, and the uncertainty introduced by aggregation.
  • KNOW-R047: A confidence state SHALL expire or be reassessed when evidence, scope, environment, consequence, or decision use changes materially.
  • KNOW-R048: Confidence SHALL NOT substitute for authority, consent, safety review, privacy control, or a required human decision.

Confidence states

Stygia may use qualitative states such as Low, Moderate, High, and Undetermined only when their definitions and assessment method are recorded. Undetermined means that the available material does not support a bounded confidence statement; it does not mean that the claim is false. Confidence states shall not replace the evidence classes in KNOW-2.

Calibration and review

Calibration compares prior confidence statements with later observed outcomes over a defined population and time horizon. It shall record selection effects, missing outcomes, changes in the underlying environment, and cases excluded from analysis. A calibration result may show overconfidence, underconfidence, or insufficient data; it shall not retroactively rewrite the original record.

Operating model and interpretation cases

Confidence review separates the claim, evidence set, method, calibration, scope, uncertainty source, decision consequence, and review trigger. A qualitative state is useful when its definition and evidence basis are visible. A numerical value is meaningful only with its interpretation, reference population, units, and calibration limits.

  • Conforming: Claim, evidence, method, calibration, scope, uncertainty, consequence, and review trigger are recorded.
  • Prohibited: Confidence is treated as permission or certainty.
  • Boundary: Undetermined confidence narrows the conclusion without declaring the claim false.
  • Failure: Distribution shift, stale data, or disagreement causes reassessment or pause.
  • Loophole: Correlated evidence is aggregated as independent support.
  • Misuse: A model score conceals a missing approval or privacy restriction.
  • Care-control: Uncertainty supports proportionate assistance while preserving consent and review.

Decision boundary

Confidence supports explanation and risk management. It does not create identity, consent, constitutional authority, access rights, or permission to act. Where uncertainty is material and recovery is difficult, the responsible authority shall prefer a reversible, bounded, or deferred course unless a separate governing rule requires otherwise.

Examples and exclusions

A well-calibrated forecast can remain wrong in an individual case. Multiple reports copied from one source are correlated evidence, not independent confirmation. This Draft does not define live model deployment, automated approval, financial risk limits, or external data collection.