# DecGuard

> DecGuard tests, verifies and gates probabilistic AI decision models: models that answer a
> typed question (choice, noul or score) and return a probability for every possible answer.
> A YAML Decision Contract and a golden dataset produce one reproducible reliability report
> with PASS / WARN / FAIL gates and CI-friendly exit codes.

- Version 0.1.1, contract and report schema 0.1. Python 3.11+, Apache-2.0. Install with `pip install decguard`.
- Commands: `decguard validate`, `test` (with `--all`), `fuzz`, `replay`, `diff`, `report`, `check` and `run`, plus a minimal `DecGuard` Python SDK.
- Measures accuracy, calibration (ECE), confidence coverage, latency and errors; checks metamorphic properties with seeded, minimized, replayable fuzzing; compares baseline and candidate runs; analyzes collected production records offline.
- Backend neutral: offline `mock`, generic `http`, `systemone` (Kev, Jev through OpenRouter) and Python plugins. DecGuard does not serve models and needs no server, database or account.
- Source and issues: https://github.com/suffro/decguard

Every page below, concatenated as one document: https://decguard.com/llms-full.txt

Each link is the page as Markdown; the rendered page is at the same path without `.md`.

## Getting Started

- [What is DecGuard?](https://decguard.com/introduction.md): What DecGuard is, the problem it solves, and where it fits next to your decision models.
- [Installation](https://decguard.com/installation.md): Install the DecGuard CLI and Python library from PyPI or from source.
- [Quickstart](https://decguard.com/quickstart.md): Write a Decision Contract and a golden dataset, validate them, and run your first reliability test in a few minutes.

## Core Concepts

- [Decisions and results](https://decguard.com/decisions.md): Probabilistic decisions, the three decision types, the Decision Contract and the normalized DecisionResult every backend produces.
- [Golden datasets](https://decguard.com/datasets.md): The golden dataset format, labeled and unlabeled cases, metadata for segments, and how to build a dataset that catches real regressions.
- [Requirements and warnings](https://decguard.com/gates.md): How requirements and warnings turn metrics into PASS, WARN and FAIL, the implicit gates, and the exit codes CI relies on.

## Testing & Reliability

- [Golden tests and metrics](https://decguard.com/metrics.md): What decguard test does, and the exact definition of every golden-test metric - accuracy, calibration, confidence coverage, latency and errors.
- [Metamorphic properties](https://decguard.com/properties.md): Metamorphic properties - option order, label format, irrelevant and repeated context, whitespace, paraphrase, noul inversion and score monotonicity - with their limits and defaults.
- [Fuzzing and replay](https://decguard.com/fuzzing.md): Run metamorphic fuzzing, read property failures, reproduce them from the seed, minimize them, and replay stored failures against any backend.
- [Regression diffs](https://decguard.com/regression.md): Compare a baseline and a candidate model run case by case, and fail CI on answer flips, calibration loss, distribution shifts, new errors or segment regressions.

## Production

- [Production checks](https://decguard.com/production.md): Analyze decisions your application already collected, offline - calibration and reliability after deployment, baseline drift, metadata segments and production gates.
- [Policies and Python SDK](https://decguard.com/policy.md): Deterministic runtime routing on decision confidence - accept, abstain, fallback and human_review - from the CLI and the Python SDK.

## Backends

- [Backends](https://decguard.com/backends.md): How DecGuard reaches decision models - built-in providers, named backends, credentials, healthchecks, error handling and the offline mock backend.
- [System One: Kev and Jev](https://decguard.com/systemone.md): Use the built-in systemone backend with a self-hosted Kev server, Jev through OpenRouter's Decisions API, or a compatible TypeSafe System One endpoint.
- [Generic HTTP](https://decguard.com/http.md): The generic http backend, its settings, and the decguard.http/0.1 JSON protocol for connecting any decision endpoint.
- [Custom backends](https://decguard.com/custom-backends.md): Connect a model through Python - wrap a function with CallableBackend, or package a DecisionBackend plugin usable from contracts and the CLI.

## Reference

- [CLI reference](https://decguard.com/cli.md): Every decguard command with its arguments, options, examples and exit codes.
- [Decision Contract reference](https://decguard.com/contracts.md): Complete reference for the Decision Contract, schema 0.1 - every section and field, defaults, validation rules and invalid combinations.
- [Reports and formats](https://decguard.com/reports.md): The JSON reports DecGuard writes - test/fuzz reports, diff, replay and production reports - their integrity checks, and every input format at a glance.

## CI and Architecture

- [CI with GitHub Actions](https://decguard.com/ci.md): Use DecGuard as a reliability gate in GitHub Actions - golden tests and properties on pull requests, regression diffs for model upgrades, and scheduled production checks.
- [Architecture](https://decguard.com/architecture.md): How DecGuard is structured, how data flows through it, the design principles it keeps, and what it deliberately does not do.