--- title: "DecGuard" description: "Reliability testing for probabilistic AI decisions: decision contracts, calibration, metamorphic fuzzing, regression gates and offline production checks." source: https://decguard.com/ --- # DecGuard > DecGuard tests, verifies and gates probabilistic AI decision models: models that answer a > typed question (choice, noul or score) and return a probability for every possible answer. > A YAML Decision Contract and a golden dataset produce one reproducible reliability report > with PASS / WARN / FAIL gates and CI-friendly exit codes. - Version 0.1.1, contract and report schema 0.1. Python 3.11+, Apache-2.0. Install with `pip install decguard`. - Commands: `decguard validate`, `test` (with `--all`), `fuzz`, `replay`, `diff`, `report`, `check` and `run`, plus a minimal `DecGuard` Python SDK. - Measures accuracy, calibration (ECE), confidence coverage, latency and errors; checks metamorphic properties with seeded, minimized, replayable fuzzing; compares baseline and candidate runs; analyzes collected production records offline. - Backend neutral: offline `mock`, generic `http`, `systemone` (Kev, Jev through OpenRouter) and Python plugins. DecGuard does not serve models and needs no server, database or account. - Source and issues: https://github.com/suffro/decguard Every page below, concatenated as one document: https://decguard.com/llms-full.txt Each link is the page as Markdown; the rendered page is at the same path with `.html` in place of `.md`. ## Getting Started - [What is DecGuard?](https://decguard.com/introduction.md): What DecGuard is, the problem it solves, and where it fits next to your decision models. - [Installation](https://decguard.com/installation.md): Install the DecGuard CLI and Python library from PyPI or from source. - [Quickstart](https://decguard.com/quickstart.md): Write a Decision Contract and a golden dataset, validate them, and run your first reliability test in a few minutes. ## Core Concepts - [Decisions and results](https://decguard.com/decisions.md): Probabilistic decisions, the three decision types, the Decision Contract and the normalized DecisionResult every backend produces. - [Golden datasets](https://decguard.com/datasets.md): The golden dataset format, labeled and unlabeled cases, metadata for segments, and how to build a dataset that catches real regressions. - [Requirements and warnings](https://decguard.com/gates.md): How requirements and warnings turn metrics into PASS, WARN and FAIL, the implicit gates, and the exit codes CI relies on. ## Testing & Reliability - [Golden tests and metrics](https://decguard.com/metrics.md): What decguard test does, and the exact definition of every golden-test metric - accuracy, calibration, confidence coverage, latency and errors. - [Metamorphic properties](https://decguard.com/properties.md): Metamorphic properties - option order, label format, irrelevant and repeated context, whitespace, paraphrase, noul inversion and score monotonicity - with their limits and defaults. - [Fuzzing and replay](https://decguard.com/fuzzing.md): Run metamorphic fuzzing, read property failures, reproduce them from the seed, minimize them, and replay stored failures against any backend. - [Regression diffs](https://decguard.com/regression.md): Compare a baseline and a candidate model run case by case, and fail CI on answer flips, calibration loss, distribution shifts, new errors or segment regressions. ## Production - [Production checks](https://decguard.com/production.md): Analyze decisions your application already collected, offline - calibration and reliability after deployment, baseline drift, metadata segments and production gates. - [Policies and Python SDK](https://decguard.com/policy.md): Deterministic runtime routing on decision confidence - accept, abstain, fallback and human_review - from the CLI and the Python SDK. ## Backends - [Backends](https://decguard.com/backends.md): How DecGuard reaches decision models - built-in providers, named backends, credentials, healthchecks, error handling and the offline mock backend. - [System One: Kev and Jev](https://decguard.com/systemone.md): Use the built-in systemone backend with a self-hosted Kev server, Jev through OpenRouter's Decisions API, or a compatible TypeSafe System One endpoint. - [Generic HTTP](https://decguard.com/http.md): The generic http backend, its settings, and the decguard.http/0.1 JSON protocol for connecting any decision endpoint. - [Custom backends](https://decguard.com/custom-backends.md): Connect a model through Python - wrap a function with CallableBackend, or package a DecisionBackend plugin usable from contracts and the CLI. ## Reference - [CLI reference](https://decguard.com/cli.md): Every decguard command with its arguments, options, examples and exit codes. - [Decision Contract reference](https://decguard.com/contracts.md): Complete reference for the Decision Contract, schema 0.1 - every section and field, defaults, validation rules and invalid combinations. - [Reports and formats](https://decguard.com/reports.md): The JSON reports DecGuard writes - test/fuzz reports, diff, replay and production reports - their integrity checks, and every input format at a glance. ## CI and Architecture - [CI with GitHub Actions](https://decguard.com/ci.md): Use DecGuard as a reliability gate in GitHub Actions - golden tests and properties on pull requests, regression diffs for model upgrades, and scheduled production checks. - [Architecture](https://decguard.com/architecture.md): How DecGuard is structured, how data flows through it, the design principles it keeps, and what it deliberately does not do.