Tender
GitHub

Verify AI-generated code.

Tender evaluates coding-agent patches against independent tests, records execution evidence, and helps developers understand what actually passed.

Each check runs in a fresh workspace built from a pinned commit. The required tests are fixed by the evaluator, not by the patch. Commands run in a network-disabled container, and the results are sealed with a SHA-256 manifest.

Explore demo View GitHub

This site is static and read-only: it shows recorded runs and reports, and nothing executes here. A pass means the configured checks passed. It does not prove the code is free of defects.

Loading a recorded report…

What Tender does

Verifies a patch

Give it a local repository, a commit, a patch from any agent or person, and acceptance criteria. It returns accepted, rejected, error, or inconclusive, with the reasons.

Evaluates an agent

Runs a bounded agent loop on pinned tasks, with tools authorized at dispatch, and verifies every candidate in a workspace the agent never touched.

Keeps the evidence

Records commands, exit codes, output, test IDs, timings, and digests. Values it never captured are shown as unknown, not as zero.

How independent verification works

Five steps, no hidden ones. The patch and the agent are untrusted, and so is the repository's own Git configuration.

Verification workflow Task or patch, then pinned repository, then isolated execution, then independent verification, then evidence report. Task or patch Pinned repository Isolated execution Independent checks Evidence report Verification workflow: task or patch, pinned repository, isolated execution, independent checks, evidence report. Task or patch Pinned repository Isolated execution Independent checks Evidence report
  1. Task or patch. An agent's output, or a .patch file from any tool. The acceptance criteria are written before it runs.
  2. Pinned repository. The commit is resolved from a sanitized copy of .git; repository hooks and helpers never run on your machine. The source is only read.
  3. Isolated execution. The patch is applied to a fresh workspace built from Git objects. Test files and test-runner settings are reset, and hidden tests are added.
  4. Independent checks. A baseline, then target and regression tests, in Docker. Required test IDs come from the base commit, so a patch cannot remove them.
  5. Evidence report. Commands, exit codes, per-test results, coverage provenance, and file hashes, with what remains unknown.

Evidence and results

Every number below is read from committed records when this page loads. Each figure names its source file. The datasets are reported separately because they were collected under different conditions.

One bug, two patches

A deterministic demonstration with no model: the first patch looks right and is rejected; the second is accepted. Both reports were produced by running scripts/run_patch_demo.py in Docker.

Loading recorded reports…

Models evaluated

Loading…

Model and provider are listed separately: openai/gpt-oss-20b is an open-weight model published by OpenAI and served here through Groq's API. The table gains a row per configuration as experiments are recorded; none are implied.

Use it on your own code

Python 3.12+, Git, and Docker. No model API key is needed to verify a patch.

1. Install and build the sandbox image
git clone https://github.com/AnvitDevadiga/Tender.git && cd Tender
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'
docker build -f Dockerfile.sandbox -t tender-sandbox:local .
2. Write criteria before looking at the patch
schema_version = 1
[task]
title = "median() is wrong for even-length input"

[[target]]
name = "even-length median"
pytest = ["tests/test_median.py"]

[[regression]]
name = "existing behaviour"
pytest = ["tests/test_core.py"]
3. Verify a patch exported from any agent
export TENDER_SANDBOX_IMAGE=tender-sandbox:local
tender verify-patch --repo ~/code/project --base HEAD \
  --patch fix.patch --criteria criteria.toml --out reports/fix-1
# exit status: 0 accepted, 1 rejected, 2 not evaluated
4. Inspect, share, and check the evidence
tender report render reports/fix-1/report.json   # Markdown
tender evidence verify reports/fix-1/evidence    # recompute hashes
TENDER_REPORT_ROOT=reports uvicorn tender.api:app
# then open http://127.0.0.1:8000/dashboard/

Where things run

Your machine

The Tender process: Git plumbing on sanitized metadata copies, patch validation, report and evidence writing, and a local API and dashboard over SQLite. It never runs repository code.

Docker

Setup, collection, baseline, target, and regression commands. Network disabled, all capabilities dropped, read-only root, non-root user, and CPU, memory, process, and time limits.

This website

Nothing executes. GitHub Pages serves recorded JSON, the dashboard bundle, and a static replay of recorded runs and reports.

Components and data flow: docs/ARCHITECTURE.md · Outcome rules: evaluation contract

Reproduce every number

From a fresh clone
python scripts/prepare_fixtures.py         # fixture commits have fixed authors and dates
pytest -q                                  # offline suite; Docker tests run when a daemon exists
python scripts/run_patch_demo.py           # regenerate the failure-and-repair reports
python scripts/build_site_data.py --check  # this page's figures, from committed records
python scripts/record_replay.py --check    # the replay rebuilds byte for byte
Recorded
Written by Tender while the run executed.
Derived
Computed from recorded values, such as counts and medians.
Unknown
Never captured. Shown as unknown, never as zero.
Verified
Re-checked in a fresh workspace at the pinned commit.

Defects found in earlier versions, and how they affect older records, are listed with dates in the evaluation errata. Historical experiment files are unchanged.

Limits of the evidence