Questions, answered briefly

It depends on four things: how often your AI is wrong, how well its risk score separates right from wrong answers, how many reviewed cases you have, and the cap you choose. A stricter cap, a weaker score or a short history means less, sometimes nothing yet. We never promise a share: it is measured on your data. The calculator gives a planning estimate, and the pilot readout gives the time saved after audits and upkeep.

Your reviewers keep working as today, with two additions: each verdict is recorded against the AI’s original answer (sent as is, edited or rewritten), and a small sample is reviewed twice to check that reviewers agree. We also need your definition of an error, the case types that must always go to people, and one contact person. No separate labeling project is needed. After go-live, a random sample of released cases keeps being checked.

Then nothing is released without review, and we tell you so. You also get an estimate of how many more reviewed cases it would take. The estimate is a plan, not a promise.

A pilot of 8 weeks on one flow costs EUR 12–15k excluding VAT; the price is fixed in the offer before we start. What comes after the pilot is priced after your readout, from your volume and the value it shows. See the pilot →

You decide, in plain language: an answer your own standard would have stopped, such as a wrong amount, an invented source or a promise you are not allowed to make. Your reviewers’ decisions are the answer key. We never claim the AI is right about the world. What we prove is that errors by your standard stay under your cap, with the confidence and conditions stated.

Among the cases of one type released without review, the share of errors stays under your cap with 95% confidence, as long as your traffic resembles the reviewed history: the same model, prompts and kind of cases. It is a statistical statement, not an insurance policy. Even when everything is done right, about one calibration in twenty ends above the cap. That rate counts every calibration, including those that release nothing, so a case type that only just carries its cap is more likely to be the one above it, which is one reason the error watch keeps running. With several case types, the chance that one of them ends above its cap is higher, unless a stricter level is agreed. How much is released is measured, never promised.^[F2]

During a pilot, people still review everything, so nothing rests on the cap yet. After go-live, you choose which flows to automate and at which cap. What we answer for is set out in the agreement before go-live: running the method correctly, keeping the watch on and recording every decision. Not every answer your AI gives.

Tell us before you change the model, the prompt or what counts as an error: that case type then goes back to people until it is recalibrated. For changes nobody announced, the watch has two parts. The score watch compares the mix of risk scores with the calibration history and reacts to changes that move it, such as a model swap; it needs no reviewed cases. The error watch counts errors in a random audit sample of released cases. A rise in errors that leaves the scores unchanged shows up only there, so the share you audit sets how fast it is caught. On an alarm, decisions go back to people.^[F4]

Every decision gets a signed receipt (Ed25519), linked to the previous one by a hash. Anyone with our public key can check offline that the receipts are authentic and the chain is unbroken; cutting off the newest receipts shows up only against a copy of the latest chain head kept elsewhere. A receipt records the case type, threshold, calibration and watch state as we attest them, and questions and answers appear only as keyed hashes. The check proves who signed what, not that each answer was right.

The gateway runs on infrastructure you choose: your cloud or on-premises. The readout needs only scores and verdicts, never the content of your cases, and they stay in the EU. Receipts carry identifiers and keyed hashes, not your content; the key stays with you.

It can provide evidence, not compliance. For high-risk AI systems the Act asks, among other things, for testing against metrics and probabilistic thresholds defined in advance (Article 9), declared accuracy (Article 15) and automatically generated logs that are kept (Articles 12 and 19). A declared error cap, a recorded calibration, the watch and signed decision records are evidence of that kind. Whether your system is high-risk, and whether this evidence meets your obligations, is for you and your legal advisers to decide.

For each case type we sort your reviewed cases by risk score and test shares from the safest share upward, with exact statistical tests at 95% confidence that the error rate in the share is under your cap. We release the largest share that passes in that order; if none passes, nothing is released and the case type stays with people. The smaller your history, the wider the margin the tests leave, so less can be released.

It depends on your error rate, your score and the cap, for each case type. To prove a 5% cap at all, the released share must hold at least 59 cases with no error among them (149 at a 2% cap); with some errors it takes more. In practice, a useful share usually takes from several hundred to a few thousand reviewed cases. The calculator estimates it for your numbers, or from your own reviewed history in the browser.

Models behind an OpenAI-compatible chat API that return token log-probabilities can be scored directly. For other models, a risk score you already compute, or one from an open judge model run next to your system, can be sent with each answer. How well a score separates right from wrong answers differs a lot between models and tasks, so we measure it on your data in weeks 0–2. If no score can be computed for an answer, the gateway sends it to a person.

You can. A threshold chosen so that the error rate in your history just meets the cap will often end up above the cap on new cases; the margin that exact tests add is what makes the limit hold. Exact tests of this kind are published, and open-source libraries offer them, so you can do that part yourself. What we add is the operation around it: calibration per case type with an honest “not yet”, a watch that hands decisions back to people, signed receipts, and a readout your risk owner can sign off.