AI Testing

Does your AI tool get it right?

An independent test of a chatbot or AI tool before your staff or the public rely on it — with pass/fail thresholds agreed before we start.

What We Test

AI tools rarely fail on the demo. They fail on the question nobody tried. We try those questions first.

Are answers accurate and supported by your own documents — or invented?

Does it say “I don’t know” when the answer isn’t in its sources, instead of guessing?

Does it reveal personal data — or miss personal data it was supposed to catch?

Does the same question, asked different ways or by different people, get the same quality of answer?

Step 1

Before testing, we write down the pass/fail thresholds with you — so the result can’t be argued after the fact.

Step 2

Questions drawn from real use — your logs, your documents, your staff — including ones the tool should refuse.

Step 3

Every answer is scored, and a second reviewer checks the scoring, so no result rests on one person’s judgment.

Step 4

What passed, what failed, examples of each, and what to fix first — in plain language.

A threshold memo, signed off before testing starts

Scored results for every test question

An independent report your legal, procurement, or board can read

The test set — yours to re-run after every update

A readout with your team

We never test a tool we built or sell

Price

From $7,500

One AI tool, about two weeks from data access.

Nonprofits, newsrooms, and schools pay 25% less. Larger or combined work is quoted as one fixed price, agreed before work begins.

Tell us what it does and who relies on it. We’ll reply within two business days with a proposed scope.