Research/Leaderboard
Reality Challenge · Leaderboard
Designed so Kybern can lose.
AI that advises on real decisions must do more than answer. It must keep missing information unknown, keep conflicts visible, abstain when it cannot know, decide when it can, and refuse to act beyond its authority. We test for exactly that, publish the scores, and say plainly what the test does not prove.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | KybernWorking prototype · October 2026 | 100.0%259 of 259 cases | |
| 2 | Naive baseline“Last record wins” summariser, the way a dashboard roll-up behaves | 28.6%74 of 259 cases | |
| – | Well-configured LLM workflowA strong modern AI baseline, configured by an independent party | Planned with partners | |
| – | Expert-written cases, scored blindCases from domain experts, scored without knowing which answer is whose | Planned with partners |
By behaviour
Eight behaviours, scored one by one.
Choose a behaviour to see what is tested and how each entrant scored.
When information is missing, Kybern must not report the condition as met. Silence is treated as unknown, never as fine.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | Kybern18 cases across 19 worlds | 100.0%18 of 18 | |
| 2 | Naive baselineSame cases, same scoring rules | 0.0%0 of 18 |
When two sources disagree, both positions must be kept and the condition marked conflicting or uncertain, not quietly resolved in favour of the newest record.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | Kybern36 cases across 19 worlds | 100.0%36 of 36 | |
| 2 | Naive baselineSame cases, same scoring rules | 0.0%0 of 36 |
When the frontline reports something the model has no category for, Kybern must flag that its model may not fit the situation.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | Kybern37 cases across 19 worlds | 100.0%37 of 37 | |
| 2 | Naive baselineSame cases, same scoring rules | 0.0%0 of 37 |
After the world changes, the answer must match the current state, not the morning report.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | Kybern37 cases across 19 worlds | 100.0%37 of 37 | |
| 2 | Naive baselineSame cases, same scoring rules | 48.6%18 of 37 |
Asked a question nothing in the World Model can answer, Kybern must abstain rather than guess.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | Kybern19 cases across 19 worlds | 100.0%19 of 19 | |
| 2 | Naive baselineSame cases, same scoring rules | 0.0%0 of 19 |
Endless caution is a failure too. When the evidence is enough, Kybern must give an answer and a usable next step.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | Kybern37 cases across 19 worlds | 100.0%37 of 37 | |
| 1 | Naive baselineSame cases, same scoring rules | 100.0%37 of 37 |
Attempts to approve a decision without authority, or to hand approval authority to an AI agent, must be refused by the platform itself.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | Kybern38 cases across 19 worlds | 100.0%38 of 38 | |
| 2 | Naive baselineSame cases, same scoring rules | 0.0%0 of 38 |
Forecasts are written before the scripted future is played, then scored against what actually happened. The naive forecast is scored the same way, and either can win.
| Rank | Entrant | Score | Pass rate |
|---|---|---|---|
| 1 | Kybern37 cases across 19 worlds | 100.0%37 of 37 | |
| 2 | Naive baselineSame cases, same scoring rules | 51.4%19 of 37 |
By world
The same engine, across every world.
Each world is a synthetic demonstration of a real kind of operation, from fleet readiness to hospital capacity. Kybern runs the same engine in all of them.
Method
How the test works, and what it does not prove.
- Cases come from the worlds themselves. For each world, the harness finds real gaps, conflicts and model limits in the loaded data, plays the scripted future in sandbox copies, and builds cases from what it finds.
- Kybern is tested through the same code people use. Abstention, decision and authority cases call Ask and the permission and delegation checks directly. The live World Model is never touched.
- The baseline is a defined rule. It reports each condition from its latest record, treats no record as no problem, takes the newer side of any conflict, answers every question, keeps its morning report, and forecasts that nothing changes.
- Forecasts are a real contest. Both forecasts are written before the future is played, and either one can win.
What this does not show. We built the harness on our own synthetic data, and many of the baseline’s failures happen by construction. This run shows that Kybern’s honesty behaviours fire consistently across worlds. It does not show that Kybern beats a modern AI workflow, or that it is right about the real world. That is what the next stage is for.
What comes next
The real test, on partners’ terms.
The next entrants on this board are added only when they have been run this way.
Cases are written by people who know the domain, not generated by us.
A well-configured modern AI workflow, set up by an independent party.
Answers are scored without knowing which system gave them, against success agreed in advance.
Where Kybern gets it wrong, we say so here, and say what we changed.
Shape the real test
Bring the cases. Agree the yardstick. See the results.
We are looking for research and pilot partners to write cases, run baselines and score blind.
- Your domain, your cases, your success measure.
- Results published, good and bad.
- Early access to Kybern for your sector.
Prefer to talk now? Call +91 98150 23001 or write to reach@requisitetech.com.