How Varyz is measured

• 98.6% of tasks sized right. A small model running on your own computer decides whether a task is new work or a small change to work already there. It got 68 of 69 held-out tasks right. The keyword rule it replaced got 60.9%, and chance is 50%. (25 September 2026)
• 72% of reworded questions answered, with 0 wrong answers. When an agent asks something you already decided, in different words, Varyz now answers 43 of 60 held-out questions from your earlier answer, up from 8 of 60 (13%). It gave no wrong answers, and matched none of the 25 questions that should not match. (25 September 2026)
• 100% of the right sources found. On cases where one long document crowds out the others, Varyz now returns all five relevant sources in its top five: up from 59.2% on 24 held-out cases, and from 57.5% to 100% on 24 fresh replication cases. (4 October 2026)
• 87% of real prompts read right. Varyz briefs your agent only when a prompt is work, not a question. On 188 real prompts from the developer's own sessions it now reads 86.7% correctly, up from 64.9% (p < 0.0001). Questions wrongly briefed as work fell from 56 to 11. (25 September 2026)
• 42% fewer questions to you. 14 questions instead of 24, for the same 8 questions across 3 projects. (24 September 2026)
• 0 answers shared without permission. In 6 held-out requests that were denied, revoked or over a sharing limit, Varyz shared nothing. Before the fix, all 6 got an answer. Withheld content leaked 0 times. (24 September 2026)
• 0 one-off choices leaked. No choice made for one project showed up in another project, across every run. (24 September 2026)
• 97% of answers filed right. When the owner did not say whether an answer was a habit or a one-off, Varyz filed it correctly 97% of the time (30 answers). (24 September 2026)
• 99 of 99 checklist items reached the agent. Every item on the owner's checklist reached the agent without the task mentioning any of them. (24 September 2026)
• 16 ms to answer from what you decided. When an agent asks something you have already settled, Varyz answers in a median of 16 ms (26 ms at most), measured live on the developer's own computer during real jobs, with no cloud call. (26 September 2026)

• Task sizing checks that Varyz tells new work from a small fix, so it steps in only when it should.
• Reworded questions check that a settled choice is answered from what the owner already decided, even in other words, without ever answering with the wrong decision.
• Sources check that the evidence behind an answer is found, not crowded out by one long document.
• Reading prompts checks that a question gets an answer, not a work briefing.
• Fewer questions checks that settled choices are not asked again.
• Permission checks that a denied, revoked or over-limit request gets nothing.
• No leaks checks that a choice made for one project stays with that project.
• Filing checks that Varyz tells a habit, which applies to every project, from a one-off, which belongs to one project or task.
• Checklist items check that the owner's standards reach the agent before it reports done.
• Speed checks that a settled answer comes back fast enough that the agent never waits on it.

• Most test cases are synthetic and labelled by us. The prompt reading result is the exception: it uses real prompts from one person, the developer.
• Where a test simulates an owner, the owner is simulated.
• The samples are small, from 6 to 188 cases per result.
• None of these is yet an independent study of whether agents finish real work better with Varyz. That study needs several owners, unseen tasks and blind judging, and has not been run.
• How quiet Varyz gets depends on how much you have taught it. A new install asks more at first.
• Each result describes the version tested on the date shown.

Read how Varyz works, or how it compares with CLAUDE.md, AGENTS.md and Cursor rules.