Why frontier coding models regress on multi-file software—and how a deterministic MCP server helps break the loop.

AI coding assistants can write working code in seconds. The trouble starts as a project grows. An assistant is a little like a gifted contractor who arrives each morning without the building’s blueprints: it does excellent work on the wall in front of it, but it can’t always see the plumbing behind that wall, and it may not remember yesterday’s change of plans. The research below measures exactly that gap.

What the studies found, in everyday terms. (1) Working is not the same as safe. In CWEval, leading models produced code that passed its tests about 80% of the time, but only about half of it was also free of well-known security flaws. BaxBench found exploitable weaknesses in roughly half of the backend programs that otherwise worked. (2) Long histories cause forgetting. In LongMemEval, a leading model answered 87% of questions correctly when handed just the relevant facts, but only 61% when it had to dig them out of a long conversation history—especially when facts had changed along the way. (3) Assistants can be misled by what they read. AgentDojo showed that instructions hidden inside documents or tool results can steer an assistant off task.

Why better instructions alone don’t fix this. Rules written into a prompt are suggestions, and they compete with everything else the model is reading. As a project grows, written rules get diluted or overlooked. The dependable fix is the one engineering has always used: check the work itself, automatically, every time.

What Resonant does about it. Resonant is a set of small, automatic checks that an AI assistant calls while it works. They don’t ask another AI for an opinion; they run ordinary code, so the same input always gets the same answer, typically in a few thousandths of a second.

• A building inspector before the drywall goes up (resonant_verify_cegar): before a change is saved, Resonant runs the edited function with tricky inputs—an empty list, a zero, a missing value—and compares the old and new behavior, flagging crashes, broken promises to other code, and common security shortcuts.
• The right folder, not the whole filing cabinet (resonant_architect_orient): instead of feeding the assistant entire files, it hands over the function being changed, the code that depends on it, and the files that have historically changed alongside it—about 87% less reading in our benchmarks, which saves cost and keeps attention on what matters.
• Sticky notes that fall off when things change (resonant_memory_jtms): design notes are tied to a fingerprint of the code they describe. Edit that code, and the note is automatically marked out of date, so the assistant isn’t guided by stale assumptions.
• Learning from each fix (resonant_evolve_guardrail): when a bug is fixed, Resonant turns the change into a reusable pattern and watches for the same mistake in future edits.
• Checking the receipts (resonant_honesty_audit): when an assistant reports “done, all tests pass,” Resonant checks the named files, functions, and test results against what actually exists.

Why it matters. For a developer, it means fewer surprises in code that “looked fine.” For a team, it means AI speed without giving up review discipline. For anyone paying the bill, it means fewer wasted AI calls. These checks catch common, well-understood classes of mistakes; they complement—not replace—thorough testing and human review.

How to read the numbers below. “@1” scores (such as pass@1) are the share of first attempts that succeed. “Sec” or “Func-Sec” scores count an attempt only if it both works and passes the security checks. The gap between the two is the hidden risk.

1. Test live in 30 seconds: Point Claude Code, Cursor, Windsurf, or Antigravity to https://resonantexo.com/api/mcp (instructions on resonantexo.com) and ask your AI to run resonant_verify_cegar (with format: "json" or format: "sarif") on a function refactor or resonant_architect_orient on a multi-file snippet.

2. What static and bytecode checks can and cannot do: Deterministic AST contract checking, boundary bytecode falsification, and AST-linked memory catch common AI regressions (dropped None/0 checks, signature drift, blocked async loops, 60fps UI memoization bugs, CWE shortcuts, and stale architectural notes) in milliseconds at zero token cost. They complement—rather than replace—domain unit tests and human engineering review.