Goodfire launched internal-signal monitors for Baseten customers on October 8, TechCrunch reports.
The approach cuts the work assigned to a second AI model. Small detectors called probes read calculations the agent’s model already makes. Only exchanges that cross a risk threshold go to an AI judge for closer review. In Goodfire’s published cyber-monitoring research, that design costs about $185 per million agent turns with judge reasoning enabled, roughly 29 times less than having the corresponding judge review every turn.
Screen every turn, review selected exchanges
Goodfire built the monitoring system for Kimi K3 and also tested a version for GLM 5.3. Its probes read internal activations, the intermediate calculations a model produces while processing and generating text. A judge then evaluates suspicious exchanges against a misuse policy and can refuse the request or let the interaction continue.
That distinction matters for long-running coding agents. Goodfire says legitimate security audits and malicious exploitation can involve similar actions. Even a harmful session can contain many ordinary steps, such as searching a directory. Reviewing all of them with another large model adds expense and delay.
Goodfire integrated probe scoring into the SGLang inference server. Its research reports less than a 2% throughput decrease across four tested workloads, within the noise of those measurements. With judge reasoning enabled, average added monitoring delay fell from 2,750 milliseconds per turn to 42 milliseconds. Without judge reasoning, it fell from 160 milliseconds to 2 milliseconds.
The company argues that this makes checks practical before an agent calls a tool. That timing gives the monitor a chance to stop an action before it happens.
What the reported results measure
Goodfire evaluated cyber misuse on about 2,400 chat and agent sessions with more than 60,000 turns. It used Claude Opus 5 labels as the reference. The dataset combined simulated interactions, real-world cyber chat and benign coding and operations tasks.
At a setting that interrupted 5.5% of harmless sessions, Goodfire reports detection of about 93% of harmful sessions. A harmless session counted as interrupted if the monitor flagged it for review at least once.
For cost and latency comparisons, however, Goodfire selected the cheapest configuration that detected at least 90% of harmful sessions while interrupting at most 10% of harmless sessions. It reports $65 per million turns without judge reasoning and $185 with reasoning. These estimates use API inference prices for its evaluation workload.
