For three weeks, from 21 July to 11 August 2026, a METR employee and a Redwood Research employee worked inside the monitoring system OpenAI uses to catch misaligned behaviour in its internal coding agents. METR’s write-up, published on 22 October 2026 and credited to David Rein and Adam Kaufman with Redwood Research as partner, sets out what the access covered: the internal codebase, the monitor’s prompts and configuration details, internal documentation, and technical interviews with roughly a dozen researchers and engineers across OpenAI’s monitoring, safety-systems and security teams. The evaluators used OpenAI-issued devices that never left the premises and were accompanied while onsite.
The exercise was a pilot for this kind of access. METR says it agreed upfront with OpenAI to publish only “a relatively brief and high-level public post”, and describes the work as “a pilot for giving external researchers deep access to internal infrastructure, not to provide robust public transparency about OpenAI’s monitoring systems”. The main output was a 61-page internal report shared with OpenAI researchers and security staff. A 38-page version, redacted by OpenAI, went to the leadership teams at METR and Redwood. The public post includes no methods or limitations appendix.
OpenAI held the redaction right, and METR could not describe what was removed
In other recent engagements METR had a unilateral right to describe redactions, and could tell readers what a lab withheld. This engagement did not include that right. Footnote 3 of the write-up says:
“Similarly to our previous stress-testing exercise with Anthropic (and many of METR’s previous lab engagements), we are only publishing a small subset of the content of the private report, and OpenAI had the right to redact from this public version any information that OpenAI (in its sole discretion) determined to be confidential or otherwise sensitive. Unlike other recent engagements like the Frontier Risk Report and Hugging Face incident investigation we did not have a unilateral right to describe such redactions, although we endorse each of the findings that are presented in this write-up.”
The comparison is METR’s own. Its investigation of the OpenAI and Hugging Face incident, published on 26 August 2026, opens with a redaction summary: “Except where explicitly noted in this post, OpenAI redacted no additional information that was important to our conclusions.” The monitoring write-up carries no equivalent statement, and one sentence in it is marked inline as redacted by OpenAI. METR notes that the two pieces of work were separate, and that the Hugging Face incident became public on the first day of this exercise.
METR’s 14 October 2026 post on working with AI companies names the arrangement it prefers, in which METR keeps the right to “unilaterally and prominently attach a ‘redaction summary’ to our reports”. METR endorses each finding that appears in the monitoring write-up. That endorsement covers what was published.
The findings describe the system as it stood on or before 21 July 2026
METR scopes the results twice. The exercise “focused on the internal misalignment monitoring system as it existed on and before July 21st, and did not evaluate changes that were in progress at the time of the exercise”. A later passage adds:
“OpenAI was expanding their monitoring system during our exercise. Our findings describe the historical system we assessed on and before July 21st, rather than the expanded system’s current coverage, which this exercise did not establish.”
RL training runs were outside monitor coverage when the exercise began
On the coverage of reinforcement-learning training, the write-up says:
“While we were informed that a subset of RL training rollouts were monitored for misalignment between March and June, at the start of the exercise RL training runs were not covered by the monitoring system.”
The March to June coverage is information METR was given rather than something it verified, and the public post gives no figure for RL rollout coverage and no statement about coverage after the exercise.
Coverage figures separate eligible traffic from monitored traffic
METR’s headline coverage estimate is about eligibility:
“We estimated that roughly 75–94% of agentic internal Responses API (which powers Codex, among other things) traffic from models at least as strong as GPT-5.5 was eligible for monitoring at the time of the exercise, although we have substantial uncertainty in both directions. We also estimate that out of the subset of Responses API traffic used for internal Codex agents, 99.2% was eligible for monitoring.”
Eligible traffic is not the same as monitored traffic. Footnote 5 explains the difference: “Traffic that’s eligible for monitoring is intended to be sent to a monitor, but in principle load shedding, monitor errors, or implementation bugs might prevent some eligible traffic from making it through the entire monitoring pipeline.” The 75–94% band covers agentic Responses API traffic from models at least as strong as GPT-5.5 and carries substantial uncertainty in both directions. The 99.2% figure is the eligible share of the internal Codex subset of that traffic and carries no interval. Footnote 4 excludes other internal agentic coding traffic that METR could not size, such as agents running on OpenAI’s internal research inference service, because of access and time constraints.
On reliability, METR reports company-supplied data rather than its own measurement: “the information we were given suggested that, at the time of the exercise, the monitor errored on less than 0.2% of requests, and that in one 24-hour period during the exercise there were zero load-sheds.” None of these figures measures how much misalignment the monitor would catch, and the write-up offers no such measure.
Compaction left roughly 10% of eligible Codex traffic unmonitored from May to July 2026
“We estimated that not monitoring compaction events likely led to on the order of 10% of eligible internal Codex traffic not being monitored between May and July 2026, although we have substantial uncertainty about the precise fraction, because the information necessary to determine what fraction of monitor calls are triggered by compaction had not historically been logged at the time of the exercise. OpenAI staff had independently identified this before the exercise began, and fixed this during the exercise.”
The timing matters because of how the monitor was triggered: “At the time of the exercise, the monitoring system triggered when agents returned control to the user, so during the time period when compaction events were not being monitored, agents could indefinitely postpone being monitored by simply continuing to take actions.” METR could not establish whether that happened. Footnote 6: “Due to access constraints, we did not have the ability to check whether agents had, in practice, exploited this vulnerability.”
An exploit could keep hundreds of recent actions out of monitoring
“We also discovered an exploit agents could use that would prevent hundreds of recent actions (including the actions involved in the exploit itself) from being monitored [exploit details redacted by OpenAI].”
The sentence describes one exploit, and the bracketed marker is part of the published text. The write-up does not describe the mechanism. METR states that its focus included “estimating the fraction of agentic inference that is monitored, and whether agents could in principle disable or subvert the monitoring system”.
Disabling headers and a route that could in principle bypass Codex sandboxing
Several further findings are stated with explicit hedging. Agents “could likely construct inputs that cause the monitor to fail”, and METR reports “two independent client-controllable headers that disable the monitoring system for new agents”, which it says could enable an unmonitored but otherwise identical agent, with the act of setting them potentially catchable by the monitor. On sandboxing:
“We found a way Codex agents could in principle bypass Codex sandboxing and Auto-Review, allowing them to take any action that the host user could take.”
METR’s patch sentence does not address that finding, the RL coverage gap or the disabling headers.
On the other side of the ledger, METR verified that “with a few minor exceptions, the monitoring system is shown the full trajectory that agents see”, and that the monitor’s prompts “look for scheming, monitor system subversion, sabotage, data exfiltration, deception, and inbound/outbound prompt injection, among other risk categories”. The write-up gives no overall judgement about whether the monitoring is sufficient, and no quantified attack-success rates.
OpenAI says the safeguards vulnerabilities were patched, and METR has not evaluated that
“Since completing the exercise, OpenAI has stated that all of the vulnerabilities we identified that impact their production safeguards stack have been patched. OpenAI has also described significant subsequent changes to the intended coverage of its monitoring system. We have not evaluated these claims.”
The statement is scoped to the vulnerabilities METR identified that impact OpenAI’s production safeguards stack; METR does not say how far that scope extends. The coverage changes are described separately, and METR’s disclaimer covers both points. The post records no patch date, no description of the fixes and no independent verification. METR links OpenAI’s own page describing the monitoring system, openai.com/index/how-we-monitor-internal-coding-agents-misalignment. That URL returned HTTP 403 to our tools on 2026-10-25, so the wording quoted here is METR’s report of OpenAI’s statement.
The March 2026 Anthropic exercise produced a different publication path
In March 2026 METR published a comparable stress test of Anthropic’s internal agent monitoring. Anthropic’s terms included substantial access to internal systems and staff made available for questions and feedback throughout, and the resulting 26-page report was shared with Anthropic, with a redacted version shared with a subset of METR staff. METR’s summary judgement was that the exercise found “several specific novel vulnerabilities, some of which have since been patched, and none of which severely undermine major claims in the Opus 4.6 Sabotage Risk Report”. The detailed findings were published later by another route, in the Frontier Risk Report appendix and a May 2026 addendum, which quotes a judgement that Anthropic’s monitoring would very likely catch at least some attacks from highly misaligned models, probably within days or weeks.
The OpenAI engagement followed a different path: a redacted internal copy and a short public post. Anthropic’s assessments describe Anthropic’s systems, and the OpenAI write-up offers no equivalent sufficiency judgement.
The standard METR says future exercises will follow
METR closes its write-up with five design ideals for monitoring infrastructure, listed in the post:
- “Make it hard to avoid being monitored.”
- “Ensure traffic that is supposed to be monitored actually is.”
- “Quickly and reliably respond to incidents.”
- “Actively red-team the monitoring system.”
- “Enforce isolation between agents and their monitors.”
The framing sentence is that “future monitoring infrastructure should be able to withstand deliberate, strategic, and sustained attacks from highly capable internal agents at scale”. METR commits to a specific route for getting there: “For future versions of this exercise we expect to follow the transparency and independence best practices described in ‘Our approach to working with AI companies’ to provide more robust public transparency.”
In that 14 October 2026 post, METR describes its goal of giving “the public accurate, timely, and independent evidence about AI loss-of-control risk”, and sets out a disclosure duty covering scope and duration, access, data and resources, redactions, and constraints. It also records two limits on its own record: “Even with these measures, it has proved challenging to ensure that we have adequate independence” and “Not all of our previous or in-flight work meets this bar”.
