Microsoft and Hugging Face built a benchmark that grades agents on the database state they leave behind. In a 12-model ablation, 67.24% of failed runs still ended with a clean tool call and no error. Here is what the executable checks measure, which numbers cover only the ablation, and what to assert before you commit agent work.
Convovo asked GPT-6.1 Sol and Grok 4.7 to name one candidate in 479 US races, through each vendor's API, with a prompt that forbids hedging. The two models picked opposite parties almost every time. A follow-up with internet search switched off suggests the two leanings have different causes: the model's weights for one, live search results for the other. The forced-choice test is a narrow measure of how the two models lean under one prompt, and it still cannot say how either model behaves in the app you open.
MedLDR's released scoring code shows how reference-label coverage, unmatched suggestions and agreement across repeated outputs measure different things.
Three evaluations in a Nature study test code, personalized answers and coaching conversations. Each supports a different kind of claim about health AI.
A Science paper published on 22 October 2026 reports that Microsoft's MAI-DxO scored 83.2% on NEJM clinicopathological conference cases against 36.1% for generalist physicians, and 80.3% on MIMIC-IV-ED cases against 46.8%. The comparison used doctors working alone, and the specialists and physician teams who normally handle such cases were not included. The paper body is paywalled, and several circulating figures come only from Microsoft's own chart.
The FT reports that nearly two-thirds of US doctors use OpenEvidence's clinical AI search, and one person close to the company says it is on track to generate $400 million to $500 million through advertising by the end of 2026. A JAMA study published on 15 October 2026 documents the scale of use. Independent evidence that the tool improves treatment decisions does not exist.
Thirteen agents built on the MIT researchers' Eko architecture ran on Claude Code with Claude Opus 4.7 for thirty hours each on a simulated island with no assigned task. The paper reports play-like behaviour under four criteria the authors applied themselves, and skills that carried over to later tasks. A soul file primed to be 'naturally curious', an automatic continue loop and the authors' own LLM judging all shaped what was recorded.
White Circle's September 2026 preprint describes a pipeline that evolved reusable strategies against commercial image APIs and reports human-verified prompt-level rates of 13.33% for DALL·E 3, 9.47% for Nano Banana 2 and 3.27% for GPT-Image-2. The same calibrated judging pushes rivals' earlier ~30% claims toward zero, 13% has no published baseline, and White Circle's claim that newer models are more breakable comes from its own unreplicated post-submission test.
A Nature study reports doubled probability skill from a modest baseline for an AI weather model. Here is what the correction changes and what the result means.
How far did the experiment go?
Follow a result from a proposed design to a physical measurement. Each experiment answers a specific question about how a method performs.
Google reports using Antigravity to curate compound data, compare molecular structures and propose new designs in an hour. Here is what those outputs establish and what scientists still need to check.
A Nature study links AI planning to physical laboratory equipment for DNA synthesis, scheduling, data storage and mutation testing, with people still servicing the system.
Anthropic's Claude Science team generated 318,484 protein designs, screened 553,052 design-toxin pairs, and had 211 designs tested at two contract research organisations by surface plasmon resonance. Seventeen bound one scorpion neurotoxin each. None was tested for neutralisation, and Anthropic says it does not claim to have produced an antivenom.
What can an independent reader verify?
Trace citations back to the original source and check what a proof or detector actually establishes. Start with these explainers on how verification works.
OpenAI’s October 2026 disclosures and corrections show why checking an AI-generated argument, establishing novelty and understanding a result require different work.
Anthropic's FlashProve checks computation against committed model weights. Model identity still needs trusted enrollment, and its benchmark measures proof production rather than chat generation.
A worked example shows how attacker access, test prompts, quality checks and detector calibration shape what a watermark robustness study can establish.
Google opened its SynthID Detector to anyone with an account on 7 October 2026, and OpenAI will add an invisible watermark to eligible ChatGPT and Codex text in the EU only. A watermark hit points to one vendor's pipeline; a miss leaves the question open.
A preprint posted on 27 October estimates that about 24% of the crawlable pages it sampled contained AI-generated or AI-edited text by August 2026, with prevalence highest in the least central pages. Pew, Dolezal et al. and Graphite report different shares, because each number belongs to one corpus, one detector and one denominator.
A SASH Fellowship team reports that Claude Fable 5.1 encoded 50 short messages in images it rendered, with a fresh Fable 5.1 session recovering 39 of them (78%) and GPT-6 Astra recovering 44 of 50 (88%). The authors call the result a precursor and say it is not currently dangerous.