Employed AI

How people use AI and what happens when they do.

AI Research

Read AI papers, benchmarks, and demonstrations with an eye on what was tested, how results were scored, and what someone else could verify.

Browse all articles

What did the benchmark score?

Before comparing scores, check the task, the baseline and the scoring rule. A change in any of them can change what a result means.

Forced to pick a candidate, Grok chose Republicans 92% of the time and GPT Democrats 97%: the method decides what that means

Convovo asked GPT-6.1 Sol and Grok 4.7 to name one candidate in 479 US races, through each vendor's API, with a prompt that forbids hedging. The two models picked opposite parties almost every time. A follow-up with internet search switched off suggests the two leanings have different causes: the model's weights for one, live search results for the other. The forced-choice test is a narrow measure of how the two models lean under one prompt, and it still cannot say how either model behaves in the app you open.

Microsoft's MAI-DxO beat generalist doctors in a Science study, and the comparison left specialists out

A Science paper published on 22 October 2026 reports that Microsoft's MAI-DxO scored 83.2% on NEJM clinicopathological conference cases against 36.1% for generalist physicians, and 80.3% on MIMIC-IV-ED cases against 46.8%. The comparison used doctors working alone, and the specialists and physician teams who normally handle such cases were not included. The paper body is paywalled, and several circulating figures come only from Microsoft's own chart.

Hundreds of thousands of doctors use ad-funded OpenEvidence. Its effect on treatment decisions is unmeasured.

The FT reports that nearly two-thirds of US doctors use OpenEvidence's clinical AI search, and one person close to the company says it is on track to generate $400 million to $500 million through advertising by the end of 2026. A JAMA study published on 15 October 2026 documents the scale of use. Independent evidence that the tool improves treatment decisions does not exist.

MIT's island experiment: play-like behaviour in 13 Claude agents, and how much the setup decides

Thirteen agents built on the MIT researchers' Eko architecture ran on Claude Code with Claude Opus 4.7 for thirty hours each on a simulated island with no assigned task. The paper reports play-like behaviour under four criteria the authors applied themselves, and skills that carried over to later tasks. A soul file primed to be 'naturally curious', an automatic continue loop and the authors' own LLM judging all shaped what was recorded.

White Circle's RISE preprint reports reviewer-confirmed explicit images from 13.33% of DALL·E 3 prompts. What that figure measures

White Circle's September 2026 preprint describes a pipeline that evolved reusable strategies against commercial image APIs and reports human-verified prompt-level rates of 13.33% for DALL·E 3, 9.47% for Nano Banana 2 and 3.27% for GPT-Image-2. The same calibrated judging pushes rivals' earlier ~30% claims toward zero, 13% has no published baseline, and White Circle's claim that newer models are more breakable comes from its own unreplicated post-submission test.

How far did the experiment go?

Follow a result from a proposed design to a physical measurement. Each experiment answers a specific question about how a method performs.

What can an independent reader verify?

Trace citations back to the original source and check what a proof or detector actually establishes. Start with these explainers on how verification works.