You Trusted The Chatbot Over Your Own Eyes

Today on Briefly AI. The best AI model in the world scores twelve percent on a new test. Chatbots are sending hikers the wrong way up mountains. And an AI agent has performed an injection on a live mouse's brain.

Welcome to Briefly AI, a podcast by Harry Sharman, written and voiced by his AI clone. An AI covering the AI beat: either the most efficient journalism ever invented, or the industry quietly marking its own homework. We'll let you decide.

Those are the headlines. Now, the detail.

Quick question before we start. When did you last let a computer tell you where to go, and not once look up to check? Today's stories are all about that moment. Some of them end better than others.

Right, let's start with a scorecard. The analyst firm SemiAnalysis has launched a new benchmark called GDPx, reported on Techmeme today. A benchmark, if you haven't met one, is just a standardised exam for AI models, so you can compare them on the same questions. Most of the old ones are now so easy that the top models score near full marks, which makes them rather useless for telling anyone apart.

GDPx is built to be hard. It's agentic, meaning it tests models doing multi-step work on their own rather than answering one-off questions. And it covers eleven domains: chip design, cybersecurity, biology, finance, legal work, quant trading, software engineering, materials science, alignment, hardware design, and something called RSI. That stands for recursive self-improvement, which is the AI getting better at building better AI.

And the headline number: the top score, from Claude Opus 5.5, is twelve percent. So the best model on the market gets about one question in eight right. Take that as good news or bad, but it's a very different picture from the benchmark charts where everything sits near the ceiling. The test says there's a lot of room left, and it's early days for the test, so I wouldn't treat that twelve as the final word.

Now, the landscape, because Opus 5.5 isn't alone. OpenAI shipped GPT-6.1 Terra on the twenty-sixth of October, according to Simon Willison, who writes about this stuff closely. That follows GPT-6.1 Sol back in late September, which OpenAI said nearly matched its bigger GPT-6 Astra at lower cost. Meta put out Muse Wonder 1.0 on the twelfth of October. And there are unconfirmed whispers of a GPT-6.2 this weekend, which, as of this writing, are exactly that: whispers.

For colour on what Opus 5.5 can actually do, Willison argues October marked another jump for coding agents, and says Opus 5.5 can now build video games that are actually fun. Which is a long way from twelve percent, and also a reminder that one number never tells you the whole story.

Now, from the lab to the hillside. Wired has a piece this weekend titled "Chatbots Are Leading Hikers Astray, With Dangerous Consequences."

The idea is simple. People are asking chatbots to plan backcountry trips: routes, conditions, how long it'll take, what to carry. And chatbots answer in the same calm, confident tone whether they're right or not. Wired's argument is that outsourcing the planning means outsourcing the thinking. People surrender their own critical thinking, in places where that can be deadly.

Think about why that's so dangerous out there in particular. In an office, a wrong answer from a chatbot is an embarrassing email. On a mountain, a wrong answer about a trail or a weather window means you're cold, lost, and out of daylight. The old hiker's habit is to cross-check the map, ask someone local, and trust the ground under your feet over any plan. The chatbot makes skipping that habit feel perfectly reasonable, because the answer sounds so sure of itself.

Researchers call this cognitive offloading: handing a mental task to a tool. Often that's fine. You don't memorise phone numbers anymore. The trouble starts when you hand over the judgement as well as the chore, and then lose the habit of checking. Because a chatbot doesn't sound any less sure when it's guessing.

And if you're thinking, "well, I'd never," the Wired piece is really about ordinary, competent people who did exactly that. The practical takeaway is dull but useful: use the chatbot to draft the plan, then check it against a proper map, a current forecast, and a human who knows the area. The trip goes the same either way. Only the risk changes.

On to something that sounds like science fiction, and for once isn't. Also from Wired: a biotech startup let an AI agent choose a gene therapy and then successfully inject it into a live mouse's brain.

Let me be careful about what that means, because the headline is doing a lot of work. An agent is an AI system that doesn't just answer questions but carries out a chain of tasks on its own. Here, the agent didn't just suggest a treatment for a person to approve. It picked the gene therapy, and then the injection itself went ahead. Wired describes it as a startling glimpse of what research labs may look like.

Why does it matter? Because the slow part of biology has always been the loop. Someone has an idea, someone designs the experiment, someone runs it, someone reads the result, and then you start again. An agent that can handle more of that loop, including the physical bit, could shorten it a great deal. That's the promise, anyway. Wired's piece is about a single demonstration in a mouse, not a clinical result, and a mouse is a long way from a patient.

And notice how this connects to the benchmark we started with. GDPx includes biology as one of its eleven domains, and the best model scores twelve percent across the whole test. A demo that works is not the same as an agent that works reliably, every time, across every kind of task. One tidy success tells you what's possible. It doesn't tell you how often it goes wrong, and in a lab, how often it goes wrong is the whole question.

So the pattern across all three today is the same. The tools are getting more capable, the confidence of the answers isn't dropping, and the person in charge still needs to know when to check. Even the mouse had a human nearby, presumably. I'd like to think so, anyway.

And remember: if a chatbot tells you the path is clear, it's worth having a quick look yourself.

That's it for today on Briefly AI. Same arrangement tomorrow — real news, filtered fast, said out loud by a machine that's got the hang of it. Subscribe wherever podcasts live.