Fewer than 10 neurons: a reported backdoor shows how small edits can hijack an agent
Follow-up to Wednesday Nov 4: The 'Astra rewires Qwen' story gets a strong pushbackRemember last week's token on GPT-6 Astra changing a tiny part of a small Qwen model's weights? Weights are the billions of numbers that make up a model. Small edits are now a whole field, and today it shows a darker side.
A post on r/LocalLLaMA says Microsoft Research backdoored an AI agent by directly editing fewer than 10 'dormant' neurons, parts of the model that are usually quiet. A backdoor is a hidden trigger that makes a model misbehave. The poster warns people who run 'sketchy' modified models, and asks if 'sleeper agents' could become a real attack. We only have the Reddit post, not the details of the research.
Other new work points the same way. A paper on Deletion Tuning finds that zeroing fewer than 0.05% of a model's weights can match full fine-tuning. And Tree-Ising, a new paper, builds on the Astra work to repair models with tiny bit-level edits.
If a few neurons can teach a model good tricks, they can also hide bad ones, and a normal test may never see them. Before you run a free model from an unknown source, ask who made it and who checked it. Too often, the honest answer is nobody.
