Big AI on small machines: a 284B model with no GPU
Running a large AI model at home usually needs an expensive graphics card, or GPU. That is changing fast. Llama-modchip, a free project on GitHub, claims to run a 284-billion-parameter DeepSeek model on 64GB of RAM, with no GPU at all. It got 423 points and 191 comments on Hacker News.
People are also shopping. A post on r/LocalLLaMA says users are buying RTX 5060 Ti 16GB cards in large numbers for AI. Another user tested Intel's Arc Pro B70 32GB, bought for $1,299, against an RTX 5090 that sells for $6,500 and more. After tuning, the Intel card wrote text at about half the speed, for about one fifth of the price.
For people who want to see what is happening inside their machine, gputaskman is a new free GPU task manager for Windows, Linux and WSL. It shows memory use per program and details about the AI model running.
Local AI means your data stays on your computer, a real privacy win after today's Gemini story. But the same power also runs models with their guardrails removed, so freedom comes with responsibility. My old laptop read this news and asked for a raise.
