After spending a few days testing every local LLM and coding agent I could get my hands on with an RTX 4060 (8GB VRAM), here’s what I found: the new generation of small models, granite4.1:8b, qwen3.5:9b, and gemma4:e2b, are shockingly good, scoring perfect marks on both bug-finding and tool-calling benchmarks where last-gen models like phi4-mini and gemma3:4b fell flat, but the model is only half the story. The agent framework matters just as much, and most of them can’t actually use local models properly.
Out of 12 frameworks tested, only Aider, Cline, Goose, and Fabric could reliably find and fix bugs with local models, while OpenHands, OpenCode, and Open Interpreter failed with every model I threw at them because their tool-calling interfaces are too complex for 7B-class models to drive. Aider was the most model-agnostic (it just works with everything), Goose went from completely broken to perfect once I swapped in qwen3.5:9b, and granite4.1:8b turned out to be the speed king for agent workflows at 0.2-0.8 seconds per tool call.
The bottom line: you can absolutely run a useful AI coding agent locally on a mid-range GPU today, but you have to pick the right model-framework pairing, and most of the flashy agent projects out there aren’t ready for local models yet.