7 posts
In a WSL environment with Ubuntu 26.04 LTS and an RTX 4060 Laptop GPU, this note uses micromamba to prepare CUDA 12.4, GCC 13, and cuBLAS without modifying system directories, builds the llama.cpp CUDA backend, and compares CPU and CUDA speed with Qwen3 4B.
On a local development machine with an RTX 4060 Laptop GPU, 8GB VRAM, and 32GB memory assigned to WSL, I tested Qwen3 4B, Qwen3 8B, Gemma 4 E4B, and Gemma 4 12B for load time, generation speed, VRAM usage, multimodal capability, and practical trade-offs.
The real value of local open models is not a benchmark score, but whether they can enter tasks that happen every day. This article puts Qwen3 4B, Qwen3 8B, Gemma 4 E4B, and Gemma 4 12B into three workflows: development assistance, image understanding, and writing organization.
A continuously updated local open-model installation note. Starting from a Windows + Ubuntu 26.04 LTS development machine with 64GB host memory, 32GB assigned to WSL, and an RTX 4060 Laptop GPU with 8GB VRAM, this article first decides whether Ollama should run on Windows or WSL, compares llama.cpp, LM Studio, vLLM, and other options, and then lists models worth keeping locally.
Starting from a Windows development machine with an NVIDIA GPU, this article explains how to verify CUDA in WSL, run a small PyTorch test, place local model caches, and debug cases where the GPU is not actually being used.
Edge AI is not a smaller copy of cloud LLMs. It is about latency, power, privacy, caching, product boundaries, and engineering constraints that live much closer to the device.
A practical test of Gemma 4 12B through Ollama: generating Canvas code, connecting it to OpenClaw, calling QVeris tools, and judging where a 12B local model is useful and where it still needs guardrails.