NVIDIA DGX Spark is easy to misunderstand. It is not a toy for ordinary users to run every chatbot locally, nor a shrunken data-center training cluster. What it really sells is 128GB unified memory, Grace Blackwell, the NVIDIA software stack, and a desktop form factor.
In a WSL environment with Ubuntu 26.04 LTS and an RTX 4060 Laptop GPU, this note uses micromamba to prepare CUDA 12.4, GCC 13, and cuBLAS without modifying system directories, builds the llama.cpp CUDA backend, and compares CPU and CUDA speed with Qwen3 4B.
On a local development machine with an RTX 4060 Laptop GPU, 8GB VRAM, and 32GB memory assigned to WSL, I tested Qwen3 4B, Qwen3 8B, Gemma 4 E4B, and Gemma 4 12B for load time, generation speed, VRAM usage, multimodal capability, and practical trade-offs.
The real value of local open models is not a benchmark score, but whether they can enter tasks that happen every day. This article puts Qwen3 4B, Qwen3 8B, Gemma 4 E4B, and Gemma 4 12B into three workflows: development assistance, image understanding, and writing organization.
A continuously updated local open-model installation note. Starting from a Windows + Ubuntu 26.04 LTS development machine with 64GB host memory, 32GB assigned to WSL, and an RTX 4060 Laptop GPU with 8GB VRAM, this article first decides whether Ollama should run on Windows or WSL, compares llama.cpp, LM Studio, vLLM, and other options, and then lists models worth keeping locally.
A short introduction to Google's lightweight Gemma LLM family, including Kaggle resources, common model formats, 2B and 7B variants, and basic hardware expectations.
A simple local Gemma setup note: download the PyTorch 2B checkpoint from Kaggle, clone gemma_pytorch, set PYTHONPATH, install dependencies, and run the official script directly.
7 published posts are not listed in the outline.