The previous article set up the basic runtime choices for local open models: run Ollama inside WSL, keep model files in the Linux home directory, build llama.cpp first as a CPU baseline, and add the CUDA build later. This article does not discuss concepts. It pulls down four models and runs them once to see whether they can become daily tools on a mobile GPU with 8GB of VRAM.

The models tested are qwen3:4b, qwen3:8b, gemma4:e4b, and gemma4:12b. None of them is a giant model, so in theory they are suitable for a personal development machine. In actual use, however, the differences are more interesting than parameter count: some are fast, some are steadier, some have multimodal capabilities worth keeping locally, and some default templates need special care.

Short Version

If I could keep only one daily text model locally, I would start with qwen3:8b. Its Chinese output is more stable, VRAM pressure stays within the acceptable range of an RTX 4060 Laptop GPU with 8GB VRAM, and after warm-up it responds quickly enough.

For lightweight classification, title generation, and short text rewriting, qwen3:4b is more resource-efficient. It is not suitable for complex reasoning, but it works well as a small node in a pipeline.

For images, screenshots, and chart explanation, gemma4:e4b is more useful than I expected. It is noticeably faster than gemma4:12b on this machine, and its image-understanding results are already good enough for daily assistance.

gemma4:12b has more detailed descriptions, but the downside is direct: it is slow, cold starts are obviously slower, and long outputs test your patience. It is suitable for occasional, more complex visual understanding, not as the default model for every call.

Test Machine

All tests were run on the same local development machine:

• Host memory: 64GB
• WSL memory: 32GB
• WSL system: Ubuntu 26.04 LTS
• GPU: NVIDIA GeForce RTX 4060 Laptop GPU, 8GB VRAM
• Ollama: 0.30.10
• llama.cpp: CPU build for this round
• Model directory: ~/.ollama/models

After Ollama started, WSL could see the NVIDIA GPU. Whether nvcc exists does not affect Ollama inference, but it does affect building the CUDA version of llama.cpp. So the llama.cpp comparison here is only a CPU baseline, not a final performance conclusion.

First-round Numbers

These are not laboratory-grade benchmarks. They are first-round measurements on the same machine and runtime, intended to judge daily usability.

Model	Local Size	Cold Load	Warm Load	Generation Speed	Warm VRAM	First Impression
qwen3:4b	2.5GB	about 12.3s	0.13s - 0.16s	about 38 - 43 tok/s	about 2.7GB - 2.8GB	Fast, good for light tasks
qwen3:8b	5.2GB	about 16.8s	about 0.23s	about 22 - 26 tok/s	about 4.9GB	Daily Chinese workhorse
gemma4:e4b	9.6GB	about 14.4s	0.66s - 0.72s	about 54 - 56 tok/s	about 3.1GB	Strong multimodal value
gemma4:12b	7.6GB	about 43.0s	about 0.72s	about 7 - 9 tok/s	about 5.8GB	More detailed, but slow

A few observations are worth keeping.

First, local file size and actual VRAM usage are not linearly related. gemma4:e4b has a larger local file, but warm VRAM usage is lower than qwen3:8b.

Second, cold start and warm start are completely different experiences. Once the model remains in memory, qwen3:8b loading is almost negligible. But the first call still involves a visible ten-plus-second wait.

Third, the speed of gemma4:12b means it should not be the default entry point. It can be a model for “call it when a more detailed visual description is needed,” not a model placed in the first step of every workflow.

Be Careful with Qwen3 Defaults

The two Qwen3 models have an easy-to-misread issue: with default calls, the answer may go into the thinking field while response is empty. When seeing this for the first time, it is easy to think the model produced no output. In reality, it is following the template and using a thinking channel.

In a default call to qwen3:8b, I saw an empty response, a long thinking field, and a stop caused by length limits. After adding think:false, short Chinese answers returned normally:

本地运行小模型可以快速响应，无需依赖网络，适合日常开发中的即时帮助。它占用资源少，能够在本地设备上流畅运行，不影响其他任务。同时，本地模型能保护用户数据隐私，更适合处理敏感信息。

This means that when Qwen3 is integrated into scripts or tools, we should not only check whether the API succeeded. We must also check output fields. For most daily tools, I would disable thinking output by default unless the task explicitly needs reasoning traces.

A reusable calling style:

curl http://127.0.0.1:11434/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3:8b",
    "messages": [
      {"role": "user", "content": "用三句话说明本地小模型适合做什么。"}
    ],
    "think": false,
    "stream": false
  }'

In engineering, this detail matters more than “the model can run.” A running model only proves the environment works. Stable output fields prove it can enter automation.

Gemma 4 Multimodal Feels Practical

The most valuable part of this round was Gemma 4’s image-understanding capability.

I tested gemma4:e4b and gemma4:12b with the same tech-style illustration. With /api/chat and think:false, gemma4:e4b could steadily return Chinese descriptions and identify people, computers, screens, CUDA, model building, AI Pipeline, and related visual elements. Its speed was around 56 tok/s, which feels close to daily usable.

gemma4:12b gave more detailed descriptions and proactively summarized visual style, screen content, background technical elements, and overall theme. The cost was much lower speed: the same image description took more than 40 seconds. It is not unusable, but it needs to be placed correctly.

After an additional multimodal test, the conclusion became clearer:

Test	Input Method	Model	Result	Time
Single image recognition	images field in /api/chat	gemma4:e4b	Identified developer, terminal, GPU, CUDA, WSL, AI Pipeline, and suitable article contexts	about 126s, mostly first-load and image processing
Multi-frame video understanding	Extract low-privacy frames from a real video with ffmpeg, then input the frames as an image sequence	gemma4:e4b	Identified video theme, frame-level content, timeline, and limitations; suitable for offline video-content analysis	about 79s

The single-image test used the image below. It was not a random decorative image; it included a person, terminal, GPU monitoring, CUDA, WSL, AI Pipeline, and multiple elements the model could identify. It is a better check of whether the model is reading the image instead of guessing from the prompt.

I did not merely ask it to “describe the image.” I asked for fixed fields. The full prompt was:

请观察这张图片，并按下面格式输出：

1. 一句话主题：概括图片在表达什么。
2. 可见元素：列出你确实能看到的对象、界面或文字。
3. 技术信息：判断图片和哪些本地模型、GPU、WSL、CUDA 或开发环境主题有关。
4. 适合用途：说明它适合放在什么文章里。
5. 不确定点：明确说明哪些内容无法仅凭图片确认。

请只根据图片回答，不要编造看不到的信息。

This is the raw result returned by gemma4:e4b:

1. 一句话主题：这张图片展示了一个开发者在一个多屏幕、高性能的计算环境中，使用WSL和CUDA技术栈进行人工智能模型开发与部署的过程。

2. 可见元素：
    *   **人物：** 一位坐在电脑前工作的年轻人。
    *   **硬件设备：** 多个显示器（至少三个），键盘，鼠标，以及一张桌面工作台面。
    *   **软件界面/文字信息：**
        *   **左侧屏幕：** 显示GPU性能监控仪表盘，包含GPU名称、风扇转速、温度、内存和利用率等指标。
        *   **中央屏幕（终端）：** 一个命令行界面（CLI），显示WSL的提示符 (`alex@zicode:~$`)，以及执行Python代码和调用CUDA库（如 `torch.cuda.is_available()`）的代码片段。
        *   **右侧屏幕：** 展示了一个关于AI工作流的流程图，包含“WSL”、“CUDA”、“models”等关键词，并描绘了从数据集到推理的完整“AI Pipeline”。
    *   **品牌/标识：** 桌面和笔记本上可见“ZiCode”字样。

3. 技术信息：
    *   **操作系统环境：** WSL (Windows Subsystem for Linux)，表明在Windows环境下运行Linux开发环境。
    *   **计算平台：** CUDA，NVIDIA的并行计算架构，用于加速深度学习和高性能计算。
    *   **编程语言/框架：** Python（通过终端执行），涉及PyTorch等深度学习库的概念。
    *   **主题流程：** AI Pipeline (人工智能流水线)，涵盖数据集处理、模型训练、推理等完整机器学习生命周期。
    *   **硬件焦点：** GPU计算和性能监控。

4. 适合用途：
    *   科技博客文章（关于AI开发环境搭建）。
    *   技术教程或指南（如“如何在Windows上配置高性能的深度学习工作站”）。
    *   产品宣传材料（用于推广NVIDIA、WSL或特定编程工具链）。
    *   学术论文配图（展示实验环境和计算流程）。

5. 不确定点：
    *   代码是否成功运行，以及当前模型训练的具体状态。
    *   图片中显示的GPU型号的详细信息（仅能看到概念图和指标）。
    *   “ZiCode”品牌或其与开发内容的具体关联性。

This output is not perfect. It understood some screen content too broadly and listed “product marketing material” and “academic-paper illustration” as suitable uses, which is too wide. But it captured the core of the image: local development environment, WSL, CUDA, GPU monitoring, and AI Pipeline. That is exactly the value of a local multimodal model in a writing workflow: first read the visible information, then let a human decide what belongs in the article.

For video, I did not send the video file directly to the model. I extracted frames first. The source was a real 26.8-second vertical video, HEVC, 720x1280, with audio. The second half contained several clear group photos, which were not suitable for a public sample. So I selected only low-privacy frames from the first half: flowers, commemorative signs, environment shots, and seat signs. This loses some narrative information, but it is more suitable for public writing.

Below is the six-frame contact sheet sent to the model. It lets readers first see what the model “saw,” then judge whether the model output is credible. Clear person frames were intentionally avoided.

Individual frames are also shown to avoid losing text and detail through contact-sheet compression.

The frame extraction command:

for ts in 0 3 4 6 9 11; do
  ffmpeg -y -ss "$ts" -i 3.mp4 -frames:v 1 \
    -vf 'scale=720:1280:force_original_aspect_ratio=decrease,pad=720:1280:(ow-iw)/2:(oh-ih)/2' \
    "frame-${ts}.png"
done

The video-frame test prompt:

这些图片是从同一个 26 秒竖屏视频中按时间顺序抽取的 6 张关键帧。为了降低隐私暴露，抽帧时刻刻意避开了清晰人物合影，只保留花草、纪念牌、环境和座位牌等画面。请基于这些帧做视频理解，按下面格式输出：

1. 视频主题：一句话概括这个视频在讲什么。
2. 画面内容：逐帧说明每一帧里能看到的主要内容。
3. 时间线：说明画面从开头到结尾大致发生了什么变化。
4. 文字信息：如果画面里有可读文字，请尽量识别并说明其作用；不确定的文字要标注“不确定”。
5. 适合用途：判断这段视频适合放进什么文章或场景。
6. 局限：只根据这些避开人物的抽帧理解，明确说明哪些信息无法确认。

请不要编造帧中看不到的信息。

This is the raw output from gemma4:e4b. Except for Markdown headings and lists, I did not rewrite it:

好的，我将根据您提供的六张关键帧图片，按照要求的格式进行视频内容分析。

---

### 视频分析报告

**1. 视频主题：**
纪念和记录一个充满春日气息的、与“思源”相关的周年庆典或校园活动。

**2. 画面内容（逐帧说明）：**

*   **第 1 帧 (第一张图)：** 户外环境，背景虚化，光线明亮，顶部有太阳图案装饰。底部中央偏右方是文字信息和Logo。
*   **第 2 帧 (第二张图)：** 户外花海场景，前景有盛开的红色花朵（如罂粟花），背景是绿色的植被和建筑物的模糊轮廓。顶部有文字装饰。
*   **第 3 帧 (第三张图)：** 广阔的户外花园/校园场景，可以看到成片的黄色和橙色花卉（可能是水仙或类似花朵），环境开阔，光线柔和。底部有文字信息。
*   **第 4 帧 (第四张图)：** 近景特写，展示了固定在树干上的纪念牌。纪念牌上有清晰的文字内容，并用红色丝带装饰。背景是绿色的植被。
*   **第 5 帧 (第五张图)：** 户外环境，拍摄角度较低，前景有盛开的红花和绿色叶子，背景能看到水面或空旷的区域，光线明亮。顶部有文字装饰。
*   **第 6 帧 (第六张图)：** 室内/教学区域的特写，展示了固定在柜子或设备上的纪念牌。纪念牌上有清晰的周年庆典信息和年份。背景是相对简洁的内部环境。

**3. 时间线（画面变化）：**
视频从一个泛景、充满春日氛围的开场（第 1 帧）开始，随后通过不同角度展示了校园内的花卉景观（第 2 帧 $\rightarrow$ 第 3 帧），重点聚焦于纪念意义的物品——首先是树上的周年纪念牌（第 4 帧），接着在更广阔的花海中记录美好瞬间（第 5 帧），最后收尾于一个室内或特定区域的设备上展示了另一个关键的周年纪念铭牌（第 6 帧）。整体节奏是从环境氛围铺垫 $\rightarrow$ 景观展示 $\rightarrow$ 核心纪念物特写，层层递进。

**4. 文字信息：**
*   **贯穿始终的文字/Logo：** “一起来春天”、“思源 Spring”（此为活动或机构名称标识）。
*   **第 2、3、5 帧（顶部）：** “一起来春天” / “思源 Spring”。
*   **第 4 帧（纪念牌）：**
    *   “十草树”、“三十而立”、“滕人”、“南大”、“1996届计算机系3312班”、“毕业三十周年纪会”、“二〇二六年四月”（核心信息：纪念某届学生毕业三十周年）。
*   **第 6 帧（纪念牌）：**
    *   “庄梓铭初心 思源传新大”、“百年历程 同窗共致远”、“1996届计算机系3312班”、“毕业三十周年纪会”（核心信息：与第 4 帧内容高度一致，再次强调了纪念和时间节点）。

**5. 适合用途：**
*   **学校/学院的校友回访视频：** 特别是为“思源”相关的学子或师生制作的怀旧、周年庆典回顾片。
*   **毕业季宣传片：** 用于纪念特定届次的优秀毕业生和深厚的校园情谊。
*   **活动预热/回顾素材：** 适用于在社交媒体（如朋友圈、视频号）发布，营造温馨、怀旧的氛围。

**6. 局限：**
由于所有画面都刻意避开了人物合影，因此无法确认以下关键信息：
*   参与活动的具体人员是谁（除了通过纪念牌推测出是1996届计算机系3312班的校友）。
*   视频的叙事重点是“周年庆典”本身，而非某个特定事件。
*   视频的情绪基调和高潮点无法从人物互动中捕捉到。

Three points are worth recording. First, even without clear group photos, the model still inferred that the video was roughly about spring, a campus-like environment, and an anniversary memorial. Second, it described frame-to-frame changes and noticed that avoiding group photos limits judgment about participants and emotional highlights. Third, its text recognition is still unstable; some Chinese on the plaques was misread. Video-frame understanding is useful for first-pass screening, summaries, and material organization, but OCR or human review is still needed for text.

To be clear: this test verifies “video understanding through extracted frames,” not native video-file parsing. For local workflows, that is already useful. Many video summaries, screen-recording explanations, and tutorial segment analyses can first extract frames by time interval and then ask a vision model to summarize them.

The test also exposed a practical issue: multi-frame input is noticeably slower. The six-frame low-privacy input took about 79 seconds, with eval_count=974. It is suitable for offline summaries, article-material checks, and screen-recording screening, not for a real-time video assistant.

My positioning for these two models is:

• gemma4:e4b: default image-understanding model for screenshots, charts, image captions, and checking whether article images drift off topic.
• gemma4:e4b + frame extraction: video summaries and screen-recording first-pass screening.
• gemma4:12b: second-pass analysis for complex images, when more detailed visual explanation is needed.

One detail matters: gemma4:e4b behaves normally in raw:true text mode, but raw:true plus images failed before. For multimodal tasks, /api/chat with explicit think:false should be preferred.

llama.cpp as a Baseline First

I also ran a CPU baseline with llama.cpp. The Qwen3 4B model downloaded by Ollama is GGUF underneath, so llama.cpp can read it directly. The llama-bench result was roughly:

Model	Backend	Threads	Prompt Processing	Generation
Qwen3 4B Q4_K_M	CPU	16	about 4.5 tok/s	about 12.4 tok/s

This result says only one thing: CPU can run it, but it is not the route this machine should rely on.

When Ollama calls a similar small model through GPU, it can steadily reach dozens of tokens per second. CPU llama.cpp is more suitable as a compatibility check.

When this article was first written, the CUDA version of llama.cpp had not yet succeeded. The machine itself could see the RTX 4060 Laptop GPU, and Ollama was already using the GPU inside WSL. But when enabling GGML_CUDA with the following command, CMake stopped at CUDA Toolkit detection:

cmake -S ~/opt/llama.cpp \
  -B ~/opt/llama.cpp/build-cuda \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_CUDA=ON

The key error was:

Unable to find cudart library.
CUDA Toolkit not found

Later, I used micromamba to install CUDA 12.4, GCC 13, and a cuBLAS environment under the user directory, and successfully built the CUDA version of llama.cpp. The full process is written separately in Building llama.cpp with CUDA in WSL: A Real Local Deployment Note. With the same Qwen3 4B GGUF, CUDA llama-bench reached about 78.7 tok/s for tg32; the previous CPU baseline was about 12.4 tok/s.

My Default Model Routing

After this round, I would not simply put four models in a flat list and choose manually. I would use a simple routing table:

Scenario	Default Model	Reason
Short text classification, titles, summaries	qwen3:4b	Fast and low VRAM usage
Daily Chinese Q&A and code explanation	qwen3:8b	Balanced stability and speed
Screenshot understanding, image captions	gemma4:e4b	Multimodal works and speed is good
Video summaries, screen-recording first pass	gemma4:e4b + ffmpeg frame extraction	Local and controllable, no native video parsing required
Complex image detail review	gemma4:12b	More detailed, but only call on demand

This is closer to daily use than asking “which model is strongest.” The goal of a personal machine is not to squeeze the leaderboard’s top model into VRAM. It is to give different tasks default choices that are fast enough, stable enough, and cheap enough.

What to Test Next

Benchmarks only answer “can it run” and “how fast is it.” The real question is whether local models can enter actual workflows.

So the next article will place these models in more concrete scenarios: development log explanation, screenshot understanding, article-image review, short-document summaries, and structured information extraction. The point is not to show off models, but to see whether they can save time in small tasks that happen every day without making the workflow more complicated.

WeChat

Follow ZiCode on WeChat

If this post was useful, you can follow later updates on WeChat as well.

X / Twitter

Follow @ax2_zicode

Faster technical notes, short thoughts, and new-post alerts are posted on X.

X Open X Profile @ax2_zicode