Qwen3-8B

2025-05-02T23:41:52+00:00

Having tried a few of the Qwen 3 models now my favorite is a bit of a surprise to me: I'm really enjoying Qwen3-8B.

I've been running prompts through the MLX 4bit quantized version, mlx-community/Qwen3-8B-4bit. I'm using llm-mlx like this:

llm install llm-mlx
llm mlx download-model mlx-community/Qwen3-8B-4bit

This pulls 4.3GB of data and saves it to ~/.cache/huggingface/hub/models--mlx-community--Qwen3-8B-4bit.

I assigned it a default alias:

llm aliases set q3 mlx-community/Qwen3-8B-4bit

I also added a default option for that model - this saves me from adding -o unlimited 1 to every prompt which disables the default output token limit:

llm models options set q3 unlimited 1

And now I can run prompts:

llm -m q3 'brainstorm questions I can ask my friend who I think is secretly from Atlantis that will not tip her off to my suspicions'

Qwen3 is a "reasoning" model, so it starts each prompt with a <think> block containing its chain of thought. Reading these is always really fun. Here's the full response I got for the above question.

I'm finding Qwen3-8B to be surprisingly capable for useful things too. It can summarize short articles. It can write simple SQL queries given a question and a schema. It can figure out what a simple web app does by reading the HTML and JavaScript. It can write Python code to meet a paragraph long spec - for that one it "reasoned" for an unreasonably long time but it did eventually get to a useful answer.

All this while consuming between 4 and 5GB of memory, depending on the length of the prompt.

I think it's pretty extraordinary that a few GBs of floating point numbers can usefully achieve these various tasks, especially using so little memory that it's not an imposition on the rest of the things I want to run on my laptop at the same time.

Tags: models, ai, generative-ai, local-llms, llm, qwen, mlx, llm-reasoning, ai-in-china

South's Design

2009-05-13T12:30:45+00:00

South's Design

Andrew Godwin explains why South resorts to parsing your models.py file in order to construct information about for creating automatic migrations.

Tags: andrew-godwin, django, models, orm, parsing, python, south

django-mptt

2007-12-29T11:33:08+00:00

django-mptt

Jonathan Buchanan’s simple utility for performing Modified Preorder Tree Traversal (efficient tree operations in SQL) on Django models.

Via Jonathan Buchanan

Tags: django, djangoorm, jonathan-buchanan, models, modifiedpreordertreetraversal, mptt, python, sql

tranquil

2007-10-09T02:30:29+00:00

tranquil

Inspired take on the Django ORM to SQLAlchemy problem: lets you define your models with the Django ORM but use SQLAlchemy to run queries against them.

Tags: django, djangoorm, models, orm, python, sqlalchemy, tranquil

Simon Willison's Weblog: models

Qwen3-8B

South's Design

django-mptt

tranquil