Imagine this: A 27-billion-parameter AI model in an 11.8 gigabyte file, scoring the same as the 54-gigabyte original on the benchmarks that actually measure reasoning. I read that claim in a YouTube video published yesterday, and I spent about ten seconds not believing it.
Then I pulled up the model card. The numbers ARE real.
The video is on Fahd Mirza’s YT. Fifteen minutes of him downloading the file on an Ubuntu box with a 48 GB Nvidia card, serving it with llama.cpp, and then throwing four genuinely hard tasks at it. I’ll get to what the model did. First, the part that matters more than the model: what actually happened, and why a 27B suddenly fits next to your game library.
A lab in Austria with a grudge against big files
The group behind this is DASLab at the Institute of Science and Technology Austria — the same lab that gave the world GPTQ, the quantization trick quietly sitting under most of the 4-bit models you have already run. They released two new methods, GSQ and RCO, and applied them to Qwen3.8-27B, which in full BF16 precision is a 53.8 GB file.
The result: four GGUF files, from 8.4 GB to 11.8 GB. Standard GGUF. No patched llama.cpp, no proprietary kernel, no “works best with our runtime” asterisk. They run unmodified in llama.cpp, Ollama, and LM Studio.
So what do they actually do?
GSQ (Gumbel-Softmax Quantization) works at the level of a single weight. Instead of assuming four bits for everyone, it asks what each weight can survive. Some weights barely move when you crush them. Others are twitchy and need more precision. GSQ tries several bit depths per weight and learns the answer with a differentiable relaxation — the Gumbel-Softmax in the name — rather than brute force, starting from a GPTQ pass. The claim, which the numbers back up, is that at 2 to 3 bits this closes most of the gap to the fancier vector-quantization methods while staying in the boring scalar format that every GGUF loader already speaks.
RCO (Riemannian Constrained Optimization) works at the level of the whole model. You hand it a budget — 11.8 gigabytes, not one more — and it decides which tensor gets which bit depth so the file lands exactly there. Sensitive tensors get the good bits. Tolerant ones get squeezed. The name sounds like graduate school. The behavior is a mastering engineer getting an album onto a cassette tape without the kick drum getting lost.
Together: GSQ finds the best precision per weight, RCO spends that precision across the model against a hard size budget. The output is a non-uniform quantization: the model’s hundreds of tensors each carrying their own bit depth in a single file, which is exactly why it can hit a size target without the accuracy bill arriving.
The numbers, because adjectives don’t fit in a VRAM budget
The lab shipped four sizes, plus a vision projector and optional speculative-decoding builds:
| File | Avg bits/param | Size | Where it lands |
|---|---|---|---|
Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf | 2.50 | 8.4 GB | Smallest. Still beats its own BF16 baseline on zero-shot tasks |
Qwen3.8-27B-GSQ-RCO-IQ2_S.gguf | 2.75 | 9.3 GB | Matches the base model on AIME25 |
Qwen3.8-27B-GSQ-RCO-IQ3_XXS.gguf | 3.00 | 10.1 GB | Strong all-round operating point |
Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf | 3.50 | 11.8 GB | The recommended build. “Task-lossless” |
mmproj-Qwen3.8-27B-BF16.gguf | 16 | 0.9 GB | Vision encoder + projector, shared by all four |
Each one also has a -mtp variant, about 0.35 GB larger, carrying the model’s Multi-Token Prediction head for speculative decoding in llama.cpp. Same weights, same quality, more tokens per second.
The IQ3_S file is the one they call task-lossless. Here is what that claim amounts to in numbers:
| AIME25 (math) | GPQA-Diamond (science) | LiveCodeBench v6 (code) | |
|---|---|---|---|
| BF16, 53.8 GB | 100.00 | 89.90 | 85.71 |
| IQ3_S, 11.8 GB | 100.00 | 89.39 | 85.71 |
Exact on math. Exact on code. Half a point on science. A 4.6x smaller file. And this is not a lab benchmark with no competition: at the same 8.4 GB size, the IQ2_XS build beats the Unsloth Dynamic IQ2_S by 10 points on AIME25 and 8.6 on GPQA-Diamond. That is the gap between “impressive for a 2-bit model” and “looks like the model it came from”.
Getting the model
The repository is ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF on Hugging Face, Apache-2.0, same as the base model. One command:
pip install -U "huggingface_hub[cli]"
hf download ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF \
Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf \
--local-dir ./modelsSwap the filename for whatever fits your card. My reading of the VRAM arithmetic, which is guidance rather than law:
- 8 GB card. IQ2_XS. The weights fit; the context barely does. Chat, yes. Agentic work, on a shoestring.
- 12 GB card. IQ2_S. The sweet spot for small cards, with quantized KV cache.
- 16 GB card. IQ3_S, the recommended build, with a 16–32K context. Community reports on 16 GB 50-series cards push much further — one user on an RTX 5060 Ti reports around 30 tokens per second on the base build and 43 on the MTP build at large context, with llama.cpp b10472 or newer.
- 24 GB card. IQ3_S with a big context, or the MTP build for speed.
- 32 GB or more. You are in luxury. Add the mmproj and run it with vision.
Running it on what you already have
The video’s flow is the flow I’d use. llama.cpp server, everything on the GPU, and the KV cache is the dial you turn when the model does not fit:
llama-server \
-m ./models/Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf \
--alias qwen38-27b \
--host 127.0.0.1 \
--port 8080 \
-ngl 99 \
--ctx-size 32768 \
--cache-type-k q8_0 \
--cache-type-v q8_0-ngl 99 pushes every layer onto the GPU. If you are short of VRAM, the KV cache is the first thing to trim: q8_0 on both K and V is a good default, q4_0 when you are desperate. In the video the server sat at just over 28 GB of VRAM at his context settings on a 48 GB card — context is the dial, and it is yours.
Prefer not to touch a launch flag? ollama run hf.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF and pick the file that fits, or search LM Studio for the repo name and choose a GSQ-RCO build. For images, the multimodal CLI takes the same file plus the projector:
llama-mtmd-cli -m ./models/Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf \
--mmproj ./models/mmproj-Qwen3.8-27B-BF16.gguf \
--image photo.jpg -p "Describe this image."One security note, because I wrote about this two weeks ago and it still bears repeating: local does not mean locked. Keep that server bound to localhost unless you have a reason, put Tailscale in front of it if you do, and keep the llama.cpp build current. The privacy promise of this whole exercise is that your prompts never leave the building — that promise only holds while the door is bolted.
What it actually did
The video’s four tests are better than most benchmark tables, because they are ugly in the way real work is ugly.
First, a doner kebab: write an HTML file that renders a kebab rotating over flames, in pure canvas, no libraries. The model thought for over eleven minutes — you can watch it checking and rechecking — then produced something with real flame gradients, sear marks on the flame-facing side, and drip physics. First shot.
Second, a SQL bug hunt. A long query with nested CTEs that silently returns the worst customers instead of the best, because a rank window is sorted ascending where it should be descending — with the correct descending sitting right next to it in a sibling CTE, which is exactly where shallow models look and stop. The full-precision base model catches this bug — that was the standard — and the 11.8 GB file caught it too. It found it, named it, and suggested improvements.
Third, one sentence about saving money, translated into 75+ languages in a single pass, including Elder Futhark runes and two constructed languages. Here is the honest part: the major languages came out solid, but the low-resource ones are barely coherent — and that is the base model’s weakness, not the quantization’s. The video is direct that Qwen 3.8’s multilingual coverage has slid versus 3.6, and the quant just faithfully preserved the slide. Notably, it did the right thing in the constructed languages: it flagged the missing vocabulary instead of confidently hallucinating fluence it does not have.
Fourth, a chained chemistry problem — solubility equilibrium, common ion effect, mass balance — where one early arithmetic slip cascades into a wrong final answer, so pattern-matching will not carry you through. Thirteen minutes of reasoning, and the final number checks out. Every setup, per the video, was correct.
So the state of play: a 27B-class model that reasons like the 54 GB original, runs on a card you can buy at a store, and does not phone home.
If you want to run the method yourself
This is the part most coverage will skip, and it is the part that makes the next eighteen months interesting. Both methods are open source: GSQ and RCO, with the papers behind them (arXiv 2604.18556 and arXiv 2605.00649).
The pipeline, in three stages. One: build a per-tensor database by quantizing every weight tensor at every candidate bit width. That is the heavy part, the multi-GPU grind. (The released files ship their importance matrix and per-tensor allocation dumps so you can audit exactly how each one was built.) Two: the RCO search, which assigns one bit width per tensor under your exact size budget. This stage is fast — the paper reports matching or beating evolutionary-search baselines at 3 to 16x less wall-clock. Three: assemble the selected tensors into a single standard GGUF.
Now, the honest hardware talk. GSQ is tested on Hopper-class cards; training has been run on Ada, but the reference recipes assume 80 GB cards, and a dense 27B is a multi-card job. For most of us, “running the method” means renting a node for a weekend.
But there is a homelab-scale door in this. GSQ can also take an existing GGUF K-quant and refine its discrete assignments in place, projecting the result back into the same format. Their own number: a public Qwen3-8B Q2_K file going from a 50.03 average benchmark score to 56.28, with no format change, running on the same llama.cpp. A single-GPU job, and the closest the rest of us will get to running the research instead of just downloading its leftovers.
The part that is the actual point
Every few months someone makes a model that is close enough to flagship quality, and the honest question is never “is it as smart as the frontier API?” It is “does it change what I can keep in my own rack?”
This one does. A 27B that does real multi-step reasoning, real code review, and real document work, in a file smaller than a couple of Blu-ray rips, on a card that is not a server part. Your prompts, your tax documents, your photos — with the vision projector, yes, the photos — stay on hardware you can point a finger at.
The file is 11.8 gigabytes. The electricity bill is still yours, and mine. That has always been the trade with a rack, and it just got a better exchange rate.
That said, I have only had a couple of days of real work through the 11.8 build so far. The benchmarks are the lab’s, the video is Fahd’s, and my own verdict is still cooking. I will tell you when the first thing it gets wrong is interesting enough to write about.