- Qwopeful 🤞🏼
1,213 points247 comments4 d ago
- I removed guardrails from Qwen and it is performing EPICALLY
Ok ok, I am still doing tests but here is the story: I have a small custom harness (who doesn't!) that does pretty complicated stuff. I am using Claude (subscription) and everything is great. In...
934 points202 comments6 d ago
- Running GLM 5.3 Flash on a 40 mini PC cluster via USB 3 interconnects at 14 tps
Hello! Here is a rack I made using 40 mini PCs I happened to have from a different project: Intel N150 12 GB DDR5 4800 - physical bandwidth limit around 33GBs Diskless nodes and they are connected...
606 points172 comments2 d ago
- Probably old news, but any website you have open can reach your local LLM server while you browse
Maybe this is obvious to everyone but me, but it only clicked yesterday. If you run llama.cpp / vLLM / or whatever on localhost with the default config, a random website you have open in another...
513 points149 comments4 d ago
- Is bro right? Local is a pipe dream?
https://x.com/matt503ea5sf9z5/status/2118155162376458298?a=4bUWTuzbF\LcVwWBUDk&s=19 Interesting post, I hear a different story with dgx sparks how people are optimizing models to run upwards of...
477 points618 comments7 d ago
- DDR4 512GB vs. switching to DDR5 for MoE offload on a multi-GPU rig?
Setup: (if finished - still under construction, partly watercooled) Threadripper PRO 5995WX, ASUS WRX80E-SAGE SE WIFI II 6× RTX PRO 6000 8× RTX 5090 4× RTX 3090 128 GB DDR4 (that’s the problem) I...
472 points171 comments6 d ago
- 2x RTX PRO 6000 running DeepSeek-V4-Flash (284B) fully offline: 1,170 tok/s peak
We build these boxes, so here's the honest test log from our own unit, network cable unplugged. Setup: 2x RTX PRO 6000 Blackwell (192 GB), EPYC 9124, 377 GB RAM, vLLM, TP2, no NVLink. The model...
362 points89 comments7 d ago
- Me when Chinese models
346 points31 comments2 d ago
- Qwen3.8 27B FP8 on 4× Radeon AI PRO R9700: 7K tok/s prefill, 814 tok/s C8 decode
Working on a full inference stack for 4xr9700, wanted to share progress so far. All GPUs are capped at 210w(so plugs into any normal 15a outlet). BF16 KV, MTP enabled(no spec). \ Latency...
297 points157 comments6 d ago
- Opus 5.5 wrote its own quantiser for Qwen3.5-0.8B, no llama-quantize, no imatrix. 5.9× better than Unsloth's same-size file.
Inspired by Weight Space Programming (MIT CSAIL, Nov 2), but in the end Opus 5.5 did its own thing: no weight-space technique was used. It wrote a quantiser from scratch in NumPy and made every...
231 points44 comments5 d ago