個人學習路線圖,從模型量化一路往下摸到kernel層,在自己的Mac mini上做。目標不是填完進度條,是真的學到東西、留下能重現的benchmark筆記——有機會的話,推一個真正上游專案的PR。目前才剛開始。 A personal learning roadmap — model quantization down to kernel-level work, run on my own Mac mini. The goal isn't finishing a checklist, it's actually learning the material and leaving reproducible benchmark notes behind — and if it goes far enough, landing a real PR upstream. Just getting started.
起點不是規劃出來的,是一連串提問逼出來的:一開始只想知道這台Mac mini上該跑哪個本地模型最快,一路往下追到記憶體頻寬牆、HBM是什麼,才冒出真正的問題——作為工程師,這件事上有沒有施力點?第一版路線圖(量化→投機解碼→kernel)建立在「MLX投機解碼生態是藍海」的判斷上,後來查證發現Ollama、LM Studio其實都已經內建投機解碼,原本設想的空白根本不存在。這次修正留下一句話當篩選判準:如果真的這麼簡單,一定早就有人做了。 This didn't start as a plan — it got forced out by a chain of questions. It began with just wanting to know which local model would run fastest on this Mac mini, which led down into the memory-bandwidth wall and what HBM even is, until the real question surfaced: as an engineer, is there actually a place to push here? The first version of the roadmap (quantization → speculative decoding → kernel work) rested on a bet that MLX's speculative-decoding ecosystem was underexplored. That bet turned out wrong — Ollama and LM Studio already ship it built in. The correction left behind a sharper filter: if it were really this easy, someone would have already done it.
判斷一個優化能不能同時對訓練跟推論有價值,看的是它鎖定什麼:鎖定共用計算核心、保證輸出不變(FlashAttention是典型)泛化好;利用單邊獨有的不對稱性(投機解碼利用decode序列化、KV-cache分頁利用推論才有的跨請求cache)不會泛化,但這不是缺陷,是這類優化價值的來源。分工邊界也很明確:同一件事第一次做,不管看起來多機械,自己動手做懂;做過幾輪、講得出每一步在幹嘛之後,規模化重複的部分才交給agent。 Whether an optimization generalizes across training and inference comes down to what it locks onto: locking onto a shared compute kernel while guaranteeing identical output (FlashAttention is the textbook case) generalizes well; exploiting an asymmetry that only exists on one side (speculative decoding exploits decode being sequential, paged KV-cache exploits inference-only cross-request caching) doesn't generalize — and that's not a flaw, it's where those optimizations get their value from. The division of labor is just as deliberate: the first time doing something, no matter how mechanical it looks, gets done by hand; only after a few rounds of being able to explain exactly what each step does does the repetitive scaling-up get handed to an agent.