Variance
用RAG回答SEC財報相關的問題，但重點在另一半：每個含數字的答案，都會和SEC官方的XBRL結構化資料比對，而不是模型說了算。目前只做蘋果一家公司，模型全部在本機執行；評估結果會附上信賴區間，不會只給一個準確率數字就結束。 Answers questions about SEC filings with RAG — but the real point is the other half: every numeric claim gets checked against SEC's own structured XBRL data instead of just being trusted. Scoped to one company (Apple) for now, runs entirely on local models, and the eval results show honest confidence intervals instead of a single accuracy number pretending to settle it.
背後的假設來自「lazy prices」：財報文字（風險因子、MD&A）以及它每季的變化，帶有市場尚未完全反映的資訊，這是有文獻支持的市場無效率。這個工具分成兩部分：一部分用混合檢索（向量 + BM25，以倒數排名融合）加上本機LLM回答問題，並附上原文出處；另一部分、也是更關鍵的，是評估：自動從XBRL資料產生問題，跑過RAG流程，再把答案和XBRL的真實數字比對。 The underlying thesis is "lazy prices": filing text (risk factors, MD&A) and how it changes quarter to quarter carries information the market hasn't fully priced in yet — a documented market inefficiency. The tool has two halves: hybrid retrieval (dense + BM25, fused via reciprocal rank fusion) plus a local LLM that answers questions with citations back to the filing text, and — the half that actually matters — an evaluator that auto-generates questions from XBRL facts, runs them through the RAG pipeline, and checks the answers against XBRL's real numbers.
最近一次實際測試（AAPL，qwen3.6:35b-mlx，共55題）：單一事實題準確率68.6%（95% CI 52.0–81.4%），複合題（先拆解、再分頭檢索）66.7%（95% CI 41.7–84.8%）。兩個信賴區間重疊，樣本數也還小，因此不能說複合路由比較差。路由本身把每一題都拆對了，拆解準確率100%。 Latest real run (AAPL, qwen3.6:35b-mlx, 55 questions): 68.6% accuracy on single-fact questions (95% CI 52.0–81.4%), 66.7% on compound questions that get decomposed and retrieved separately (95% CI 41.7–84.8%). The two intervals overlap and the sample is still small, so this isn't evidence the router makes things worse — and the router itself split every compound question into exactly the right parts, a 100% decomposition match rate.