建立在Syco Eval開源研究上的稽核服務，用來檢查AI有沒有為了討好使用者而說出不實的內容。做法很單純：另外訓練一個獨立的分類器來檢查，而不是讓AI自己回報有沒有諂媚，因為這種自我評估本來就不可靠。 An audit service built on the open-source Syco Eval research, made to catch AI flattering you instead of telling the truth. The method isn't fancy: train a separate classifier to check, rather than asking the AI to self-report its own sycophancy — self-grading like that was never going to be reliable.
核心差異在於：由外部、獨立訓練的分類器檢查AI的輸出，而不是讓被稽核的AI對照規則自己標記。市面上至少有一套開放規格採用後者，讓AI agent自己讀規則、自我監控、自我標記，這和「自我監控不等於驗證」是同一種結構性弱點。這個差異站不站得住，取決於分類器是否真的訓練出來，而不只停留在規格文件上。 The differentiation is direct: an external, independently trained classifier checks the AI's output, instead of letting the AI being audited grade itself against a rulebook. At least one open spec in this space works exactly that way — an AI agent reads the rules and self-monitors, self-flags — which is the same structural weakness as self-report not counting as verification. Whether that differentiation actually holds up depends on the classifier getting built for real, not staying at the spec-document stage.
初期不打算做全自動判定的SaaS，而是從稽核與顧問服務開始：每個案例都由創辦人親自複查，模型負責初步篩選、加快標記速度，不取代人的判斷。成效評估分三層：以人類標記者之間的一致率作為上限；和最便宜的基準線（直接問前沿模型「這算不算諂媚」）比較；以及在客戶真實的對話紀錄上，確認能不能找出現有流程漏掉的案例。 Rather than an autonomous, scaled-up SaaS from day one, this starts as an audit/advisory service: the founder personally reviews every case, with the model doing triage and speeding up labeling, not replacing human judgment. Performance gets evaluated on three layers — inter-rater agreement among human labelers as the ceiling, comparison against the cheapest possible baseline (just asking a frontier model if something's sycophantic), and field validation on a client's real conversation logs to see if it catches cases their existing process misses.
目前還在很早期的階段，介面還沒做出來。等有實際可以看的東西再放上來，不會拿其他畫面充數。 Still early — there's no interface built yet. I'll put one up once there's something real to show, not a stand-in from something else.