TOON Won’t Cut Your LLM Bill in Half: Fix Bloated Responses First
TOON is genuinely useful for big structured blobs. But most LLM cost in real apps doesn’t come from JSON in your prompts – it comes from over-polite, overlong answers and chat history bloat. If you want a smaller bill, don’t just compress data. Teach your stack to shut up sooner.