Post

Amazon saves millions by replacing Sonnet 4.6 with Strands Decider

Amazon reports replacing a general-purpose LLM with a smaller decision model for high-volume ticket classification, cutting estimated inference costs from about $22,200 to $500 per million tickets at published prices.

The comparison uses a shorter prompt and drops older comments from long tickets to fit the smaller model’s context; it is not a like-for-like general-purpose model comparison. The team reports comparing both paths on live traffic and judging disagreements. It also found JSON key order could change output probabilities.

HN readers discussed the longstanding role of lightweight classifiers and the trade-off between their data-labeling costs and the convenience of a general model.