TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Two Amazon engineers say they replaced Claude Sonnet 4.6 with Strands Decider E4B to label tickets in a large internal system. They report comparable labeling results on a separate test set and estimate a 98% reduction in model costs, from about $22,200 to $500 per million tickets at stated usage assumptions.
Two Amazon engineers say they replaced Claude Sonnet 4.6 with Strands Decider E4B for labeling tickets in an internal system, reporting comparable results on a separate test set at an estimated 98% lower model cost. Their published comparison puts the cost at about $500 per million tickets for Decider, versus $22,200 for Sonnet, using specified AWS list prices and the team’s production utilization assumptions.
The engineers, identified in the Strands Agents report as Carr and Mehul, describe a system that sorts company tickets into 20 categories. Those tickets can include customer escalations, capacity requests, automated alarms and security findings. Previously, the system used Sonnet 4.6 through Amazon Bedrock, with a long prompt asking the model to reason over the available labels and return JSON.
Strands Decider uses a different format: it receives a ticket as the state and the label choices as options, then returns probabilities and a confidence score rather than generating a text response. The team says it began with Strands Decider 2B, then selected an E4B Gemma 4 build after it performed better in its tests. After tuning the confidence threshold, the engineers report that E4B beat the production Sonnet setup on a separate set of tickets. The source does not publish the test-set size or an accuracy score.
The reported average call time fell from 4,215 milliseconds to 897 milliseconds, a 78% decrease. Input tokens per ticket dropped from 6,554 to 2,009, while output tokens fell from 170 to one. The team also says average Lambda execution duration dropped 74% on the day Sonnet use was turned off. These are figures from the engineers’ account, rather than an independently audited comparison.
Lower Costs for High-Volume Labeling
The reported savings matter because the system handles tickets at high volume, where small per-item costs can add up. Using the post’s stated prices and estimates, a million tickets cost about $21,700 less with Strands Decider than with Sonnet 4.6. The authors say the change could save millions of dollars in annual token costs, but they do not provide the system’s annual ticket count or an audited annual savings figure.
The result also shows a practical trade-off in model selection: a generative language model may be more capable than a task requires. For a fixed set of choices, a model that selects among options can avoid producing a lengthy explanation or structured response on every request. Faster calls may also reduce time spent in the surrounding processing path. The engineers report a shorter Lambda duration, though the published cost comparison focuses on model costs and excludes Lambda.
The figures are not a universal price comparison. Sonnet’s per-ticket estimate comes from its token usage and listed Bedrock rates. Decider’s estimate depends on how fully its SageMaker instance is used: the team says a continuously busy instance could process about 5,600 tickets an hour, while its practical estimate of about $500 per million allows capacity for traffic bursts. Results elsewhere could differ with workload, hardware utilization, ticket length and label quality requirements.
Products Worth Considering
Amazon Kindle Paperwhite (2024, 16 GB) – 7-inch Glare-Free Display, Black
Amazon Kindle Paperwhite Signature Edition (12th Generation, 2024), 32 GB, Metallic Raspberry
Amazon Kindle Scribe (3rd Generation), 64GB, 11-inch Display, Graphite, with Premium Pen
As an affiliate, we earn on qualifying purchases.
How the Team Tested the Switch
The engineers did not move directly from one model to the other. They describe a three-setting rollout controlled through AWS AppConfig. In BEDROCK mode, Sonnet labels tickets alone. In DUAL mode, the two systems run in parallel, Sonnet’s result is applied, and the outputs are compared. In DECIDER mode, Strands Decider labels tickets without calling Bedrock.
That comparison path let the team examine disagreements on live ticket traffic before relying on the new model. The report says the switch could be reversed with a configuration change taking about 45 seconds. The rest of the described setup uses AWS services including SQS, Lambda, SageMaker, CloudWatch and CDK, with deployment stages for Beta, Gamma and production.
The team ran the E4B model on SageMaker ml.g5.xlarge instances with an A10G GPU and 24 GB of memory. It says the model used about 19 GiB and took about 640 milliseconds per request on one instance. To fit the decision-model format, the engineers shortened descriptions for the 20 labels into one- or two-sentence criteria and added a “none” option. For long tickets, the model input drops the oldest comments to stay within its context limit.
“The results showed E4B beating production Sonnet on a separate set of our tickets.”
— Carr and Mehul, Amazon engineers, in the Strands Agents report
Limits of the Published Comparison
The report does not state the test-set size, publish category-by-category accuracy, or explain how “performed just as well” was measured. It says E4B beat production Sonnet on a separate ticket set, but supplies no numerical quality result or independent evaluation. The cost and performance claims should therefore be understood as the engineers’ account of their internal deployment.
The annual savings claim also lacks the ticket volume and accounting period needed to verify a total. The post describes million-ticket costs using list prices, with Decider’s practical estimate depending on instance utilization and burst capacity. It excludes Lambda costs from the comparison, although the team says Lambda ran for less time after the change.
It is also unclear how often the ticket catalog changes, how the team will monitor classification quality over time, or whether it plans to apply the approach to other workflows. The engineers caution that the method may not suit tasks needing explanatory text, frequently changing labels, or more useful context than can fit into a few thousand tokens.
Monitoring the New Labeling Path
The engineers’ report describes Strands Decider as the live labeling path after the team turned off Sonnet for this workload. It does not announce a broader rollout or give a schedule for one. Continued monitoring of label quality, confidence thresholds, latency and instance utilization will be relevant to whether the reported savings hold as traffic and ticket content change.
The team says it sorted JSON keys after finding that key order could shift the model’s output probabilities. That is a reminder that evaluation depends on keeping request formatting consistent, not just comparing model names. Readers considering a similar change would need to compare both systems on representative traffic, inspect disagreements and account for capacity headroom as well as list prices.
The source points readers to the Strands Decider repository and serving notes, but does not provide a follow-up date or promise additional results. Further details about accuracy, workload volume and realized annual savings remain unreported.
Key Questions
What did Amazon replace Sonnet 4.6 with?
The engineers say they replaced Claude Sonnet 4.6 with Strands Decider E4B for labeling tickets in an internal system. The model selects from a fixed set of labels rather than generating a JSON answer.
How much cheaper did the engineers say it was?
The report estimates about $500 per million tickets for Strands Decider under the team’s practical utilization assumptions, compared with about $22,200 for Sonnet 4.6 at the listed prices. The authors describe that as a 98% cost reduction; it is not an independently audited figure.
Did the new model label tickets as accurately?
The engineers say E4B performed comparably and beat production Sonnet on a separate set of tickets after they tuned its confidence threshold. The post does not provide the test-set size, accuracy scores or category-level results.
Why might this approach not work for every task?
The team says decision models fit tasks with a fixed set of choices and compact input. They may be a poor fit when labels change often, results need explanatory text, or relevant context cannot fit within the model’s input limit.
Source: strandsagents.com
