Amazon Web Services (AWS) customers got an unpleasant surprise early Friday morning when their billing dashboards started showing charges that made no sense.
Accounts that normally cost a few cents a month suddenly displayed estimated bills in the millions, billions, and in some extreme cases, trillions of dollars.
The trouble started late Thursday night, around 5:38 AM local time, when AWS’s billing console began showing what the company later called inaccurate estimated billing data.
Amazon said it started investigating about six hours later and traced the problem to an issue with unit pricing inside the system that calculates estimated bills. The company never explained exactly what went wrong with that pricing logic.
The numbers people saw were absurd. Bill Radjewski, who runs the site CollegeFootballData.com, told reporters he’d had his AWS account for more than six years without his monthly spend ever topping two cents.
Friday morning he got an alert saying he’d racked up over $1.5 billion in usage, with his next bill projected to hit $3 billion. He wasn’t close to alone. Other users reported estimated charges of $75 billion, and $110 billion.
One Reddit user’s account showed a monthly bill of nearly $2.5 billion for a person whose actual usage typically cost less than a dollar. Another user’s dashboard claimed they owed $7.1 trillion in fees since the start of the month, more than twice Amazon’s entire market value.
Amazon tried to fix things by rolling back the recent change that seemed to have caused the bug, but that didn’t work. The company then paused the billing estimate calculations entirely, meaning the inflated numbers already on screen would stay frozen rather than climb any higher.
Throughout the ordeal, Amazon repeated the same reassurance: these were estimates, not real charges, and no customer needed to do anything or pay anything as a result.
Amazon didn’t say how many customers were affected, though its own status dashboard described the outage as global.
A company spokesperson largely declined to comment beyond pointing reporters to the status page, and Amazon never detailed whether any accounts had been suspended or paused because of the erroneous figures.
By today evening, Amazon said it had found and fixed the underlying cause and started correcting the numbers in customers’ accounts. The company said everyone should see accurate billing figures by Saturday, July 18.
