Amazon strives to outpace Nvidia with cheaper, faster AI chips

Amazon’s chip lab is churning out a constant stream of innovation in Austin, Texas. A new server design was put through its paces by a group of devoted engineers on July 26th.

During a visit to the facility in Austin, Amazon executive Rami Sinno shed light on the server’s use of Amazon’s AI chips. This development is a bold step toward competing with Nvidia, the current leader in the field.

The main reason Amazon is developing its own processor is this: it doesn’t want to rely on Nvidia and buy the company’s chips. The expensive Nvidia chips power a big part of the AI cloud business at Amazon Web Services. This business is the most significant growth engine of the company. Thus, the so-called “Nvidia tax” was pushing the company to look for a cheaper option.

Amazon’s chip development program has a dual purpose. Firstly, the project is meant to provide customers with more affordable opportunities for complex calculations and large data volume processing. Secondly, the initiative was developed to preserve Amazon’s competitiveness in the volatile cloud computing and AI industry. This move was also supported by the directions of tech giants such as Microsoft and Alphabet, which are developing custom-made chips to maintain their leadership in the market.

Rami Sinno, director of engineering for Amazon’s Annapurna Labs, a key element of the AWS ecosystem, emphasised that customer demand for more economical solutions to Nvidia’s products is growing. The acquisition of Annapurna Labs in 2015 was a savvy move by Amazon as it enabled the company to lay the groundwork to begin developing popular chips.

Although Amazon’s chips for AI are in their early days, the company has been making and refining chips for other mainstream applications for nearly a decade, most notably its general-purpose chip, Graviton, which is now in its fourth generation. Amazon has announced that its Trainium and Inferentia chips, the company’s latest and strongest, are still in their early days and are specially designed processors.

The impact is potentially huge because the impressive performance underscores the reports by David Brown, vice president of compute and networking at AWS. In this light, it should be acknowledged that Amazon’s in-house chips could deliver up to a 40-50% price-performance ratio improvement compared to Nvidia-based solutions. In turn, this potential improvement could mean considerable savings for AWS clientele deploying their AI workloads.

AWS’ significance to Amazon’s overall business cannot be underestimated. In the first quarter of this year, AWS made up a little under a fifth of Amazon’s total revenue, as its sales soared by 17 per cent year over year to reach $25 billion. At the moment, AWS holds about a third of the global cloud computing market, and Microsoft’s Azure covers about a quarter, or 25%.

Amazon’s commitment to its custom chip strategy was demonstrated during the recent Prime Day, a two-day sales event at Amazon.com. To handle the highly elevated level of shopping as well as streaming video, music, and other content, Amazon deployed an impressive 250,000 Graviton chips and 80,000 of its custom AI chips across its platforms. Adobe Analytics announced record Prime Day results of $14.2 billion in sales.

It seems that as Amazon intensifies its work on the development of AI chips, the industry leader, Nvidia, is not going to remain at the same level. Nvidia’s CEO, Jensen Huang, has presented Nvidia’s latest Blackwell chips, which are scheduled for release later in the year. Their performance has increased significantly, and Huang promised that the new chips are twice as powerful for AI model training and five times faster for inference.

Nvidia’s dominant position in the AI chip market is underscored by its impressive client list, which includes tech giants like Amazon, Google, Microsoft, OpenAI, and Meta. The company’s focus on AI has propelled its market value to a staggering $2 trillion, making it the third most valuable company globally, behind only Microsoft and Apple.

As the AI chip race intensifies, Nvidia is also diversifying its offerings. The company has introduced new software tools to facilitate AI integration across various industries and is developing specialised chips for emerging applications such as in-car chatbots and humanoid robots.

(Image by Gerd Altmann)

See also: Nvidia: World’s most valuable company under French antitrust fire

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Tags: ai, Amazon, Nvidia, processors