Certain startups have observed that Amazon’s AI chips are not performing as well as Nvidia’s GPUs. AWS Trainium chips are positioned as a more cost-effective alternative to Nvidia GPUs. Amazon currently has a small market share in AI chips, with Nvidia holding over 78% dominance. Amazon is relying on its in-house AI chips to drive its next phase of growth. An internal document indicates that Amazon is still trying to catch up to Nvidia in terms of GPU performance.
AI startup Cohere found that Amazon’s Trainium 1 and 2 chips were not performing as well as Nvidia’s H100 GPUs. Cohere reported limited access to Trainium 2 and frequent service disruptions. Amazon and its chip group, Annapurna Labs, are investigating the performance issues with Cohere. Another startup, Stability AI, expressed similar concerns about the performance of Amazon’s Trainium 2 chips compared to Nvidia’s H100 GPUs.
Amazon’s internally developed Trainium chips are crucial for the company’s competitiveness in the AI-cloud sector. The success of Amazon Web Services hinges on the design of its own data centre chips. If some AWS customers reject Trainium, it could impact Amazon’s future cloud profits. Customer complaints internally highlighted by Amazon demonstrate the challenge it faces in matching Nvidia’s performance.
An Amazon spokesperson appreciates customer feedback to enhance the quality and usage of its chips. AWS claims that its in-house AI chips offer better price performance compared to current-generation GPUs. During an earnings call, Amazon CEO Andy Jassy mentioned that Trainium 2 chips are highly sought after and have become a significant business for the company.
Other AWS customers have also raised issues about Amazon’s AI chips. For instance, a startup called Typhoon found Nvidia’s A100 GPUs to be more cost-efficient than AWS’s Inferentia 2 chips. Similarly, research by AI Singapore revealed that AWS G6 servers with Nvidia GPUs provided better cost performance than Inferentia 2.
The struggle between Amazon and Nvidia is evident in market share dynamics, with Nvidia dominating the market share. A recent partnership worth $38 billion between AWS and OpenAI underscores Amazon’s challenges in this area. The partnership involves AI cloud servers exclusively equipped with Nvidia GPUs without any mention of Trainium processors, which has been viewed as disappointing by analysts.
The technical limitations and comparative issues between Amazon’s custom AI chips and Nvidia GPUs have become critical blockers for customers, according to an internal document from July. Anthropic, a prominent customer of Trainium, is expected to deploy over 1 million Trainium 2 chips by year-end, which could significantly benefit Amazon’s efforts.
Investors were surprised by Anthropic expanding its partnership with Google last month, but Amazon reassured them that Anthropic continues to increase its use of Trainium chips. Jassy emphasised AWS’s commitment to providing multiple chip options during the earnings call with analysts last week, leading to a surge in Amazon shares following strong AWS revenue growth reported last quarter.