OpenAI Explores Alternatives to Nvidia AI Chips Amid Strategic Shift
OpenAI, the creator of ChatGPT, has expressed dissatisfaction with certain artificial intelligence chips from Nvidia and has actively sought alternatives since last year, according to eight sources familiar with the matter. This development could potentially strain the relationship between two of the most prominent players in the ongoing AI boom.
Strategic Shift Towards Inference Chips
The details of OpenAI's strategic shift, reported here for the first time, revolve around an increasing emphasis on chips designed for specific elements of AI inference. Inference refers to the process where an AI model, such as the one powering the ChatGPT application, responds to customer queries and requests. While Nvidia continues to dominate the market for chips used in training large AI models, inference has emerged as a new competitive frontier in the industry.
This decision by OpenAI and other companies to explore alternatives in the inference chip market represents a significant test of Nvidia's dominance in artificial intelligence. The timing is particularly noteworthy as the two companies are engaged in investment negotiations.
Investment Talks and Changing Requirements
In September, Nvidia announced its intention to invest up to $100 billion in OpenAI as part of a deal that would grant the chipmaker a stake in the startup while providing OpenAI with the necessary funds to purchase advanced chips. Reuters initially reported that the deal was expected to close within weeks, but negotiations have instead extended for several months.
During this period, OpenAI has entered into agreements with AMD and other companies for GPUs designed to compete with Nvidia's offerings. However, OpenAI's evolving product roadmap has altered its computational resource requirements, complicating talks with Nvidia, according to a person familiar with the discussions.
Public Statements and Private Concerns
On Saturday, Nvidia CEO Jensen Huang dismissed reports of tension with OpenAI, labeling the idea as "nonsense" and reaffirming Nvidia's plans for a substantial investment in the AI company. Nvidia stated in an official response, "Customers continue to choose NVIDIA for inference because we deliver the best performance and total cost of ownership at scale."
An OpenAI spokesperson separately acknowledged that the company relies on Nvidia for the majority of its inference fleet, noting that Nvidia offers the best performance per dollar for inference tasks. Following the Reuters report, OpenAI Chief Executive Sam Altman posted on X, describing Nvidia as producing "the best AI chips in the world" and expressing hope that OpenAI would remain a "gigantic customer for a very long time."
Performance Issues and Alternative Solutions
Seven sources revealed that OpenAI is dissatisfied with the speed at which Nvidia's hardware processes responses for ChatGPT users, particularly for specific problem types such as software development and AI communication with other software. One source indicated that OpenAI requires new hardware to eventually meet approximately 10% of its inference computing needs in the future.
The ChatGPT maker has explored partnerships with startups including Cerebras and Groq to provide chips for faster inference capabilities. However, Nvidia's $20 billion licensing agreement with Groq reportedly halted OpenAI's discussions with the company, according to one source.
Technical Challenges and Market Dynamics
Nvidia's graphics processing chips excel at the massive data processing required for training large AI models like ChatGPT, which have driven the explosive global growth of artificial intelligence. However, recent AI advancements increasingly focus on using trained models for inference and reasoning, potentially representing a new and larger phase of AI development that has inspired OpenAI's efforts.
OpenAI's search for GPU alternatives since last year has concentrated on companies developing chips with substantial amounts of memory embedded directly into the silicon, known as SRAM. Maximizing SRAM on each chip can provide speed advantages for chatbots and other AI systems as they handle requests from millions of users simultaneously.
Inference typically requires more memory than training because chips spend relatively more time retrieving data from memory than performing mathematical operations. Nvidia and AMD GPU technology depends on external memory, which can increase processing time and slow user interactions with chatbots.
Internal Challenges and Competitive Landscape
Within OpenAI, these performance issues became particularly evident with Codex, the company's product for generating computer code, which has been aggressively marketed. OpenAI staff attributed some of Codex's limitations to Nvidia's GPU-based hardware, according to one source.
During a January 30 call with reporters, Altman emphasized that customers using OpenAI's coding models would "put a big premium on speed for coding work." He noted that OpenAI plans to address this demand through its recent agreement with Cerebras, adding that speed is less critical for casual ChatGPT users.
Competing products such as Anthropic's Claude and Google's Gemini benefit from deployments that rely more heavily on Google's in-house tensor processing units (TPUs). These TPUs are specifically designed for inference calculations and can offer performance advantages over general-purpose AI chips like Nvidia's GPUs.
Nvidia's Strategic Moves
As OpenAI made its reservations about Nvidia technology clear, Nvidia approached companies working on SRAM-heavy chips, including Cerebras and Groq, regarding potential acquisitions. Cerebras declined the offer and instead entered into a commercial agreement with OpenAI, announced last month.
Groq engaged in discussions with OpenAI to provide computing power and received investor interest for funding at a valuation of approximately $14 billion. However, by December, Nvidia moved to license Groq's technology in a non-exclusive all-cash deal. Although this arrangement allows other companies to license Groq's technology, Groq is now focusing on selling cloud-based software as Nvidia hired away its chip designers.
Chip industry executives observed that Nvidia's decision to acquire key talent from Groq appeared to be an effort to strengthen its technology portfolio and better compete in the rapidly evolving AI industry. Nvidia stated that Groq's intellectual property was highly complementary to its product roadmap.
