Nvidia has dominated the AI chip market for years, but the era of total dependence might be ending. OpenAI just shared its plans to spice things up with Jalapeño, its custom inference chip built with Broadcom, joining Google, Apple, and SpaceX in a growing list of companies building their way out of single-supplier risk.
The rising cost of Nvidia dependence
Nvidia’s H100 and upcoming B200 GPUs have become the de facto standard for training and running large AI models, but they come at a premium. A single H100 can cost upwards of $30,000, and hyperscalers like Microsoft and Meta have spent billions acquiring them. This has created a bottleneck: supply is constrained, lead times are long, and the cost structure leaves little room for experimentation.
Also read: Claude is quietly winning over paying AI consumers, data shows, even as ChatGPT still dominates
For companies like OpenAI, which runs massive inference workloads for ChatGPT, the economics of relying solely on Nvidia are becoming unsustainable. Custom silicon, even with high upfront development costs, offers a path to lower per-query expenses and better power efficiency.
Jalapeño: OpenAI’s custom inference play
OpenAI’s Jalapeño chip, developed in partnership with Broadcom, is designed specifically for inference — the process of running a trained AI model to generate responses. Unlike Nvidia’s general-purpose GPUs, a custom inference chip can be optimized for the exact mathematical operations OpenAI’s models use, potentially delivering higher throughput at lower cost.
Also read: Databricks' former AI chief claims his new chip can cut AI's energy bill by 1,000x
The move mirrors Google’s strategy with its Tensor Processing Units (TPUs), which power many of its own AI services. By controlling the hardware, these companies can also better manage their supply chains and avoid the kind of shortages that have plagued the industry.
Who else is building their own chips?
OpenAI is far from alone. Google has been using TPUs since 2016, and its latest Trillium chip is designed for both training and inference. Apple has integrated custom neural engines into its A-series and M-series chips for years, powering on-device AI features. SpaceX is reportedly developing custom chips for its Starlink satellite network, which relies on AI for beamforming and network management. Amazon has its Trainium and Inferentia chips for AWS, and Meta is working on its own AI accelerator.
This trend represents a significant shift in the semiconductor industry. Historically, companies like Intel and AMD dominated the CPU market, and Nvidia dominated GPUs. Now, the largest tech firms are increasingly designing their own silicon, often contracting with manufacturers like TSMC for production.
What this means for Nvidia
Nvidia’s revenue from data center chips has soared, reaching over $47 billion in its most recent fiscal year. However, the company faces a strategic challenge: its largest customers are also becoming its competitors. While Nvidia’s software ecosystem, CUDA, remains a powerful moat, custom chips are becoming more viable as AI workloads mature and standardize.
Nvidia is not standing still. The company continues to push performance with new architectures and has expanded its offerings with Grace CPUs and networking hardware. But the days of a single, dominant supplier for AI chips are likely numbered. The question is not whether the market will diversify, but how quickly.
The broader implications
This wave of custom chip development has several consequences. For the AI industry, it could lower the cost of inference, making AI services more accessible and enabling new applications. For the semiconductor supply chain, it increases demand for advanced manufacturing capacity at TSMC and other foundries. For investors, it signals a shift in where value is captured — from chip vendors to the companies that integrate hardware and software.
Regulators are also paying attention. The U.S. government has restricted exports of advanced AI chips to China, and the push for domestic chip manufacturing through the CHIPS Act could accelerate this trend. Companies building custom chips may also seek to source production from U.S.-based fabs for geopolitical reasons.
Frequently Asked Questions
What is OpenAI’s Jalapeño chip?
Jalapeño is a custom inference chip being developed by OpenAI in partnership with Broadcom. It is designed specifically to run AI models after they have been trained, aiming to reduce costs and reliance on Nvidia hardware.
Why are companies moving away from Nvidia?
Companies want to reduce single-supplier risk, lower costs, and optimize hardware for their specific workloads. Nvidia’s GPUs are general-purpose and expensive, so custom chips can offer better performance per dollar for dedicated tasks.
Which other companies are building their own AI chips?
Google has its Tensor Processing Units (TPUs), Apple uses custom silicon in its devices, and SpaceX is reportedly developing chips for its Starlink satellite network. Amazon and Meta also have custom chip projects.
Be the first to comment