AI labs move inference onto custom silicon

Organizations

Updates

1. Aug 28

   OpenAI's first custom inference chip, Jalapeño, delivered 1.5-1.9x more work per watt and 1.7-3.6x lower latency than GB200/GB300 systems on InferenceX and deploys internally by year-end; Nvidia's $12.9B Hugging Face deal backs open models against labs cutting GPU dependence.

   OpenAI's Jalapeño chip posts first results · Nvidia to acquire Hugging Face for $12.9B

2. Aug 21

   Stripe agreed to buy OpenRouter, which routes more than 10 trillion tokens a day across about 400 models, for a reported $7.5B, while Etched doubled its valuation to $21B in 26 days on inference-ASIC demand and Velaura AI raised $110M for low-power AI silicon.

   Stripe to acquire OpenRouter for $7.5B · Etched raises $700M at $21B · Velaura AI raises $110M at $1B+

3. Aug 7

   Anthropic is assembling a team to co-design an inference-optimized chip with its future models, and AMD agreed to buy Taalas, whose HC1 chip casts Llama 3.1 8B weights into transistors to serve about 17,000 tokens per second.

   Anthropic to build a custom inference chip · AMD to acquire Taalas

4. Jul 24

   Etched raised $300M at a $10.3B valuation in a Sequoia-led round with SK Hynix participating, for Sohu, a transformer-specific inference ASIC on TSMC's 4nm process already backed by roughly $1B of signed orders.

   Etched raises $300M at $10.3B

5. Jul 10

   Reuters and The Information reported that DeepSeek and Zhipu are both designing custom inference chips to cut dependence on Nvidia and Huawei, with DeepSeek's move coinciding with its first outside raise of about $7B.

   DeepSeek and Zhipu design in-house inference chips

6. Jun 26New

   OpenAI unveiled Jalapeño, its first LLM-inference accelerator, co-developed with Broadcom and Celestica, taped out in nine months with help from its own models, and aimed at gigawatt-scale deployment with Microsoft and other partners by end-2026.

   OpenAI and Broadcom unveil the Jalapeño chip

Predictions

1. Dec 2026

   Owned inference economics (performance per watt and serving cost) become an explicit competitive and procurement axis, with more lab-designed accelerators and disclosed efficiency figures surfacing as audited unit-economics pressure builds ahead of frontier-lab listings. Jun 26

2. Jan 2027

   Owned inference silicon and disclosed efficiency figures spread further, with Chinese labs pairing chip efforts with domestic fabrication and large capital raises to fund them. Jul 10

3. Jan 2027

   Purpose-built inference silicon moves from thesis to deployment, and cost per completed task remains the primary axis of model competition. Jul 24

4. Feb 2027

   Every frontier lab pursues co-designed or model-specific inference silicon, and multi-silicon strategies spanning GPUs, TPUs, ASICs, and model-specific chips become the default posture rather than the exception. Aug 7

5. Feb 2027

   Token routing, metering, and spend governance consolidate into a strategic layer, and control of model-consumption economics becomes as contested as model capability itself. Aug 21

6. Feb 2027

   Every major lab pairs GPUs with co-designed inference chips, and performance-per-watt at a fixed latency becomes the headline metric operators optimize. Aug 28