Google ships four TPU variants in one year as it ramps up competition with Nvidia

Google ships four TPU variants in one year as it ramps up competition with Nvidia

Key Points

- Google is introducing two new systems in its eighth-generation AI chip lineup, the TPU 8tx and TPU 8i Max, both built on existing processors.

- The TPU 8tx keeps confidential data encrypted during processing, while the TPU 8i Max doubles memory and compute power within the same rack by combining two chips on a single package.

- Together with software tools like TorchTPU, Google aims to lower the barrier for developers and increasingly market its hardware to external customers for day-to-day model inference.

With two new TPU variants, Google is targeting confidential AI workloads and memory-hungry applications. The expansion cements Google's strategic shift toward positioning its in-house AI chips as a broad platform for external customers.

Google has added two new systems to its eighth-generation AI accelerator lineup: the TPU 8tx for confidential computing and the TPU 8i Max for memory-intensive workloads. With these additions, Google says it has shipped four TPU systems within a single year for the first time.

Both new variants are built on existing silicon. Google isn't introducing a new chip generation here, but new system configurations: different packaging, different networking, and different memory setups around the same processor. The company frames this modular approach as a deliberate design choice that lets it deliver new hardware variants in months rather than years.

TPU 8tx keeps sensitive data encrypted even during processing

The TPU 8tx targets enterprises and government agencies working with confidential data. Normally, data is encrypted during storage and transmission but has to be decrypted in memory once a processor works with it.

The TPU 8tx is designed to close that gap. According to Google, data and model weights stay protected and isolated even during active use. Nvidia already offers a comparable feature for GPUs through its confidential computing architecture.

The system uses the same low-latency Boardfly network Google developed for real-time inference and is supposed to deliver up to twice the latency reduction compared to the TPU 8t. Google says enterprises can use a single system to fine-tune models, distill large models into smaller ones, and train AI agents with reinforcement learning, all without moving sensitive data between different clusters.

TPU 8i Max doubles compute power in the same rack

The TPU 8i Max tackles a different problem: the memory bottleneck. When AI agents work simultaneously across large codebases or hours of video, memory bandwidth becomes the chokepoint. Google addressed this by placing two TPU 8i chips on a single package.

The result is double the compute power, double the memory, and double the memory bandwidth in the same rack format, which means significantly more performance per square meter of data center space.

Google lowers the bar for TPU adoption

Beyond hardware, Google is working to make TPUs easier to use. The company has released TorchTPU as open source, so PyTorch code should run on TPUs with minimal changes. Popular inference frameworks like vLLM and SGLang are also supported.

The inference market is driving the custom chip race

The expansion is a direct response to Google's strategic shift from treating TPUs as a purely internal tool to marketing them as a competitive platform for outside customers. The company now counts nearly 100 external TPU customers.

A broader market trend is driving this push. As AI applications spread, the focus is shifting from expensive training runs to inference, meaning the day-to-day operation of models. Specialized chips like TPUs can play to their strengths in exactly this area. That's also why Amazon, Microsoft, and OpenAI are all developing their own accelerators.

Nvidia has responded by bolstering its own inference offering through its acqui-hire and licensing deal with Groq.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now