Google has opened its TPU Raiden, an inference optimization library for its AI chips, to the public as open source on GitHub. It uses the Apache license, allowing anyone to use and modify it for free. Semiconductor analysis firm SemiAnalysis was the first to report the release on the 9th (local time), noting that “Google is increasingly making TPU software available outside the company.”
The tug-of-war in AI chip competition is shifting from hardware to software. Google has now made public a core TPU technology that had previously been used only internally.
Google has released “TPU Raiden,” an inference optimization library for its own AI chip TPU, as open source on the developer-sharing site GitHub. It is available under the Apache license, which allows anyone to use and modify it for free. SemiAnalysis first reported the release on the 9th (local time), saying, “Google is increasingly putting TPU software into the hands of outsiders.”
◆ Key technology for generating AI answers made public
To explain the terminology, TPU is a semiconductor directly designed by Google for AI computing. It is a competing chip that performs the same role as Nvidia GPUs. Inference refers to the stage in which an AI model that has already been trained receives a question and generates an answer. Every moment a chatbot responds is inference, and a large share of global data center costs is spent here.
Raiden handles the logistics of that inference process. It quickly moves temporary memory created as AI reads a question, known as KV cache, between chips.
It includes functions for direct chip-to-chip transfer within a single computer, sending data to another device over the network, and moving memory that is not immediately needed into general memory. Google said it is still an early version under development and not intended for official service use yet.
◆ AI inference runs like kitchen labor division
The usefulness of Raiden is closely tied to how AI services operate today. The process of generating an answer is divided into two stages.
One stage reads the entire question at once, and the other writes the answer letter by letter. Because reading requires heavy computation while writing uses a lot of memory, large services these days split the two tasks across different chip groups.
It is like dividing work in a kitchen. If one team handles ingredient prep and another handles cooking, each can move faster with tools suited to its task. But the success of that division depends on quickly passing the prepared ingredients to the cooking station. In AI, the ingredient equivalent is the KV cache, and the software that delivers it is Raiden.
Nvidia’s ecosystem already has NIXL, a tool that plays the same role. It is built into widely used open-source tools such as vLLM, which companies use to run AI services. If Raiden is integrated into such tools, companies operating services on Nvidia equipment could shift the same workloads to TPU without major redevelopment.
◆ The chips are not sold, but the ecosystem is opening
Google has never sold TPU as a standalone chip. Users had to go through Google Cloud, and the software that made TPU work well was kept internal for services such as Search and Gemini. That began to change over the past year. Google signed a deal to supply 1 million TPUs to Anthropic, and even OpenAI, its biggest rival in model competition, has started using TPU for some tasks.
Analysts say the opening strategy is based on the calculation that as external customers grow, public tools become necessary. No matter how powerful a chip is, companies will not leave Nvidia if they lack engineers, documentation, and tools that know how to use it. One of the main objections to adopting TPU was that businesses did not want to be tied to unfamiliar technology with little documentation, and the release of Raiden is seen as an attempt to chip away at that barrier.
There are also limits. Raiden is TPU-specific, so it is only useful to customers who rent TPU. That is why some developers have described it as a steering wheel useful only to someone who has already brought the ship. Nvidia has argued that its GPUs are still one generation ahead and remain the only platform capable of running all AI models everywhere.
The issue is also relevant to South Korea’s semiconductor industry. Since AI accelerators such as TPU also require large amounts of high-bandwidth memory (HBM), observers say memory demand will continue even if the competitive landscape for compute chips shifts from Nvidia dominance to a multi-player structure. No matter which camp leads, memory must still sit next to the compute chip.
Whether Google can catch up to Nvidia’s software ecosystem, built over 10 years, by gradually opening each piece is the remaining question to watch. The battle over AI chips is being seen as moving beyond performance tables and into a contest over who makes life easier for developers.
