Google announced today that it will be adding two additional Tensor Processing Units (TPUs) to its existing family of artificial intelligence hardware products. This addition represents a further expansion of Google’s presence within the rapidly evolving space of artificial intelligence computing infrastructure. Today’s announcement is consistent with the general movement of the technology industry toward developing application-specific processors capable of supporting the increasing complexity of artificial intelligence applications.
The two new systems (TPU 8tx and TPU 8i Max) each serve a unique purpose in terms of computing needs. One system is intended to provide secure artificial intelligence processing to support government operations and regulated industries. The second system will be able to handle workloads requiring significant memory capacity and the ability to analyze and interpret vast amounts of data.
This is a very important development, as AI companies are now competing with each other based upon both the capabilities of the models that they develop, as well as the equipment that is needed to run and train those models in an efficient manner.
Why Google is developing specialised AI processors
For years, Google’s TPUs have supported many of the company’s internal products and services (such as Search, YouTube, and Gemini). In addition, Google has expanded access to these processors via Google Cloud, providing businesses with the ability to leverage Google’s computing power to support their own AI workload.
Google is putting itself into a more competitive position with respect to Nvidia. Nvidia’s GPUs are now at the center of how many organizations in the technology world do AI training and inference.
There are many ways to design specialized artificial intelligence (AI) processors. Each design can be tailored to efficiently execute specific types of computations. As such, each design has the potential to enhance performance, reduce energy consumption, and/or lower operating costs associated with running AI-based software.
Developing proprietary chips may also enable cloud providers to decrease dependence upon third-party chip manufacturers and provide them with increased control regarding the design of their computer architecture.
Google’s announcement is made at a time when companies are looking into the use of artificial intelligence (AI) systems for such applications as writing code, analyzing large documents, aiding researchers and performing multiple steps in a process with little or no human interaction.
TPU 8tx focuses on secure processing
The TPU 8tx was developed for use by federal, state and local government agencies and for-profit companies which are subject to regulations requiring them to keep their data secure while they are working on it.
According to Google, the system uses both of their own TPU technology along with AMD processors and has built-in hardware based encryption. They also stated that this system is able to provide twice the amount of performance of their previous system while having little effect on performance.
Security is especially critical for organizations handling sensitive financial records, government information, medical data, or proprietary business documentation.
Artificial intelligence implementation into these systems will require more than just an advanced algorithm. Businesses must take into consideration access controls, encryption, data storage policies, governmental regulations and whether there may be a risk that some sensitive data could be disclosed when processed.
While hardware-level security measures may be part of an overall security strategy, there will always be some degree of risk. Software vulnerabilities, improper access control, and improperly configured systems continue to pose risks to the security of an organization.
In order for this platform to be effective, it will rely on the way in which the hardware protections provided by the platform interact with both the security controls used in the cloud and an organization’s internal processes.
TPU 8i Max targets demanding AI applications
Google has announced the availability of a second TPU system, called TPU 8i Max. This new system is targeted for applications that require processing of very large amounts of data simultaneously. According to Google, this new system uses two chips within one package to increase both memory and computing capabilities.
This approach may have value when applying artificial intelligence to complex problems with large amounts of contextual information.
Software development could be an area where AI coding assistants could potentially use their abilities. Coding assistants based on AI will most likely need to review many different files containing code, recognize how these files interact with each other, find problems within the code, and suggest ways to fix those problems in various areas of the software.
As well as being useful in scientific research, such as drug discovery, mathematical reasoning, etc., memory-intensive computing has many applications that require analyzing large amounts of data or require heavy computational resources.
However, even if you increase your hardware resources, that does not mean that your performance will improve. Performance can be influenced by several other factors such as the way the model is constructed, how well the software has been optimized, the quality of the data used for training and testing, and the way in which the AI application itself is designed.
Businesses need to examine real world outcomes, cost of operation and how well the new technology works within the framework of their current hardware systems prior to determining if there are advantages in using customized hardware.
What this means for the AI industry
Google’s recent news signals an industry wide shift in how artificial intelligence (AI) is being developed. In addition to developing individual components, many companies have started creating full system solutions which incorporate processing units, network capabilities, storage devices, software and cloud services.
As such, this creates a potential market for cloud providers to provide customized computing platforms tailored to individual customer needs. Additionally, it provides greater competition between chip manufacturers, providing enterprise users with more choices as to where they can deploy and execute their AI workloads.
As a result of increased competition, there is a possibility that businesses may have access to more AI computing resources. However, depending upon the type of task being performed by the business and which company provides the service, there will be variations in pricing, availability and performance.
A key concern over time is whether custom hardware will offer enough benefits to convince more people to switch from using their current GPU-based systems.
The next step of this battle is Google’s new TPU options. The result of this battle will be determined by how well the new hardware performs with real-world workloads, and if developer’s consider adopting the new hardware.