Apple's paper revealed that a large-scale server language model, the server Apple Basic Model (AFM), was trained on 8,192 Google TPUv4 chips to perform 6.3 trilliontoken training; end-side AFM was trained on 2048 TPUv5p chips; and AFM and AFM services were trained on a “cloud TPU cluster”.
Author of this article: Li Dan
Source: Hard AI
According to public documents, Apple Intelligence, the development of its own artificial intelligence (AI) system, is inseparable from the support of Google's customized chips.
On Monday July 29th EST, Apple's official website published a technical paper detailing some basic language models developed to support Apple Intelligence, Apple's personal intelligence system, including an approximately 3 billion parameter model for efficient operation on devices — the end-side “Apple Foundation Model” (AFM), and a large-scale server language model designed for Apple's cloud AI architecture “Private Cloud Compute” (Private Cloud Compute) — server AFM.
In the paper, Apple explains that end-side AFM and server AFM are members of the generative model family developed by Apple. These models are all used to support users and developers. In the paper, Apple revealed that the training model used the fourth-generation AI ASIC chip TPUv4 developed by Google and the newer generation chip TPUv5. The article reads:
“We trained the 6.3 trilliontoken from scratch on 8,192 TPUv4 chips using a sequence length of 4096 and a batch size of 4096 sequences.”
“End-side AFM trains on 2048 TPUv5p chips.”
In this 47-page paper, Apple did not mention the names of Google or Nvidia, but stated that its AFM and AFM services are trained on “cloud TPU clusters.” This means that Apple leases servers from cloud service providers to perform calculations.
In fact, during the World Developers Conference (WWDC) in June of this year, the media already discovered in the details of technical documents released by Apple that Google has become another winner of Apple's efforts in the field of AI. Apple's engineers used the company's self-developed framework software and various hardware to build the basic model, including a tensor processing unit (TPU) that can only be used on Google Cloud. However, Apple did not disclose how much Apple relies on Google's chips and software compared to other AI hardware vendors such as Nvidia.
As a result, comments on social media X this Monday indicated that news of Apple's use of Google chips came out in June, and now we have more details about the training stack.
Some commentators say that Apple doesn't hate Nvidia; it's just that TPU is faster. There are also reviews that TPU is faster, so it makes sense for Apple to use it, and of course it may be cheaper than Nvidia's chip.
Google's TPU was initially created for internal workloads and is now being used more widely, according to a media review this Monday. Apple's decision to use Google's chip training model suggests that in terms of AI training, some tech giants may be looking for and have found alternatives to Nvidia's AI chips.
Wall Street News mentioned that last week, Meta CEO Zuckerberg and Alphabet and Google CEO Pichay both hinted in their speeches that their company and other tech companies may be overinvesting in AI infrastructure and “probably investing too much in AI.” But at the same time, they all acknowledge that the business risk would be too high if this were not done.
Zuckerberg said:
“The consequence of falling behind is that you will be disadvantaged in the most important technology for the next 10 to 15 years.”
Pichay said:
AI is expensive, but the risk of underinvestment is greater. Google may be investing too much in AI infrastructure, mainly including buying Nvidia's GPUs. Even if the AI boom slows down, data centers and computer chips purchased by companies can be used for other purposes. For us, the risk of underinvesting far outweighs the risk of overinvesting.