What other surprises can artificial intelligence chips bring to the market in 2024?

来源：半导体行业观察

2023年，随着以大语言模型为代表的人工智能市场持续火爆，我们看到了人工智能成为了半导体行业的最大推动力，也见证了Nvidia惊人的销售业绩以及其市值创下新高。随着新年的到来，我们也对2024年人工智能芯片做一个展望。

市场需求：人工智能仍将继续火热

从市场需求侧来看，我们认为人工智能在2024年仍然将继续火热，这也将继续推动相关的芯片行业保持良好的态势。然而，与2023年不同的是，我们认为在2024年，人工智能市场的需求会同时从云端慢慢扩展到终端，这也将驱动相应的芯片市场发展。

首先，从云端需求来看，大语言模型仍然将是主要的增长点。同时，图像生成类模型也会保持快速增长的势头。具体来说，大语言模型仍然是各大科技公司竞相研发的核心技术，包括OpenAI，微软，谷歌，华为，阿里巴巴，百度等中外科技公司都在大力研发下一代的大语言模型，而包括中国移动等传统行业的公司也在入局大语言模型领域，同时还有大量的初创公司在依靠风险投资的支持也在大力开发大语言模型。大语言模型的百家争鸣时代才刚刚开始并且远远没有落下帷幕，在这样的群雄逐鹿的时间节点，预计对于芯片的需求也会快速增长。大语言模型的特点是需要海量的数据和训练芯片资源，而且同时由于格局尚未尘埃落定有大量的公司在研发新的模型，因此总体来说训练芯片的需求会非常大。

而随着云端人工智能交互进入多模态时代，聊天机器人已经不仅仅可以用文字回答，而且还可以完成看图说话以及图像甚至视频生成等任务，因此我们认为，图像生成类的模型，以及图像和语言结合的多模态模型也将会成为云端人工智能的一个重要增长点。

除了云端之外，我们认为终端（包括手机和智能车）也将会成为人工智能的新增长点。手机上的人工智能虽然早已不再新鲜，但是随着生成模型的成熟，我们可望看到这类模型落地手机端并赋能新的用户体验。手机端生成类模型又分为两种，一种是图像生成类模型，即以扩散模型（diffusion model）为代表的模型，这类模型可以实现高质量超分辨以及高质量修图，从而可望给用户的拍照和照片编辑带来革命性的变化。另一类应用是语言模型——与运行在云端的大语言模型（LLM）相对地，过去几个月我们看到了小语言模型（SLM）的兴起。小语言模型和大语言模型一样，都是主要用于语言的理解和生成（或者说，与人对话）；小语言模型在参数量变少之后，可以更灵活地应用在一些专用的场景中（而不是像大语言模型一样试图覆盖所有场景）并且提供很高的准确性，同时还有可能运行在终端设备中。

从智能车领域，一方面随着端到端多任务大模型带来的革命性性能提升（例如BEVFormer带来的俯瞰场景识别性能提升，以及商汤在2023发布的UniAD带来的多任务性能大幅提升）会推动这类模型进一步落地并且推动芯片的需求，另一方面则是来自于语言模型人机交互这类源自于云端的人工智能应用迁移到智能车场景。

因此，我们预测，2024年将会是人工智能持续火热的一年，与2023年不同的是除了云端人工智能保持热门之外，我们预计终端应用场景也会成为新的人工智能需求增长点。

云端市场格局分析

云端人工智能芯片市场，我们预计Nvidia仍然将保持领跑地位，但是AMD等竞争者预计也将获得更多市场份额。

首先，如前所述，云端市场目前主要的需求在于大语言模型和生成式图像的训练和推理。由于这些模型对于计算资源需求非常大，而且训练任务占的比例很大，因此给相应的芯片设了一个很高的门槛。这里的门槛包括了：

- 芯片算力：为了支持巨大的计算量，芯片需要有足够的计算单元，内存容量和带宽

- 分布式计算支持：对于大模型来说分布式计算是必须的

- 软件兼容性和生态：对于训练来说，对于模型进行反复快速迭代是一个很强的需求，因此必须有足够好的生态来支持不同的模型算符快速迭代

目前而言，Nvidia在这一个领域仍然是领先，无论是其芯片和分布式计算性能，还是软件生态兼容性来说，都是首选。这也是为什么2023年Nvidia的H100成为了人工智能相关公司最宝贵的资源，在市场上供不应求。在2024 H2，Nvidia将会开始出货H200，相对于H100来说，H200拥有40%和内存带宽提升以及80%的内存容量提升，因此可望会被人工智能公司争相购买。

在2024年，我们预计AMD在云端人工智能领域也将更加站稳脚跟，并由此开始慢慢走向更大的市场份额。2023下半年，AMD发布了最新的MI300X用于高性能计算的GPU模组，该芯片包含大芯片粒（12个处理器/IO芯片粒），并且相比于H200来说，拥有更高（1.6倍）的FP8算力，以及更大的内存容量和带宽（1.2倍）。从AMD公布的实测数据来看，MI300X的推理能力大约比H100强20%-40%，训练性能则和H100持平。我们认为，软件生态（包括编译器性能）将成为决定AMD在云端人工智能市场能否成功的决定性因素，而这一点预计在2024年将会有所改善：OpenAI将在最新发布的Triton框架中加入对于AMD MI300X的支持，而各大初创公司的人工智能加速软件框架也在加强对于AMD GPU的支持。随着芯片性能和软件生态的提升，以及各大科技公司对于Nvidia GPU一家独大地位的担忧态度，我们预计2024年对于AMD的GPU在人工智能市场将会是重要的一年，预计将会看到更多客户的应用。

从供应链角度，由于云端人工智能芯片对于HBM3等高带宽内存有着强力的需求，我们认为HBM内存以及高级封装（如CoWoS）的产能仍然将会火热，这也将推动相应半导体企业扩大产能，以及积极研发下一代内存和高级封装技术。从这个角度看，人工智能应用仍然将会是推动半导体新技术高速发展的核心动力。

终端市场格局分析

除了云端市场外，我们预计人工智能对于终端市场也有更强的需求，这也将推动人工智能成为终端计算芯片中越来越重要的差异化元素。

在手机端，人工智能的使用频率将会进一步增强，这也会推动芯片加入更多相应的算力，并且将人工智能支持作为SoC的核心亮点。例如高通发布的Snapdragon 8 Gen 3就以“低于一秒的延迟实现图像生成任务”作为一个重要卖点，估计这样的人工智能能力将会深度集成到手机厂商的操作系统中。除了高通这样的第三方芯片公司之外，自研手机芯片的系统厂商预计也会继续加码人工智能；苹果虽然在这方面保持低调，但是预计在未来会以各种方式（加大NPU算力，或者是加大软件支持）的方法来开始赋能更多人工智能在iPhone上的新拍摄用户体验。vivo在自研ISP芯片领域已经有了数年积累，而目前生成式人工智能对于拍摄体验的提升恰好和ISP有很强的协同作用，这也是2023年8月vivo发布的v3 ISP芯片强调了生成式人工智能作为一个核心亮点。未来，预计会看到越来越多这样的芯片强调人工智能对于用户体验上的赋能作用。

在智能汽车领域，Nvidia虽然不如在云端强势，但是其Orin系列芯片仍然是各大车厂考虑的标准芯片模组。我们认为，随着人工智能大模型对于智能驾驶的赋能作用逐渐加强，无论是第三方芯片还是车厂自研的芯片都将会进一步强化对于人工智能算力的投入，同时也推动芯片性能的快速提升——最近，无论是蔚来发布新自研芯片的算力规格，还是特斯拉传出将使用TSMC 3nm作为下一代芯片的生产工艺，都在提示着我们2024年人工智能将在智能车芯片领域扮演越来越重要的角色。

哪些新技术值得关注？

除了以上讨论的芯片之外，有哪些新技术可望会为人工智能芯片领域带来新的变化？

首先，存内计算和近内存计算/处理技术可望会得到越来越多的关注。对于云端人工智能来说，内存访问开销一直是一个性能瓶颈，而随着大模型的参数量越来越大，内存访问带来的开销也越来越大。存内计算和近内存计算/处理技术的主要目的就是为了降低这样的开销，让一些计算和处理任务能够在内存中就完成。在这个领域，三星的PIM（process in memory）和PNM（process near memory）技术非常值得我们关注，而这些技术也可望会成为三星未来进一步提高自身存储器技术差异化竞争力的关键。

对于终端人工智能来说，智能汽车场景中由于对于延迟有着较强的需求，因此新技术有比较多的机会能产生影响。在云端，以GPU为代表的加速芯片都是主要基于优化吞吐量的考虑，而不是延迟，因此在智能车领域必须有新的架构设计。对于车载应用来说，数据都是以数据流的形式（而不是批量的形式）进入处理器，因此人工智能芯片必须能高速低延迟处理这些数据流。另一方面，大模型正在进入智能车应用，因此如何在低延迟的情况下支持大模型推理，将会是智能车芯片中新技术需要重点突破的方向。

编辑/Jeffrey

Source: Semiconductor Industry Watch

In 2023, as the artificial intelligence market represented by big language models continues to boom, we saw artificial intelligence become the biggest driving force in the semiconductor industry, and also saw Nvidia's amazing sales performance and market capitalization reach new highs. With the arrival of the new year, we are also looking forward to 2024 artificial intelligence chips.

Market demand: Artificial intelligence will continue to be popular

From the market demand side, we believe artificial intelligence will continue to be popular in 2024, which will continue to drive the relevant chip industry to maintain a good trend. However, unlike 2023, we believe that in 2024, demand in the artificial intelligence market will slowly expand from the cloud to terminals at the same time, which will also drive the corresponding chip market development.

First, judging from cloud demand, the big language model will still be the main growth point. At the same time, image generation models will continue to grow rapidly. Specifically, the big language model is still the core technology being developed by major technology companies. Chinese and foreign technology companies such as OpenAI, Microsoft, Google, Huawei, Alibaba, and Baidu are vigorously developing the next generation of big language models, while companies in traditional industries such as China Mobile are also entering the big language model field. At the same time, a large number of startups are also vigorously developing big language models with the support of venture capital. The era of great language models has only just begun and is far from over. Demand for chips is also expected to grow rapidly at this point in time. The characteristics of big language models are that they require massive amounts of data and training chip resources. At the same time, since the dust has not yet settled on the pattern, a large number of companies are developing new models, so the overall demand for training chips will be very large.

As artificial intelligence interaction in the cloud enters the multi-modal era, chatbots can not only answer with words, but can also complete tasks such as reading pictures, speaking, and generating images and even videos. Therefore, we believe that image generation models, as well as multi-modal models combining images and language, will also become an important growth point for cloud artificial intelligence.

In addition to the cloud, we believe that terminals (including mobile phones and smart cars) will also become a new growth point for artificial intelligence. Although artificial intelligence on mobile phones is no longer new, as generative models mature, we can expect to see such models land on mobile phones and enable new user experiences. Mobile phone generation models are also divided into two types. One is an image generation model, that is, a model represented by a diffusion model (diffusion model). This type of model can achieve high-quality super-resolution and high-quality retouching, which is expected to revolutionize users' photography and photo editing. Another type of application is language models — as opposed to big language models (LLMs) running in the cloud, we've seen the rise of small language models (SLM) over the past few months. The small language model, like the big language model, is mainly used for language understanding and generation (or conversation with people); after the number of parameters is reduced, the small language model can be applied more flexibly in specific scenarios (rather than trying to cover all scenarios like the big language model) and provides high accuracy, and is also likely to run on terminal devices.

In the field of smart cars, on the one hand, revolutionary performance improvements brought about by the large end-to-end multi-tasking model (such as the improvement in panoramic scene recognition performance brought by Bevformer and the significant increase in multitasking performance brought by Shang Tang in 2023) will drive the further implementation of such models and drive the demand for chips. On the other hand, artificial intelligence applications originating from the cloud, such as human-computer interaction from language models, migrate to smart car scenarios.

Therefore, we predict that 2024 will continue to be a hot year for artificial intelligence. Unlike 2023, in addition to maintaining popularity of artificial intelligence in the cloud, we also expect that terminal application scenarios will also become a new growth point for artificial intelligence demand.

Cloud market pattern analysis

In the cloud-based artificial intelligence chip market, we expect Nvidia to maintain its leading position, but competitors such as AMD are also expected to gain more market share.

First, as mentioned earlier, the main demand in the cloud market currently lies in the training and inference of large language models and generative images. Since these models require a lot of computational resources and account for a large proportion of training tasks, a high threshold has been set for the corresponding chips. The thresholds here include:

- Chip computing power: In order to support huge computational volumes, chips need sufficient computing units, memory capacity, and bandwidth

- Distributed computing support: distributed computing is a must for large models

- Software compatibility and ecology: For training, iterating models over and over again is a strong requirement, so there must be a good enough ecosystem to support rapid iteration of different model operators

Currently, Nvidia is still leading in this field, and is the first choice in terms of its chip and distributed computing performance, as well as software ecosystem compatibility. This is why in 2023, Nvidia's H100 became the most valuable resource for artificial intelligence-related companies, and is in short supply in the market. In 2024 H2, Nvidia will begin shipping the H200. Compared to the H100, the H200 has 40% increase in memory bandwidth and 80% increase in memory capacity, so it is expected that artificial intelligence companies will compete to buy it.

In 2024, we expect AMD to gain a stronger foothold in the field of artificial intelligence in the cloud, and slowly start moving towards a larger market share as a result. In the second half of 2023, AMD released the latest MI300X GPU module for high-performance computing. The chip contains a large chip (12 processors/IO chips), and compared to the H200, it has higher FP8 computing power (1.6 times), and greater memory capacity and bandwidth (1.2 times). Judging from the measured data released by AMD, the MI300X's reasoning ability is about 20%-40% better than the H100, and the training performance is on par with the H100. We believe that the software ecosystem (including compiler performance) will be a decisive factor in whether AMD can succeed in the cloud AI market, and this is expected to improve in 2024: OpenAI will add support for the AMD MI300X to the newly released Triton framework, and AI acceleration software frameworks from major startups are also strengthening support for AMD GPUs. With the improvement of chip performance and the software ecosystem, and the concerns of major technology companies about the dominant position of Nvidia GPUs, we expect 2024 to be an important year for AMD GPUs in the artificial intelligence market, and we expect to see more customer applications.

From a supply chain perspective, since cloud-based artificial intelligence chips have strong demand for high-bandwidth memory such as HBM3, we believe that production capacity for HBM memory and advanced packaging (such as CoWoS) will still be hot. This will also drive corresponding semiconductor companies to expand production capacity and actively develop next-generation memory and advanced packaging technology. From this perspective, the application of artificial intelligence will still be the core driving force for the rapid development of new semiconductor technologies.

Terminal market pattern analysis

In addition to the cloud market, we expect artificial intelligence to also have stronger demand for the terminal market, which will also push artificial intelligence to become an increasingly important differentiating element in terminal computing chips.

On mobile phones, the frequency of use of artificial intelligence will be further increased, which will also push the chip to add more corresponding computing power, and use artificial intelligence support as the core highlight of the SoC. For example, the Snapdragon 8 Gen 3 released by Qualcomm uses “less than one second to perform image generation tasks” as an important selling point. It is estimated that such artificial intelligence capabilities will be deeply integrated into mobile phone manufacturers' operating systems. In addition to third-party chip companies such as Qualcomm, system makers that develop their own mobile phone chips are expected to continue to increase artificial intelligence; although Apple keeps a low profile in this regard, it is expected that in the future, it will use various methods (increase NPU computing power or increase software support) to begin empowering more artificial intelligence to create new shooting user experiences on iPhones. Vivo has accumulated several years in the field of self-developed ISP chips, and currently generative artificial intelligence has a strong synergy with ISP in improving the shooting experience. This is also the v3 ISP chip released by vivo in August 2023, which emphasizes generative artificial intelligence as a core highlight. In the future, it is expected that more and more such chips will emphasize the enabling role of artificial intelligence in the user experience.

In the smart car field, although Nvidia is not as strong as in the cloud, its Orin series chips are still the standard chip modules considered by major car manufacturers. We believe that as the AI model gradually strengthens its enabling role in intelligent driving, whether it is third-party chips or chips developed by car manufacturers, it will further strengthen investment in artificial intelligence computing power, and also drive rapid improvements in chip performance — recently, whether NIO released the computing power specifications of new self-developed chips or Tesla's announcement that it will use TSMC 3nm as the next generation chip production process, all suggest that artificial intelligence will play an increasingly important role in the field of smart car chips in 2024.

What new technologies are worth watching?

In addition to the chips discussed above, what new technologies are expected to bring new changes to the field of artificial intelligence chips?

First, in-memory computing and near-memory computing/processing techniques are expected to receive more and more attention. For artificial intelligence in the cloud, memory access costs have always been a performance bottleneck, and as the number of parameters in large models increases, so does the cost of memory access. The main purpose of in-memory computing and near-memory computing/processing techniques is to reduce such overhead so that some computational and processing tasks can be completed in memory. In this field, Samsung's PIM (process in memory) and PNM (process near memory) technologies are very worthy of our attention, and these technologies are also expected to become the key to further improving the differentiated competitiveness of Samsung's memory technology in the future.

For terminal artificial intelligence, since there is a strong demand for latency in the smart car scenario, there are many opportunities for new technologies to have an impact. In the cloud, acceleration chips represented by GPUs are mainly based on optimized throughput considerations rather than latency, so new architecture designs are necessary in the field of smart cars. For automotive applications, data enters the processor in the form of data streams (not in batches), so artificial intelligence chips must be able to process these data streams at high speed and with low latency. On the other hand, large models are entering smart car applications, so how to support large model inference with low latency will be a key breakthrough direction for new technologies in smart car chips.

Editor/Jeffrey

The translation is provided by third-party software.

The above content is for informational or educational purposes only and does not constitute any investment advice related to Futu. Although we strive to ensure the truthfulness, accuracy, and originality of all such content, we cannot guarantee it.

2024年，人工智能芯片还能给市场哪些惊喜？