Source: Hard AI

In the post-Moorish era, where data is growing exponentially, AI technology based on strong computing power support is booming, and the demand for computing power is also increasing day by day.

As AI training and inference costs continue to rise, the number of LLM (large language model) parameters has grown from 94 million parameters in 2018 to GPT-3 with 175 billion commercially available parameters, and GPT-4 is expected to exceed 1 trillion. According to some data, the computing power required to train an AI model will increase 275 times every 2 years.

Advances in data processing technology have driven the evolution of computers, but traditional processing units and large computing clusters cannot break through the boundaries of computational complexity. Although Moore's Law is still evolving and being reborn, it doesn't explain the need for faster, more powerful computational power.

Nvidia CEO Hwang In-hoon once confessed: Moore's Law is dead. As computing power continues to break through boundaries, what are the opportunities for AI?

Bank of America Merrill Lynch stated in an in-depth report released on March 21 that the next generation of computers will include: high performance computing (HPC), edge computing, spatial computing, quantum computing, and biological computing.

High Performance Computing (HPC)

High performance computing refers to computing systems that use supercomputers and clusters of parallel computers to solve advanced computing problems.

According to the report, high-performance computing systems are generally more than 1 million times faster than the fastest desktop computers, laptops, or server systems, and are widely used in established and emerging fields such as autonomous vehicles, the Internet of Things, and precision agriculture.

Bank of America believes that the development trend of high-performance computing brings room for growth in hyperscale systems (including LLM) accelerators.

Although high-performance computing accounts for only a small portion (about 5% share) of the total data center availability market (TAM), the future trend is to become a leading indicator for cloud-/enterprise applications.

In particular, as LLM's demand for computing power increases, 19 of the 48 new systems use accelerators, representing an accelerator addition rate of about 40%. However, according to a survey of the world's top 500 companies, there is room for improvement in the use of accelerators in hyperscale service systems. Currently, only about 10% of servers are accelerated.

Bank of America pointed out that another trend is that with the help of coprocessors (processors developed and applied to help CPUs complete processing tasks that cannot execute or perform inefficient and inefficient), computing methods will increasingly shift from serial to parallel.

Moore's Law/The maturity of serial computing is shifting more workloads to parallel computing, which is achieved through the use of independent coprocessors/accelerators such as GPUs, custom chips (ASICs), and programmable chips (FPGAs)).

As of November 2023, 186 machines in the world's top 500 companies used coprocessors, an increase from 137 systems five years ago; the use of coprocessors/accelerators in the top 500 remained flat month-on-month, increasing by about 5% year-on-year; the total computing performance of the top 500 supercomputers increased to 7.0 exaflops, an increase of 45% year over year.

Spatial computation

Spatial computing refers to a computer that changes human-computer interaction by integrating the user's graphical interface into the real physical world by using AR/VR technology.

In fact, we're reaching a turning point in human-computer interaction: moving from traditional keyboard and mouse configurations to the edge of touch gestures, conversational AI, and enhanced visual computational interaction.

Bank of America believes that after PCs and smartphones, spatial computing has the potential to drive the next wave of disruptive changes — making technology part of our everyday behavior, connecting our physical and digital lives with real-time data and communication.

Apple's Vision Pro, for example, has taken a critical step.

edge computing

Compared to cloud computing, edge computing refers to processing data at a location closer to the physical location of the terminal device, which has advantages in terms of latency, bandwidth, autonomy, and privacy. According to research agency Omdia, “edge” refers to a location where the round trip time with the end user is at most 20 ms (ms).

Bank of America said many companies are investing in edge computing and edge locations (from internal IT and OT to external, remote sites) to be closer to end users and where data is generated.

Tech giants such as Facebook, Amazon, Microsoft, Google, and Apple are all investing in edge computing, and the return on this investment is expected to drive these companies' stock performance over the next 5 years.

It is estimated that 75% of enterprise-generated data will be created and processed at the edge by 2025.

According to data from research firm IDC, the edge computing market is expected to reach 404 billion US dollars by 2028, with a compound annual growth rate of 15% in 2028.

It is estimated that between 2022 and 2025, the development trajectory of the edge computing market will be roughly as follows:

Phase I (2022): Use Case — Highly Customized; Phase II (2023): Vertical Sector — Vertical Kit/Package; Phase III (2024): Horizontal Sector — Cross-Vertical Technology; Phase IV (2025): IT Strategy — Vertical Strategy.

In the future, Bank of America believes that AI opportunities come from reasoning, and for edge computing reasoning, CPUs will be the best choice.

Unlike training in core computing, inference requires a distributed, scalable, low latency, and low cost model, which is what the edge computing model provides. The current divide in the edge computing industry is whether to use CPUs or GPUs to support edge inference. While all major vendors support GPU and CPU features, we think CPUs are the best choice to support edge inference.

Under the GPU model, only 6-8 requests can be processed at a time. However, the CPU is able to segment servers by user, making it a more efficient processing system at the edge. Instead, CPUs provide cost efficiency, scalability, and flexibility, and allow edge computing vendors to overlay proprietary software in the computing process.

Fog calculation

In the field of edge computing, there is another introductory branch concept: fog computing (Fog computing).

Fog computing is a network architecture that is used to store, communicate, and transmit data when using terminal devices to perform extensive edge computing in the field.

Bank of America believes that fog computing and cloud computing are complementary and may form a hybrid/multi-cloud deployment format in the future.

As applications move to the cloud, a hybrid/multi-cloud approach is being deployed. Cloud computing and edge computing complement each other. Using distributed methods can solve different needs in different ways to create value.

According to an IDC survey, 42% of enterprise respondents have difficulty designing and implementing key components, including infrastructure, connectivity, management, and security. In the long run, the combination of edge data aggregation and analysis with the large-scale capabilities of cloud access (such as analysis and model training) will create a new economy built on digital edge interactions.

quantum computing

Quantum computing refers to calculations that use subatomic particles to store information and use superposition to perform complex calculations.

Bank of America believes that quantum computing is important because it has an inherent irreplaceable advantage in solving problems that traditional computers cannot solve — this is also known as “quantum hegemony.” However, at present, the commercialization process of quantum computing is still in its infancy.

Quantum computing can almost instantly solve problems that traditional computers would take billions of years to solve. We are in the very early stages of adoption, and only a few machines deployed in the cloud are commercially available, mainly for research. However, the commercialization process is progressing rapidly.

Bank of America believes that quantum computers have broken the boundaries of computation, and that the combination of the two strongest technologies, AI and quantum computers, can fundamentally change the physical world and the mathematical world.

In the short to medium term, the life sciences, chemicals, materials, finance, and logistics industries have benefited the most. In the long run, when AI reaches human cognitive ability or even self-awareness, general artificial intelligence (AGI) will lead to a fundamental shift in technology.

The report points out that quantum computers are not suitable for routine tasks such as using the Internet, office tasks, or email, but for complex big data computations such as blockchain, machine, and deep learning or nuclear simulation. The combination of quantum computers and 6G mobile networks will change the rules of the game in all walks of life.

Big data analysis: Untapped big data has huge potential. Untapped big data has huge potential. It is expected that the amount of data created will double, from 120ZB in 2022 to 183 ZB.

According to IDC data, currently, due to computing power bottlenecks, we only store, transmit, and use 1% of global data. But quantum computing can change that and unlock real economic value — potentially using 24% of global data and doubling global GDP.

Cybersecurity: With parallel processing capabilities of up to 1 trillion calculations per second (Bernard Marr), quantum computing can technically challenge all current cryptographic methods, including blockchain. This also opens the door to new cryptographic technology based on quantum computing elements.

Artificial intelligence and machine learning: Advances in machine learning and deep learning are limited by the speed of underlying data computation. Quantum computers can accelerate machine learning capabilities by using more data to more quickly resolve connections between complex data points.

Cloud: This is probably one of the winners because the cloud is probably the platform where all data creation, sharing, and storage happens. Once commercialization of quantum computers begins, cloud access will be required, and data generation should grow exponentially. So a cloud platform would be the solution.

Autonomous vehicle fleet management: A connected autonomous vehicle will generate the same amount of data as 3,000 Internet users; for two vehicles, the amount of data generated will jump to about 8,000-9,000 users. As a result, the data growth generated by autonomous vehicles alone will be exponential.

Brain-computer interface

Brain-computer interface refers to direct interaction with the outside world through human and animal brain waves.

Bank of America pointed out that startups such as Neuralink are studying human-robot collaboration through implants (BCI). Brain wave control devices have already been implemented in animal experiments, and early human clinical trials are still ongoing.

Currently, brain-computer interface (BCI) and brain-brain interface (CBI) technologies are being developed, and there are examples of controlling hand movements through thought.

Syncron's solution is to place a grid tube with sensors and electrodes in the blood vessels that supply the brain. Neuron signals can be received from it. After the signals are transmitted to an external unit, they will be translated and transmitted to the computer. In clinical trials, paralyzed individuals were able to text, email, and bank and shop online.

Neural implants include nerve lines that are inserted into the brain through neurosurgical robots to pick up nerve signals. Clinical patients can now move computer mice by thinking.

editor/tolk