Source: Wall Street News
Author: Ge Jiaming

GPUs, GPUs, or GPUs. The “computing power revolution” initiated by AI not only brought Yingwei, which almost monopolizes the GPU market, to the spotlight, but also made the OpenAI CEO talk about the shortage of GPUs in interviews over and over again.

On May 29, Humanloop, a startup engaged in machine learning and artificial intelligence, CEO Raza Habib invited 20 developers, including OpenAI CEO Sam Altman, to discuss the future of artificial intelligence.And the most important topic in the whole discussion was that the GPU shortage was the biggest bottleneck.

Altman said that currently OpenAI's customers are least satisfied with the reliability and speed of the API, and this problem is mainly caused by a shortage of GPUs.

OpenAI relies heavily on GPUs

Altman said that their next plan is to continuously update GPT-4 to make it cheaper and faster, and the biggest bottleneck they have encountered in the development process is the shortage of GPUs. The interview stated:

The 32k-long context window is not yet ready for promotion, and OpenAI has yet to overcome technical barriers, mainly because of the high complexity of the algorithm. Although there will soon be 100,000 to 1 million new 32k contextual window tokens,However, providing more access tokens requires breakthroughs in technology and research.

The Fine-Tuning API is currently also limited by GPU availability.They haven't used efficient fine-tuning methods like Adapters or LoRa, so fine-tuning operation and management requires high-precision computation. There will be better fine-tuning support in the future. They might even host a marketplace for community contribution models.

The dedicated capacity services provided by OpenAI are limited by GPUs. When users request dedicated capacity, OpenAI needs to ensure that enough GPUs are available to allocate to users. However, supply is limited due to limited GPU resources. To use this service, customers must be willing to pay $100,000 upfront.

Regarding the future development of OpenAI, Altman said that the goal of OpenAI in 2023 is to reduce the “cost of intelligence” as much as possible and continue to reduce the cost of the API:

Cheaper and faster GPT-4 — top priority.

Longer context windows — up to 1 million context window tokens in the near future.

Fine-Tuning API — The Fine-Tuning API will be extended to the latest models, but the exact form will be shaped later by developers indicating what they really want.

API that supports session state — Now when you call the chat API, you have to repeat the historical conversation over and over again and pay for the same tokens. In the future, there will be a version of the API that remembers the conversation history.

By 2024, GPT-4's multi-mode — the current multi-mode has been announced as part of GPT-4, but it won't be extended to everyone until the GPU computing power bottleneck is broken.

At the AI congressional hearing held on May 16, Altman already expressed concerns about computing power bottlenecks:It would be better if fewer people were using ChatGPT due to bottlenecks in computing power.

In addition to concerns about future computing power bottlenecks, regarding the future of ChatGPT, Altman also said that although many developers have expressed interest in incorporating the ChatGPT plugin into the API, it may not be released soon:

The plugin doesn't currently reach Product Market Fit (PMF) and won't be appearing in the API anytime soon. Other than browsing, the usage of plug-ins shows that they haven't reached the best fit with the market.

A lot of people think they want their apps to be in chatGPT, but what they really want is to be able to use chatGPT in their apps.

Altman said that OpenAI will avoid competing with their customers, and their goal is to turn ChatGPT into a super intelligent work assistant:

Many developers say that when they use OpenAI's API to build products, they are worried that OpenAI will eventually release their competitors. However, OpenAI will not release more products than ChatGPT in the future.

Historically, every great platform company has had a “killer” app, and ChatGPT's role is to help customers make their products better. ChatGPT's vision is to be a super smart work assistant, but there will also be many other GPT use cases that OpenAI won't touch.

As for future regulation, Altman called for greater regulation of future models, but he doesn't think the existing big models are dangerous.Regulating them (big language models) now or prohibiting their development is a big mistake.

Altman also reiterated the importance of open source and said that OpenAI is considering open source GPT-3.

Altman said that the “era of giant models” is over, and this statement is not accurate:

OpenAI's internal data shows that the proportional law of model performance still exists: when the model size increases exponentially, the corresponding model performance increases linearly.

But because OpenAI has expanded the model millions of times in just a few years,Doing this will not be sustainable in the future.

This doesn't mean that OpenAI won't continue to try to scale up models in the future; it just means they may only double or triple each year instead of increasing many orders of magnitude.

The fact that the law of scaling continues to work has an important impact on the timeline of AGI development. The scaling assumption is that we probably already have most of the parts needed to build AGI, and most of the work left will be to extend existing methods to larger models and larger data sets. If the era of giant models ends,Then we should probably expect AGI to go further. The continued validity of the law of scaling suggests a shorter timeline.

Editor/Hoten