Source: Wall Street News

The next leap in AI seems to be unable to report on time.

According to the Wall Street Journal on the 20th local time, OpenAI's next-generation AI project GPT-5 (codename Orion) is facing numerous difficulties. This project has been in development for over 18 months at a great cost, yet it has not achieved the expected results.

Sources reveal that OpenAI's biggest financial backer, Microsoft, initially expected to see the new model around mid-2024. OpenAI has conducted at least two large-scale trainings, each lasting several months and consuming Beijing Vastdata Technology, but each time new problems arose, and the Software could not meet the researchers' expectations.

Analysts believe there may not be enough data in the world to make it smart enough.

The enormous costs are staggering, and the progress of the GPT-5 project is not smooth.

Analysts previously predicted that technology giants might invest 1 trillion dollars in AI projects in the coming years. Estimates indicate that the cost of a 6-month training session for GPT-5 alone could reach about 0.5 billion dollars. OpenAI's CEO Sam Altman has stated that the costs of future AI models are expected to exceed 1 billion dollars. However, sources familiar with the project have indicated:

While Orion's performance has improved compared to OpenAI's current products, it is still not enough to justify its huge Operation costs.

In October of this year, the $157 billion valuation given to OpenAI by investors is largely based on Altman's prediction, in which he previously stated that GPT-5 would be a "significant leap", and he also mentioned that GPT-4 performed like a clever high school student, but the final GPT-5 would actually resemble someone with a doctorate in certain tasks.

Reports indicate that GPT-5 should be able to unlock new scientific discoveries and perform everyday human tasks such as making appointments or booking flights. Researchers hope that it will make fewer mistakes than existing AIs or at least acknowledge "doubt", as current models may produce hallucinations.

However, there is no fixed standard for "when AI can be smart enough"; it is more based on feelings.

So far, the in-development GPT-5 still does not feel strong enough. Altman stated in November, "No products named GPT-5 will be released within 2024."

Data shortages have become a major bottleneck.

To avoid wasting large investments, researchers try to minimize the chances of failure through small-scale trial runs.

However, the plan for GPT-5 seems to have had problems from the start. In mid-2023, OpenAI began a training run, which was also a test of Orion's proposed new design. But this process progressed slowly, indicating that larger-scale training may take an exceptionally long time, which in turn would make costs soar.

OpenAI's researchers decided to make some technical adjustments to enhance Orion, and they also found that to make Orion smarter, more high-quality and diverse data is needed. Testing the model is a continuous process, and large-scale training runs can take months, with trillions of tokens being fed to the model.

However, the data from news articles, social media posts, and scientific papers on the public Internet is no longer sufficient to meet the requirements. DatologyAI CEO Ari Morcos stated:

"This has become very expensive and difficult to find more equally high-quality data."

To address this issue, OpenAI chose to create data from scratch. They hired professionals such as software engineers and mathematicians to write new code or solve mathematical problems to serve as training data.

The company also collaborated with experts in fields such as theoretical physics to explain how they would tackle the most challenging problems in the field, but this process is very slow, and GPT-4's training used approximately 13 trillion tokens. Even with 1,000 people writing 5,000 words each day, only 1 billion tokens could be generated in a few months.

OpenAI also began developing "synthetic data" using data generated by AI to train Orion, believing that faults could be avoided by using data generated from another AI model, o1.

Is Google catching up, and is OpenAI in a panic?

This year, with Google launching its most popular new AI application NotebookLM, OpenAI became even more anxious.

Due to the stagnation of Orion, the company began to develop other projects and applications, including a streamlined version of GPT-4 and Sora, which can create AI-generated videos. However, insiders indicate that this has led to a need for teams developing new products and Orion researchers to compete for limited computing resources.

Additionally, OpenAI is also developing more advanced reasoning models, believing that allowing AI to 'think' for longer periods can solve complex problems not encountered during training.

However, these new strategies are also facing challenges. Researchers at Apple have found that reasoning models, including OpenAI's o1, are likely just mimicking training data rather than truly solving new problems. Moreover, the method o1 uses to generate multiple answers has significantly increased Operation costs.

Nevertheless, OpenAI is still persistently advancing the development of GPT-5. On Friday, Altman announced a new reasoning model plan smarter than any previous product, but did not disclose when or if a model that could be called GPT-5 would be launched.

编辑/jayden