具体而言，DualPipe是一种双向流水线并行算法，它通过“双向管道调度”和“计算通信重叠”，旨在减少分布式训练中的流水线“气泡”（空闲时间），让训练过程像流水线一样顺畅，提升GPU利用率。
DualPipe and EPLB are two core technologies for large-scale AI model training, which improve model training speed and GPU utilization, respectively. The profile-data clearly presents the full-link performance data from training to inference of DeepSeek.
Three big breakthroughs announced all at once! DeepSeek's "Open Source Week" fourth release, open-source latest optimized parallel strategy, including DualPipe, Expert Parallel Load Balancer (EPLB), and full process performance data (profile-data).
According to reports, DualPipe and EPLB are two core technologies aimed at large-scale AI model training, focusing on distributed training efficiency optimization and expert parallel load balancing, both designed for V3/R1.
Specifically, DualPipe is a bidirectional pipeline parallel algorithm that aims to reduce pipeline "bubbles" (idle time) in distributed training through "bidirectional pipeline scheduling" and "computation-communication overlap," allowing the training process to run smoothly like a pipeline and improving GPU utilization.
For example, in traditional AI training, the "pipeline bubbles" caused by GPUs waiting for data transfer can occupy more than 30% of the time—whereas DualPipe allows AI training to have the ability to "cook while washing dishes," directly enabling the pipeline to "operate in both directions."
It is worth mentioning that DualPipe was jointly developed by three individuals—Jiashi Li, Chengqi Deng, and Liang Wenfeng.
In a mixture of experts model (MoE), uneven expert load often leads to GPU utilization below 60%. EPLB is an algorithm designed to solve this problem, which can allocate copies of high-load experts to idle GPUs, similar to how DiDi Global Inc schedules more vehicles in high-demand areas during peak times, thereby improving resource utilization.
Actual tests have shown that the GPU load difference decreased from 20%-30% to within 5%, and training speed increased threefold.
The final big surprise is that DeepSeek has directly released the full-link performance data from training to inference, essentially providing an "X-ray" of the AI training process, clearly demonstrating how DeepSeek-AI meticulously optimizes computation and Communications.
At the same time, the accompanying open-source performance analysis toolkit provides complete data tracking for the training, pre-filling, and decoding stages, allowing developers to visually analyze the computation-Communications overlap efficiency through a browser.
Editor/lambor
Today, after hitting the high near 24,000 in the early session and entering bearish positions, the index fell sharply by nearly over 600 points, immediately recouping yesterday's losses significantly.
Moreover, today it broke the new high again, reaching a maximum of 24,076, but by the end of the market, it fell back by about 70 points, producing a bearish candle. The current trend has not yet been broken, but from the previous low until now, it has risen close to 6,000 points. It is believed that those with positions can continue to hold until there is a clear trend reversal for profit-taking. Those without positions can wait for a pullback to get in. Actually, it is hoped for a quick pullback, as it allows for entry and also provides a healthy breath.
Currently, the outlook remains the same as before. It is believed that even if there is a pullback, it shouldn't be too deep. However, if Futures fail to stabilize and close below 22,350, there may still be room for decline. The chance of Futures falling below 21,400 in the short term should be low, so it is considered that if a significant pullback occurs, it presents a good opportunity to incrementally go long. Recently, there has been a consistent approach to not hold positions overnight, only focusing on immediate trades, as there is no high chasing and no casual short selling.
Comment(0)
Reason For Report