Source: Geek Park Author: Zhou Yongliang

May 15, for$Tesla (TSLA.US)$On the news that the FSD (Full-Self Driving) paid option rate is only 2%, Tesla CEO Musk replied on the X platform that the actual situation far exceeds this figure.

The response was in response to prominent Tesla investor Gary Blake's questions about future FSD acceptance rates. According to data provided by credit card data provider YipitData, only 2% of US Tesla owners who tried FSD for a month chose to continue subscribing to the service, far lower than the expected 6%.

Blake believes this may be because the FSD service doesn't have much added value or the subscription price ($99 per month) is too high. He called on Tesla to carefully evaluate these factors to better meet the needs of car owners.

Meanwhile, Musk's visit to China at the end of April sparked discussions about Tesla's FSD implementation in China. However, according to a report by China Daily, although Tesla proposed the launch of “driverless taxis” in China, the Chinese government has yet to fully approve the full implementation of the FSD in China and may first support its domestic testing and demonstration.

So, what is the actual experience with Tesla FSD at the cusp? Will it trigger the pangasius effect again?

01. ChatGPT moment for intelligent driving

As new energy vehicles enter the second half of intelligence, intelligent driving has always been one of the technological high points pursued by the industry. People are not only concerned about the passing rate of driving scenarios and commuting efficiency, but are also more concerned about whether intelligent driving is more humane.

The previous V11 version of Tesla's FSD, like many intelligent driving systems, had blunt speed control, sudden braking or acceleration when dealing with unexpected situations, and had an obvious sense of mechanics. Especially in special situations such as narrow roads or bad weather, this blunt speed control can make users uneasy.

But now, the advent of Tesla's FSD V12 has changed that landscape. Zhihu blogger “EatelePhant”, who has experienced Tesla FSD V12, said that one of its greatest strengths is its ability to personify. The biggest improvements in the V12 version are speed and smooth steering control. Even when seated in the back row, passengers hardly felt any frustration when starting and stopping traffic lights and turning at intersections.

Second, V12 has greatly improved its handling of unstructured scenarios (such as lack of restrictions on lane lines and traffic rules), such as the timing and amplitude of turns and the exact degree of deceleration. For example, in the V11 version, the intelligent driving system responds to a significant deceleration when the vehicle encounters a vehicle that turns left far in the distance while going straight. Although this can avoid the risk of collision, the deceleration is usually too large, causing obvious frustration and increasing the risk of being overrun by a rear car.

In the V12 version, in the face of a similar situation, the system seems to be able to more accurately determine the driving route and speed of the vehicle in front. Therefore, it can slow down at a very suitable speed, making the passengers in the car almost unable to feel it, and at the same time leave a sufficient safe distance to avoid unnecessary discomfort and the risk of being rear-ended by the vehicle behind.

Third, V12's ability to cope with various scenarios has been significantly improved, greatly reducing the number of times manual intervention is required. For example, the Tesla FSD V12 can not only recognize and bypass obstacles such as iron sheets on the road surface, but can also drive on temporary roads according to guide signs during road construction, responding as flexibly as a human driver. Even if you need to drive in the opposite lane, you can be unrestricted by the retrograde sign and the yellow line in the center. When driving at night, it was able to cleverly avoid oncoming cars by turning right like a human, and then turning the steering wheel to the left before moving on to the left, showing excellent driving skills.

At the same time, Zhihu blogger “EatelePhant” also mentioned that FSD V12 has demonstrated the ability to generate some similar intelligence. During the test, the vehicle faced a problem: there was a lane in front of the intersection, and the V12 was unable to use the reverse gear to complete a U-turn. While passengers were waiting, V12 spotted a small parking lot and resolutely changed course to bypass in an attempt to replace the standard U-turn. Although the system at the last exit suggests takeover, this behavior is a major breakthrough in autonomous driving technology, because usually the system strictly follows the navigation route, and the act of deviating from navigation on its own is almost unacceptable.

These improvements are also reflected in the number of manual interventions. Compared to the previous version, the V12.3 version significantly increased the cruising range without critical takeover in urban environments, from about 100 miles (about 160 km) to 386.7 miles (about 622 km).

In contrast, Xiong Lu, a professor at Tongji University and vice dean of the Automobile School, once said that autonomous driving companies in Beijing, Shanghai, Guangzhou and other places require an average of 3.5 to 10 takeover times per 100 kilometers. Basically, they need to be taken over manually every ten or tens of kilometers traveled.

Currently, Tesla FSD may be ushering in its “ChatGPT moment,” although there are still some shortcomings. For example, when driving, the system sometimes makes people feel that they are too close to the side of the road, which causes some concern; it is not timely enough to recognize and avoid incoming cars, such as when it encounters a fire engine; it may be wrong to recognize some special road signs or lanes, and there are also problems with identifying and avoiding potholes.

However, this does not prevent Tesla from drastically increasing its investment in intelligent driving. According to information, Tesla plans to invest a total of 10 billion US dollars in autonomous driving technology by the end of this year. Considering that the total spending from 2016 to 2023 was around $2 billion, this means Tesla's spending on smart driving will reach around $8 billion this year. This is a huge investment and shows Tesla's determination to further improve FSD technology.

02. How is FSD V12 made?

Behind the significant improvement in the Tesla FSD V12 experience is the result of the gradual convergence of its technology path.

Since introducing the FSD feature in 2020, Tesla has been leading the way in the development of smart driving technology. Unlike traditional methods that rely on lidar and high-precision maps, Tesla insists on using pure vision technology to enable vehicles to better understand the surrounding environment.

Over the next two years, Tesla announced a number of proposals on FSD technology through a series of events such as AI Day. These solutions involve concepts such as closed data loops, shared backbone networks, BEV perception, and occupied networks. Although these technologies are in an industry-leading position, previous versions have been criticized by users, who think that their user experience improvements are not obvious enough.

However, as FSD progressed to the V12 phase, things changed. Compared to FSD 11, the biggest change in FSD V12 is the use of End-to-End Neural Network (E2E NN) technology. This technology enables the system to better understand and handle complex driving environments, reduces driver intervention, and improves the accuracy and degree of automation of autonomous driving.

In the past, the basic process of FSD usually involved three stages: perception, decision making, and execution. In earlier versions, the perception phase required obtaining information about surrounding objects through vision or radar, and identifying and classifying them, while the decision-making phase relied on pre-written control rules.

However, in FSD V12, using end-to-end neural network technology, these steps have been revolutionized: the perception phase no longer requires manual object identification and classification, and the decision phase no longer requires pre-written control rules. The system only needs a large amount of video input to learn the neural network to be able to make the right decisions in different situations. This allows Tesla to reduce a lot of code in FSD V12, making the system lighter and more flexible, and able to operate properly in an unfamiliar environment even without an internet connection.

Musk said at the end of last year that Tesla's FSD Beta V12 had no programming from beginning to end, and no programmers wrote a single line of code to identify concepts such as roads or pedestrians; everything was left to the neural network to think for itself. The C++ code was only 2000 lines, while V11 had 300,000 lines.

Actually, the end-to-end model is not a new concept; it has been proposed a long time ago. However, many people have always had doubts about the explainability and reliability of neural networks. Although the end-to-end system increases the upper limit of the model's capabilities, it also amplifies the unexplainable problem of neural networks as a “black box”, which poses a huge challenge for R&D iteration and problem solving. Therefore, many companies are afraid to try it easily.

Tesla FSD lets everyone see the potential of an end-to-end model. Of course, this process didn't happen overnight. Tesla has always emphasized end-to-end “purity” in autonomous driving technology. Starting with the V10.9 version, they removed the post-processing code for lane line detection and instead directly output the lane line from the model. On AI Day, Tesla also showed how to introduce learning trajectory generation and neural network decision models into planning control modules.

However, Tesla's technical updates are mainly focused on the middle and upper stages of the technology stack, such as perception and prediction, while the decision planning module that controls driving functions has rarely changed. As a result, the improvement in the user experience was not significant enough.

V12's major breakthrough is that it opens up the last link (decision planning) of the entire technology stack, enables the system to be data-driven from end to end, and ultimately achieves more natural and intelligent driving behavior.

As V12 adopts an end-to-end technical architecture, the planning control output is directly optimized, so the user experience will improve more rapidly. At the beginning of May, Musk announced that Tesla's FSD system will soon receive three major updates, namely V12.4, V12.5, and V12.6.

Among them, the V12.4 version is expected to be launched in mid-May. This version will comprehensively update model training to improve the accuracy and reliability of the system. Second, in response to the problems of excessive acceleration and rapid braking reported by users, V12.4 and subsequent versions will focus on optimizing driving comfort, thereby improving the passenger's driving experience.

Back to the beginning of this article, Musk's visit to China seems to be making the FSD “landing in China” more credible. There is even widespread news on the extranet that BYD and Tesla will cooperate in FSD, further unleashing FSD's potential to enter China — what kind of results will the cooperation between the world's two largest NEV companies produce?

If the rumor comes true, will Tesla FSD change the current state of smart driving? How will the smart driving “story” of Chinese car companies continue to develop?

This is probably a question that everyone fantasizes about, but is afraid to ask.

edit/lambor