Elon Musk has confirmed that Tesla FSD Beta 12 is now a software build at Tesla. In a reply on X to @WholeMarsCatalog, he announced that he tested an alpha build of FSD Beta 12 and reflected on the experience, saying it was mind-blowing.
FSD Beta 12 is slated to be a significant upgrade over the current builds of FSD Beta. The latest build, released to customers in the US in the past couple of days is V11.4.6. While we don’t know how many builds will be released between now and V12, we do have some indication of what’s changing.
To drive the car, Tesla’s vision-only system uses data from their fleet of millions of cars across the world to train an AI model on how to perform a number of functions.
Perception
The first step is perception which involves taking input from the cameras around the car and comparing that to objects in the model and inferring each object in front of and around the car.
Because the cameras provide a constant input to the system, this is constantly re-evaluated at up to 30 frames per second.
For the occupants, the car’s perception is communicated through the UI on the center screen, showing cars, signs, bins, etc on the display.
Occupancy network
Part of Tesla’s autonomous system is a concept called the occupancy network. This takes camera input and converts it from the rgb values of pixels, into digital bits called voxels. We can think of this as similar to how we experience a driving game. Many game engines have structures that detect when objects overlap and a collision occurs and developers may signify this in a number of ways, with damage to the object, colours, sounds etc.
What Tesla can model is what space around the vehicle is occupied and if the car’s path is going to intersect with any occupied space. It doesn’t at this stage concern itself with what that object is, a person, a mailbox, a tree, just that it knows the car can not drive on a path where those two would intersect. This requires the car to understand its own dimensions and properties like speed to calculate where it will be a few seconds from now to avoid a collision.
Path planning
Understanding the playspace available allows the system to plot a safe path through the environment. This historically would have meant keeping in your lane, but driving is much more complex than this and leveraging the drivable space concept, the car can cross lane lines, although it typically prefers to stay within, as long as it doesn’t breach the occupied space rule.
Navigation
While it is possible to initiate the FSD beta system without lane lines, it is helpful to enter a destination so the navigation can route the car there, rather than make random turns. The navigation provides the overall journey information, almost like a task, but the car may have to be dynamic and adjust to the environment around it, in order to reach the destination.
This could occasionally mean taking turns you wouldn’t typically make, but the car is calculating the optimal route a number of times throughout the drive. In recent builds, we’ve seen FSD Beta improve on lane selection, particularly where you have a tight left, right combo that involves switching lanes in a short distance. Should that not occur, the navigation can re-route.
Rules and Laws
Operating any vehicle on the road requires following the legislation for that state or country. The rules of the road are defined in a number of ways, through signs, lights, and painted road markings, but also just knowledge of how road systems and traffic flow.
If we take a 4-way stop for example, understanding whose turn it is next is really important to all road users working through the environment safely.
Only by knowing and understanding the rules and laws can an autonomous car really move safely and efficiently through a service area.
Command and control
After the system has an understanding of all the above, the car needs to actually drive. This means the application of the throttle, the brake, the steering wheel, and the indicators as a basic starting point. The car also needs to perform maneuvers during the drive such as changing lanes, which involves a complex sequence of checking the available space around the car, predicting available space by the time the move is executed and constantly monitoring if the inputs result in the outputs expected.
As with human driving, there are different tolerances for the speed and rate at which these moves are executed before passengers feel nervous, or even sick, meaning control of the vehicle is incredibly important to overall passenger satisfaction.
So what is FSD V12?
Musk spoke on Tesla’s Q2 2023 Earnings call and indicated that they had made a major breakthrough.
He wasn’t willing to go into detail at the time, but it is expected that Tesla FSD V12 will be End to end.
“End-to-end” refers to a type of artificial intelligence (AI) architecture that uses a single neural network to learn all aspects of a task. In the case of Tesla’s Full Self-Driving (FSD) system, this means that the neural network would be responsible for everything from interpreting the car’s surroundings to controlling its steering, acceleration, and braking.
Traditionally, FSD systems have used a two-step approach, with one neural network responsible for perception and another responsible for control as discussed above. This approach has the advantage of being easier to develop and debug, but it can also be less efficient.
While hundreds of FSD Beta builds have shipped to customer cars since October 2020, Musk indicated recently that FSD version 12, would drop the beta tag, a sign of the maturity expected in the next major release.
The reality is, we are likely to still see a beta release, in software development this typically means when software is mature enough to go outside the company. Internally releases are often known as alpha which is what Musk says he tested today.
Once the software is ready for wide release, that’s when the Beta tag will drop away and it’ll simply be known as FSD.
So when is FSD V12 coming?
Many Tesla fans would love the answer to be tomorrow, but the reality is, we often see FSD Beta build numbers that indicate software branches for a new build are actually created at least a few weeks before they reach the public.
Without knowing what the V12 software build is, it’s difficult to know with any certainty when V12 will arrive, but the fact there is now a code branch at Tesla, and it’s in a car Elon has tested, we’re definitely getting close.
Musk recently spoke confidently that FSD would be safer than a human by the end of this year. While this is a statement he’s made previously, we’re getting other signs that he’s not alone in this line of thinking.
Tesla’s Phil Duan and Ashok Elluswamy, both recently spoke about autonomy during CVPR, an autonomous driving conference in Canada. These also referenced a breakthrough of some kind and during the Q&A section were asked about when this would make it into the product, to which they simply said, before the end of the year. It’s likely this was also a reference to V12.
As unclear as Tesla’s plans for FSD V12 are in the US and Canada, their international rollout plans are even more vague. The company did start hiring ADAS test drivers in Australia, which is a positive sign they’re working on localisations for our market.
So, in essence, this article is a lo of hype and speculation with a clickbait title. Nothing here that reveals v12 architecture over v11.
Sorry you feel that way and no the titled was not clickbait.
In simplest terms, end-to-end will be video in, command and control of the vehicle out.
Yes, and that is exactly what v11 is already doing right now. Nothing in your article details any differences between v11 and v12. That is why your title is clickbait.