Is Autopilot actually "Learning" on V9?

  • Please post everything related to v9 (2018.39.x) into the version 9 MEGATHREAD. Please do your part to keep this site organized for everyone by looking around or doing a search BEFORE posting. Thanks

  • Winter is here and the forum is starting to get flooded with cold weather threads and posts. Please heed the suggested threads and posts before you post something related to cold weather. This is a great place to start: https://model3ownersclub.com/threads/teslas-in-cold-weather.5271
  • It's OK to discuss software issues here but please report bugs to Tesla directly at servicehelpna@teslamotors.com if you want things fixed.

3V Pilot

Top-Contributor
Joined
Sep 15, 2017
Messages
1,191
Location
Oro Valley, AZ
Tesla Owner
Model 3
Country
Country
#1
I've noticed some interesting new Autopilot behavior since upgrading to V9 and I'm wondering if anyone else has had similar experiences. I know there is a lot of talk about Tesla's "Neural Net" and how it will help make autopilot better but does that mean that our cars now "Learn" on their own and in collaboration with other Tesla's on the road? From what I've seen over the first few days of V9 I'm starting to think the learning features are now active, let me explain.....

On V8 and prior the Autopilot behavior was always pretty static, by that I mean it would always react pretty much the same to any given road. For example on my morning commute there is one spot where the fast lane of the freeway splits and becomes 2 lanes. V8 Autopilot would always veer left as the lane lines widened, then snap back right to stay in the same lane it was in to begin with. I just started disengaging auto-steer prior to that section so I didn't look like a half crazed maniac...lol.

On V9 the first day I left Autopilot engaged and in that same spot there was much less of a left input and a minor correction back to the right. Then, the very next time I noticed there was almost no movement at all, like the car "knew" what was coming and didn't swerve at all. This has never happened before, Autopilot changing behavior without a software update.

I also have another example where V9 was getting much too close to a curb on a right hand bend one day and I had to disengage auto-steer twice, the next day around the same bend it stayed right in the middle of the lane and seemed smoother at that. Again, same road, different behavior, no update.

I don't think I'm imagining things and maybe this whole Neural Net thing is actually active now but I'd just like to know if anyone else has seem similar events? Or did I just miss the fact that this is what Elon has been talking about all along?
 

Gavyne

Well-Known Member
Joined
Jul 7, 2018
Messages
286
Location
SoCal
Tesla Owner
Model 3
Country
Country
#2
Neural net was always working, V9 just made it a whole lot better.
 

3V Pilot

Top-Contributor
Joined
Sep 15, 2017
Messages
1,191
Location
Oro Valley, AZ
Tesla Owner
Model 3
Country
Country
#3
Neural net was always working, V9 just made it a whole lot better.
I was under the impression that the Neural Net was helping Tesla come out with better Autopilot upgrades, basically back at the factory. Was it your impression that the cars actually got better in between updates? This is the first I've noticed it and that is why I'm assuming it's actually is integrated into the car now and having a direct effect.
 

babula

Well-Known Member
M3OC Supporting Member
Joined
Aug 26, 2018
Messages
362
Location
New York
Tesla Owner
Model 3
Country
Country
#4
If what you reported is true, thats truly incredible.

Not sure if it always existed in V8 but I recall reading something about it getting 500x better in V9.
 

Gavyne

Well-Known Member
Joined
Jul 7, 2018
Messages
286
Location
SoCal
Tesla Owner
Model 3
Country
Country
#5
Neural net is about processing the data, such as V9 increased the capacity in which the data can be processed. But neural net is also about training the data. Each driver using the autopilot trains the data, the new learned data then is applied to all cars. If your question is whether autopilot is learning, then yes, it most definitely is learning. What we don't know is their process for validation, like who's validating the data, how much are they doing it manually, etc..
 

GDN

Moderator
Moderator
M3OC Supporting Member
Joined
Oct 30, 2017
Messages
1,713
Location
Far North Dallas, TX
Tesla Owner
Model 3
Country
Country
#6
Neural nets are something I don't understand, but reading about them I've been led to believe it will get smarter on it's own. I'm not sure if that happens in each individual car, or if the feedback from cameras and car paths is just fed back to the mothership at night which then improves maps or other tidbits that could be fed back to the navigation system to improve its behavior. I am definitely led to believe however that it doesn't take the next release to come out to make the car/nav smarter, it is an ongoing improvement at all times. Just don't know the details of how.
 

Bernard

Well-Known Member
M3OC Supporting Member
Joined
Aug 3, 2017
Messages
243
Location
Hawaii, HI
Tesla Owner
Model 3
Country
Country
#7
Neural net is about processing the data, such as V9 increased the capacity in which the data can be processed. But neural net is also about training the data. Each driver using the autopilot trains the data, the new learned data then is applied to all cars. If your question is whether autopilot is learning, then yes, it most definitely is learning. What we don't know is their process for validation, like who's validating the data, how much are they doing it manually, etc..
In the case discussed, there cannot have been any person validating -- Tesla does not have tens of thousand of employees just monitoring Tesla drivers ;-) But the system can learn perfectly well on its own, through corrections. If you take over from Autopilot (and do not have an accident and do not end up the way of another car ;-), the system can learn from that; in fact, it could also learn from your not taking any action -- by concluding it is doing fine and reinforcing its current settings. Given that the task is car driving, where errors can be lethal, human corrections are likely to be given a lot more weight than if no correction is applied -- in fact, I would guess that reinforcement through lack of action is not happening, since it may be a bit too risky.
But, yes, doing that requires a lot of data -- the system has to have enough information to characterize the situation that led you to take over, and, more challengingly, it must infer what the correct action should have been -- next time, it cannot just mimic exactly what happened, since that would be jerking you around (at the time when you took over in the first occurrence), and that takes a lot of computing power, even if (as I describe below) it is just recomputing a few calibration values rather than adjusting the net's weights.
A neural net is a very simple concept (originally inspired by early -- and somewhat mistaken -- descriptions of the brain as a collection of neurons, each neuron connected to a fairly large number of others, using connections that can be "weighted" -- some are more influential than others); the concept is quite old and the first "artificial" neural nets date back 50 years, but the issue has always been one of scale: the brain has billions of neurons and early research was using neural nets with tens, or at best hundreds of nodes, because anything larger would have taken forever to train (and also because data were scarce). I would not be surprised if Tesla turned up the rate of per-vehicle learning (through recalibration).
I would be rather surprised if individual cars regularly recompute the weights of their own nets -- this is a very time-consuming computation and it would be risky to allow modification of the entire net based on some non-validated data such as a correction. More likely, what is happening is some local recalibration. For instance, it would be much simpler to have some parameters describing how close to the inside of a turn the car should drive and recalibrate those parameters after the driver corrects the trajectory in a curve rather than rerun the computation of marginal probabilities using the correction data. (Another inexpensive measure would be to memorize each correction so as to be ready to deviate from the otherwise automatic course of action when presented with the same circumstances -- a sort of patch between firmware updates.)
In any case, it's great and fascinating!
 

Gavyne

Well-Known Member
Joined
Jul 7, 2018
Messages
286
Location
SoCal
Tesla Owner
Model 3
Country
Country
#8
I believe I read in an earlier article where they mentioned people going through the data. So I think there are people looking at the data. What we don't know is the exact details, like what data they are looking at, how much of it, and what they're doing with the data. They won't release the details either, just as Google won't release their search engine algorithm.

I do agree the machine is learning on its own. But at this point in time I do believe Tesla still is tweaking and inputting data in the neural net.
 

KFORE

Active Member
Joined
May 19, 2018
Messages
39
Location
Detroit, MI
Tesla Owner
Model 3
Country
Country
#9
Each car is certainly not learning on its own. The trained NN's are included in OTA updates and your car runs the trained nets locally, but does not self-learn for each specific car. That would actually be a nightmare in terms of having an entire fleet learning in a non homogeneous way. No two cars would react the same. Tesla doesn't want that. Instead, your car may flag an issue automatically and send it to Tesla once it gets back on wifi, which they then can use to train the larger NN that all cars get during each OTA update.

My guess what you're experiencing is the NN's are quite new in V9 and are going to produce some dynamic results as Tesla works out the kinks of using all cameras as a part of the network.
 

kort677

Well-Known Member
Joined
Sep 17, 2018
Messages
261
Location
Ponte Vedra Beach FLA
Tesla Owner
Model 3
Country
Country
#10
auto pilot is always learning, many of us share the data from the car with the mothership, the mothership receives vast amounts of data that is then used to make adjustments. I have noticed changes in the AP system. most of us who drive regular routes have certain spots where the AP can get a bit wonky, and sometimes that wonkiness is lessened, why? because the AP refined itself and "figures" out whatever was causing that wonkiness
 

garsh

Dis Member
Moderator
M3OC Supporting Member
Joined
Apr 4, 2016
Messages
7,620
Location
Pittsburgh PA
Tesla Owner
Model 3
Country
Country
#11
Each car is certainly not learning on its own. The trained NN's are included in OTA updates and your car runs the trained nets locally, but does not self-learn for each specific car. That would actually be a nightmare in terms of having an entire fleet learning in a non homogeneous way. No two cars would react the same. Tesla doesn't want that. Instead, your car may flag an issue automatically and send it to Tesla once it gets back on wifi, which they then can use to train the larger NN that all cars get during each OTA update.
KFORE's got it. :)

"Machine learning" is an extremely CPU-intensive offline process. Tesla will use thousands of machines running in parallel for this. You feed in a ton of data (collected from all the cars of the fleet), and out comes a new neural network (aka neural net, or NN).

The neural net is then included in the next version of vehicle software. It is also CPU-intensive, but can be handled by the car's single computer. This allows the car to make decisions based upon all of its input. But there is no "learning" happening at this point. The car will just be making decisions based on the programming of the neural net.

If you notice any difference in behavior in a single car from one drive to the next, then I think it's more likely due to calibrating the cameras. Each car's cameras will be aimed slightly differently, and it may take the car a little while to determine exactly how each camera is aimed and compensate for the differences.
 

MelindaV

☰ > 3
Moderator
Joined
Apr 2, 2016
Messages
6,367
Location
Vancouver, WA
Tesla Owner
Model 3
Country
Country
#12
If what you reported is true, thats truly incredible.

Not sure if it always existed in V8 but I recall reading something about it getting 500x better in V9.
You are thinking of the AP3 hardware (next year) being capable of x better performance than AP2.5
 

Ed Woodrick

Well-Known Member
Joined
May 25, 2018
Messages
229
Location
Atlanta, GA
Tesla Owner
Model 3
Country
Country
#13
Each car is certainly not learning on its own. The trained NN's are included in OTA updates and your car runs the trained nets locally, but does not self-learn for each specific car. That would actually be a nightmare in terms of having an entire fleet learning in a non homogeneous way. No two cars would react the same. Tesla doesn't want that. Instead, your car may flag an issue automatically and send it to Tesla once it gets back on wifi, which they then can use to train the larger NN that all cars get during each OTA update.

My guess what you're experiencing is the NN's are quite new in V9 and are going to produce some dynamic results as Tesla works out the kinks of using all cameras as a part of the network.
I'll agree, and to slightly translate. Each car does not learn. What occurs is that driving information is accumulated in the Tesla cloud and with the data, they use the neural nets to create sets of parameters that then become part of a specific release. So everyone's car drives essentially the same, but upon each update, they drive better, using information from other drivers.

Would we really want cars to learn the bad driving habits of some drivers?
 

Rick59

Top-Contributor
Joined
Jul 20, 2016
Messages
543
Location
Ottawa, Ontario, Canada
Tesla Owner
Model X
Country
Country
#14
Everyone is making very persuasive arguments so let me put in my nickel’s worth.
I doubt that Tesla is using valuable central processing time to correct a small aberration in one location, in one car.
There is no reason to think that the car’s computer can’t process some simple transactions and remember them for the next time. I don’t see why a world-wide update would be needed to remedy a situation in my remote corner of the planet.
 

Jay79

Well-Known Member
Joined
Aug 18, 2018
Messages
246
Location
Michigan
Tesla Owner
Model 3
Country
Country
#15
If your car will auto raise the suspension on the S or X or open your garage door based Geographical Data by remembering your input, its plausible to assume it can remember exactly where you disengaged autopilot and what manual correction was made at the time and place of the incident. Perhaps V9 has opened up this kind of self calibration and fine tune refinement. I think the NN will be an over all data push to continue to smooth AP out and implement enhanced AP features. Due to the million scenarios each driver will experience, a fine tune adjustment within each car makes perfect sense. FYI, this is all guesses and speculation
 

JWardell

Legendary Member
Joined
May 9, 2016
Messages
2,315
Location
Boston
Tesla Owner
Model 3
Country
Country
#16
In my understanding, the entire system learns as a whole, and if you have a high number of interventions or corrections in a spot, it will learn the proper behavior, then encode that in the high-resolution maps that are updated and downloaded from time to time. Round-trip time could be weeks or months, but it should eventually stop veering into an exit ramp or braking for a bridge once the map tells it not to
 

3V Pilot

Top-Contributor
Joined
Sep 15, 2017
Messages
1,191
Location
Oro Valley, AZ
Tesla Owner
Model 3
Country
Country
#17
Wow, some really great responses here from people who really understand this stuff. Thanks everyone! It all just fascinates me and I like to see the car improving all the time. How it all works, well that's mirrors and magic to me! Sounds like my first assumption was correct in that the main learning happens back at Tesla's top secret underground lair were they lock away the most knowledgeable computer geeks and never let them see the light of day.

I'm just glad it does learn somehow and since I like to believe the car is my own personal AI droid I'll just keep on believing that it's little brain is learning all on it's own....lol.
 
Last edited:

PNWmisty

Top-Contributor
Joined
Aug 19, 2017
Messages
1,699
Location
Anacortes, WA
Country
Country
#18
If your car will auto raise the suspension on the S or X or open your garage door based Geographical Data by remembering your input, its plausible to assume it can remember exactly where you disengaged autopilot and what manual correction was made at the time and place of the incident. Perhaps V9 has opened up this kind of self calibration and fine tune refinement.
That's a good point. While I agree very much with those who say the NN learns as a fleet, I think it might be overstating what we know to assume every car will behave exactly the same in every situation. I think it's likely that individual cars could set "flags" at certain points. These points would probably be at spots where the EAP has a decision to make and either decision seems relatively safe and somewhat equally suitable. The "flag" could then develop a preference for one behavior over the other, over time, based upon whether EAP was disengaged or whether it was necessary to swerve shortly after the flag or not.
 

PNWmisty

Top-Contributor
Joined
Aug 19, 2017
Messages
1,699
Location
Anacortes, WA
Country
Country
#19
I don't think I'm imagining things and maybe this whole Neural Net thing is actually active now but I'd just like to know if anyone else has seem similar events? Or did I just miss the fact that this is what Elon has been talking about all along?
I don't have enough repeat miles to say whether each car is actively learning but I can say the capabilities of version 9 are miles ahead of the previous version, especially in terms of being able to successfully navigate curvy, single lane roads/highways requiring varied speeds. I was in a 55 mph zone that had two consecutive 90 degree turns (the highway navigates around farmer's fields). Locals normally take these corners at 25-30 mph, speed up to about 45 mph in the middle section and then resume 55 mph driving after the second corner. On version 8 the EAP would enter the corners too fast and the car would cross wide into the oncoming lane (tested when there was no oncoming traffic). Yesterday, version 9 drove that section just like a local would. It also drove 8-9 miles of the same curvy rural highway slowing down appropriately for tighter corners and accelerating back to speed at corner exit, all without driver intervention.

So, while there is plenty to be unhappy with in terms of dashcam file corruption, etc, the real upgrade here was the capabilities of EAP.
 

Bernard

Well-Known Member
M3OC Supporting Member
Joined
Aug 3, 2017
Messages
243
Location
Hawaii, HI
Tesla Owner
Model 3
Country
Country
#20
KFORE's got it. :)

"Machine learning" is an extremely CPU-intensive offline process. Tesla will use thousands of machines running in parallel for this. You feed in a ton of data (collected from all the cars of the fleet), and out comes a new neural network (aka neural net, or NN).

The neural net is then included in the next version of vehicle software. It is also CPU-intensive, but can be handled by the car's single computer. This allows the car to make decisions based upon all of its input. But there is no "learning" happening at this point. The car will just be making decisions based on the programming of the neural net.

If you notice any difference in behavior in a single car from one drive to the next, then I think it's more likely due to calibrating the cameras. Each car's cameras will be aimed slightly differently, and it may take the car a little while to determine exactly how each camera is aimed and compensate for the differences.
The NN does not change except through firmware updates, clearly -- a neural net of the size used by Tesla takes enormous computational power to train.
But there is somewhat more to the car's Autopilot than just the monolithic NN.

For instance, we know Autopilot uses a number of calibration parameters (although we don't know what those are -- Tesla refers to initial camera calibration and little else). Some of these parameters could be adjusted as owners drive the car -- this is a form of learning if it is done in reaction to driving inputs. (Again, we know it occurs in the first 20-50mi or so for parameters affecting the cameras, but we do not know whether additional system parameters may be involved nor whether additional changes can be made after these initial 20-50mi.)
Automated Autopilot calibration for a particular driver (not just for a particular car) is of course desirable, so I would expect that Tesla is at least thinking about it and perhaps even introducing some minor aspects of it in new firmwares. (Even once we have true level 5 autonomy, not everyone will want to be driven around in the same manner, however safe; and while driver input remains essential, interpreting that input is important.)

There are other ways to modifiy the Autopliot that do not require recomputing the parameters of the NN, but I doubt that Tesla would use them, as frequent firmware updates almost always provide a better solution.
(For instance, memorizing certain special cases where the driver overrode the software, so as not to repeat the error that had to be corrected. Think of it as a patch on the current firmware, until a later firmware release fixes the problem by providing new weights for the NN itself. This is doable, but it might slow down Autopilot, since it would have to check whether one of these cases applies, and it is also somewhat risky.)