Tesla's Computer Vision Master Plan

Discussion in 'Tesla Discussions' started by Trent Eady, Apr 17, 2018.

More threads by Trent Eady
  1. Trent Eady

    Member
    Expand Collapse

    Joined:
    Sep 3, 2017
    Messages:
    13
    Location:
    Baccalieu
    Country:
    Country
    My latest work: an in-depth outline of Tesla's plan to give cars the ability to perceive the world and therefore drive autonomously throughout it.

     
    • Like Like x 2
  2. John

    Tech CEO
    M3OC Supporting Member
    Expand Collapse

    Joined:
    Apr 15, 2016
    Messages:
    1,185
    Location:
    California
    Tesla Owner:
    Model 3
    Country:
    Country
    Thoroughly enjoyed this piece, Trent.

    I'm especially curious as to how Tesla partitions the self-driving problem between one or more NNs and traditional logic. For instance, does a NN just do lane interpretation and sign/signal recognition, and then the rest is traditional logic? Or is a NN learning how to drive more holistically? How do they chunk out the problem in their control diagram, and are there any "critic" NNs observing and correcting in parallel? One day I'd like to hear Andrej discuss this.
     
    • Agree Agree x 1
    • Love it! Love it! x 1
  3. KarenRei

    Top-Contributor
    Expand Collapse

    Joined:
    Jul 27, 2017
    Messages:
    1,056
    Location:
    Reykjavík
    Country:
    Country
    #3 KarenRei, Apr 18, 2018
    Last edited: Apr 18, 2018
    Good article.

    One thing you left out, and which I wrote about recently, concerns examining the road surface. This was one of several reasons for me previously having pessimism about full self driving. We constantly examine the road ahead, looking for things that look wet, icy, potholed, gravel, debris, etc. And when there's water on the road, we try to assess its depth (despite not being able to see the road beneath) using context to see if it's safe to drive through, or just a puddle. These things involve very complex reasoning involving a lot of back knowledge, and are not readily offloaded to a neural net for simple visual pattern matching. AI-Hard, so to speak.

    However, I think there's two *excellent* solutions for them. Probably without any changes to the current hardware suite:

    https://model3ownersclub.com/threads/autopilot-seeing-the-road-in-ways-that-we-cant.6403/#post-86747

    I still think there's a massive body of knowledge that AP needs to incorporate before the general public can feel safe that it'll handle most all situations as good or better than they would. If you live in the countryside it might be "recognize that this is a ewe, that's a lamb, and the lamb will likely run to the ewe as you get close". In a suburban area it might be "recognize that this is a child in that yard and determine from his behavior and the presence or lack of a parent how likely he is to run out in front of you". In a bad area it might be "recognize whether these people you see ahead of you are harmless or whether they're acting suspiciously and might be planning a carjacking". In a rugged area it might be "recognize how weak the shoulder looks to be ahead and evaluate how severe the consequences would be if you drive off". Etc, etc, etc.** These things can all be incorporated eventually, but there's a massive corpus of knowledge that we use when driving.

    But as for evaluating the road... radar returns and fleet-sourced high-res topography data should let a vehicle evaluate the upcoming road surface and assess water depth, respectively, better than humans can.


    ** These challenges can even be combined. For example, the worst part of the route to my house has a short but relatively steep slope on a gravel road, and you can't see over the top. The shoulders are weak, so you prefer to drive toward the centre, and there's a ravine on the south side (but not the north). If the gravel is wet or the road is very icy, you want to have momentum going into the slope, as you certainly don't want to have to stop on the slope when climbing (or brake on the downslope), but you have to balance that off against not knowing if someone is coming who might also be driving in the middle of the road to avoid the shoulder on the ravine side.

    These decisions are not simple. An AI simply will not make them as good as a human when presented with equivalent information. The AI needs much better information to compensate for its reduced reasoning power. But I'm increasingly convinced that it can get said information.
     
    • Informative Informative x 1
  4. John

    Tech CEO
    M3OC Supporting Member
    Expand Collapse

    Joined:
    Apr 15, 2016
    Messages:
    1,185
    Location:
    California
    Tesla Owner:
    Model 3
    Country:
    Country
    Trent you should post your latest article on the Cambridge NN stuff. That split video showing semantic interpretation of the roadway and other objects is awesome.
     
  5. Trent Eady

    Member
    Expand Collapse

    Joined:
    Sep 3, 2017
    Messages:
    13
    Location:
    Baccalieu
    Country:
    Country
    I assume you mean this? The same split screen video from the Cambridge/Oxford researchers is already in the Medium article that I posted originally!
     
    • Like Like x 1
  6. Trent Eady

    Member
    Expand Collapse

    Joined:
    Sep 3, 2017
    Messages:
    13
    Location:
    Baccalieu
    Country:
    Country
    Thank you, John!

    I imagine that Tesla at the very least does not use one big end-to-end neural network for everything, for reasons described in this video. I haven’t really heard any refutation of Shashua’s argument in that video; people generally seem to agree. So at a minimum, I imagine Tesla must split different tasks into different neural networks.

    Some people on the Tesla Motors Club forum who have hacked into their cars claim that the perception is done by neural networks, and the actual driver input (braking, steering, acceleration) is traditional logic, but I don’t know if what they’re claiming is true. Hard to verify these claims.

    On the topic of critic neural networks, have you seen this paper?
     
    • Informative Informative x 1
  7. John

    Tech CEO
    M3OC Supporting Member
    Expand Collapse

    Joined:
    Apr 15, 2016
    Messages:
    1,185
    Location:
    California
    Tesla Owner:
    Model 3
    Country:
    Country
    Oooh. I'll have to give a that a read. Thanks!
     
  8. John

    Tech CEO
    M3OC Supporting Member
    Expand Collapse

    Joined:
    Apr 15, 2016
    Messages:
    1,185
    Location:
    California
    Tesla Owner:
    Model 3
    Country:
    Country
    Did you see the post on TMC about the multiple NNs someone dug up?
    Check it out here.
     
    • Like Like x 1
    • Informative Informative x 1
  9. EVfusion

    M3OC Supporting Member
    M3OC Supporting Member
    Expand Collapse

    Joined:
    Mar 10, 2017
    Messages:
    157
    Location:
    Canberra, ACT
    Country:
    Country
    Trent, I thoroughly enjoyed your piece. It is both a very informative and a lucid account. I must commend you both for the depth of your underlying research and for the very clear way you have presented it.
     
    • Agree Agree x 1
    • Love it! Love it! x 1

Share This Page

Model 3 Owners Club © 2018. All rights reserved.
Model 3 Owners Club (M3OC) is an enthusiast club and is not affiliated with Tesla Motors or the Tesla Motors Club Forum (TMC).
All Tesla logos are trademarks or registered trademarks of Tesla Motors.