# Enhanced Summon / FSD more camera's needed?



## adam m (Feb 1, 2019)

I was just thinking, how can Enhanced summon / FSD ever work if the car can't see to the left and right of it. As a child, and then learning to drive your always taught to look both ways but the cars currently don't have a forward left and right facing camera. How will the car navigate 'T's or intersections?


----------



## ajl710 (Mar 1, 2019)

The front radar could most likely detect the distance and speed of objects to the left and right at an intersection.


----------



## iChris93 (Feb 3, 2017)

A couple things to keep in mind.
1) There are wider front facing cameras than we can see with dash cam.
2) Tesla can have the car turn slightly to use one of the side cameras to see perpendicularly too.


----------



## DocScott (Mar 6, 2019)

It would be really interesting to do the following experiment (this would either require Tesla to do it or some mighty hacking skills):

1. Choose a test car with HW 2.5 or 3 (doesn't matter which for this purpose).

2. Modify that car so that it can beam video from all camera views, along with radar data, ultrasonic data, and the usual displays (e.g. speed) to a nearby user not in the car. (Alternatively, the user could be in the car, but wearing a VR rig, if transmission lag is a problem. For the rest of this post, I'll still refer to them as the "remote user," although they might be physically in the car.)

3. Give the remote user a set-up where they can monitor all of the data streams at once (e.g. multiple monitors, one for each camera, plus one for radar and the usual display for ultrasonics).

4. Put a safety driver in to the car.

5. Set it up so that the remote user can drive the car (steering, accelerator, brake, signals, horn).

6. See how well the remote driver can drive the car using just what the car itself sees.

There would presumably be a bit of a learning curve for the remote driver getting used to looking at the interface. But once that was done, can the person drive as well as they could if they were sitting in the car? Maybe better, because it's easier to look in multiple directions at once? Or would it be worse, because there were some kind of blind spots, or because the camera quality isn't high enough, or because there's a penalty for not being able to move your head around to see things from slightly different angles?

If the remote driver in that situation can't drive the car effectively, then no amount of software and fancy chips will _ever_ allows the Teslas to achieve FSD. If the remote driver can drive effectively, though, then we know the hardware's good enough.

I hope Tesla tried experiments like that. It seems like an obvious test, but I haven't heard any reports of remote controlled Tesla test vehicles (with safety drivers) out on the street.


----------



## NOGA$4ME (Sep 30, 2016)

While I can see a wide-angle front-facing camera being able to conceivably navigate a regular T, but how about a slant T with a fairly acute slant to the left or the right?


----------



## Ed Woodrick (May 26, 2018)

Left and right are handled by the b pillar cams.
The car can see 360 degrees


----------



## SkipperOFMO (Mar 15, 2019)

NOGA$4ME said:


> While I can see a wide-angle front-facing camera being able to conceivably navigate a regular T, but how about a slant T with a fairly acute slant to the left or the right?


How do you do it with just two eyes sitting in a seat looking through a windshield? This car has far better visibility ALL the time in all situations.


----------



## garsh (Apr 4, 2016)

adam m said:


> but the cars currently don't have a forward left and right facing camera.


The B-pillar cameras cover left and right. You can see the view in the upper left and right portions of this video.






When all the cameras are combined, the car has a full 360 degree view.


----------



## NOGA$4ME (Sep 30, 2016)

SkipperOFMO said:


> How do you do it with just two eyes sitting in a seat looking through a windshield? This car has far better visibility ALL the time in all situations.


Umm...I turn my head (and at this one particular intersection I am thinking of, I have to partially turn my torso as well to look "back").

Another situation that I am concerned about are regular T's where you have to lean forward to see past an obstacle. In some cases these are signs that builders helpfully put up on corners to advertise housing developments, and sometimes they are plants that are either intentionally put on corners, or in some cases not properly maintained. And then there is the case of pulling out of a perpendicular parking spot with SUVs on both sides Not only do I have to lean forward, but I actually have to slowly pull my nose out slightly into traffic before I can get a good view of the travel lane. A sufficiently wide-angle front facing camera might be able to get a good view in this scenario, but I really have my doubts about the B pillar cameras given that they are WAY behind my head when I am leaning forward.


----------



## garsh (Apr 4, 2016)

NOGA$4ME said:


> Another situation that I am concerned about are regular T's where you have to lean forward to see past an obstacle.


No doubt, those situations are difficult. But the combination of the wide-angle front camera (there are 3 front cameras below the rear-view mirror) and the b-pillar cameras should have it covered. The fish-eye wide-angle front camera is still further forward than you leaning forward behind the windshield.


----------



## adam m (Feb 1, 2019)

Ed Woodrick said:


> Left and right are handled by the b pillar cams.
> The car can see 360 degrees


Looks like it's pointing at the ground not out the front of the car.


----------



## NOGA$4ME (Sep 30, 2016)

garsh said:


> No doubt, those situations are difficult. But the combination of the wide-angle front camera (there are 3 front cameras below the rear-view mirror) and the b-pillar cameras should have it covered. The fish-eye wide-angle front camera is still further forward than you leaning forward behind the windshield.


What is the field of view of the wide-angle front camera? Is it a full 180 degrees?


----------



## garsh (Apr 4, 2016)

adam m said:


> Looks like it's pointing at the ground not out the front of the car.


Try looking at the video I included in my post. The cameras have a very wide angle of view.


----------



## garsh (Apr 4, 2016)

NOGA$4ME said:


> What is the field of view of the wide-angle front camera? Is it a full 180 degrees?


No, it doesn't appear to be quite that far from the example videos I've seen. Probably more like 160.
But the B-pillar cameras capture everything from almost completely forward to slightly behind.


----------



## Rick Steinwand (May 19, 2018)

iChris93 said:


> 2) Tesla can have the car turn slightly to use one of the side cameras to see perpendicularly too.


That's what humans with stiff necks do!


----------



## airbusav8r (Feb 24, 2019)

The car can see close to 360, the lines are filled in much like google maps (complex lambda calculus + vectoring). The cameras do not mean much, the primary go/no-go system is radar which can determine object, speed, direction, etc... and is exponentially more reliable than Tesla cams and... 1000x faster than you turning your head left and right using judgment to base a car accelerating towards you. You are using system 1 (approximations). In the end, it probably is a wash and is about as safe as someone with 20/30+ years of driving, but the computer will not build bad habits.


----------



## airbusav8r (Feb 24, 2019)

What Tesla has not enabled yet (HW3) is echo relay (time to bounce off everything) essentially mapping an entire area with Sonar (not radar, this is important to note). Lidar will also be introduced, I’m assuming it will be Omni-directional and most likely will work much like adaptive headlights moving towards the direction you want to turn. I suspect that is a hard requirement for full FSD and legal, the car will need at a minimum 3 systems. Primary, secondary (redundant), manual intervention. The individuals designing much of this are from Boeing/Airbus/NG/Honeywell, individuals that worked on the 3-rule FMC (flight management computer). It’s quite interesting to see how much they are leveraging from aviation now, the car even has an internal inertia computer as of v10 to make turns more acceptable regarding G-Force on the APEX of a turn, cool stuff!


----------



## FF35 (Jul 13, 2018)

airbusav8r said:


> What Tesla has not enabled yet (HW3) is echo relay (time to bounce off everything) essentially mapping an entire area with Sonar (not radar, this is important to note). Lidar will also be introduced, I'm assuming it will be Omni-directional and most likely will work much like adaptive headlights moving towards the direction you want to turn. I suspect that is a hard requirement for full FSD and legal, the car will need at a minimum 3 systems. Primary, secondary (redundant), manual intervention. The individuals designing much of this are from Boeing/Airbus/NG/Honeywell, individuals that worked on the 3-rule FMC (flight management computer). It's quite interesting to see how much they are leveraging from aviation now, the car even has an internal inertia computer as of v10 to make turns more acceptable regarding G-Force on the APEX of a turn, cool stuff!


Lidar? Do you have a source for that information?


----------



## iChris93 (Feb 3, 2017)

airbusav8r said:


> the primary go/no-go system is radar which can determine object, speed, direction, etc...


I believe they want to move away from using radar.


----------



## Mr. Spacely (Feb 28, 2019)

airbusav8r said:


> What Tesla has not enabled yet (HW3) is echo relay (time to bounce off everything) essentially mapping an entire area with Sonar (not radar, this is important to note). Lidar will also be introduced, I'm assuming it will be Omni-directional and most likely will work much like adaptive headlights moving towards the direction you want to turn.


Tesla will never use Lidar. Musk used Lidar on SpaceX and said: "Other car companies are all going to dump lidar. Anyone relying on lidar is doomed. Lidar is really a shortcut."


----------



## Caulin (Sep 2, 2018)

I remember reading an article somewhere about Teslas approach vs others using Lidar. It had a great schematic around a Tesla for where radar and cameras can "see". That image confirmed what I see IRL, our cars are almost blind when looking behind us. I really dont see a path to FSD without having a *long range radar looking behind us*. How can we trust the car to change lanes without seeing cars coming up from the rear??? what happens if a car is flying down the left lane at 90 mph and my car wants to change lanes to pass someone? I imagine its going to pull out infront of the speeding car.


----------



## Mr. Spacely (Feb 28, 2019)

Caulin said:


> I remember reading an article somewhere about Teslas approach vs others using Lidar. It had a great schematic around a Tesla for where radar and cameras can "see". That image confirmed what I see IRL, our cars are almost blind when looking behind us. I really dont see a path to FSD without having a *long range radar looking behind us*. How can we trust the car to change lanes without seeing cars coming up from the rear??? what happens if a car is flying down the left lane at 90 mph and my car wants to change lanes to pass someone? I imagine its going to pull out infront of the speeding car.


Our cars have been successfully driving themselves on the highway without rear radar on NOA for months...


----------



## Skelly (Aug 15, 2018)

If you are using V10, you can spin the view around and see what the car "sees" behind you. I am actually pretty impressed with the number of cars it sees behind me. With regards to a car speeding up at 90 mph... That is certainly a concern of mine as well, but if you think about it, the car may have a better chance of handling that than a human driver. A human driver looks over their shoulder once, maybe twice, maybe a few times before changing lanes (although I have seen plenty not look at all!), so it's very possible that from the time they take their final look to the time they start changing lanes a car could come speeding up behind them...whereas the car is always looking and hopefully will react accordingly.


----------



## teslarules (Apr 16, 2017)

IMO, if they do not add more sensors the car will never be fully self driving. Look at the images provided, there are many situations pulling out onto a highway, etc. the camera cannot see far enough to detect moving cars.....I have been in numerous situations where you can barely make a safe decision to pull out, much less a car with cameras mounted half way back on the pillars? Just isn't going to be 100% as it is. Again my opinion but I have significant experience with this tech. I hope for many people who have high hopes and plopped down good hard earned money. I will not purchase FSD for my (3) or my (X) until I see it happening in real time.


----------



## garsh (Apr 4, 2016)

teslarules said:


> I have been in numerous situations where you can barely make a safe decision to pull out, much less a car with cameras mounted half way back on the pillars?


One of the three front cameras is a wide-angle camera with about a 160-degree view. That's further forward than you could ever put your own eyeballs. I think Tesla has everything pretty well covered with the current assortment of cameras.


----------



## DocScott (Mar 6, 2019)

Again, this is testable (by Tesla). Give a human driver a feed that shows only what the various sensors show, and see if they can drive the car effectively. If they can, then FSD is "just" an AI problem. If they can't, then it's also a sensor problem.


----------



## garsh (Apr 4, 2016)

DocScott said:


> Again, this is testable (by Tesla). Give a human driver a feed that shows only what the various sensors show, and see if they can drive the car effectively. If they can, then FSD is "just" an AI problem. If they can't, then it's also a sensor problem.


That would be a relevant test if you had 360° eyeballs in your skull.


----------



## Klaus-rf (Mar 6, 2019)

Mr. Spacely said:


> Our cars have been successfully driving themselves on the highway without rear radar on NOA for months...


 Actually, No. The cars still require a 100% attentive, human driver to constantly supervise and take over in an instant.

"successfully driving themselves" implies and requires that no human is involved. We're not there yet.


----------



## Mr. Spacely (Feb 28, 2019)

Klaus-rf said:


> Actually, No. The cars still require a 100% attentive, human driver to constantly supervise and take over in an instant. "successfully driving themselves" implies and requires that no human is involved. We're not there yet.


No. My car DOES drive itself many many miles at a time without intervention changing lanes and finding the correct exits. Whether or not I am there to take over doesn't negate the fact that it is driving itself...


----------



## Caulin (Sep 2, 2018)

What about the fact that there are almost NO redundancy sensors correct?? Multiple times last winter I couldnt even use the ACC because my radar sensors had ice on them. They really need a deicing system. I had heard rumers that there was one but from my experience it didnt seem to work.



Klaus-rf said:


> Actually, No. The cars still require a 100% attentive, human driver to constantly supervise and take over in an instant.


LEGALLY yes thats true. But like @Mr. Spacely stated, my car has driven thousands of miles with zero intervention needed.


----------



## iChris93 (Feb 3, 2017)

Caulin said:


> Multiple times last winter I couldnt even use the ACC because my radar sensors had ice on them. They really need a deicing system. I had heard rumers that there was one but from my experience it didnt seem to work.


Last year Elon Tweeted that they were working on Vision only. 

__ https://twitter.com/i/web/status/1069000094479876096


----------



## Caulin (Sep 2, 2018)

iChris93 said:


> Last year Elon Tweeted that they were working on Vision only.
> 
> __ https://twitter.com/i/web/status/1069000094479876096


It would be nice if they added a regular cruise control, even if its just for when ACC is unavailable. I was on a 5 hour trip and was super annoyed I had zero cruise control. On a side note, I thought cameras were a bad sensor for speed, or is that totally false?

edit: grammar


----------



## DocScott (Mar 6, 2019)

garsh said:


> That would be a relevant test if you had 360° eyeballs in your skull.


I have been driving with 360 degree vision for many years, thanks to rear and side view mirrors.


----------



## Klaus-rf (Mar 6, 2019)

Possible game changer:
https://www.autoblog.com/2020/01/05/bosch-introducing-cheaper-lidar-system/


----------

