# Gaze tracking using interior camera?



## DocScott (Mar 6, 2019)

bwilson4web said:


> I've grown used to adjusting the volume up/down one click instead of applying torque. The speed vernier works the same way. Have you tried them?
> 
> Bob Wilson


I do wish Elon would just give in and switch to gaze-tracking (in the model 3, at least--do the S and X have inward facing cameras?). It's both safer and easier for the driver.


----------



## ibgeek (Aug 26, 2019)

DocScott said:


> I do wish Elon would just give in and switch to gaze-tracking (in the model 3, at least--do the S and X have inward facing cameras?). It's both safer and easier for the driver.


The camera in the car is a fish-eye lens, it wont work, plus you need infra red for night time. It would be a nice option, but it can't happen with the current hardware.


----------



## DocScott (Mar 6, 2019)

ibgeek said:


> The camera in the car is a fish-eye lens, it wont work, plus you need infra red for night time. It would be a nice option, but it can't happen with the current hardware.


Are you sure about that?

https://electrek.co/2020/10/04/tesla-hacker-driver-facing-camera-looking-for/

(Mods: feel free to move this discussion to another thread; it clearly has nothing to do with a firmware version that's more than a year old.)


----------



## JWardell (May 9, 2016)

DocScott said:


> Are you sure about that?
> 
> https://electrek.co/2020/10/04/tesla-hacker-driver-facing-camera-looking-for/
> 
> (Mods: feel free to move this discussion to another thread; it clearly has nothing to do with a firmware version that's more than a year old.)


 Yes...he had ambient light when taking that picture


----------



## Ksb466 (Oct 22, 2018)

JWardell said:


> Yes...he had ambient light when taking that picture


Maybe will use it to lessen nags when possible. it does return "dark" when that's the case and could retain nags in dark case. It'd be better then nothing For older models and a HW fix for future models


----------



## JasonF (Oct 26, 2018)

Ksb466 said:


> Maybe will use it to lessen nags when possible. it does return "dark" when that's the case and could retain nags in dark case. It'd be better then nothing For older models and a HW fix for future models


Remember, the screen gives off quite a bit of light when it's on, even when the background color is black. If you want to see just how much, park it in a completely dark garage, and try to take a selfie with your phone while sitting in the driver's seat with the display on.


----------



## DocScott (Mar 6, 2019)

It certainly seems as if the interior camera is good enough for supplemental gaze-tracking during the day, although not on dark roads at night: 

__ https://twitter.com/i/web/status/1379928419136339969.

Teslas are apparently already capable of evaluating what the driver is doing, and there are some reports that it may be doing that to increase the frequency of steering torque prompts if you're not looking straight ahead.

It would be great if Tesla moved to a hands-free L2 mode for daytime-driving in situations where AP is appropriate. Essentially decrease the prompts to once every couple of minutes if it can tell you're looking straight ahead. If it loses that ability--say, you enter a dark tunnel--then it could immediately prompt aggressively for steering wheel torque.

Mods: It would be nice to move this discussion to its own thread.


----------



## garsh (Apr 4, 2016)

DocScott said:


> Mods: It would be nice to move this discussion to its own thread.


If you can send us a report with exactly which post #s from the thread to move into a new thread, that would greatly simplify our task.


----------



## Needsdecaf (Dec 27, 2018)

DocScott said:


> It certainly seems as if the interior camera is good enough for supplemental gaze-tracking during the day, although not on dark roads at night:
> 
> __ https://twitter.com/i/web/status/1379928419136339969.
> 
> ...


Mine would get: "Camera Blocked 100%" and not much else.


----------



## tivoboy (Mar 24, 2017)

DocScott said:


> It certainly seems as if the interior camera is good enough for supplemental gaze-tracking during the day, although not on dark roads at night:
> 
> __ https://twitter.com/i/web/status/1379928419136339969.
> 
> ...


F. Ing AI!… looks pretty competent. I will say this could allow action against the obvious and blatant physical phone usage, either calling by hand with phone up to your head (who does that today still, really) and actual phone interaction, but…

At this point, what stops me from putting an iPad mini right behind the steering wheel and use apples excellent Accessibility controls to interact with the iOS device through voice or even some eye, head movement interaction. At this point I can sit there and open apps, scroll, compose messages, compose emails, have nearly anything read to me and start stop it, open web pages, scroll news feeds, twitter, etc. I don't see any way that the car could interrupt that capability using the onboard camera and AI assesement and controls.


----------



## Enginerd (Aug 28, 2017)

tivoboy said:


> what stops me from putting an iPad mini right behind the steering wheel


No offense to you personally, but there will always be an idiot that can one-up the system. I am somewhat reassured that it would be unnatural to do this... it would take some effort to evade the system, which wouldn't come as easily to the average distracted idiot.

The driver-facing NN is an interesting cul-de-sac along the AP/FSD development highway. Tesla has emphasized that the end game is FSD, at which time most of these driver attention-related attributes are moot. It does make you wonder what _else_ they're going to do with this information in the meantime.


----------



## Madmolecule (Oct 8, 2018)

I do not believe the camera is really there to monitor whether you can use FSD or not. I believe as I posted earlier the camera is part of Tesla's new phone that they will be releasing. I believe it will all be part of their "life pilot" system Which will of course be able to monitor the drivers attentiveness but more importantly provide continuous health screening and reaction monitoring. It will be the link to manage your personalize experience, infotainment, routing, situational awareness, but most important health.

I do believe there's much better chance that teslas phone with "life pilot" will be released this year than FSD

copy of my earlier phone rumor


Madmolecule said:


> Since my wildest prediction about Telsa and Bitcoin came somewhat true I will continue with the real "made-up" reason the app has stagnated and does not have the capabilities of some 3rd party apps. Also why they have seemed to spend more time on game development than the Tesla App development.
> 
> The Tesla phone or as I call it Musk Mobile Communicator and Life Pilot.
> 
> ...


----------



## tivoboy (Mar 24, 2017)

I think the camera is mostly meant to be used to monitor and record the cabin, IF and WHEN Tesla ever moves to the robotaxi model for users to put their cars on the network. They would need to capture the passengers and record them in the case that they purposely vandalize the car. If it was really meant to capture the driver actions for FSD, it probably wouldn’t have been put in the center stack and put possibly in front of the driver.


----------



## Needsdecaf (Dec 27, 2018)

tivoboy said:


> I think the camera is mostly meant to be used to monitor and record the cabin, IF and WHEN Tesla ever moves to the robotaxi model for users to put their cars on the network. They would need to capture the passengers and record them in the case that they purposely vandalize the car. If it was really meant to capture the driver actions for FSD, it probably wouldn't have been put in the center stack and put possibly in front of the driver.


Then why does it have all kinds of code identifying whether the driver is engaged, looking down, holding a phone, using a phone, wearing sunglasses, etc? That would be a completely unnecessary waste of resources to put in that programming for simply having a security camera recording malfeasance.


----------



## tivoboy (Mar 24, 2017)

Needsdecaf said:


> Then why does it have all kinds of code identifying whether the driver is engaged, looking down, holding a phone, using a phone, wearing sunglasses, etc? That would be a completely unnecessary waste of resources to put in that programming for simply having a security camera recording malfeasance.


I never said it was THE reason, just that's I THINK that the center cam was "mostly" implemented for future robotaxi or cabin monitoring. That they have been able to, either as an initial objective or secondary, to use AI to interpret a drivers actions vis a vis attention, mobile device (or other) usage doesn't invalidate that. Who knows, we may find that there is additional AI that is trying to interpret physical actions of the passengers, (I can think of a few right now) someone with a weapon, someone attacking the car or another passenger with a tool or weapon. All of this could certainly be in the AI and that one driver view we saw simply didn't trigger any of it.


----------



## GDN (Oct 30, 2017)

I honestly believe that most of us truly believed it wouldn't have the resolution or capabilities to do anything more than monitor the cabin. Tesla either knew all along or have been able to get it to do a decent job of tracking, and this may truly be required to get to any kind of next level by regulators. I think it to be impressive what they are able to track from it. It does seem there will be a problem with a dark cabin - night time, etc.

Also noted above someone placing a phone in front of the driver behind the steering wheel, I don't care who creates the system or what it does, there will always be a way created to cheat it.


----------



## TesLou (Aug 20, 2016)

Has anyone confirmed whether or not the refreshed X/S has a cabin cam?


----------



## tivoboy (Mar 24, 2017)

TesLou said:


> Has anyone confirmed whether or not the refreshed X/S has a cabin cam?


I don't think any are really in the wild yet. Though I can't imagine they would make this uniform update from a supplier standpoint .


----------



## GDN (Oct 30, 2017)

TesLou said:


> Has anyone confirmed whether or not the refreshed X/S has a cabin cam?


I don't have a link, but yes it was noted it has the interior camera.


----------



## garsh (Apr 4, 2016)

TesLou said:


> Has anyone confirmed whether or not the refreshed X/S has a cabin cam?


This official interior photo appears to show it.


----------



## TesLou (Aug 20, 2016)

garsh said:


> This official interior photo appears to show it.


Indeed it does. Good spot, Garsh.


----------



## tivoboy (Mar 24, 2017)

garsh said:


> This official interior photo appears to show it.


Not to beat a dead horse, but I believe that photo is actually a rendering? And well, it does show a wheel that isn't going to come out first as well? but yes, I'm pretty sure the RVM will have the camera module as do the 3 and Y.


----------



## garsh (Apr 4, 2016)

tivoboy said:


> Not to beat a dead horse, but I believe that photo is actually a rendering?


Yes, it's probably a rendering.



tivoboy said:


> And well, it does show a wheel that isn't going to come out first as well?


I know we've seen a few Model S test mules with round steering wheels, but I don't think that Tesla has backtracked at all on offering the Yoke steering wheel.


----------

