# Soooo... the selfie camera isn't for eye tracking? Then what is it for?



## KarenRei (Jul 27, 2017)

__ https://twitter.com/i/web/status/996102919811350528


----------



## garsh (Apr 4, 2016)

It's for the Tesla Network.

When you rent out your car to the Tesla Network, it'll record what's going on in the interior.


----------



## jsmay311 (Oct 2, 2017)

If eye tracking is ineffective, how does GM manage to do it with Super Cruise?
(Cadillac Super Cruise Eye Tracking Technology)

I always kinda thought that "hands on wheel" was a poor proxy for "driver paying attention". Privacy concerns aside, making sure the driver's eyes are actually on the road seems like it would be a more effective solution.


----------



## SoFlaModel3 (Apr 15, 2017)

garsh said:


> It's for the Tesla Network.
> 
> When you rent out your car to the Tesla Network, it'll record what's going on in the interior.


^^^ this!


----------



## BluestarE3 (Oct 12, 2017)

How well would eye tracking work if the driver wears sunglasses (especially reflective ones)? Maybe they really mean head orientation/position? For example, head is turned aside or down (distracted or asleep) rather than facing forward?


----------



## Spiffywerks (Jul 30, 2017)

Super-cruise works with sunglasses on. If it cannot see your eyes, then it won't turn on. It will also turn off if you are not facing forward with your eyes looking out in front of you. Camera looks right at your face from the steering wheel. Supercruise is also limited to certain highways and does not do lane changes.

The interior camera on the Model 3 is not situated in a position to be effective at determining you are looking forward.

Check out this video to see what Super Cruise does. It's a good review. But forget about the Autopilot problems. They did update the video and saying it no longer has the issues they first reported.


----------



## 3V Pilot (Sep 15, 2017)

jsmay311 said:


> If eye tracking is ineffective, how does GM manage to do it with Super Cruise?


Okay, it's really hard to say this without it sounding very snarky but I'm no fan of GM so take that for what it's worth. Just because GM did it doesn't mean it's effective. Just my opinion. Maybe Tesla engineers didn't like the fact it couldn't see through mirrored sunglasses or whatever limitations it places on the driver. Or maybe they realized it was more of a waste of time that what they could get out of it. In any case I'm really glad that autopilot doesn't track my eyeballs!


----------



## jsmay311 (Oct 2, 2017)

As already mentioned, Super Cruise does work with sunglasses.

And the WSJ quote says "Tesla engineers wanted to add eye tracking and other sensors", so the question isn't just about the capability of the _existing_ camera, but rather what a _different_ suite of hardware _could've_ done.



Mike Land said:


> Just because GM did it doesn't mean it's effective. Just my opinion. Maybe Tesla engineers didn't like the fact it couldn't see through mirrored sunglasses or whatever limitations it places on the driver.


Oh right. I forgot that Tesla is always right and GM is always wrong. Silly me. I guess I can ignore all of the Super Cruise reviews that say the eye tracking works very well.


----------



## KarenRei (Jul 27, 2017)

jsmay311 said:


> As already mentioned, Super Cruise does work with sunglasses.


Actually, what was stated was just the opposite: "Super-cruise works with sunglasses on. If it cannot see your eyes, then it won't turn on."


----------



## jsmay311 (Oct 2, 2017)

jsmay311 said:


> As already mentioned, *Super Cruise does work with sunglasses*.





KarenRei said:


> Actually, what was stated was just the opposite: "*Super-cruise works with sunglasses on.* If it cannot see your eyes, then it won't turn on."


Now I'm confused. Did the definition of the word "opposite" change?  J/k.

I'm no expert on infrared cameras, but it appears that the Super Cruise camera can "see through" sunglasses.

http://www.thedrive.com/tech/17083/...esla-autopilot-vs-gm-supercruise-head-to-head


> "The driver's face is illuminated by six infrared lamps positioned on the steering wheel at 11 and 1 o'clock, while an infrared camera on top of the steering column monitors the driver's face. Turn your head or take your eyes off the road for too long-which is not that long at all-and disengagement warnings will begin.
> I tried to trick it. I held my phone and began texting, first in front of the steering wheel, then off to the side. I wore four different pairs of sunglasses. No matter what I did, the disengagement warning went off."


----------



## KarenRei (Jul 27, 2017)

jsmay311 said:


> Now I'm confused. Did the definition of the word "opposite" change?  J/k.
> 
> I'm no expert on infrared cameras, but it appears that the Super Cruise camera can "see through" sunglasses.
> 
> http://www.thedrive.com/tech/17083/...esla-autopilot-vs-gm-supercruise-head-to-head


----------



## MelindaV (Apr 2, 2016)

can we move on now please? If you care to start a separate thread in the Off Topic section about the wonders of GMs Supercruise, please feel free.


----------



## John (Apr 16, 2016)

Tesla has used NVidia hardware in the past for Autopilot processing, though not necessarily off-the-shelf stuff.
Here's a video of NVidia's work on eye tracking, and an interesting paper that claims that eye tracking can be reduced to a trained neural net that can run on phones and tablets, just like face recoginition already runs on just about every camera these days.






Paper:

http://gazecapture.csail.mit.edu/cvpr2016_gazecapture.pdf


----------



## John (Apr 16, 2016)

I tend to think that eye tracking just got pushed aside by Tesla for other concerns, in a time where it has been most important to get Autopilot itself working, and that it's a little embarrassing that GM (actually, Seeing Machines) got this to work before them.

So I guess I'm second guessing the claim that eye tracking was set aside because it doesn't work well. I think it's true that it *didn't work well for Tesla when they were last working on it*, but not true that it *can't*.


----------



## garsh (Apr 4, 2016)

John said:


> I tend to think that eye tracking just got pushed aside by Tesla for other concerns, in a time where it has been most important to get Autopilot itself working, and that it's a little embarrassing that GM (actually, Seeing Machines) got this to work before them.


I really don't think the interior camera in the 3 was ever meant for eye tracking. A camera designed for eye tracking would be aimed at the driver's head. This camera is aimed at the center of the interior. I really think it's purpose is just to record passengers for when the Tesla Network becomes available.


----------



## jsmay311 (Oct 2, 2017)

MelindaV said:


> can we move on now please? If you care to start a separate thread in the Off Topic section about the wonders of GMs Supercruise, please feel free.


It's not off topic.

The original post quotes Elon disputing the accounts of Tesla Autopilot engineers who reported that eye tracking was not added to Autopilot despite its safety benefits because Tesla execs objected to the added cost.

Musk instead argues that eye tracking was not added because it's "ineffective", but Super Cruise provides a concrete counter-example of eye tracking working effectively.


----------



## Spiffywerks (Jul 30, 2017)

Here's a thought.. I'm sure Chevy tested it, and although it may sound funny....

How does the eye camera system work with Asian drivers? Some of my Asian friends eyes you can barely see. Yes, sounds silly, but I'm serious. Would the Autopilot/Supercruise not work if it didn't see your eyes cause you barely open your eyes to see?

Maybe Elon was thinking of the huge market in China? 

Also another thought - I could see the use of a driver facing camera being redundant to the ultimate goal of the car: Full Self-Driving. The car's intent is to drive on it's own, so having a driver monitoring system would not be necessary and would be a waste of money if the plan is to have FSD in less than 5 years. Adding the additional equipment wouldn't only be needed on the Model 3, but S and X as well - adding another dividing line between newer models and older ones.

After all, Tesla is a company to make money. And Elon is human and can make mistakes. (OMG!) But, if it turns out not including is a mistake, I'm sure he'll man up to it and make it right.


----------



## KarenRei (Jul 27, 2017)

"Racist AIs" is actually a legitimate problem that's come up to bite many projects. You have to be very careful with your training data. For example, if you're training an image recognition AI and your images of people are overwhelmingly caucasian, and you have a bunch of animals as the rest of the training set, it might start classifying people of African descent as gorillas because it's not familiar with the concept of humans with dark skin.

In the case of "asians with eyes closed", this was an actual issue with a New Zealand passport photo checker :

https://uk.reuters.com/article/uk-newzealand-passport-error-idUKKBN13W0RE


----------



## garsh (Apr 4, 2016)

KarenRei said:


> "Racist AIs" is actually a legitimate problem that's come up to bite many projects


Also a problem for non-AI products.


----------



## roflwaffle (Sep 25, 2017)

I'm 97+% sure Tesla is at least doing something similar to eye-tracking with the selfie camera. I've noticed on a few occasions the 3's steering wheel would vibrate when driving. I mentioned this to my wife, and she was surprised because she never noticed it. 

She tried to replicate the situations where I said it had vibrated and she couldn't. The next time I drove us someplace, I noticed the wheel vibrating again and I think figured it out. To get the steering wheel to vibrate, I need to reduce my grip on the steering wheel, drift towards another lane, and look at something inside the car instead of on the road. The wheel won't vibrate if I only loosen my grip on the steering wheel and drift towards another lane.

My guess is Elon was calling out the use of eye-tracking alone, not saying that eye-tracking isn't worthwhile.


----------



## GDN (Oct 30, 2017)

roflwaffle said:


> I'm 97+% sure Tesla is at least doing something similar to eye-tracking with the selfie camera. I've noticed on a few occasions the 3's steering wheel would vibrate when driving. I mentioned this to my wife, and she was surprised because she never noticed it.
> 
> She tried to replicate the situations where I said it had vibrated and she couldn't. The next time I drove us someplace, I noticed the wheel vibrating again and I think figured it out. To get the steering wheel to vibrate, I need to reduce my grip on the steering wheel, drift towards another lane, and look at something inside the car instead of on the road. The wheel won't vibrate if I only loosen my grip on the steering wheel and drift towards another lane.
> 
> My guess is Elon was calling out the use of eye-tracking alone, not saying that eye-tracking isn't worthwhile.


At this point the camera is not active. At least that is what we are told. It will only take a small piece of electrical tape to change the behavior if you think it is active.

The steering wheel will vibrate, it is designed to so so when you cross a lane marker without using your blinker or drift across a lane. The feature can be turned off. It didn't do it because the camera could see your eyes and knew you weren't watching.


----------



## roflwaffle (Sep 25, 2017)

GDN said:


> At this point the camera is not active. At least that is what we are told. It will only take a small piece of electrical tape to change the behavior if you think it is active.
> 
> The steering wheel will vibrate, it is designed to so so when you cross a lane marker without using your blinker or drift across a lane. The feature can be turned off. It didn't do it because the camera could see your eyes and knew you weren't watching.


I'll test that out. It might activate based on only steering wheel inputs and steering angle compared to the direction of the lane markers, but my wife couldn't replicate it by drifting towards or crossing across a lane markers alone. She may be holding the steering wheel too tightly though.


----------



## John (Apr 16, 2016)




----------



## roflwaffle (Sep 25, 2017)

Doh! I'll check DW's settings tomorrow.


----------



## Uricasha (Feb 19, 2017)

Maybe in the future the selfie camera can be used for initial seat adjustment for ride sharing. Head location relative to seat height is proportional to leg length?



roflwaffle said:


> Doh! I'll check DW's settings tomorrow.


 When I first got the car, I thought there was an issue with the wheels being off balance at certain speeds causing the steering wheel to vibrate. Realized later that it was just the lane departure warning. (in my opinion, it's too subtle, hoping they add audible warning in future)


----------



## garsh (Apr 4, 2016)

Uricasha said:


> Maybe in the future the selfie camera can be used for initial seat adjustment for ride sharing.


For ride-sharing, I hope that your Tesla account (associated with your Tesla key-card) will have your preferred driver preferences saved for each type of vehicle - seat position, steering wheel position, mirror positions, radio presets, etc.


----------



## Kizzy (Jul 25, 2016)

garsh said:


> For ride-sharing, I hope that your Tesla account (associated with your Tesla key-card) will have your preferred driver preferences saved for each type of vehicle - seat position, steering wheel position, mirror positions, radio presets, etc.


I'd assume the key would be a Tesla account and one's cell phone. Model 3 seemed to me to pilot that process a bit (though maybe it's not reliable enough).


----------



## viperd (Feb 17, 2017)

roflwaffle said:


> I'm 97+% sure Tesla is at least doing something similar to eye-tracking with the selfie camera. I've noticed on a few occasions the 3's steering wheel would vibrate when driving. I mentioned this to my wife, and she was surprised because she never noticed it.
> 
> She tried to replicate the situations where I said it had vibrated and she couldn't. The next time I drove us someplace, I noticed the wheel vibrating again and I think figured it out. To get the steering wheel to vibrate, I need to reduce my grip on the steering wheel, drift towards another lane, and look at something inside the car instead of on the road. The wheel won't vibrate if I only loosen my grip on the steering wheel and drift towards another lane.
> 
> My guess is Elon was calling out the use of eye-tracking alone, not saying that eye-tracking isn't worthwhile.


When you depart your lane over solid lines, the steering wheel vibrates.

You could test by covering the camera with electrical or painters tape.


----------



## GDN (Oct 30, 2017)

viperd said:


> When you depart your lane over solid lines, the steering wheel vibrates.
> 
> You could test by covering the camera with electrical or painters tape.


It is more than solid lanes. It is any lane change where you don't use your signal. Using your signal will keep the warning from happening.


----------

