Here’s a weird AP issue: Autopilot mistakes moon for a traffic light

shareef777

Top-Contributor 👍🏻
TOO Supporting Member
Joined
Mar 9, 2019
Messages
1,580
Location
Chicago
Country
Country
Tesla Owner
Model 3
So driving on AP and it kept thinking the moon was a yellow light 😂

3EE5715E-83CD-4EA1-8E23-540301AEAD1E.jpeg
 

garsh

🐦🦆🦅🐧 🐓🦃🦉🦢🦜
Moderator 📌
TOO Supporting Member
Joined
Apr 4, 2016
Messages
15,543
Location
Pittsburgh PA
Country
Country
Tesla Owner
Model 3
I also had an issue with the car seeing phantom traffic signals.

I just went on a 400-mile drive with 2021.4.18.2. While I experienced several instances of unwanted, incorrect braking, I never experienced "phantom" braking. That is, I was always able to figure out the cause of the braking.

The most infuriating instance I found was when encountering these little orange diamond-shaped reflectors on signs:
View attachment 38805
It took me a while to figure out what was going on. When I glanced at the screen, I saw flickering traffic signals with a yellow light on! With the sun somewhat behind me and reflecting off of these, the car thought it was a bright yellow light!

The other common (and much more understandable) occurrence was encountering yellow flashing lights for various reasons. They didn't even have to be on - the car would begin to slow down.

View attachment 38806
 

Long Ranger

Top-Contributor 👍🏻
TOO Supporting Member
Joined
Jun 1, 2018
Messages
872
Location
Seattle
Country
Country
Tesla Owner
Model 3
So driving on AP and it kept thinking the moon was a yellow light 😂
How’s the air quality there? I saw the same thing last summer when we had bad smoke from forest fires. In my case it was the sun instead of the moon.
 

shareef777

Top-Contributor 👍🏻
TOO Supporting Member
Joined
Mar 9, 2019
Messages
1,580
Location
Chicago
Country
Country
Tesla Owner
Model 3
How’s the air quality there? I saw the same thing last summer when we had bad smoke from forest fires. In my case it was the sun instead of the moon.

I'm in the Chicagland area, so didn't think it'd be from the west coast fires, but who knows 🤷‍♂️
 

Nancat999

Member
Joined
Jun 2, 2021
Messages
8
Location
TN
Country
Country
Tesla Owner
Reservation
So driving on AP and it kept thinking the moon was a yellow light 😂

View attachment 39309
Better check the moon cycle on the weather app, so you know when the full moon is out. Unless of course you unfortunately work in customer service, all the crazies seems to call in. Joking aside, I can see by the photo why the car software was confused. I would think since meteorologist know when that will occur for the 10 yrs, maybe somehow they can get that calculated into their algorithms.
 

JasonF

Legendary Member 🏆
Joined
Oct 26, 2018
Messages
2,723
Location
Orlando FL
Country
Country
Tesla Owner
Model 3
All they need to do is have FSD grab two consecutive video frames and compare the relative speed of the car vs the “traffic light”. If the distance isn’t shrinking comparatively to the speed of the car, ignore it.
 
  • Agree
Reactions: DocScott

garsh

🐦🦆🦅🐧 🐓🦃🦉🦢🦜
Moderator 📌
TOO Supporting Member
Joined
Apr 4, 2016
Messages
15,543
Location
Pittsburgh PA
Country
Country
Tesla Owner
Model 3
All they need to do is have FSD grab two consecutive video frames and compare the relative speed of the car vs the “traffic light”. If the distance isn’t shrinking comparatively to the speed of the car, ignore it.
IIRC, autopilot currently reads two frames. But with the car itself bouncing around a little bit, it could look like the moon is moving at times, which is probably why a traffic light keeps appearing and disappearing on the screen.

Hopefully the pure-vision re-write will use more frames to allow the car to make better judgements.
 

JasonF

Legendary Member 🏆
Joined
Oct 26, 2018
Messages
2,723
Location
Orlando FL
Country
Country
Tesla Owner
Model 3
IIRC, autopilot currently reads two frames. But with the car itself bouncing around a little bit, it could look like the moon is moving at times, which is probably why a traffic light keeps appearing and disappearing on the screen.

I noticed something in the video from a coder point of view - and it’s that someone got lazy and made and assumption about traffic lights to hack them in quickly. It looks like it’s designed to assume traffic lights are stationary and close by, and the hack is that they estimate where it is by how fast the car is moving. That, I believe, is why you see the moon traffic light appear and move toward, and then disappear overhead.

We know AP/FSD can judge closing distance to moving cars, they simply didn’t use that mechanic for traffic lights. Maybe to save processing time?
 
  • Like
Reactions: DocScott

tencate

Top-Contributor 👍🏻
TOO Supporting Member
Joined
Jan 10, 2018
Messages
734
Location
Los Alamos, NM
Country
Country
Tesla Owner
Model 3
I recently took a long 3000 mile trip to Portland and back. At one point the car became very confused, hesitating and starting to stop every few seconds or so, and then it'd go away and then come back. Phantom braking? But why? After about 1/2 hour of this rather frustrating behavior (glad traffic was light!), I finally figured out why.

The car thought it was seeing a flashing yellow traffic light, on the interstate (where the speed limit was 80 mph)! I first saw a traffic light showing up on the display and then looked up to see what it THOUGHT it was seeing. There it was, the moon had just risen and was about the right height for the car to think it was a yellow traffic light. I snapped a quick photo with the phone which I've attached. Betcha this is an "edge" case no one thought about! :)
MoonriseNSelfDriving.JPG
 

lance.bailey

Top-Contributor 👍🏻
Joined
Apr 1, 2019
Messages
1,146
Location
cloverdale, BC
Country
Country
Tesla Owner
Model 3
lots of issues like that. I bet you did not have a "slowing down tap wand ..." messages. BC has a number of "edge case" situations throughout the province that cause slowing down without a means to bypass them because there is no " ... tap wand..." message.
 

TrevP

TOO Founder / Technoking
Founder
Joined
Oct 20, 2015
Messages
7,962
Location
Toronto
Country
Country
Tesla Owner
Model 3
Evidently Tesla needs to train the AI with some more data sets to weed that stuff out. I'm sure it will happen in due time
 
  • Agree
Reactions: FRC

tencate

Top-Contributor 👍🏻
TOO Supporting Member
Joined
Jan 10, 2018
Messages
734
Location
Los Alamos, NM
Country
Country
Tesla Owner
Model 3
This was on an Interstate where FSD has been brilliant to date. Love the auto lane changes too, those were almost flawless the entire trip. This was the first phantom brake "edge case" I've had in a long time. How often does the moon rise in the direction you happen to be travelling? :)

Now it's a different story when I got to single lane US Highways, there's the constant addition of passing lanes up hills in Utah, Idaho and Oregon and then the passing lane goes away... the car doesn't handle that situation very gracefully, trying to figure out what lane to be in a LOT. But this discussion probably belongs in a different thread so I'll just leave it with the moon edge case. Wonder if driving into a sunrise or sunset would cause something similar, the car thinking it was encountering a red light? Hmmm.