# Here’s a weird AP issue: Autopilot mistakes moon for a traffic light



## shareef777 (Mar 10, 2019)

So driving on AP and it kept thinking the moon was a yellow light 😂


----------



## garsh (Apr 4, 2016)

I also had an issue with the car seeing phantom traffic signals.



garsh said:


> I just went on a 400-mile drive with 2021.4.18.2. While I experienced several instances of unwanted, incorrect braking, I never experienced "phantom" braking. That is, I was always able to figure out the cause of the braking.
> 
> The most infuriating instance I found was when encountering these little orange diamond-shaped reflectors on signs:
> View attachment 38805
> ...


----------



## iChris93 (Feb 3, 2017)

2 weeks away from FSD though.


----------



## garsh (Apr 4, 2016)

Someone else just tweeted about this same issue.


__ https://twitter.com/i/web/status/1418413307862585344


----------



## Long Ranger (Jun 1, 2018)

shareef777 said:


> So driving on AP and it kept thinking the moon was a yellow light 😂


How's the air quality there? I saw the same thing last summer when we had bad smoke from forest fires. In my case it was the sun instead of the moon.


----------



## shareef777 (Mar 10, 2019)

Long Ranger said:


> How's the air quality there? I saw the same thing last summer when we had bad smoke from forest fires. In my case it was the sun instead of the moon.


I'm in the Chicagland area, so didn't think it'd be from the west coast fires, but who knows 🤷‍♂️


----------



## bsunny (Oct 8, 2018)

Long Ranger said:


> How's the air quality there? I saw the same thing last summer when we had bad smoke from forest fires. In my case it was the sun instead of the moon.


Orange moon across U.S. from fires in the west. We saw it here in New England too. 😫


----------



## Nancat999 (Jun 2, 2021)

shareef777 said:


> So driving on AP and it kept thinking the moon was a yellow light 😂
> 
> View attachment 39309


Better check the moon cycle on the weather app, so you know when the full moon is out. Unless of course you unfortunately work in customer service, all the crazies seems to call in. Joking aside, I can see by the photo why the car software was confused. I would think since meteorologist know when that will occur for the 10 yrs, maybe somehow they can get that calculated into their algorithms.


----------



## JasonF (Oct 26, 2018)

All they need to do is have FSD grab two consecutive video frames and compare the relative speed of the car vs the “traffic light”. If the distance isn’t shrinking comparatively to the speed of the car, ignore it.


----------



## garsh (Apr 4, 2016)

JasonF said:


> All they need to do is have FSD grab two consecutive video frames and compare the relative speed of the car vs the "traffic light". If the distance isn't shrinking comparatively to the speed of the car, ignore it.


IIRC, autopilot currently reads two frames. But with the car itself bouncing around a little bit, it could look like the moon is moving at times, which is probably why a traffic light keeps appearing and disappearing on the screen.

Hopefully the pure-vision re-write will use more frames to allow the car to make better judgements.


----------



## JasonF (Oct 26, 2018)

garsh said:


> IIRC, autopilot currently reads two frames. But with the car itself bouncing around a little bit, it could look like the moon is moving at times, which is probably why a traffic light keeps appearing and disappearing on the screen.


I noticed something in the video from a coder point of view - and it's that someone got lazy and made and assumption about traffic lights to hack them in quickly. It looks like it's designed to assume traffic lights are stationary and close by, and the hack is that they estimate where it is by how fast the car is moving. That, I believe, is why you see the moon traffic light appear and move toward, and then disappear overhead.

We know AP/FSD can judge closing distance to moving cars, they simply didn't use that mechanic for traffic lights. Maybe to save processing time?


----------



## tencate (Jan 11, 2018)

I recently took a long 3000 mile trip to Portland and back. At one point the car became very confused, hesitating and starting to stop every few seconds or so, and then it'd go away and then come back. Phantom braking? But why? After about 1/2 hour of this rather frustrating behavior (glad traffic was light!), I finally figured out why.

The car thought it was seeing a flashing yellow traffic light, on the interstate (where the speed limit was 80 mph)! I first saw a traffic light showing up on the display and then looked up to see what it THOUGHT it was seeing. There it was, the moon had just risen and was about the right height for the car to think it was a yellow traffic light. I snapped a quick photo with the phone which I've attached. Betcha this is an "edge" case no one thought about!


----------



## lance.bailey (Apr 1, 2019)

lots of issues like that. I bet you did not have a "slowing down tap wand ..." messages. BC has a number of "edge case" situations throughout the province that cause slowing down without a means to bypass them because there is no " ... tap wand..." message.


----------



## TrevP (Oct 20, 2015)

Evidently Tesla needs to train the AI with some more data sets to weed that stuff out. I'm sure it will happen in due time


----------



## tencate (Jan 11, 2018)

This was on an Interstate where FSD has been brilliant to date. Love the auto lane changes too, those were almost flawless the entire trip. This was the first phantom brake "edge case" I've had in a long time. How often does the moon rise in the direction you happen to be travelling?  

Now it's a different story when I got to single lane US Highways, there's the constant addition of passing lanes up hills in Utah, Idaho and Oregon and then the passing lane goes away... the car doesn't handle that situation very gracefully, trying to figure out what lane to be in a LOT. But this discussion probably belongs in a different thread so I'll just leave it with the moon edge case. Wonder if driving into a sunrise or sunset would cause something similar, the car thinking it was encountering a red light? Hmmm.


----------

