# Disengagement Bug Reports



## Deadbattery (May 8, 2017)

I watched much of the autonomous driving day presentations. One of the things that Elon and Andrew Karpathy talked about at some length was how they use the fleet to help gain training data for their autonomous AI training. They spoke specifically that they focus on disengagements as a signal that the system is not working well... ok so I go home and my kid tells me that every time I come home, the Wifi quits ( it shouldn't but was able to confirm that this is in fact happening) I looked at the google wifi data and it turns out we are uploading data at volume (700 mb in the last week). 

So I was thinking it would be fun (and conceivably useful) to add metadata to disengagements by submitting bug reports right after a disengagement. the data will be timestamped and location matched to the disengagement. ... ok so Not sure If my comments will make driving a Tesla any better but I feel sure that whoever is forced to listen to them will enjoy themselves.

The Tesla voice AI is .... not good. It shuts off mid-sentence, it misses words. Also, after disengaging is not the best time to try to not crash the car, start the voice recorder and be able to voice cogently anything useful. 

Maybe some of you are better at this than I.


----------



## Bokonon (Apr 13, 2017)

Deadbattery said:


> ok so I go home and my kid tells me that every time I come home, the Wifi quits ( it shouldn't but was able to confirm that this is in fact happening) I looked at the google wifi data and it turns out we are uploading data at volume (700 mb in the last week).


Funny you should mention this, just before reading your post I had opened my Google WiFi and noticed that my car uploaded 137 MB of data immediately upon connecting to my router this evening.  Usually it only uploads something like 10-20 MB over the course of an hour after connecting. I only drove 25 miles today, steered out of autopilot twice (once by accident), and manually disengaged fewer than 10 times via the lever, all of which is typical for me. But part of me wonders whether Tesla is going to become more aggressive about collecting disengagement clips the closer we get to FSD "feature complete" status...

Anyway, I like your idea of annotating disengagements. If the disengagement data uploaded to the mothership included any bug reports logged within (say) a minute of a disengagement event, it could add a new dimension of (human!) input to the Autopilot training data. Imagine aggregating all of the telemetry and video for disengagements that include the words "pothole" -- BAM, there's a good starting point for teaching the car how to identify potholes. (I guess you could also help teach it to avoid them by incorporating data from the chassis sensors and narrowing the dataset to the cases where the driver successfully avoided the hit. )


----------



## Deadbattery (May 8, 2017)

Bokonon said:


> Funny you should mention this, just before reading your post I had opened my Google WiFi and noticed that my car uploaded 137 MB of data immediately upon connecting to my router this evening.  Usually it only uploads something like 10-20 MB over the course of an hour after connecting. I only drove 25 miles today, steered out of autopilot twice (once by accident), and manually disengaged fewer than 10 times via the lever, all of which is typical for me. But part of me wonders whether Tesla is going to become more aggressive about collecting disengagement clips the closer we get to FSD "feature complete" status...
> 
> Anyway, I like your idea of annotating disengagements. If the disengagement data uploaded to the mothership included any bug reports logged within (say) a minute of a disengagement event, it could add a new dimension of (human!) input to the Autopilot training data. Imagine aggregating all of the telemetry and video for disengagements that include the words "pothole" -- BAM, there's a good starting point for teaching the car how to identify potholes. (I guess you could also help teach it to avoid them by incorporating data from the chassis sensors and narrowing the dataset to the cases where the driver successfully avoided the hit. )


Exactly! 
As you note many disengagements are not related to autopilot annotating them as by choice or whatever relative to disengagements because of lane confusion or trying to kill me by driving 70 into an off-ramp a foot away from jersey barriers ( I wasn't scared at all).

I could come up with a couple of easy ones that might be both instructive to Tesla and easy for my small brain to remember so I am not so fumble tongued.

"Bug report, disengaged, inappropriate lane change"
"Bug report, disengaged, car swerved/broke/accelerated"


----------



## Frully (Aug 30, 2018)

My concern would be as 'data' they could aggregate bajillions of disengagements and corroborate likely relationships or commonalities...such as known bad locations or visual data uncertainties for example. Adding the voice annotations would be something they'd have to manually parse for the most part. Voice recognition whenever I file a bug report usually has a large handful of typos that wouldn't break a fairly standard spell-checker...but does completely change what I was trying to note. If it became a human operation it would be an enormous task.


----------



## MelindaV (Apr 2, 2016)

Deadbattery said:


> Exactly!
> As you note many disengagements are not related to autopilot annotating them as by choice or whatever relative to disengagements because of lane confusion or trying to kill me by driving 70 into an off-ramp a foot away from jersey barriers ( I wasn't scared at all).
> 
> I could come up with a couple of easy ones that might be both instructive to Tesla and easy for my small brain to remember so I am not so fumble tongued.
> ...


Most every one of my disengagements are from comfort level with the adjacent car/truck and how Close to the line they are, and avoiding known potholes. Following those, mostly unintentional disengagements with me thinking the center of the lane is different than where AP thinks it is.


----------



## Deadbattery (May 8, 2017)

MelindaV said:


> Most every one of my disengagements are from comfort level with the adjacent car/truck and how Close to the line they are, and avoiding known potholes. Following those, mostly unintentional disengagements with me thinking the center of the lane is different than where AP thinks it is.


Yes, the comfort level when next to an 80,000 lb truck is ... interesting. My wife and I made our first extended trip (350 mile rt) since automatic lane changes became a thing and must say it is transformational. It helped that we were in Vermont and NH so were not traffic challenged on this trip.

We had one place where we were heading into cones in a construction zone and another where we were going 70 as it pulled into the exit lane a foot off the guardrail. SLOW Your roll Tesla! My bug report for that one will make someone laugh. "Bug report, 70 onto an off ramp? are you guys kidding me?!"


----------

