Tesla Owners Online Forum banner

Sentry Mode: Empty Files, Corrupt Video

24K views 97 replies 39 participants last post by  GDN 
#1 ·
Roughly 1/6 of my Sentry Mode recordings are blank (0-byte) files. Each video stream is only 4 Mbps (total 1.2 MB/s) so it's certainly not a speed issue (any flash drive on the market will easily handle many times that). Plus, the recent clips (from the past hour) are all okay.

Many of the non-empty files have corrupt video (bottom half is a giant green blob and the rest is extremely pixelated).

The drives were freshly formatted (GPT partition) and a new TeslaCam folder was created before being re-inserted. These problems occur on multiple different flash drives.

Is anyone else experiencing this? I'm on 2019.8.2 but I saw this on 2019.5.15, too.
 
#3 ·
Roughly 1/6 of my Sentry Mode recordings are blank (0-byte) files. Each video stream is only 4 Mbps (total 1.2 MB/s) so it's certainly not a speed issue (any flash drive on the market will easily handle many times that). Plus, the recent clips (from the past hour) are all okay.
I'm seeing something similar. After the first few minutes of Sentry recording, about 1/6 - 1/8 of the left-repeater camera files are zero bytes. And I've seen video clips that are missing keyframes so they end up discolored or pixelated.

I suspect sustained (rather than burst) write-speed is the issue here... after a while, the flash drive gets too hot to handle the I/O. I'm considering either a UHS SD card or some kind of SSD setup.
 
#5 ·
yesterday about half of my passenger side repeater camera videos had similar distortion as this one (on 2019.8.2). I've emailed it to Tesla to ask if this is a MCU processing issue, camera hardware, USB recording, etc issue. Videos start out mostly ok, then get a little wavy, then totally garbled before returning to mostly ok.
 
#7 ·
It's just bad/missing blocks in the file saved to the stick. Nothing to worry over with the car.
I bet eventually Tesla will combine these in the computer and write one clean video that will work better with most usb sticks. Question is if HW3 will be required to do so.
 
#8 ·
Roughly 1/6 of my Sentry Mode recordings are blank (0-byte) files. Each video stream is only 4 Mbps (total 1.2 MB/s) so it's certainly not a speed issue (any flash drive on the market will easily handle many times that).
I could easily imagine that Tesla's early coding attempts try to create a new file, and "time out" too quickly, then give up on writing to that file. In the meantime, the file creation actually finishes shortly after the code gives up waiting on it, leaving you with a nice, fresh 0-length file.

As you say, almost any USB drive nowadays can handle the requested throughput. But 0-length files probably aren't a throughput issue. But it could still be speed-related if the USB drive isn't very quick at "seeking" for some reason.

The "missing file blocks" that @MelindaV and @SoFlaModel3 are seeing sound like a possible throughput issue.

In either case, the quick & easy thing to try is a different USB drive. So far I appear to be 2-for-3. The first one I tried actually appeared to fry from overheating and I had to pitch it. The other two actually seem to be holding up well, even though they're probably at least 5 years old.
 
#13 ·
I have some familiarity with digital video recording in Linux-based OS's. The video above looks like it's being recorded at a variable bit rate (VBR), but the final output bit rate is too inconsistent for clean playback. You can see that effect if you watch a badly edited compilation video on youtube - as the bit rate changes abruptly from one to another, you get that pixellation/color shift/pause. If I'm correct about it, that's often done on low-power CPU devices (like dashcams, ironically enough!) to keep the video from freezing when CPU power runs out.

The good news is that might be fixable in software by increasing caching so the bit rate doesn't have to change so abruptly when CPU usage goes up. Down side to that is if the MCU crashes, you can lose a few minutes of footage.
 
#15 ·
I love this sentry mode feature. I caught someone cleaning their car at work put a ding on my door. I know its a Nissan and have the person on video but haven't seen him since or the car. I guess he realized that I may be looking for him and started to park somewhere else. I wish the pillar cameras were also active as part of the sentry mode.
 
#16 ·
@Mehul,

Great! Have the vid? You can always cruise the lot and see if you recognize his car.

Ski
 
#19 ·
Another data point that this may actually be a firmware bug...

Last night I was reviewing some fun dashcam footage (which I will share later :) ), and noticed that the corruption in my repeater cameras follows a very simple and consistent pattern. When the car is in motion, there are no issues with the side cam videos, but as soon as the car stops, BAM, corruption starts to seep in. When the car starts moving again, the corruption immediately vanishes. This happens sometimes with the front camera video as well, but I never saw this happen when only the front camera was recording in 50.6 and prior.

It's as though there's a firmware (over-)optimization in 2019.5 and later to minimize dashcam I/O by writing very few (if any) keyframes to the video files while the car is at rest. Or at the very least there seems to be some significant change/reduction in I/O activity that only happens while the car is at rest, and it can produce some buggy side effects.

Makes me wonder whether some of the zero-byte files have a similar cause, rather than being an issue with drive write speed that the firmware doesn't handle gracefully.
 
  • Like
Reactions: JWardell
#21 ·
I also have this issue of empty files from Sentry Mode recordings. Not sure why many are suggesting the speed of the drive would be the issue. The same drive records from all the same cameras for the regular dashcam feature right?
 
#22 · (Edited)
Another data point that this may actually be a firmware bug...

Last night I was reviewing some fun dashcam footage (which I will share later :) ), and noticed that the corruption in my repeater cameras follows a very simple and consistent pattern. When the car is in motion, there are no issues with the side cam videos, but as soon as the car stops, BAM, corruption starts to seep in. When the car starts moving again, the corruption immediately vanishes. This happens sometimes with the front camera video as well, but I never saw this happen when only the front camera was recording in 50.6 and prior.

It's as though there's a firmware (over-)optimization in 2019.5 and later to minimize dashcam I/O by writing very few (if any) keyframes to the video files while the car is at rest. Or at the very least there seems to be some significant change/reduction in I/O activity that only happens while the car is at rest, and it can produce some buggy side effects.

Makes me wonder whether some of the zero-byte files have a similar cause, rather than being an issue with drive write speed that the firmware doesn't handle gracefully.
I have been obsessively reviewing Sentry Mode and Dashcam videos for some time. I am seeing the same kinds of distortion that you all are describing, especially in Sentry mode.

I have a theory about this: In Sentry Mode the car is constantly writing the three cameras to a limited memory buffer, which is written out to our USB drives when Sentry Mode is triggered. It seems to tend to retain only video that has motion, although it seems to almost always retain the front view. Hence the side cameras frequently are zero. Sometimes the side cams only show a few seconds. This motion selectivity is not consistent--the right camera seems to be zero much more often than the left, but when there is a lot of large scale motion they both seem more likely to have good images.

I think this limitation is designed to use limited buffer memory efficiently. I am guessing that they do not want to allocate much memory so it can be used for more important functions--looking forward to Hardware 3.

The side cams are highly motion compressed with an apparent variable bit rate and seem to produce the best images when there is a lot of large scale motion in Dashcam mode as well as Sentry. When driving the side cams have good video quality, but if I come to a stop light and there is not much large scale motion, the image shows the same kind of deterioration as in the video above. I think the compression bit rate on the side cams is biased toward motion and does not deal well with static images.

In addition, since the 3 camera upgrade I have found that my USB drives, even the fastest, tend to need repair when I scan them on Mac Disk Utility. They don't stop working, and the video plays fine. I wouldn't notice it except I have been in the habit of verifying every time I look at the contents of a drive ever since the early Dashcam days when we got a lot of unreadable REC files. Later they became readable and then stopped occurring in 50.6.

It may be that we will have to go to even faster SSD drives, but at the moment I think most of the problems have to do with how they have written the software, and I expect it to get better like everything else--especially with Hardware 3.

Most of this is speculation. I am very interested in what the rest of you are able to figure out.
 
#23 ·
I tried an SSD to prove that it's not a bandwidth issue per my original post; sure enough, it fared no better than a regular, modern thumb drive.

17.45% of the files in my SavedClips folder had zero-byte files. This is a brand-new 128GB Kingston A400 SSD in a Sabrent USB enclosure. Interior temperature ranged from 54.2 to 77.4 degrees F (digital thermometer placed next to it).

This was within a few percent of the Sandisk Cruzer Fits and Silicon Power flash drives I used before.

I also tried GPT and MBR partitions but they had about the same failure rate.

I noticed that recordings from the font camera are always okay—no 0-byte files there. My right camera consistently has nearly twice as many empty files as the left camera.

At no time did I get a grey dot on the dashcam icon; it always reported proper operation with a red dot.

Remember that the first few firmware versions with dashcam had even worse corruption problems (dashcam would stop working entirely). They eventually fixed it and so I'm hoping that they will soon fix it via software update this time around, too.
 
#24 ·
I tried an SSD to prove that it's not a bandwidth issue per my original post; sure enough, it fared no better than a regular, modern thumb drive.

17.45% of the files in my SavedClips folder had zero-byte files. This is a brand-new 128GB Kingston A400 SSD in a Sabrent USB enclosure. Interior temperature ranged from 54.2 to 77.4 degrees F (digital thermometer placed next to it).

This was within a few percent of the Sandisk Cruzer Fits and Silicon Power flash drives I used before.

I also tried GPT and MBR partitions but they had about the same failure rate.

I noticed that recordings from the font camera are always okay-no 0-byte files there. My right camera consistently has nearly twice as many empty files as the left camera.

At no time did I get a grey dot on the dashcam icon; it always reported proper operation with a red dot.

Remember that the first few firmware versions with dashcam had even worse corruption problems (dashcam would stop working entirely). They eventually fixed it and so I'm hoping that they will soon fix it via software update this time around, too.
As I reported, earlier, my findings are consistent with yours.

I tested with 64 GB iDiskk USB drives, an SD card designed for video cameras which tested with a 50% faster write speed than the iDiskk, and a SanDisk Extreme GO 3.1 USB drive which has one of the highest speed ratings for USB drives--my tests showed it twice as fast as the iDiskk.

In all cases, I got frequent zero byte files on the side cams. What I am not sure about is whether this is by design--there is no motion in those cams for that minute--or is it due to write errors to the disk. I think we have established that it is not related to disk speed.

Sometimes one or more side cam views are just not present, which may be intentional. When there are zero byte files, I always find that Mac Disk Utility indicates that the disk needs repair--there are errors in the directory map--even though the disk is still working with a red dot and is readable. This suggests that the zero byte files may be due to write errors.

I would be interested to know if your SSD drive shows errors needing repair even though you are getting a red dot.

One way or the other I am sure we will see this all get better, as it has in the past.
 
  • Like
Reactions: simpsonhomer
#25 ·
I continue to get good video from all three cameras, so this is somehow a bug that fixed itself. I would suggest rebooting your car.

Also, I can't remember if it coincided, but with my wireless charger plugged into the other port, it was always resetting once or twice a minute. Maybe the whole hub was resetting and killing the files being written as well. I can't remember the timing, but I moved that charger to a pass-through battery and everything has been working great since.
 
#26 ·
To be clear, we are talking about video from Sentry Mode. I am getting all three cameras fine in Dashcam mode, with the caveat that the image in the side cameras tends to degrade if the car is stopped for very long--apparently due to the variable bit rate on the MPEG compression being biased toward motion.

What are you seeing when you look at footage saved from Sentry Mode? Do you always get all three cameras with no zero byte files? Have you run repeated disk checks on the drive with no repair suggested? What version of firmware do you have? What I am describing has been with 5.4 and 5.15.

I have the drive plugged into a hub connected to the right port with nothing else plugged into the hub or the other port when in Sentry Mode. I haven't tried plugging the drive directly into the USB port. Is that what you are suggesting?

I always reboot after an update and have rebooted a few times since the last update for other reasons.
 
  • Like
Reactions: simpsonhomer
#27 ·
Yes...while I had several Zero K and corrupted repeater cam files when I first tried Sentry, since that day, I have had clean and complete files from all cameras.

I have the Sandisk Connect 32GB Wireless drive along with a few other things plugged into an Anker USB3 hub plugged into the right port.

I am on a....later firmware that I can't mention.
 
#31 ·
I’ve had two sessions with Sentry Mode, one on 5.4 and the other on 5.15. When reviewing the files, I did not notice any 0KB files from the repeater cams.
 
  • Like
Reactions: simpsonhomer
#32 · (Edited)
Last night I was reviewing some fun dashcam footage (which I will share later :) ), and noticed that the corruption in my repeater cameras follows a very simple and consistent pattern. When the car is in motion, there are no issues with the side cam videos, but as soon as the car stops, BAM, corruption starts to seep in.
Yes, movement is okay, stationary is problematic. Interestingly, for dashcam clips that start stationary and have the annoying green blocking the bottom half, if there is movement within that minute clip, and if I slide the playback video back to the beginning, the green is gone. Still pixelated and poor quality, but better than the green band. Green remains no matter what for clips that are all stationary.

And I just had a sentry mode session this morning that most of my left repeater files are 0 kB. I'm on 2019.8.3.
 
  • Like
Reactions: simpsonhomer
#33 ·
Does anyone know how long you have to leave the flash drive in for it to put the file on it?


So far I'm testing with a brand new Kingston DataTraveler 100 G3 64GB flash drive, I put TeslaCam folder in the flash drive after formatting to Fat32 and, admittedly I've only tried it twice but so far do not see any files at all in the flash drive.

I see that Sentry is active, I wait a few minutes, turn car off, pull out flash drive, put it in computer, and no files :(
 
#34 ·
Does anyone know how long you have to leave the flash drive in for it to put the file on it?

So far I'm testing with a brand new Kingston DataTraveler 100 G3 64GB flash drive, I put TeslaCam folder in the flash drive after formatting to Fat32 and, admittedly I've only tried it twice but so far do not see any files at all in the flash drive.

I see that Sentry is active, I wait a few minutes, turn car off, pull out flash drive, put it in computer, and no files :(
When you plug in your drive, a camera icon will pop up in the top right corner of your screen. If you see the camera icon with a red dot then it is currently recording. If you see a camera icon with no red dot (just all white or gray I think), then you need to press it to initiate the recording, the icon will then show a red dot to show it is now recording. While it is recording, tap the icon to save the latest footage, the icon will change to a save icon and then green check to confirm the save was complete.

To stop recording, hold down on the camera icon and it should change from the red dot to white/gray. Then you can safely remove the drive.

If you see a camera with an x then that usually means the drive is full or there are issues.
 
#37 ·
Is anyone having issues where your USB doesn't even recognize the thumb drive? I've had two that work from time to time, but I feel like they only work if I (1) reformat them, (2) plug it into the right port, (3) let the car go into deep sleep, and (4) when I get back in it will recognize it. I'm not getting the red X or the grey dot, in the upper portion of the display... I litterally get nothing.
 
#41 ·
So quick question....haven’t had quite the opportunity to play with Sentry Mode yet. Leaving it at the airport for a few days (armed) so we’ll see what happens. Anticipating 1mph draw/drain while gone. My question is....if you get an ‘alert’ due to Sentry Activation or a so called “triggered state”......how LONG does the alarm/horn sound? A specified duration then it quits? One minute? Two Minutes? OR do I have to physically get the alert......then enter the app and turn off or disable Sentry to stop the lights/horn from blaring? So as to not drain the battery?
Again I’m going to be out of town and not sure how it all works, do we deactivate to stop the Horn/lights and then ‘reactivate’ Sentry? Or does it time out after a specified time duration? What is the Protocol?

Regards,

Skione65
 
#42 ·
So quick question....haven't had quite the opportunity to play with Sentry Mode yet. Leaving it at the airport for a few days (armed) so we'll see what happens. Anticipating 1mph draw/drain while gone. My question is....if you get an 'alert' due to Sentry Activation or a so called "triggered state"......how LONG does the alarm/horn sound? A specified duration then it quits? One minute? Two Minutes? OR do I have to physically get the alert......then enter the app and turn off or disable Sentry to stop the lights/horn from blaring? So as to not drain the battery?
Again I'm going to be out of town and not sure how it all works, do we deactivate to stop the Horn/lights and then 'reactivate' Sentry? Or does it time out after a specified time duration? What is the Protocol?

Regards,

Skione65
IIRC, someone posted in this or the other forum that the alarm sounds for 30 seconds and then stops and rearms itself.
 
#44 ·
I love this sentry mode feature. I caught someone cleaning their car at work put a ding on my door. I know its a Nissan and have the person on video but haven't seen him since or the car. I guess he realized that I may be looking for him and started to park somewhere else. I wish the pillar cameras were also active as part of the sentry mode.
So I found the guy. He denied it initially but after watching the video all he could say was "oh!" 😀. Estimate came out to be $262 which he ended up paying.

Love this feature. Wish it used the pillar cameras and the backup cameras as well. I have a 256 SSD so it should be good enough to record min 2 weeks.
 
#45 ·
Chalk me up to another one who's thumb drive bit the dust. Worked fine yesterday morning. I went to save a clip on the way home yesterday and noticed the gray X. Brought it inside and plugged it in, the computer said "corrupted" then asked if I wanted to fix, which I did. I was able to view the files from yesterday. A few blanks, but mostly there.

Plugged it in this morning and still the gray X.

Any good solution to this?
 
#46 ·
Chalk me up to another one who's thumb drive bit the dust. Worked fine yesterday morning. I went to save a clip on the way home yesterday and noticed the gray X. Brought it inside and plugged it in, the computer said "corrupted" then asked if I wanted to fix, which I did. I was able to view the files from yesterday. A few blanks, but mostly there.

Plugged it in this morning and still the gray X.

Any good solution to this?
Are you able to format or delete the files? My drive that went bad couldn't do either. Look into using SSD drives or dash friendly microsd cards with USB adapter (Samsung Pro Endurance) instead. What flash drive were you using?
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top