Weird Quirk - Getting miscompares on verify, on ONE ISO, on ONE file

[Number Cruncher]

I posted this over at SlySoft... I was also using ImgBurn and at this point I would be VERY interested to hear ANY ideas from ANYONE. This is the weirdest thing I have ever seen!

 

--------------------

 

I'm pretty sure at this point that this isn't really an AnyDVD problem, but it might be interesting as a case study (bear with me it's kind of long). I've been working on it for several days now. I'm just not sure where to report this, but I feel like I want to say something somewhere. This almost seems like a computer science problem.

 

Here's what happened: I picked up a $10 Blu-Ray - "Hills Have Eyes (1977)" on Amazon, and tried to back it up. (Which I now read is a crappy DVD upscale AND I've used several blanks but I am more interested in troubleshooting at this point... just wanted to get that out of the way...)

 

I used ImgBurn 2.5.6.0 and 2.5.7.0, AnyDVD 7.0.2.4 and 7.0.2.6, and blank Verbatim BD-25's (source is single layer).

 

When I first tried it, on verify, I got several miscompares. Most all were either one or two sectors with the same checksums.

 

First thought... bad blank. So, I tried another blank from a different spindle. Same result, some miscompares in the same sectors, some in different sectors.

 

Next... it could be a bad optical drive. Replaced the drive. It's a Sony BWU-100A IDE. (I have some spares.) SAME result. So, probably not the drive.

 

Miscompares can be caused by RAM or CPU stability problems. So, I went to check that next. I ran a Memtest86 for an hour and CPU Stability for an hour. NO errors! So, probably not that.

 

Next... I/O comms issue? I backed the disc up to a network attached storage drive (My Book World). I ALSO backed it up to a locally attached IDE hard disk. Still getting miscompares.

 

At this point I am noticing that the miscompares are always happening on the same 14 gig file - 00006.m2ts - seemingly throughout the whole file. Most, but not all, of the miscompare checksums look the same!

 

I next tried backing up, burning, and verifying a different title ("A Lonely Place to Die" Region A Blu-Ray). It worked fine, no miscompares!

 

Next, I backed up Hills Have Eyes again to the MyBook, and verified the resulting ISO to the ORIGINAL *WITH* AnyDVD running. Miscompare still there.

 

Still can't totally rule out bad error correction on a drive, or I/O Comm. So, this is what I did:

 

I made an AnyDVD decrypted ISO of Hills Have Eyes to the MyBook (with whatever miscompares are probably on it). Then, I CLOSED AnyDVD, CLOSED ImgBurn, then used Windows Explorer to COPY the ISO from the MyBook to a shared directory on the desktop IDE drive. THAT should be the same file! Then, using a laptop, I mounted the ISO on the MyBook using Virtual Clone Drive, started ImgBurn, and did a VERIFY to the ISO on the shared directory on the IDE drive on the desktop. I got miscompares! And again, just in the one 14 gig 00006.m2ts file! This is straight Windows copy!!

 

OK, so I tried this: I attached a SECOND MyBook World network attached storage drive to my LAN, then COPIED the ISO from the first MyBook drive to the second using the LAPTOP. Did the mount/verify as above, and got NO miscompares!

 

Next, I tried doing the SAME thing... COPYING the ISO from the first MyBook drive to the second MyBook drive using the DESKTOP (where the problem appears to be). I got two miscompares! AGAIN, **ONLY** in 00006.m2ts. I tried it again, and this time got one miscompare in 00006.m2ts, but in a different sector. Finally, I tried it a third time, and I got no miscompares.

 

SO... it seems to ONLY happen on the DESKTOP. And it happens when being completely off the local buses, but apparently, not as often when not.. but it STILL happens. But ONLY on the desktop, NOT on the laptop.

 

So, what could it be? Drive, I/O comm, RAM, CPU, seem to be ruled out. So does AnyDVD or any software other than windows. Bad drive filter maybe? I checked the registry.. ALL standard filters for optical AND hard drives, nothing weird. But so far, this seems to affect ONLY the AnyDVD decrypted ISO of the Hills Have Eyes Blu-Ray, and ONLY in the 00006.m2ts file!

 

I then decrypted to ISO and backed up another Blu-Ray ("A Serbian Film" Region A Blu-Ray)... and it was fine! I verified the copied disc to the ISO a second time... no miscompares. I put the other disc I successfully backed up earlier in and verified THAT a second time... no miscompares. Finally, I made an ISO backup of Hills Have Eyes, but DIDN'T use AnyDVD... I left it encrypted. THAT gives no miscompares!

 

The ONLY hypothesis I have right now... the desktop has ECC RAM, the laptop doesn't. If bad error correction can cause this, and it's not the drive, then it very well could be the ECC RAM. But... this ECC RAM passes Memtest and the system seems to back up other Blu-Rays fine. I've backed up my entire collection of over 1,000 on this system, and since getting the miscompare, it seems it's ALWAYS the same file, ONLY on the decrypted ISO of Hills Have Eyes, and not on other discs (which I admit I've only tried three so far; the two I mentioned plus Hills Have Eyes encrypted).

 

SO... could this actually be a fluke?? Could it possibly be, that JUST the right sequence of bytes in a file could trigger ECC RAM to correct a byte error that does not exist? Or could SOMETHING on Windows affect this?

 

As part of troubleshooting, I did remove all AntiVirus before all the testing above. (I only had WebRoot SecureAnywhere.) This is a Windows XP SP3 system with the latest updates applied. (Both Desktop + Laptop.)

 

Has ANYONE ever heard of ANYTHING like this happening? Am I the first to encounter this weirdness?

 

I doubt any developers would want to spend serious time on this, unless of course they found it interesting. I have no problems uploading the ISO to SlySoft if they want to take a look... or they could just buy it from Amazon. This is the USA Blu-Ray disc of Hills Have Eyes.

 

I've attached the following for anyone who might want to poke around (the logs are arranged in the sequence they were created):

 

\ImgBurn Logs

01_ImgB_LP2D_Success1.txt

This is a backup of "A Lonely Place to Die" Blu-Ray. It was made after several attempts to back up Hills Have Eyes and I got miscompares. It was successful.

 

02_ImgB_HHE_Burn_Fail1_Verify1.txt

This is the typical result I get when trying to decrypt to ISO and back up Hills Have Eyes on the desktop using AnyDVD + ImgBurn. Four miscompares then I aborted.

 

03_ImbB_HHE_Any_Fail.txt

This happened when making an ISO of Hills Have Eyes and verifying it to the original with AnyDVD running. I aborted after a miscompare.

 

04_ImgB_ASF_Burn_Success1.txt

05_ImgB_ASF_Burn_Success1.txt

Two successful burns and verifies of "A Serbian Film" Blu-Ray using AnyDVD and ImgBurn.

 

06_ImgB_HHE_Burn_Fail1_Verify2.txt

Another verify of Hills Have Eyes. Miscompares in some of the same sectors, plus some other ones as well.

 

07_ImgB_HHE_Burn_Fail1_Verify3_CDIO.txt

Tried using Elby CDIO in ImgBurn for another Hills Have Eyes verify. Made no difference.

 

08_ImgB_HHE_Burn_Fail1_Verify4.txt

Another miscompare verify of Hills Have Eyes.

 

09_ImgB_LP2D_Success2.txt

A second verify of the ISO of "A Lonely Place to Die" Blu-Ray. Success, as before.

 

10_ImgB_HHE_Burn_Fail2_1xClean.txt

Just to try it, I cleaned the Hills Have Eyes disc, and read, burned, and verified at 1x speed. This actually seems to have caused MORE miscompares. But, again, ONLY in 00006.m2ts.

 

11_ImgB_HHE_Copy_Fail2_Laptop_Verify5.txt

This was where I copied the Hills Have Eyes ISO from the MyBook to the Desktop IDE via Windows, then mounted the MyBook ISO and verified to the Desktop IDE via a Laptop. Miscompares in 00006.m2ts.

 

12_ImgB_HHE_Copy_NAS2NAS_Fail1.txt

13_ImgB_HHE_Copy_NAS2NAS_Fail2.txt

Here I copied the Hills Have Eyes ISO from the MyBook to a SECOND MyBook using the DESKTOP. First log has two miscompares, second has one miscompare. All in 00006.m2ts. Another attempt of this was successful (so far the only one from the desktop, but it's on an ISO **created** on the desktop, so it probably had the miscompare thing happening when it was created); log from this isn't attached. Also not attached is a successful verify of a NAS to NAS copy using the LAPTOP to do the copying.

 

14_ImgB_HHE_EncryptedBurn_Success.txt

Here is a SUCCESSFUL copy, burn, and verify of Hills Have Eyes when AnyDVD is NOT used and the copy is ENCRYPTED. No miscompares. It seems some sequence of data in the decrypted 00006.m2ts is triggering something, perhaps ECC RAM???

 

 

\AnyDVD Logs

AnyDVD_7.0.2.6_Info.zip

AnyDVD_7.0.2.6_Info_Z_THE_HILLS_HAVE_EYES.zip

Miscompare_Logs.zip

Edited April 10, 2012 by Number Cruncher

[Number Cruncher]

Another thing I should mention here... this was long and complicated enough already... one thing that occurred to me was that this COULD be a bug in ImgBurn. In other words... did the CRC's of the ISO files match, but ImgBurn still report miscompares? I did use Express Checksum Calculator throughout. (And it was a PAIN IN THE *SS). The checksums of the files didn't match when there were miscompares. The files really were different... it's not an ImgBurn bug. The bytes in the ISO file actually do change when copied via Windows on that desktop. (I even tried TeraCopy, that had no effect.) IN FACT... when Express Checksum Calculator is used on the Desktop on the decrypted Hills Have Eyes ISO, it can produce a DIFFERENT checksum each time because of this whole miscompare quirkiness!!

Edited April 9, 2012 by Number Cruncher

[Number Cruncher]

I've got it down to 120 bytes. The attached binary file contains:

 

00000000h: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
00000010h: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
00000020h: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
00000030h: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
00000040h: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
00000050h: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
00000060h: FF FF FF FF FF FF FF FF FF FF 00 00 28 00 00 00
00000070h: 00 79 DF FF FF 3E 7F FF

 

If I copy and paste that file say 100 times on any SCSI, IDE, or USB drive from that desktop computer, about 5 - 15% of the time (not sure what the depending factor is), byte offset 114 (the DF in "00 79 DF ...") changes to FF. Always that offset, always that byte. No error or any notification from the system at all.

 

All the FF's I put in. The only string that came from that Hills Have Eyes Blu-ray is FF 00 00 28 00 00 00 00 79 DF FF FF 3E 7F FF. (I don't even remember at this point which LBA it came from now.) If I insert that string into any binary file at offset 105 to 119 (just as it is above) those files will also change from 00 79 DF to 00 79 FF about 5 - 15% of the time. If I put that string in different offset positions, it doesn't seem to happen.

 

It also doesn't seem to happen if the copying and pasting is done on a mapped network drive. Based on my earlier testing, it does look like there's a chance it can still happen, but it is a lot lot less likely.

 

I booted the system with ERD Commander and tried the copying and pasting there. That let me do it outside of the machine's Windows install. The DF -> FF still happens, so it HAS to be hardware.

 

I'm just not sure if this is FAULTING hardware, or BUGGED hardware (meaning, even if I go to the trouble to replace it, this anomaly will still happen). If it's faulted hardware, I would think surely either RAM, CPU or Motherboard.

 

I did try going into CMOS and turning off the CPU's L2 cache. That made no difference.

 

This is the weirdest thing I've ever seen. Has ANYONE ever heard of something like this?!?!?

ctest33.zip

Edited April 10, 2012 by Number Cruncher

[LIGHTNING UK!]

How many passes did you actually get through in Memtest86+ when you did that?

 

It sure does just look like a bit is getting flipped somewhere.

[Number Cruncher]

How many passes did you actually get through in Memtest86+ when you did that?

 

Several. All passed. I don't think RAM at this point since it's ECC. You're right, DF -> FF is one bit (11011111 -> 11111111).

 

At this point I'm leaning towards either faulty or buggy motherboard (perhaps the chipset). If not that, maybe the power supply... but I wouldn't expect that to produce behavior that only acts up with specific bits at specific positions and produce exact consistant results.

 

So far tonight I ran Windows Update and backed up some more Blu-Ray's. No problems.

Edited April 11, 2012 by Number Cruncher

[Number Cruncher]

On the off chance anyone ever reads this thread in the future (I realize it's more of a personal journal ), I wanted to update and report that I solved the problem.

 

It dawned on me that one thing that can cause weird system quirkiness is a weak CMOS battery. I replaced it, then cleared the CMOS using the motherboard's onboard CCMOS jumper, then went into CMOS on starting back up, loaded setup defaults, saved and rebooted, then put back in my normal settings, saved and rebooted. Problem is now gone!

 

I don't think my CMOS battery (a regular 2032) had ever been replaced. It tested dead on a battery tester, but the system was still retaining CMOS settings when powering down completely. It's a wonder it wasn't way more unstable.

[LIGHTNING UK!]

Thanks for the update and I'm glad you appear to have sorted it.

 

The problem/fix seems a little weird though. I never thought a cmos battery did anything more useful than allowing the machine to remember a few things when it's powered down.

[Number Cruncher]

The battery provides power to the CMOS chip which stores the nonvolatile BIOS memory, which holds all settings and config data. The BIOS is like an .exe, the CMOS is like the files, folders, and registry keys that hold all the config data which can change. Anytime the computer's main power supply is shut off, it is the battery that keeps enough power to the CMOS chip to keep the config data intact. (A CMOS chip is used because of its extremely low power requirements.)

 

IF the battery gets REALLY REALLY weak but is still putting out a TINY charge, it can glitch the data stored on the CMOS chip (flip bits and such). If that data is glitched, it can cause weird behavior. If for example you mess with a program's config files directly and now the program behaves strangely, if you delete all the config files, restart the program, and then reset all the settings within the program, now it's normal again. (Or if you reinstall Windows or reformat.)

 

Fun, right?

[LIGHTNING UK!]

If the power from the cell battery was low enough to somehow cause bits to flip in CMOS data, a checksum failure would occur - and of course that would mean the defaults get loaded and you get prompted at bootup to deal with the issue.

 

So I still can't really see it causing a problem.

 

But anyway, if you say the problem is now fixed then it's all good.