File checksum integrity verifier12/28/2023 ![]() Various file copy utilities can perform additional cross-checking of integrity on copies - or you can perform it manually. If you're concerned, I would suggest skipping MD5 and use sha256 or 512. Typically the filesystem is responsible for (and capable of) maintaining integrity at the file level. (But back in the day, Windows XP didn't always format disks as NTFS unless you told it to.) NTFS is pretty good and has been pretty good for a pretty long time. That said, what filesystem is the data stored on and what file systems has it been stored on in the past? If FAT(12/16/32) was involved. It cost something like $20K to have the drives shipped out and the data recovered. I have personally witnessed a 2 drive failure in a RAID 5. Here's a thread from r/sysadmin discussing the issue from 2012.Įven RAID6 is being questioned with large drives. I'm not opposed to a paid software for my integrity checking but I'd like to keep my enclosure for the time being.Įvery major enterprise storage vendor is clear: the bit error rates on large drives (>~1TB) is not safe enough for single parity. Lastly I need to make sure any new files once archived stay corruption free. I'm unsure if this is possible as I don't have original "perfect" data to check against anymore. This way I can get new copies of the files. I know I have old files that are corrupt so I need some way to check and identify them. From another users post I'm thinking something along DrivePool and SnapRAID is my best bet or checksum software. Being a windows user if I'm not mistaken puts ZFS and BTRFS out of reach and leaves MD5 or checksum as the option for me. No experience with Linux or Ubuntu at all. Spent all my life on Windows, 7 currently 10 eventually. Moving towards a solution I'm feeling a bit out of my league and don't know which direction to go. Now that I'm into the TB's of storage I have to be more aware. Never had a tool to mark them as bad and not write to them. I had never experienced silent corruption, only corruption from bad transfer, damage, etc, so I never checked the sectors on my drives before. I've found some of my archived movie, music and image files look good, but in the middle of a movie it'll freeze, audio glitches, or images have some bars going across them. My biggest mistake was on file integrity. For 3-2-1 all my archived files live on Tier 3, and a redundant copy is stored on another enclosure as well as offsite both updated regularly. I'm aware of the mistakes I made and want to prevent them in the future. Over the past year my world has expanded quite a bit. The tower is now my main source of data gathering and transferring. I moved all data from my individual enclosures over. Each tier is setup individually using RAID 5. I used to have a large collection of external drives (27) but recently made the move to a new Addonics Tower IX and (10) 8TB and (5) 10TB all HGST drives. But I'm hoping someone has a recommendation for my situation. Basically I searched DataHoarder for "integrity" and started reading. ![]() I've found a lot of info but still only partially understand it, I am googling quite a bit of the lingo on here. I'm looking for ways to mitigate silent data corruption in my set up. Longtime lurker of reddit, this subreddit is what actually made me join.
0 Comments
Leave a Reply.AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |