Wipe Out Existance of Deleted Files - Page 1
Forum section: electronics / general

author: bostonman
author_rank: Super Contributor
timestamp: December 14, 2020, 01:18:33 am
content: This is somewhat of a continuation of a question I asked some time ago. I have old hard drives with files that were deleted and used software (can't think of the name at the moment) to see whether certain ones could be resurrected.As I guessed and knew, I was able to retrieve deleted files. Since then I used the same software to wipe them out, however, they are still showing, but are corrupt.From what I learned, for some reason the TOC (?) for whatever reason still contains the names. I'd like to eliminate them from existence. I'm thinking of writing a loop program that would just dump a never ending text file to the hard drive until it's full; and then manually delete that file to make space. This way any existence would be overwritten with useless text code and hopefully eliminate the file names from showing when trying to recover files with programs.Can this be done with a Linux command?

author: retiredfeline
author_rank: Frequent Contributor
timestamp: December 14, 2020, 01:26:57 am
content: Here are 4 suggestions:https://linoxide.com/linux-command/commands-wipe-disk-linux/

author: rsjsouza
author_rank: Super Contributor
timestamp: December 14, 2020, 01:44:44 am
content: A bootable solution is DBAN. I have used it in the past, and it wipes using multiple passes to meet specific standards. Worth a try.Link with this and many other utilities:https://www.lifewire.com/free-data-destruction-software-programs-2626174

author: bostonman
author_rank: Super Contributor
timestamp: December 14, 2020, 01:47:54 am
content: Just to clarify, I want to keep existing files and only write to "empty" places on the drive so the old deleted stuff will hopefully not show in the TOC thus making it cease to exist from existence.

author: retiredfeline
author_rank: Frequent Contributor
timestamp: December 14, 2020, 01:58:43 am
content: That's a different kettle of fish. What you call TOC are directories in Linux parlance. I can think up of various untested solutions but the possibility of accident is too great. You'd be better off copying the stuff you want off the disk, formatting the disk, and copying stuff back. And no you can't erase the deleted directory entries by filling the disk with a file. The space for the directory entries have already been allocated and won't be affected by creating a file.

author: bostonman
author_rank: Super Contributor
timestamp: December 14, 2020, 02:00:29 am
content: This is also a Windows based drive. I figured Linux can do this easier and why I mentioned it.

author: golden_labels
author_rank: Super Contributor
timestamp: December 14, 2020, 04:17:50 am
content: Due to a misunderstanding the post was written with a wrong assumption about OP’s goals. The content is mostly irrelevant for this discussion.Depends on what you want, but may be technically impossible at software level.Even if it would be possible, it would require a specialized program, which has little practical use but relatively huge maintenance costs. That’s because handling any file system type concerning you is required, with any possible extensions and options. All the effort just to perform an operation useful only in rare circumstances. Not saying such software does not exist, but paint me surprised if it doesand really works. If it can’t handle file systems at low level, it will fail to remove data from partially used blocks, some metadata, areas used for journalising &c.However, the reason any of it will fail is because no program has full access to the storage medium. That problem is particularly visible in flash memory, but not exclusive to it. As a part of wear leveling and to offer remapping the device contains more blocks than what is advertised or addressable. Even if you perform a proof-of-capacity attack by filling available space with random data (unlike zeros or TRIM calls it forces actual writes), some areas will contain old data. It will not be accessible from the outside, but it is still there and anyone with the right equipment may read it. This is why normally ATA Secure Erase command is issued after filling the device with garbage: the only option to even hope those blocks are erased too. Some manufacturers have either failed to implement that properly or outright faked it, but that’s the best we have.The problem is it overwrites the whole medium and you want some data to be left untouched. That will not happen. What I would suggest in this case is backup files to another medium, properly erase the old one and restore the backup. Otherwise the best you can get is mounting the drive from another system, filling a file with/dev/urandom(cp /dev/urandom /mounted/drive/filling-file), unmounting it (do not skip that part!), mounting it again and removing the file. That will remove most of data from the addressable unused space, but will leave metainformation, possibly some fragments in partially filled blocks and data in non-addressable blocks.

author: Nominal Animal
author_rank: Super Contributor
timestamp: December 14, 2020, 04:34:10 am
content: If you are using an SSD, you can usefstrim. This tells the drive to discard all unused blocks.Debian provides at least two packages, zerofree (zerofreecommand) and secure-delete (sfillcommand) that are available on all Debian derivatives (Debian, Devuan, Ubuntu, Mint, etc.) from standard repositories, both writing zeroes to unused blocks.I recommend usingsudo sfill -llzmountpoint/.as it works on mounted filesystems just fine. It does not harm the current contents of the disk, just writes zeroes (once) over unused blocks. If you want a more secure wiping of the unused blocks, usesudo sfillmountpoint/.The same secure-delete package also providessrmwhich basically tries hard to wipe the file contents before actually deleting the file itself.

author: rsjsouza
author_rank: Super Contributor
timestamp: December 14, 2020, 09:11:12 am
content: Ah, for wiping directory entries and file contents in FAT or NTFS systems there are several utilities (one ishere), but I couldn't find one that does it from the command line.A couple of decades ago I used to do this using Norton Utilities... Good times.

author: ledtester
author_rank: Super Contributor
timestamp: December 14, 2020, 09:23:44 am
content: Search for a "secure delete" utility.This Linux version overwrites a file multiple times and then overwrites data in the inode:https://www.howtogeek.com/425232/how-to-securely-delete-files-on-linux/A Windows command-line version is mentioned here:https://www.techrepublic.com/article/how-to-securely-and-completely-delete-files-in-windows-10-without-third-party-software/

author: Microdoser
author_rank: Frequent Contributor
timestamp: December 14, 2020, 10:23:34 am
content: I read somewhere that even multiple overwrites were not 100% effective due to minute changes in the head position which could leave a tiny sliver of data at the edge of the track remaining and this could be 'read' by ultra precise measurement of the entire disk although this would be very very expensive, slow, and require military/lab level equipment.At the end of the day, the question to ask is: How much is this data worth to someone else?Then you just have to put enough security on it so it costs more than that to retrieve. For cheap data (like mine) that is just deleting the file header. More expensive data might warrant overwriting the file data, there is not much data in the world that is not made secure by multiple overwrites of the file.IIRC there are free and easy to use apps which could do what you need:https://www.groovypost.com/howto/7-free-ways-securely-delete-files-windows/

author: Haenk
author_rank: Super Contributor
timestamp: December 14, 2020, 01:25:33 pm
content: I use Hard Disk Sentinel (on Windows) to erase HDDs and SSDs (and also test them and read parameters).The "destructive" test writes random data to each block and reads it back, so 1) data is deleted and 2) the device is checked for bad blocks in one session.This method is also great when reusing oddly partitioned disks, like Mac formatted disks or RAID-disks - as the disk will be fully overwritten, all partitions will be erased, too.

author: Ian.M
author_rank: Super Contributor
timestamp: December 14, 2020, 03:04:01 pm
content: Even if the drive is totally wiped, there is still likely to be trace evidence of filenames and paths on any OS drive that was previously used to access the wiped drive. e.g. for Windows, seehttps://frsecure.com/blog/windows-forensics-execution/If you *need* total plausible deniability of the previous existence of a specific deleted file, the only certain solution is to securely wipe the whole drive, any backups of that drive, and any OS drives previously used to access them, even if the OS in question was only used to browse to the containing folder!

author: bostonman
author_rank: Super Contributor
timestamp: December 14, 2020, 03:15:38 pm
content: I didn't realize hard drives (SSDs) could be larger than advertised.This is a standard 3.5". This isn't necessarily about making sure every single bit of a potential file is erased, it's more about the TOC is showing these files once existed. To exaggerate a bit, say I HAD a file named 'My Checking Account Number.doc', although the file got wiped out (or assuming it did) by wiping out the unused space using software after it had been physically deleted, the file name still shows.This would let some hacker (assuming someone stole this drive and/or I didn't smash it with a hammer before tossing it) know at one point such a file existed and give reason to try finding bits and pieces of that file.If the file name ceased to exist in the TOC, then finding that file would be much harder because someone would have to sift through fragments of files and guess that X random numbers are X out of Y digits needed to steal my checking account numbers.

author: bw2341
author_rank: Regular Contributor
timestamp: December 14, 2020, 04:05:51 pm
content: https://www.eevblog.com/forum/general-computing/how-to-wipe-out-deleted-files/I found your original topic. In that thread, I suggested a way to generate a large number of files by typing a single-line command in Command Prompt. I guess you didn't try it? If you don't trust random commands from strangers on the Internet, you could do this manually in File Explorer.Make a new folder and a new text file in that folder. Press Select All on the ribbon menu. Press Copy on the ribbon menu. Press Paste on the ribbon menu. Windows will make a copy of the file with "- Copy" postpended to the filename.Now you just have to do this over and over again. If you are comfortable with keyboard shortcuts, you just hit Ctrl-A, Ctrl-C and Ctrl-V. After 10 times, you will have 1024 files. After 20 times, you will have 1048576 files.I think tens of thousands of files should be sufficient. The goal is to create enough files so that entries in the Master File Table (MFT) corresponding to previously deleted files will be overwritten. You only need to make enough files so that the total number of files surpasses the highest number of files that were previously on the disk.The MFT is for NTFS filesystems. If I remember correctly, this would not be effective on FAT, FAT32 or exFAT filesystems. With FAT, subdirectories are stored in directory table files among the data files. To erase old deleted filenames you would need to create new files in every subdirectory to overwrite the old names.

author: golden_labels
author_rank: Super Contributor
timestamp: December 14, 2020, 04:50:23 pm
content: bostonman:Initially I got a bit confused by your goals, driven away from the proper interpretation by the initial answers. However, now I am completely confused.What exactly do you want to achieve? What is your threat model? Unless you are charged with a bombing attempt and have a bunch of files named “Anachist cookbook.pdf” or “How to make bombs from common household items.txt” the file names do not important.(1)If you want to prevent discovery of what was stored there in the past — or rather to reduce that risk to negligible values — properly wiping out the whole storage medium is the only option. If some files are to be preserved, they must be restored from backups.Properly wiping means doingboththose steps:Filling the whole storage medium with pseudorandom garbage and syncing it.Executing ATA Secure EraseIf you are short on time, the second one may be skipped at the risk of some small amount of data remaining in unaddressable areas.____(1)They would be if the disk is used to store particular types of data, but then all of them would be — not only the files already deleted.

author: David Hess
author_rank: Super Contributor
timestamp: December 14, 2020, 09:02:23 pm
content: The sure way is to copy the files individually to another drive, and then overwrite the old drive sector by sector.

author: emece67
author_rank: Frequent Contributor
timestamp: December 14, 2020, 09:33:58 pm
content: .

author: golden_labels
author_rank: Super Contributor
timestamp: December 15, 2020, 12:58:22 am
content: Quote from: emece67 on December 14, 2020, 09:33:58 pmNo, it will not. For the reasons given in… uh,the post I have erased.That does not erase all the data on the medium. At best it fills a large portion of it (but not all), at worst it heats up air in you room. Software can’t directly access blocks/sectors on the device, because they are not expossed in the logical addressing. In the most optimistic scenario this command will still leave some blocks not overwritten. Even then those, which will be filled with pseudorandom data, may be different than those which contained the data to be erased. This is merely due to extra space available to deal with wear leveling and relocation. Then come features like HPA or caching. The method that gives a reasonable chance of beig successful at nuking data is giventwo posts above.Also: stop thinking ofddeach time word “disk” is mentioned. That tool has nothing to do with disks, has association with them merely for historical reasons and is considerably misunderstood. The “disk destroyer” backronym should be enough of a warning. Unless you wish to use very specific functions ofdd, there is really no reason to use it.

author: Jr460
author_rank: Regular Contributor
timestamp: December 15, 2020, 01:21:25 am
content: Quote from: golden_labels on December 15, 2020, 12:58:22 amI don't buy it. If /dev/hdX, or /dev/sdY don't allow you address all blocks on the device, then when you put a filesystem on it and mount it, the file system can not access those blocks either, and thus nothing is ever written to them.You could use a decent OS and file system to start with that has a data-erase on file delete option.Just read up on how it is done in DoD. Simple program, boot off a CD/DVD floppy whatever. It writes to all blocks at least 7 times. I think the first pass is all zero, then all ones, then random stuff, all zeros, all ones, random, and a final all zeros. I may have the order wrong, but you get the idea.And then in some more important cases you put the whole drive in degausser. Problem with that is it takes off the servo data also, so the drive will not work again.And then in high-end cases it is crushed, then mixed in some ratio with other scrap metal in a batch. Then that batch gets melted down and then sold as scrap.

author: bostonman
author_rank: Super Contributor
timestamp: December 15, 2020, 01:53:09 am
content: QuoteYou had me laughing. I can assure you that none of my old files revolve around bombs. Simply put, I found old deleted files that had full names of people. I'd like to delete any history of these files existing because they contain names.Whether or not the files can be retrieved is another story. The bigger question is why are these file names still showing after I used a disk destroyer to write 0s or 1s to the empty sections of the hard drive.To answer your question, we can ignore whether the original files can be retrieved and focus on getting rid of the old file names in the TOC or whatever it's called where they are stored.From my understanding, when a file is normally deleted, the file doesn't physically get deleted, but the pointer to that section. After, when writing new data to the drive, the TOC that once pointed to that section shows as empty and writes to that section of the hard drive; in reality it's writing over the data but the TOC allows it because the pointer shows that section as empty.Hope I answered your question.

author: golden_labels
author_rank: Super Contributor
timestamp: December 15, 2020, 02:30:01 am
content: Quote from: bostonman on December 15, 2020, 01:53:09 amFirst I will answer with the assumption of honesty and no grave mistakes from authors of the program used. If the utility only erases unused space by writing a lot of data through normal file system access methods, it must have skipped directory tables that are at least partially used and fragments of former directory tables that are left in clusters used by any new files. They can never be touched in that scenario, as that would damage existing data. This is my educated guess — but without manually investigating the low level structure of the file system it’s impossible to tell. The second answer is more cynical: everyone can write a program, but not everyone is willing to first understand the problem.Quote from: bostonman on December 15, 2020, 01:53:09 amIn the caase of FAT file systems, there are two separate structures. The eponymous FAT keeps track of which clusters are linked together to form file contents. There are also special clusters (stored in the data area, not in FAT) — directory tables — that describe directory contents. That includes file names, their attributes and the index of the first cluster of a file. That index in combination with FAT allows the system to determine the complete list of clusters tha form a file.Now, if you delete a file its entry in the corresponding directory table ismarkedas removed. And that’s it. The directory table itself remains intact, as it still describes other files. It just has some unused data in the middle. Even if the directory table itself is no longer needed, it still remains in place — it’s merely not referenced by anything. And then some other file may use the same cluster, but it may remain only partially overwritten.Quote from: Jr460 on December 15, 2020, 01:21:25 amThere is nothing to buy in here. Just learning the basics of operation of any of the devices involved.Quote from: Jr460 on December 15, 2020, 01:21:25 amThe file system has no access to them either. The device has.Quote from: Jr460 on December 15, 2020, 01:21:25 amThere is no such OS or file system. As explained above, software on the CPU side has no direct access to data on the storage medium. Not userspace programs, but any software.Quote from: Jr460 on December 15, 2020, 01:21:25 amYou are referencing the rainbow books method? You do realize it was designed in early 1990s and talks about ancient magnetic media? It was never suitable for modern HDDs or flash-based memories. Both because it doesn’t address current threats and because it suggests operations making no sense for those devices. It’s a piece of history. The current policies, like theStorage Device Sanitization Manual, no longer allowanymethod of overwriting: only physical destruction or using an approved degausser. Of course we do not need that level of security here: overwriting is enough for an average person. But proper overwriting.

author: rsjsouza
author_rank: Super Contributor
timestamp: December 15, 2020, 03:00:43 am
content: Quote from: golden_labels on December 15, 2020, 02:30:01 amThat was a very nice and entertaining reading. Thanks for sharing.

author: golden_labels
author_rank: Super Contributor
timestamp: December 15, 2020, 05:17:08 pm
content: To anyone reading that memo: there is no single, nation-wide policy. Neither the Rainbow Series nor this document are authoritative. While obviously different recommendations may be better or worse, we should not automatically assume that NSA or DoD are in some way privileged in being right. I’m not saying that authorship is not important, but how trustworthy the authors are should be judged just like any other thing. They are experts, but so are your local CERT teams and their word should be taken as seriously. Unless, of course, it contradicts basic knowledge, ignores known vulnerabilities &c.I have chosen that memo only because the Rainbow Series sanitization method (or some minimally mangled version of it) has been referenced by Jr460. Those books had no direct descendant, but this recommendation is probably the closest you can get: it shows how policies of the same agencies have changed over years with respect to the same topic.Those recommendations are also way beyond what most people need. So far the opinions I see suggest that wiping is enough. It just has to be done properly to minimize the risk of data being retained in blocks used for wear leveling or relocation, due to caching or in things like HPA.

author: David Hess
author_rank: Super Contributor
timestamp: December 16, 2020, 09:47:45 am
content: For most users wiping the drive sector by sector will be sufficient. The threat model which includes an adversary recovering data from sectors hidden by relocation is pretty extreme. If even law enforcement does this, I have not heard about it.But if your drive supports and properly executes secure erase which depends on encryption, then that is a sure way to render even hidden sectors undecipherable. On drives which do not support secure erase, whole disk encryption allows the same thing.

author: AntiProtonBoy
author_rank: Frequent Contributor
timestamp: December 17, 2020, 12:25:09 am
content: Quote from: David Hess on December 16, 2020, 09:47:45 amAs far as I understand, zero formatting should also overwrite those hidden sectors as well.

author: David Hess
author_rank: Super Contributor
timestamp: December 17, 2020, 01:17:21 am
content: Quote from: AntiProtonBoy on December 17, 2020, 12:25:09 amThe problem is that sector which were removed from use are no longer directly accessible, and likely contain old data.

author: Doctorandus_P
author_rank: Super Contributor
timestamp: March 03, 2021, 09:02:09 pm
content: A simple and secure way is:1. Copy all your valuable files to another disk.2. Verify the copy is correct.3. Reformat the HDD (This wipes the "TOC" (as you call it)).4. Copy big data files to it. such as HD movies, until your OS complains the disk is full.5. Copy some smaller files to it until the disk is full.6. Reformat the drive.7. Put your data back.8. Verify you correctly put the data back.This method can be done from any OS, does not need any special software and is reasonably secure even when done by people with limited computer knowledge. The most difficult part will be to reformat the drive.

author: Stray Electron
author_rank: Super Contributor
timestamp: March 04, 2021, 01:43:16 am
content: Quote from: rsjsouza on December 14, 2020, 09:11:12 amBack when Norton Utilities were actually useful!In the Pre-Windows days, I would write a short batch file that would take an existing file such as Command.com and copy it over and over and concatenate each copy into one giant file. The way that DOS (later versions) worked, any new file would written to never used areas of the disk and only after those were full it would starting writing to areas that had been used but marked as deleted. So eventually that would overwrite everything on the disk except areas that were marked as being in current use. Then I would delete the newly created giant file and I was done. It would be child's play to write a batch file that would do everything automatically and could repeat that as many times as desired. The beauty of using a batch file was that I could use it on any computer that had MS-DOS compatible OS and it didn't require any additional SW or even an External (to DOS) command.It's been YEARS but I think "label1:" "Copy Command.com+newfile newfile" "goto label1" was all that was needed in the batch file and I could use "Copy Con Loopy.bat" to create the batch file. Totally simple and unsophisticated but it would run until the drive was full and then stop. You could write a second batch file that would erase the "newfile" and then restart the Loopy.bat batch file.

author: rsjsouza
author_rank: Super Contributor
timestamp: March 04, 2021, 03:30:59 am
content: Interesting approach, and all the batch commands are still there in a typical windows install, even the "copy from console" (copy con), which I used in many recovery situations where edlin or edit (qbasic) were not available.Good reference that I have been using for years is:https://ss64.com/

author: bostonman
author_rank: Super Contributor
timestamp: March 05, 2021, 03:29:11 am
content: QuoteThis was my original idea on how to approach the problem.I'm uncertain I understand your commands though, they don't appear to be DOS format.

author: salbayeng
author_rank: Frequent Contributor
timestamp: March 05, 2021, 10:07:24 am
content: Sorry to be noticing the elephant in the room, but if you spend 2 hrs at $100/hr wiping the disk clean, it would be cheaper to spend $100 on a virgin disk, and toss the unwanted disk in an incinerator (domestic oven would do, the curie temperature is only around 300C) or a bucket with $10 of battery acid.

author: rsjsouza
author_rank: Super Contributor
timestamp: March 05, 2021, 10:14:31 am
content: Quote from: bostonman on March 05, 2021, 03:29:11 amOne of the limitations of this approach is the slack area (clusters allocated for active files but not fully used) are not overwritten, although for recently deleted files this would work well.Another detail is that, if the maximum file size is smaller than the free space, there'll still be areas left with the original data.Quote from: bostonman on March 05, 2021, 03:29:11 amWhat he describes is inside a batch file. From a command prompt, type the following:C:\>copy con loopy.batlabel1:Copy command.com+newfile newfilegoto label1^Z1 file(s) copiedC:\>The ^Z is the EOL character and can be typed using the F6 key or the keypress combination Ctrl + ZCheck the Copy command in the reference I sent before.

author: salbayeng
author_rank: Frequent Contributor
timestamp: March 05, 2021, 11:18:41 am
content: Re " Copy command.com+newfile newfile "is this the right way around? it would make a file that gets bigger like 1 2 4 8 16 32.... leaving < 49% of the HDD unwritten on the last write assuming DOS checks available space before writing.... hmm maybe it doesn't, DOS is pretty dumb.Maybe it should be" Copy newfile+command.com newfile " to append just one copy each time. But does the "+" cause the file to be loaded into memory first? or does it do an append on the fly.?You could also pipe it , this is a true append I think?"command.com >> newfile"Been 4 decades since I messed around with this stuff.Another thought, you could copy a handful of Jpgs there to get more entropy,then do a "copy *.jpg combo.jpg" in the loop

author: rdl
author_rank: Super Contributor
timestamp: March 05, 2021, 04:00:34 pm
content: When I want a file gone, not just deleted, I use Eraser. Been using it for years with very few problems. It may not cover every situation but for most people it's probably good enough. A few times I've tried, but I was never able to recover a file I erased.

author: bostonman
author_rank: Super Contributor
timestamp: August 19, 2021, 01:44:44 pm
content: Digging up an old thread.As some suggested, it made sense to buy new drives and combine the contents of the drives onto one large one.My new goal is buy a secondary new drive and make a mirror image should something happen to the first.Also, since I have these old drives, it makes sense to write the combined contents back to one or two as backups (also as someone suggested). Having said this, I've done a search to see which files are "recoverable" and found 1000s. While none of them contain any vital information, I'd like make the drive so that once I write back the contents, "recoverable" file names can't be located nor can the file itself.As an example, maybe I have a file named: my passwordsI'd like this to not be found because hypothetically it would give someone the desire (should someone actually steal my drive) to try retrieving that file. The file itself being somewhat present on the drive is secondary in my opinion because the average person who would steal a drive from my house or go through my trash should I toss these away without destroying them isn't going to do forensic analysis trying to piece together bits and pieces of a file to see what it contains.Would writing a text file on a continuous loop until the drive is full be the best way?

author: mariush
author_rank: Super Contributor
timestamp: August 19, 2021, 01:54:20 pm
content: Western Digital Data LifeGuard Diagnostics :https://support.wdc.com/downloads.aspx?p=3Use the ERASE > FULL and it will erase the whole disk, without possibility of recovery.Works with any drive, not just WD.

author: bostonman
author_rank: Super Contributor
timestamp: August 19, 2021, 02:01:02 pm
content: This should be perfect.I'll try it either later or this weekend.Thanks