Yes and no. If you’re talking about deleting the reference to the original data (the icon and everything it represents), then overwrite does indeed mean delete. But if you’re concerned with the actual data stored on the hard drive, then overwrite doesn’t necessarily mean delete—at least not immediately.
In many cases, it takes days, weeks, and even months for the original data to be overwritten. Until that happens, the data can still be recovered with the help of special data recovery software, which can be extremely useful if you don’t create backups on a regular basis.