This definition is an oversimplification. Usually the data blocks are added to a free list, so that they can be allocated to other files. An inode is a data structure that is used to manage most of the metadata for a file on a UNIX like filesystem. This allows one to have multiple links to a file. It turns out that most filesystems use this feature extensively to support the directory structure.
Directories are just inodes that are mostly just like files.
Somewhere you have a parent directory. It contains a link to you. It seems pretty self explanatory to me. You're removing a link. If that link is the last one to that file, then you've remove the file as well.