Cats might not be able to feel the emotion that comes with being sorry, but if they do feel bad for making you upset, they show you with affection. They will sidle up close to you and rub their head against you, much like they do when showing love for you. This is pretty much as close as cats get to saying sorry, so if your cat does this to you, then know you should be grateful for your furry friend.