Don't consider king, knight or pawns attacks