Filter adjacent tiles based on which tile we are getting them from