Although the almonds add to the texture of the dessert, you can skip them if you're allergic. If you're not allergic and simply don't like almonds, you can substitute them with peanuts, broken cashew nuts, pine nuts, chopped walnuts, or any other nuts of your choice.