Certainly, leave the almonds out and double the amount of flour in the streusel. You can also leave the streusel off entirely.