Yes. Feel free to leave out the candied walnuts completely. You could use raw walnuts plus an extra tablespoon of brown sugar to the crumble. To make this loaf nut free, substitute the nuts for 1/4 cup of all purpose flour and a tablespoon of brown sugar.