Of course not. You can definitely leave out the food coloring. Your mint layer just won't be green. It will still taste every bit as awesome.