You can, but it will change the flavor slightly. If you like mushrooms, I say try it!