We did not. If you want that smokey flavor on the outside of your meat and a crustier/browned exterior you shouldn't. If you want to you can alternatively sear outside in a large skillet before or after cooking.