You can, and this might be a good idea if you decide to cook a type of rice that requires a shorter or longer cooking time than long-grain white rice. You may need to adjust the amount of chicken broth when baking the chicken without the rice to avoid the chicken boiling and becoming rubbery and dry.