Yes, you can use water if you don't have store-bought or homemade chicken stock or broth on hand. You could also make up some stock with chicken stock cubes or powder.