You can thicken rice pudding by adding a cornstarch slurry while on the stovetop. Mix 1 tablespoon of cornstarch with 1 tablespoon of water until combined in a small bowl. Pour the slurry into the pudding then heat for up to 15 minutes, stirring often, until the desired consistency is reached.