Package: image-reward
ImageReward🤗HF Repo• 🐦Twitter• 📃Paper• 🖼Dataset• 🌐中文博客ImageReward: Learning and Evaluating Human Preferences for Text-to-Image GenerationImageReward is the first general-purpose text-to-image human preference RM, which is trained on in total137k pairs of expert comparisons, outperforming existing text-image scoring methods, such as CLIP (by 38.6%), Aesthetic (by 39.6%), and BLIP (by 31.6%), in terms of understanding human preference in text-to-image synthesis.Additionally, we introduce Reward Feedback Learning (ReFL) for direct optimizing a text-to-image diffusion model using ImageReward. ReFL-tuned Stable Diffusion wins against untuned version by 58.4% in human evaluation.Both ImageReward and ReFL are all packed up to Pythonimage-rewardpackage now!Tryimage-rewardpackage in only 3 lines of code for ImageReward scoring!# pip install image-rewardimportImageRewardasRMmodel=RM.load("ImageReward-v1.0")rewards=model.score("<prompt>",["<img1_obj_or_path>","<img2_obj_or_path>",...])Tryimage-rewardpackage in only 4 lines of code for ReFL fine-tuning!# pip install image-reward# pip install diffusers==0.16.0 accelerate==0.16.0 datasets==2.11.0fromImageRewardimportReFLargs=ReFL.parse_args()trainer=ReFL.Trainer("CompVis/stable-diffusion-v1-4","data/refl_data.json",args=args)trainer.train(args=args)If you findImageReward's open-source effort useful, please 🌟 us to encourage our following developement!ImageRewardQuick StartInstall DependencyExample UseReFLInstall DependencyExample UseDemos of ImageReward and ReFLTraining code for ImageRewardIntegration into Stable Diffusion Web UIFeaturesScore generated images and append to image informationUsage:Demo video:Automatically filter out images with low scoresUsage:Demo video:View the scores of images that have been scoredUsage:Example:Other FeaturesMemory ManagementReproduce Experiments in Table 1Reproduce Experiments in Table 3CitationQuick StartInstall DependencyWe have integrated the whole repository to a single python packageimage-reward. Following the commands below to prepare the environment:# Clone the ImageReward repository (containing data for testing)gitclonehttps://github.com/THUDM/ImageReward.gitcdImageReward# Install the integrated package `image-reward`pipinstallimage-rewardExample UseWe provide example images in theassets/imagesdirectory of this repo. The example prompt is:a painting of an ocean with clouds and birds, day time, low depth field effectUse the following code to get the human preference scores from ImageReward:importosimporttorchimportImageRewardasRMif__name__=="__main__":prompt="a painting of an ocean with clouds and birds, day time, low depth field effect"img_prefix="assets/images"generations=[f"{pic_id}.webp"forpic_idinrange(1,5)]img_list=[os.path.join(img_prefix,img)forimgingenerations]model=RM.load("ImageReward-v1.0")withtorch.no_grad():ranking,rewards=model.inference_rank(prompt,img_list)# Print the resultprint("\nPreference predictions:\n")print(f"ranking ={ranking}")print(f"rewards ={rewards}")forindexinrange(len(img_list)):score=model.score(prompt,img_list[index])print(f"{generations[index]:>16s}:{score:.2f}")The output should be like as follow (the exact numbers may be slightly different depending on the compute device):Preference predictions:

ranking = [1, 2, 3, 4]
rewards = [[0.5811622738838196], [0.2745276093482971], [-1.4131819009780884], [-2.029569625854492]]
          1.webp: 0.58
          2.webp: 0.27
          3.webp: -1.41
          4.webp: -2.03ReFLInstall Dependencypipinstalldiffusers==0.16.0accelerate==0.16.0datasets==2.11.0Example UseWe provide example dataset for ReFL in thedata/refl_data.jsonof this repo. Run ReFL as following:bashscripts/train_refl.shDemos of ImageReward and ReFLTraining code for ImageRewardDownload data: 🖼Dataset.Make dataset.cdtrain
pythonsrc/make_dataset.pySet training config:train/src/config/config.yamlOne command to train.bashscripts/train_one_node.shIntegration intoStable Diffusion Web UIWe have developed acustom scriptto integrate ImageReward into SD Web UI for a convenient experience.The script is located atsdwebui/image_reward.pyin this repository.Theusageof the script is described as follows:Install: put the custom script into thestable-diffusion-webui/scripts/directoryReload: restart the service, or click the"Reload custom script"button at the bottom of the settings tab of SD Web UI. (If the button can't be found, try clicking the"Show all pages"button at the bottom of the left sidebar.)Select: go back to the"txt2img"/"img2img"tab, and select"ImageReward - generate human preference scores"from the "Script"dropdown menu in the lower left corner.Run: the specific usage varies depending on the functional requirements, as described in the"Features"section below.FeaturesScore generated images and append to image informationUsage:Do notcheck the "Filter out images with low scores" checkbox.Click the"Generate"button to generate images.Check the ImageReward at thebottomof the image informationbelow the gallery.Demo video:https://github.com/THUDM/ImageReward/assets/98524878/9d8a036d-1583-4978-aac7-4b758edf9b89Automatically filter out images with low scoresUsage:Check the"Filter out images with low scores"checkbox.Enter the score lower limit in"Lower score limit". (ImageReward roughly follows the standard normal distribution, with a mean of 0 and a variance of 1.)Click the"Generate"button to generate images.Images with scores below the lower limit will be automatically filtered out andwill not appear in the gallery.Check the ImageReward at thebottomof the image informationbelow the gallery.Demo video:https://github.com/THUDM/ImageReward/assets/98524878/b9f01629-87d6-4c92-9990-fe065711b9c6View the scores of images that have been scoredUsage:Upload the scored image file in the"PNG Info"tabCheck the image information on the right with the score of the image at thebottom.Example:Other FeaturesMemory ManagementImageReward model will not be loadeduntil first script run."Reload UI"will not reload the model nor unload it, butreuses the currently loaded model (if it exists).A"Unload Model"button is provided to manually unload the currently loaded model.Reproduce Experiments in Table 1Note:The experimental results are produced in an environment that satisfies:(NVIDIA) Driver Version: 515.86.01CUDA Version: 11.7torchVersion: 1.12.1+cu113
According to our own reproduction experience, reproducing this experiment in other environments may cause the last decimal place to fluctuate, typically within a range of ±0.1.Run the following script to automatically download data, baseline models, and run experiments:bash./scripts/test-benchmark.shThen you can check the results inbenchmark/results/or the terminal.If you want to check the raw data files individually:Test prompts and corresponding human rankings for images are located inbenchmark/benchmark-prompts.json.Generated outputs for each prompt (originally fromDiffusionDB) can be downloaded fromHugging FaceorTsinghua Cloud.Each<model_name>.zipcontains a directory of the same name, in which there are in total 1000 images generated from 100 prompts of 10 images each.Every<model_name>.zipshould be decompressed intobenchmark/generations/as directory<model_name>that contains images.Reproduce Experiments in Table 3Run the following script to automatically download data, baseline models, and run experiments:bash./scripts/test.shIf you want to check the raw data files individually:Test prompts and corresponding human rankings for images are located indata/test.json.Generated outputs for each prompt (originally fromDiffusionDB) can be downloaded fromHugging FaceorTsinghua Cloud. It should be decompressed todata/test_images.Citation@misc{xu2023imagereward,
      title={ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation},
      author={Jiazheng Xu and Xiao Liu and Yuchen Wu and Yuxuan Tong and Qinkai Li and Ming Ding and Jie Tang and Yuxiao Dong},
      year={2023},
      eprint={2304.05977},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}