Transformers
A2N
super-image
image-super-resolution
Inference Endpoints
Eugene Siow commited on
Commit
0a0b275
1 Parent(s): 84f4c1d

Add new training recipe.

Browse files
Files changed (1) hide show
  1. README.md +28 -22
README.md CHANGED
@@ -5,6 +5,10 @@ tags:
5
  - image-super-resolution
6
  datasets:
7
  - eugenesiow/Div2k
 
 
 
 
8
  metrics:
9
  - pnsr
10
  - ssim
@@ -42,8 +46,9 @@ preds = model(inputs)
42
  ImageLoader.save_image(preds, './scaled_2x.png') # save the output 2x scaled image to `./scaled_2x.png`
43
  ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png') # save an output comparing the super-image with a bicubic scaling
44
  ```
 
45
  ## Training data
46
- The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/), a dataset of 800 high-quality (2K resolution) images for training, augmented to 4000 images and uses a dev set of 100 validation images (images numbered 801 to 900).
47
  ## Training procedure
48
  ### Preprocessing
49
  We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
@@ -51,24 +56,20 @@ Low Resolution (LR) images are created by using bicubic interpolation as the res
51
  During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches.
52
  Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image.
53
 
54
- The following code provides some helper functions to preprocess the data.
 
 
 
 
 
55
  ```python
56
- from super_image.data import EvalDataset, TrainAugmentDataset, DatasetBuilder
 
57
 
58
- DatasetBuilder.prepare(
59
- base_path='./DIV2K/DIV2K_train_HR',
60
- output_path='./div2k_4x_train.h5',
61
- scale=4,
62
- do_augmentation=True
63
- )
64
- DatasetBuilder.prepare(
65
- base_path='./DIV2K/DIV2K_val_HR',
66
- output_path='./div2k_4x_val.h5',
67
- scale=4,
68
- do_augmentation=False
69
- )
70
- train_dataset = TrainAugmentDataset('./div2k_4x_train.h5', scale=4)
71
- val_dataset = EvalDataset('./div2k_4x_val.h5')
72
  ```
73
  ### Pretraining
74
  The model was trained on GPU. The training code is provided below:
@@ -94,18 +95,19 @@ trainer = Trainer(
94
 
95
  trainer.train()
96
  ```
 
97
  ## Evaluation results
98
  The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm).
99
 
100
  Evaluation datasets include:
101
- - Set5 - [Bevilacqua et al. (2012)](http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html)
102
- - Set14 - [Zeyde et al. (2010)](https://sites.google.com/site/romanzeyde/research-interests)
103
- - BSD100 - [Martin et al. (2001)](https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/)
104
- - Urban100 - [Huang et al. (2015)](https://sites.google.com/site/jbhuang0604/publications/struct_sr)
105
 
106
  The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
107
 
108
- |Dataset |Scale |Bicubic |msrn-bam |
109
  |--- |--- |--- |--- |
110
  |Set5 |2x |33.64/0.9292 |**37.87/0.9602** |
111
  |Set5 |3x |30.39/0.8678 | |
@@ -122,6 +124,10 @@ The results columns below are represented below as `PSNR/SSIM`. They are compare
122
 
123
  ![Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2](images/a2n_2_4_compare.png "Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2")
124
 
 
 
 
 
125
  ## BibTeX entry and citation info
126
  ```bibtex
127
  @misc{chen2021attention,
 
5
  - image-super-resolution
6
  datasets:
7
  - eugenesiow/Div2k
8
+ - eugenesiow/Set5
9
+ - eugenesiow/Set14
10
+ - eugenesiow/BSD100
11
+ - eugenesiow/Urban100
12
  metrics:
13
  - pnsr
14
  - ssim
 
46
  ImageLoader.save_image(preds, './scaled_2x.png') # save the output 2x scaled image to `./scaled_2x.png`
47
  ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png') # save an output comparing the super-image with a bicubic scaling
48
  ```
49
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Upscale_Images_with_Pretrained_super_image_Models.ipynb "Open in Colab")
50
  ## Training data
51
+ The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://huggingface.co/datasets/eugenesiow/Div2k), a dataset of 800 high-quality (2K resolution) images for training, augmented to 4000 images and uses a dev set of 100 validation images (images numbered 801 to 900).
52
  ## Training procedure
53
  ### Preprocessing
54
  We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
 
56
  During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches.
57
  Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image.
58
 
59
+ We need the huggingface [datasets](https://huggingface.co/datasets?filter=task_ids:other-other-image-super-resolution) library to download the data:
60
+ ```bash
61
+ pip install datasets
62
+ ```
63
+ The following code gets the data and preprocesses/augments the data.
64
+
65
  ```python
66
+ from datasets import load_dataset
67
+ from super_image.data import EvalDataset, TrainDataset, augment_five_crop
68
 
69
+ augmented_dataset = load_dataset('eugenesiow/Div2k', 'bicubic_x4', split='train')\
70
+ .map(augment_five_crop, batched=True, desc="Augmenting Dataset") # download and augment the data with the five_crop method
71
+ train_dataset = TrainDataset(augmented_dataset) # prepare the train dataset for loading PyTorch DataLoader
72
+ eval_dataset = EvalDataset(load_dataset('eugenesiow/Div2k', 'bicubic_x4', split='validation')) # prepare the eval dataset for the PyTorch DataLoader
 
 
 
 
 
 
 
 
 
 
73
  ```
74
  ### Pretraining
75
  The model was trained on GPU. The training code is provided below:
 
95
 
96
  trainer.train()
97
  ```
98
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Train_super_image_Models.ipynb "Open in Colab")
99
  ## Evaluation results
100
  The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm).
101
 
102
  Evaluation datasets include:
103
+ - Set5 - [Bevilacqua et al. (2012)](https://huggingface.co/datasets/eugenesiow/Set5)
104
+ - Set14 - [Zeyde et al. (2010)](https://huggingface.co/datasets/eugenesiow/Set14)
105
+ - BSD100 - [Martin et al. (2001)](https://huggingface.co/datasets/eugenesiow/BSD100)
106
+ - Urban100 - [Huang et al. (2015)](https://huggingface.co/datasets/eugenesiow/Urban100)
107
 
108
  The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
109
 
110
+ |Dataset |Scale |Bicubic |A2N |
111
  |--- |--- |--- |--- |
112
  |Set5 |2x |33.64/0.9292 |**37.87/0.9602** |
113
  |Set5 |3x |30.39/0.8678 | |
 
124
 
125
  ![Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2](images/a2n_2_4_compare.png "Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2")
126
 
127
+ You can find a notebook to easily run evaluation on pretrained models below:
128
+
129
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Evaluate_Pretrained_super_image_Models.ipynb "Open in Colab")
130
+
131
  ## BibTeX entry and citation info
132
  ```bibtex
133
  @misc{chen2021attention,