Package: aiarc
aiarcWhyI prototyped novel and good algorithms for chemistry in 2021 and it took me two years to get them publication ready. Of course I did other stuff like thesis, work, exams etc. However, the development time was far too long.WhatPreliminaries and LearningsI need my own frameworkI learned from aiarc that I need more flexibility for the major partsI need a directory structure and saving structure to properly distribute the filesthere could be an initializermethods should be imported easier through theinitfilesTesting should be more rigorous (in each new project)The package is aimed to investigate algorithms fasterThe focus is on making algorithms and sparing out the restThus the focus is on an easy interface for investigation. PytorchGeometric gives a good guideline on how to do this. It is more though, and also about producing plots to score the overall algorithm.aiarcis aimed to improve rapid prototyping of AI algorithms on graphs (timeseries, molecules, networks etc) for building novel simulation applications fast and at scale.The package shall provide production grade algorithms in an automated way but with user interaction.Lessons learned from other projectsThe lessons I learned from other projects wereNever trust propietary software that is used for politics, e.g. Microsoft, Nvidia drivers, Canonical etcAvoid trusting open-source software that is aimed to upsell a paid software distribution (e.g. Canonical, or security software)Code high quality instead of coding fast -> is faster in the endTry to code consistently instead of a lot at onceHow it should workWork on a project base -> provide a framework to start a project
A project needs differnt thingsConfigurationtrainingMonitoringLoggingAnalysisFoundationsI am having some years of experience now, trying to model in an regulatory environment such as drug design, risk assessement, health care and chemistry.Packages I have built in that context arechembeeaiarcAn architectural pattern became prevalent such that I found it intorchsr, too (which had mutiple years of development, too)torchsrNow, withaiarcwe aim to get modelling to the next stage.Packages to usejaxpytorch-gym (?)torchtorchsrSuper-Resolution Networks for PytorchSuper-resolutionis a process that increases the resolution of an image, adding additional details.
Methods using neural networks give the most accurate results, much better than other interpolation methods.
With the right training, it is even possible to make photo-realistic images.For example, here is a low-resolution image, magnified x4 by a neural network, and a high resolution image of the same object:In this repository, you will find:the popular super-resolution networks, pretrainedcommon super-resolution datasetsa unified training script for all modelsModelsThe following pretrained models are available. Click on the links for the paper:EDSRCARNRDNRCANNinaSRNewer and larger models perform better: the most accurate models are EDSR (huge), RCAN and NinaSR-B2.
For practical applications, I recommend a smaller model, such as NinaSR-B1.Expand benchmark resultsSet5 resultsNetworkParameters (M)2x (PSNR/SSIM)3x (PSNR/SSIM)4x (PSNR/SSIM)carn1.5937.88 / 0.960034.32 / 0.926532.14 / 0.8942carn_m0.4137.68 / 0.959434.06 / 0.924731.88 / 0.8907edsr_baseline1.3737.98 / 0.960434.37 / 0.927032.09 / 0.8936edsr40.738.19 / 0.960934.68 / 0.929332.48 / 0.8985ninasr_b00.1037.72 / 0.959433.96 / 0.923431.77 / 0.8877ninasr_b11.0238.14 / 0.960934.48 / 0.927732.28 / 0.8955ninasr_b210.038.21 / 0.961234.61 / 0.928832.45 / 0.8973rcan15.438.27 / 0.961434.76 / 0.929932.64 / 0.9000rdn22.138.12 / 0.960933.98 / 0.923432.35 / 0.8968Set14 resultsNetworkParameters (M)2x (PSNR/SSIM)3x (PSNR/SSIM)4x (PSNR/SSIM)carn1.5933.57 / 0.917330.30 / 0.841228.61 / 0.7806carn_m0.4133.30 / 0.915130.10 / 0.837428.42 / 0.7764edsr_baseline1.3733.57 / 0.917430.28 / 0.841428.58 / 0.7804edsr40.733.95 / 0.920130.53 / 0.846428.81 / 0.7872ninasr_b00.1033.24 / 0.914430.02 / 0.835528.28 / 0.7727ninasr_b11.0233.71 / 0.918930.41 / 0.843728.71 / 0.7840ninasr_b210.034.00 / 0.920630.53 / 0.846128.80 / 0.7863rcan15.434.13 / 0.921630.63 / 0.847528.85 / 0.7878rdn22.133.71 / 0.918230.07 / 0.837328.72 / 0.7846DIV2K results (validation set)NetworkParameters (M)2x (PSNR/SSIM)3x (PSNR/SSIM)4x (PSNR/SSIM)8x (PSNR/SSIM)carn1.5936.08 / 0.945132.37 / 0.887130.43 / 0.8366N/Acarn_m0.4135.76 / 0.942932.09 / 0.882730.18 / 0.8313N/Aedsr_baseline1.3736.13 / 0.945532.41 / 0.887830.43 / 0.8370N/Aedsr40.736.56 / 0.948532.75 / 0.893330.73 / 0.8445N/Aninasr_b00.1035.77 / 0.942832.06 / 0.881830.09 / 0.829326.60 / 0.7084ninasr_b11.0236.35 / 0.947132.51 / 0.889230.56 / 0.840526.96 / 0.7207ninasr_b210.036.52 / 0.948232.73 / 0.892630.73 / 0.843727.07 / 0.7246rcan15.436.61 / 0.948932.78 / 0.893530.73 / 0.844727.17 / 0.7292rdn22.136.32 / 0.946832.04 / 0.882230.61 / 0.8414N/AB100 resultsNetworkParameters (M)2x (PSNR/SSIM)3x (PSNR/SSIM)4x (PSNR/SSIM)carn1.5932.12 / 0.898629.07 / 0.804227.58 / 0.7355carn_m0.4131.97 / 0.897128.94 / 0.801027.45 / 0.7312edsr_baseline1.3732.15 / 0.899329.08 / 0.805127.56 / 0.7354edsr40.732.35 / 0.901929.26 / 0.809627.72 / 0.7419ninasr_b00.1031.97 / 0.897428.90 / 0.800027.36 / 0.7290ninasr_b11.0232.24 / 0.900429.13 / 0.806127.62 / 0.7377ninasr_b210.032.32 / 0.901429.23 / 0.808727.71 / 0.7407rcan15.432.39 / 0.902429.30 / 0.810627.74 / 0.7429rdn22.132.25 / 0.900628.90 / 0.800427.66 / 0.7388Urban100 resultsNetworkParameters (M)2x (PSNR/SSIM)3x (PSNR/SSIM)4x (PSNR/SSIM)carn1.5931.95 / 0.926328.07 / 0.84926.07 / 0.78349carn_m0.4131.30 / 0.920027.57 / 0.83925.64 / 0.76961edsr_baseline1.3731.98 / 0.927128.15 / 0.85226.03 / 0.78424edsr40.732.97 / 0.935828.81 / 0.86526.65 / 0.80328ninasr_b00.1031.33 / 0.920427.48 / 0.837425.45 / 0.7645ninasr_b11.0232.48 / 0.931928.29 / 0.855526.25 / 0.7914ninasr_b210.032.91 / 0.935428.70 / 0.864026.54 / 0.8008rcan15.433.19 / 0.937229.01 / 0.86826.75 / 0.80624rdn22.132.41 / 0.931027.49 / 0.83826.36 / 0.79460All models are defined intorchsr.models. Other useful tools to augment your models, such as self-ensemble methods and tiling, are present intorchsr.models.utils.DatasetsThe following datasets are available. Click on the links for the project page:DIV2KRealSRFlicr2KREDSSet5,Set14,B100,Urban100All datasets are defined intorchsr.datasets. They return a list of images, with the high-resolution image followed by downscaled or degraded versions.
Data augmentation methods are provided intorchsr.transforms.Datasets are downloaded automatically when using thedownload=Trueflag, or by running the corresponding script i.e../scripts/download_div2k.sh.Usagefromtorchsr.datasetsimportDiv2Kfromtorchsr.modelsimportninasr_b0fromtorchvision.transforms.functionalimportto_pil_image,to_tensor# Div2K datasetdataset=Div2K(root="./data",scale=2,download=False)# Get the first image in the dataset (High-Res and Low-Res)hr,lr=dataset[0]# Download a pretrained NinaSR modelmodel=ninasr_b0(scale=2,pretrained=True)# Run the Super-Resolution modellr_t=to_tensor(lr).unsqueeze(0)sr_t=model(lr_t)sr=to_pil_image(sr_t.squeeze(0))sr.show()Expand more examplesfromtorchsr.datasetsimportDiv2Kfromtorchsr.modelsimportedsr,rcanfromtorchsr.models.utilsimportChoppedModel,SelfEnsembleModelfromtorchsr.transformsimportColorJitter,Compose,RandomCrop# Div2K dataset, cropped to 256px, width color jitterdataset=Div2K(root="./data",scale=2,download=False,transform=Compose([RandomCrop(256,scales=[1,2]),ColorJitter(brightness=0.2)]))# Pretrained RCAN model, with tiling for large imagesmodel=ChoppedModel(rcan(scale=2,pretrained=True),scale=2,chop_size=400,chop_overlap=10)# Pretrained EDSR model, with self-ensemble method for higher qualitymodel=SelfEnsembleModel(edsr(scale=2,pretrained=True))TrainingA script is available to train the models from scratch, evaluate them, and much more. It is not part of the pip package, and requires additional dependencies. More examples are available inscripts/.pipinstallpiqtqdmtensorboard# Additional dependenciespython-mtorchsr.train-h
python-mtorchsr.train--archedsr_baseline--scale2--download-pretrained--imagestest/butterfly.png--destinationresults/
python-mtorchsr.train--archedsr_baseline--scale2--download-pretrained--validation-only
python-mtorchsr.train--archedsr_baseline--scale2--epochs300--lossl1--dataset-traindiv2k_bicubicYou can evaluate models from the command line as well. For example, for EDSR with the paper's PSNR evaluation:python -m torchsr.train --validation-only --arch edsr_baseline --scale 2 --dataset-val set5 --chop-size 400 --download-pretrained --shave-border 2 --eval-luminanceAcknowledgementsThanks to the people behindtorchvisionandEDSR, whose work inspired this repository.
Some of the models available here come fromEDSR-PyTorchandCARN-PyTorch.To cite this work, please use:@misc{torchsr,
  author = {Gabriel Gouvine},
  title = {Super-Resolution Networks for Pytorch},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Coloquinte/torchSR}},
  doi = {10.5281/zenodo.4868308}
}

@misc{ninasr,
  author = {Gabriel Gouvine},
  title = {NinaSR: Efficient Small and Large ConvNets for Super-Resolution},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Coloquinte/torchSR/blob/main/doc/NinaSR.md}},
  doi = {10.5281/zenodo.4868308}
}