Package: careamics-portfolio
CAREamics PortfolioA helper package based onpoochallowing
downloading various example datasets used in publications by the Jug lab,
including data featured in N2V, P(P)N2V, DivNoising, HDN, EmbedSeg, etc.The complete list of datasets can be found indatasets.json.CAREamics-portfolio tooling was generated usingpydev-guide/pyrepo-copier.InstallationTo install the portfolio in your conda environment, simply usepip:$pipinstallcareamics-portfolioUsageFollow theexample notebookfor details on how to use the package.The portfolio can be instantiated as follow:fromcareamics_portfolioimportPortfolioManagerportfolio=PortfolioManager()You can explore the different datasets easily:print(portfolio)print(portfolio.denoising)print(portfolio.denoising.N2V_SEM)Finally, you can download the dataset of your choice:frompathlibimportPathdata_path=Path('data')# to the path of your choiceportfolio.denoising.N2V_SEM.download(data_path)# or to your system's cacheportfolio.denoising.N2V_SEM.download()By default, if you do not passpathto thedownload()method, all datasets
will be saved in your system's cache. New queries to download will not cause
the files to be downloaded again (thanks pooch!!).Important: if you download all datasets of interest using the same path,poochwill maintain a regsitry of files
and you will not have to download them again!Add a dataset to the portfolioThere are a few steps to follow in order to add a new dataset to the repository::white_check_mark: 1 - Create aPortfolioEntrychild class:white_check_mark: 2 - Instantiate the portfolio entry in anIterablePortfolio:white_check_mark: 3 - Updateregistry.txt:white_check_mark: 4 - Make sure all tests passNote: To run the tests, you will need to havepytestinstalled. You can
create an environment withcareamics-portfolioandpytestby running:pipinstall"careamics-portfolio[test]"1 - Create a portfolio entryTo add a dataset, subclass aPortfolioEntryand enter the following information
(preferably in one of the current categories, e.g.denoising_datasets.py):classMyDataset(PortfolioEntry):def__init__(self)->None:super().__init__(portfolio="Denoising",# for instancename="MyDataset",url="https://url.to.myfile/MyFile.zip",file_name="MyFile.zip",hash="953a815333805a423b7342971289h10121263917019bd16cc3341",# sha256description="Description of the dataset.",license="CC-BY 3.0",citation="Citation of the dataset",files={"/folder/in/the/zip":["file1.tif","file2.tif"],# folder can be "."},size=13.0,# size in MBtags=["tag1","tag2"],is_zip=True,)To obtain sha256 hash of your file, you can run the following code and read out
the sha256 from the pooch prompt:importpoochurl="https://url.to.myfile/MyFile.zip"pooch.retrieve(url,known_hash=None)Likewise, to get the size in MB of your file:importosos.path.getsize(file_path)/1024/10242 - Add the entry to a portfolioAdd the file class to one of the categories (e.g. denoising) inportfolio.py:classDenoising(IterablePortfolio):def__init__(self)->None:self._N2V_BSD68=N2V_BSD68()self._N2V_SEM=N2V_SEM()self._N2V_RGB=N2V_RGB()self._flywing=Flywing()# add your dataset as a private attributeself._myDataset=MyDataset()[...]# and add a public getter@propertydefMyDataset(self)->MyDataset:returnself._myDataset3 - Update registryFinally, update the registry by running the following pythons script:pythonscripts/update_registry.pyor run:fromcareamics_portfolioimportupdate_registryupdate_registry()Thedatasets.jsonfile is updated using:pythonscripts/update_json.py4 - Verify that all tests passVerify that all tests pass, it can take a while:pytest