Package: hcai-datasets-nightly
This Repository is deprecated since version 0.1.14 and will not be further developed. Bugfixes might be released if necessary.DescriptionThis repository contains code to make datasets stored on the corpora network drive of the chair.
You can use this project to easily create or reuse a data loader that is universally compatible with either plain python code or tensorflow / pytorch.
Also you this code can be used to dynamically create a dataloader for a Nova database to directly work with Nova Datasets in Python.Compatible with thetensorflow dataset api.
Pytorch Datasetis also supported.Installation InformationFor efficient data loading we rely on thedecordlibrary.
Decord ist not available as prebuild binary for non x86 architectures.
If you want to install the project on other architecture you will need to compile it yourself.Currently available DatasetsDatasetStatusUrlckplus✅http://www.iainm.com/publications/Lucey2010-The-Extended/paper.pdfaffectnet✅http://mohammadmahoor.com/affectnet/faces✅https://faces.mpdl.mpg.de/imeji/nova_dynamic✅https://github.com/hcmlab/novaaudioset❌https://research.google.com/audioset/is2021_ess❌-librispeech❌https://www.openslr.org/12ArchitectureDataset implementations are split into two parts.\Data access is handled by a generic python iterable, implemented by the DatasetIterable interface.The access class is then extended by an API class, which implements tfds.core.GeneratorBasedBuilder.
This results in the dataset being available by the Tensorflow Datasets API, and enables features
such as local caching.The iterables themselves can also be used as-is, either in PyTorch native DataGenerators by wrapping them in
the utility class BridgePyTorch, or as tensorflow-native Datasets by passing them to BridgeTensorflow.The benefits of this setup are that a pytorch application can be served without installing or loading
tensorflow, and vice versa, since the stack up to the adapters does not involve tf or pytorch.
Also, when using tf, caching can be used or discarded by using tfds or the plain tensorflow Dataset
provided by the bridge.Dynamic Dataset usage with Nova ExampleTo use the hcai_datasets repository with Nova you can use theHcaiNovaDynamicIterableclass from thehcai_datasets.hcai_nova_dynamic.hcai_nova_dynamic_iterablemodule to create an iterator for a specific data configuration.
This readme assumes that you are already familiar with the terminology and the general concept of the NOVA annotation tool / database.
The constructor of the class takes the following arguments as input:db_config_path:stringpath to a configfile with the nova database config. the config file looks like this:[DB]
ip = 127.0.0.1
port = 37317
user = my_user
password = my_passworddb_config_dict:stringdictionary with the nova database config. can be used instead of db_config_path. if both are specified db_config_dict is used.dataset:stringthe name of the dataset. Same as the entry in the Nova db.nova_data_dir:stringthe directory to look for data. same as the directory specified in the nova gui.sessions:listlist of sessions that should be loaded. must match the session names in nova.annotator:stringthe name of the annotator that labeld the session. must match annotator names in nova.schemes:listlist of the annotation schemes to fetch.roles:listlist of roles for which the annotation should be loaded.data_streams:listlist datastreams for which the annotation should be loaded. must match stream names in nova.start:string | int | floatstart time_ms. use if only a specific chunk of a session should be retrieved. can be passed as String (e.g. '1s' or '1ms'), Int (interpreted as milliseconds) or Float (interpreted as seconds).end:string | int | floatoptional end time_ms. use if only a specific chunk of a session should be retrieved. can be passed as String (e.g. '1s' or '1ms'), Int (interpreted as milliseconds) or Float (interpreted as seconds).left_context:string | int | floatadditional data to pass to the classifier on the left side of the frame. can be passed as String (e.g. '1s' or '1ms'), Int (interpreted as milliseconds) or Float (interpreted as seconds).right_context:string | int | floatadditional data to pass to the classifier on the left side of the frame. can be passed as String (e.g. '1s' or '1ms'), Int (interpreted as milliseconds) or Float (interpreted as seconds).frame_size:string | int | floatthe framesize to look at. the matching annotation will be calculated as majority vote from all annotations that are overlapping with the timeframe. can be passed as String (e.g. '1s' or '1ms'), Int (interpreted as milliseconds) or Float (interpreted as seconds).stride:string | int | floathow much a frame is moved to calculate the next sample. equals framesize by default. can be passed as String (e.g. '1s' or '1ms'), Int (interpreted as milliseconds) or Float (interpreted as seconds).flatten_samples:boolif set toTruesamples with the same annotation scheme but from different roles will be treated as separate samples. only  is used for the keys.add_rest_class:boolwhen set to True an additional rest class will be added to the end the label listfrompathlibimportPathfromhcai_dataset_utils.bridge_tfimportBridgeTensorflowimporttensorflowastffromhcai_datasets.hcai_nova_dynamic.hcai_nova_dynamic_iterableimportHcaiNovaDynamicIterableds_iter=HcaiNovaDynamicIterable(db_config_path="./nova_db.cfg",db_config_dict=None,dataset="affect-net",nova_data_dir=Path("./nova/data"),sessions=[f"{i}_man_eval"foriinrange(8)],roles=["session"],schemes=["emotion_categorical"],annotator="gold",data_streams=["video"],frame_size=0.04,left_context=0,right_context=0,start="0s",end="3000ms",flatten_samples=False,)forsampleinds_iter:print(sample)Pytorch ExampleThe BridePyTorch module can be used to create a Pytorch DataLoader directly from the Dataset iterable.fromtorch.utils.dataimportDataLoaderfromhcai_dataset_utils.bridge_pytorchimportBridgePyTorchfromhcai_datasets.hcai_affectnet.hcai_affectnet_iterableimportHcaiAffectnetIterableds_iter=HcaiAffectnetIterable(dataset_dir="path/to/data_sets/AffectNet",split="test")dataloader=DataLoader(BridgePyTorch(ds_iter))forsampleindataloader:print(sample)Tensorflow ExampleThe BridgeTensorflow module can be used to create a Pytorch DataLoader directly from the Dataset iterable.fromhcai_dataset_utils.bridge_tfimportBridgeTensorflowfromhcai_datasets.hcai_affectnet.hcai_affectnet_iterableimportHcaiAffectnetIterableds_iter=HcaiAffectnetIterable(dataset_dir="path/to/data_sets/AffectNet",split="test")tf_dataset=BridgeTensorflow.make(ds_iter)forsampleintf_dataset:print(sample)Tensorflow Dataset API (DEPRECATED)Example Usageimportosimporttensorflowastfimporttensorflow_datasetsastfdsimporthcai_datasetsfrommatplotlibimportpyplotasplt# Preprocessing functiondefpreprocess(x,y):img=x.numpy()returnimg,y# Creating a datasetds,ds_info=tfds.load('hcai_example_dataset',split='train',with_info=True,as_supervised=True,builder_kwargs={'dataset_dir':os.path.join('path','to','directory')})# Input output mappingds=ds.map(lambdax,y:(tf.py_function(func=preprocess,inp=[x,y],Tout=[tf.float32,tf.int64])))# Manually iterate over datasetimg,label=next(ds.as_numpy_iterator())# Visualizeplt.imshow(img/255.)plt.show()