Package: osculari
Exploring artificial neural networks with psychophysical experiments.OverviewTheoscularipackage provides an easy interface for different techniques to explore and interpret
the internal presentation of deep neural networks.Supporting the following pretrained models:All classification and segmentation networks
fromPyTorch's official website.All OpenAICLIPlanguage-vision models.AllTaskonomynetworks.Managing convolution and transformer architectures.Allowing to readout the network at any given depth.Training a linear classifier on top of the extract features from any network/layer.Experimenting with 2AFC and 4AFC paradigms.At a granular level, Kornia is a library that consists of the following components:ModuleDescriptionosculariOpen source library to explore and interpret pretrained deep neural networks.osculari.datasetsA module to create datasets and dataloaders to train and test linear probes.osculari.modelsA module to readout pretrained networks and add linear layers on top of them.osculari.paradigmsA module to implement psychophysical paradigms to experiment with deep networks.InstallationFrom pippipinstallosculariAlternative installation optionsFrom source with symbolic links:pipinstall-e.From source using pip:pipinstallgit+https://github.com/ArashAkbarinia/osculariExamplesPlease check theexample pageof our
documentation with many notebooks that can also be executed on Google Colab.Quick startPretrained featuresLet's create a linear classifier on top of the extracted features from a pretrained network to
perform a binary classification task (i.e., 2AFC – two-alternative-force-choice). This is easily
achieved by calling theparadigm_2afc_merge_concatenatefrom theosculari.modelsmodule.architecture='resnet50'# network's architectureweights='resnet50'# the pretrained weightsimg_size=224# network's input sizelayer='block0'# the readout layerreadout_kwargs={'architecture':architecture,'weights':weights,'layers':layer,'img_size':img_size,}net_2afc=osculari.models.paradigm_2afc_merge_concatenate(**readout_kwargs)DatasetsTheosculari.datasetsmodule provides datasets that are generated randomly on the fly with
flexible properties that can be dynamically changed based on the experiment of interest.
For instance, by passing aappearance_funto theShapeAppearanceDatasetclass, we can
dynamically merge foreground masks with background images to generate stimuli of interest.defappearance_fun(foregrounds,backgrounds):# implementing the required appearance (colour, texture, etc.) on foreground and merging# to background.returnmerged_imgs,ground_truthnum_samples=1000# the number of random samples generated in the datasetnum_imgs=net_2afc.input_nodes# the number of images in each samplebackground=128# the background typedataset=osculari.datasets.geometrical_shapes.ShapeAppearanceDataset(num_samples,num_imgs,img_size,background,appearance_fun,unique_bg=True,transform=net_2afc.preprocess_transform())Linear probeTheosculari.paradigmsmodule implements a set of psychophysical paradigms. Thetrain_linear_probefunction trains the network on a dataset following the paradigm passed to the function.# experiment-dependent function to train on an epoch of dataepoch_fun=osculari.paradigms.forced_choice.epoch_loop# calling the generic train_linear_probe functiontraining_log=osculari.paradigms.paradigm_utils.train_linear_probe(net_2afc,dataset,epoch_fun,'./osculari_test/')Psychophysical experimentTheosculari.paradigmsmodule also implements a set of psychophysical experiments similar to the
experiments conducted with human participants. In this example, we use thestaircasefunction
to gradually measure the network's sensitivity.# experiment-dependent function to test an epoch of datatest_epoch_fun=osculari.paradigms.forced_choice.test_dataset# the test dataset implementing desired stimuli.classTestDataset(TorchDataset):def__getitem__(self,idx):returnstimulitest_log=osculari.paradigms.staircase(net_2afc,test_epoch_fun,TestDataset(),low_val=0,high_val=1)ContributionWe welcome all contributions to the project that extend or improve code and/or documentation!
Please read theCONTRIBUTINGpage and follow theCode of Conduct.