Package: tensordict
TensorDictInstallation|General features|Tensor-like features|Distributed capabilities|TensorDict for functional programming|**TensorDict for parameter serialization|Lazy preallocation|Nesting TensorDicts|TensorClassTensorDictis a dictionary-like class that inherits properties from tensors,
such as indexing, shape operations, casting to device or point-to-point communication
in distributed settings.The main purpose of TensorDict is to make code-bases morereadableandmodularby abstracting away tailored operations:fori,datainenumerate(dataset):# the model reads and writes tensordictsdata=model(data)loss=loss_module(data)loss.backward()optimizer.step()optimizer.zero_grad()With this level of abstraction, one can recycle a training loop for highly heterogeneous task.
Each individual step of the training loop (data collection and transform, model prediction, loss computation etc.)
can be tailored to the use case at hand without impacting the others.
For instance, the above example can be easily used across classification and segmentation tasks, among many others.FeaturesGeneralA tensordict is primarily defined by itsbatch_size(orshape) and its key-value pairs:>>>fromtensordictimportTensorDict>>>importtorch>>>data=TensorDict({..."key 1":torch.ones(3,4,5),..."key 2":torch.zeros(3,4,5,dtype=torch.bool),...},batch_size=[3,4])Thebatch_sizeand the first dimensions of each of the tensors must be compliant.
The tensors can be of any dtype and device. Optionally, one can restrict a tensordict to
live on a dedicated device, which will send each tensor that is written there:>>>data=TensorDict({..."key 1":torch.ones(3,4,5),..."key 2":torch.zeros(3,4,5,dtype=torch.bool),...},batch_size=[3,4],device="cuda:0")>>>data["key 3"]=torch.randn(3,4,device="cpu")>>>assertdata["key 3"].deviceistorch.device("cuda:0")But that is not all, you can also store nested values in a tensordict:>>>data["nested","key"]=torch.zeros(3,4)# the batch-size must matchand any nested tuple structure will be unravelled to make it easy to read code and
write ops programmatically:>>>data["nested",("supernested",("key",))]=torch.zeros(3,4)# the batch-size must match>>>assert(data["nested","supernested","key"]==0).all()>>>assert(("nested",),"supernested",(("key",),))indata.keys(include_nested=True)# this works too!You can also store non-tensor data in tensordicts:>>>data=TensorDict({"a-tensor":torch.randn(1,2)},batch_size=[1,2])>>>data["non-tensor"]="a string!">>>assertdata["non-tensor"]=="a string!"Tensor-like featuresTensorDict objects can be indexed exactly like tensors. The resulting of indexing
a TensorDict is another TensorDict containing tensors indexed along the required dimension:>>>data=TensorDict({..."key 1":torch.ones(3,4,5),..."key 2":torch.zeros(3,4,5,dtype=torch.bool),...},batch_size=[3,4])>>>sub_tensordict=data[...,:2]>>>assertsub_tensordict.shape==torch.Size([3,2])>>>assertsub_tensordict["key 1"].shape==torch.Size([3,2,5])Similarly, one can build tensordicts by stacking or concatenating single tensordicts:>>>tensordicts=[TensorDict({..."key 1":torch.ones(3,4,5),..."key 2":torch.zeros(3,4,5,dtype=torch.bool),...},batch_size=[3,4])for_inrange(2)]>>>stack_tensordict=torch.stack(tensordicts,1)>>>assertstack_tensordict.shape==torch.Size([3,2,4])>>>assertstack_tensordict["key 1"].shape==torch.Size([3,2,4,5])>>>cat_tensordict=torch.cat(tensordicts,0)>>>assertcat_tensordict.shape==torch.Size([6,4])>>>assertcat_tensordict["key 1"].shape==torch.Size([6,4,5])TensorDict instances can also be reshaped, viewed, squeezed and unsqueezed:>>>data=TensorDict({..."key 1":torch.ones(3,4,5),..."key 2":torch.zeros(3,4,5,dtype=torch.bool),...},batch_size=[3,4])>>>print(data.view(-1))torch.Size([12])>>>print(data.reshape(-1))torch.Size([12])>>>print(data.unsqueeze(-1))torch.Size([3,4,1])One can also send tensordict from device to device, place them in shared memory,
clone them, update them in-place or not, split them, unbind them, expand them etc.If a functionality is missing, it is easy to call it usingapply()orapply_():tensordict_uniform=data.apply(lambdatensor:tensor.uniform_())apply()can also be great to filter a tensordict, for instance:data=TensorDict({"a":torch.tensor(1.0,dtype=torch.float),"b":torch.tensor(1,dtype=torch.int64)},[])data_float=data.apply(lambdax:xifx.dtype==torch.floatelseNone)# contains only the "a" keyassert"b"notindata_floatDistributed capabilitiesComplex data structures can be cumbersome to synchronize in distributed settings.tensordictsolves that problem with synchronous and asynchronous helper methods
such asrecv,irecv,sendandisendthat behave like theirtorch.distributedcounterparts:>>># on all workers>>>data=TensorDict({"a":torch.zeros(()),("b","c"):torch.ones(())},[])>>># on worker 1>>>data.isend(dst=0)>>># on worker 0>>>data.irecv(src=1)When nodes share a common scratch space, theMemmapTensorbackendcan be used
to seamlessly send, receive and read a huge amount of data.TensorDict for functional programmingWe also provide an API to use TensorDict in conjunction withFuncTorch.
For instance, TensorDict makes it easy to concatenate model weights to do model ensembling:>>>fromtorchimportnn>>>fromtensordictimportTensorDict>>>importtorch>>>fromtorchimportvmap>>>layer1=nn.Linear(3,4)>>>layer2=nn.Linear(4,4)>>>model=nn.Sequential(layer1,layer2)>>>params=TensorDict.from_module(model)>>># we represent the weights hierarchically>>>weights1=TensorDict(layer1.state_dict(),[]).unflatten_keys(".")>>>weights2=TensorDict(layer2.state_dict(),[]).unflatten_keys(".")>>>assert(params==TensorDict({"0":weights1,"1":weights2},[])).all()>>># Let's use our functional module>>>x=torch.randn(10,3)>>>withparams.to_module(model):...out=model(x)>>># an ensemble of models: we stack params along the first dimension...>>>params_stack=torch.stack([params,params],0)>>># ... and use it as an input we'd like to pass through the model>>>deffunc(x,params):...withparams.to_module(model):...returnmodel(x)>>>y=vmap(func,(None,0))(x,params_stack)>>>print(y.shape)torch.Size([2,10,4])Moreover, tensordict modules are compatible withtorch.fxand (soon)torch.compile,
which means that you can get the best of both worlds: a codebase that is
both readable and future-proof as well as efficient and portable!TensorDict for parameter serialization and building datasetsTensorDict offers an API for parameter serialization that can be >3x faster than
regular calls totorch.save(state_dict). Moreover, because tensors will be saved
independently on disk, you can deserialize your checkpoint on an arbitrary slice
of the model.>>>model=nn.Sequential(nn.Linear(3,4),nn.Linear(4,3))>>>params=TensorDict.from_module(model)>>>params.memmap("/path/to/saved/folder/",num_threads=16)# adjust num_threads for speed>>># load params>>>params=TensorDict.load_memmap("/path/to/saved/folder/",num_threads=16)>>>params.to_module(model)# load onto model>>>params["0"].to_module(model[0])# load on a slice of the model>>># in the latter case we could also have loaded only the slice we needed>>>params0=TensorDict.load_memmap("/path/to/saved/folder/0",num_threads=16)>>>params0.to_module(model[0])# load on a slice of the modelThe same functionality can be used to access data in a dataset stored on disk.
Soring a single contiguous tensor on disk accessed through thetensordict.MemoryMappedTensorprimitive and reading slices of it is not onlymuchfaster than loading
single files one at a time but it's also easier and safer (because there is no pickling
or third-party library involved):# allocate memory of the dataset on diskdata=TensorDict({"images":torch.zeros((128,128,3),dtype=torch.uint8),"labels":torch.zeros((),dtype=torch.int)},batch_size=[])data=data.expand(1000000)data=data.memmap_like("/path/to/dataset")# ==> Fill your dataset here# Let's get 3 items of our dataset:data[torch.tensor([1,10000,500000])]# This is much faster than loading the 3 images independentlyPreprocessing with TensorDict.mapPreprocessing huge contiguous (or not!) datasets can be done viaTensorDict.mapwhich will dispatch a task to various workers:importtorchfromtensordictimportTensorDict,MemoryMappedTensorimporttempfiledefprocess_data(data):images=data.get("images").flip(-2).clone()labels=data.get("labels")//10# we update the td inplacedata.set_("images",images)# flip imagedata.set_("labels",labels)# cluster labelsif__name__=="__main__":# create data_preproc heredata_preproc=data.map(process_data,num_workers=4,chunksize=0,pbar=True)# process 1 images at a timeLazy preallocationPre-allocating tensors can be cumbersome and hard to scale if the list of preallocated
items varies according to the script configuration. TensorDict solves this in an elegant way.
Assume you are working with a functionfoo() -> TensorDict, e.g.deffoo():data=TensorDict({},batch_size=[])data["a"]=torch.randn(3)data["b"]=TensorDict({"c":torch.zeros(2)},batch_size=[])returndataand you would like to call this function repeatedly. You could do this in two ways.
The first would simply be to stack the calls to the function:data=torch.stack([foo()for_inrange(N)])However, you could also choose to preallocate the tensordict:data=TensorDict({},batch_size=[N])foriinrange(N):data[i]=foo()which also results in a tensordict (whenN = 10)TensorDict(
    fields={
        a: Tensor(torch.Size([10, 3]), dtype=torch.float32),
        b: TensorDict(
            fields={
                c: Tensor(torch.Size([10, 2]), dtype=torch.float32)},
            batch_size=torch.Size([10]),
            device=None,
            is_shared=False)},
    batch_size=torch.Size([10]),
    device=None,
    is_shared=False)Wheni==0, your empty tensordict will automatically be populated with empty tensors
of batch-sizeN. After that, updates will be written in-place.
Note that this would also work with a shuffled series of indices (pre-allocation does
not require you to go through the tensordict in an ordered fashion).Nesting TensorDictsIt is possible to nest tensordict. The only requirement is that the sub-tensordict should be indexable
under the parent tensordict, i.e. its batch size should match (but could be longer than) the parent
batch size.We can switch easily between hierarchical and flat representations.
For instance, the following code will result in a single-level tensordict with keys"key 1"and"key 2.sub-key":>>>data=TensorDict({..."key 1":torch.ones(3,4,5),..."key 2":TensorDict({"sub-key":torch.randn(3,4,5,6)},batch_size=[3,4,5])...},batch_size=[3,4])>>>tensordict_flatten=data.flatten_keys(separator=".")Accessing nested tensordicts can be achieved with a single index:>>>sub_value=data["key 2","sub-key"]TensorClassContent flexibility comes at the cost of predictability.
In some cases, developers may be looking for data structure with a more explicit behavior.tensordictprovides adataclass-like decorator that allows for the creation of custom dataclasses that support
the tensordict operations:>>>fromtensordict.prototypeimporttensorclass>>>importtorch>>>>>>@tensorclass...classMyData:...image:torch.Tensor...mask:torch.Tensor...label:torch.Tensor......defmask_image(self):...returnself.image[self.mask.expand_as(self.image)].view(*self.batch_size,-1)......defselect_label(self,label):...returnself[self.label==label]...>>>images=torch.randn(100,3,64,64)>>>label=torch.randint(10,(100,))>>>mask=torch.zeros(1,64,64,dtype=torch.bool).bernoulli_().expand(100,1,64,64)>>>>>>data=MyData(images,mask,label=label,batch_size=[100])>>>>>>print(data.select_label(1))MyData(image=Tensor(torch.Size([11,3,64,64]),dtype=torch.float32),label=Tensor(torch.Size([11]),dtype=torch.int64),mask=Tensor(torch.Size([11,1,64,64]),dtype=torch.bool),batch_size=torch.Size([11]),device=None,is_shared=False)>>>print(data.mask_image().shape)torch.Size([100,6117])>>>print(data.reshape(10,10))MyData(image=Tensor(torch.Size([10,10,3,64,64]),dtype=torch.float32),label=Tensor(torch.Size([10,10]),dtype=torch.int64),mask=Tensor(torch.Size([10,10,1,64,64]),dtype=torch.bool),batch_size=torch.Size([10,10]),device=None,is_shared=False)As this example shows, one can write a specific data structures with dedicated methods while still enjoying the TensorDict
artifacts such as shape operations (e.g. reshape or permutations), data manipulation (indexing,catandstack) or calling
arbitrary functions through theapplymethod (and many more).Tensorclasses support nesting and, in fact, all the TensorDict features.InstallationWith Pip:To install the latest stable version of tensordict, simply runpipinstalltensordictThis will work with Python 3.7 and upward as well as PyTorch 1.12 and upward.To enjoy the latest features, one can usepipinstalltensordict-nightlyWith Conda:Installtensordictfromconda-forgechannel.condainstall-cconda-forgetensordictCitationIf you're using TensorDict, please refer to this BibTeX entry to cite this work:@misc{bou2023torchrl,
      title={TorchRL: A data-driven decision-making library for PyTorch}, 
      author={Albert Bou and Matteo Bettini and Sebastian Dittert and Vikash Kumar and Shagun Sodhani and Xiaomeng Yang and Gianni De Fabritiis and Vincent Moens},
      year={2023},
      eprint={2306.00577},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}DisclaimerTensorDict is at thebeta-stage, meaning that there may be bc-breaking changes introduced, but
they should come with a warranty.
Hopefully these should not happen too often, as the current roadmap mostly
involves adding new features and building compatibility with the broader
PyTorch ecosystem.LicenseTensorDict is licensed under the MIT License. SeeLICENSEfor details.