Package: terrace
TerraceDocumentation|Code|PyPIWelcome to Terrace, a collection of high-level utilities for writing concise and maintainable PyTorch code. I've been using PyTorch in my own work for a while now, and I developed these tools to boost my productivity. I'm now ready to share them with the world -- I hope you find Terrace to be as helpful as I have.Terrace provides two major features: Modules and Batches. Terrace Modules allow you to more concisely define your PyTorch models entirely in theforwardmethod. Batches allow you to place all your data in nice classes rather than swinging around a bunch of raw tensors like a barbarian.ModulesIf you're writing a vanilla PyTorch model, you need to populate both the__init__andforwardmethods of your model.__init__specifies all the submodules of your model, including all the input and output tensor shapes.forwardspecifies how to use all these submodules in order to run your computation. A simple neural network with a single hidden layer might look something like this:importtorchimporttorch.nnasnnimporttorch.nn.functionalasFclassSimpleNetwork(nn.Module):def__init__(self):super().__init__()self.hidden_nn=nn.Linear(128,64)self.out_nn=nn.Linear(64,1)defforward(self,x):hid=F.relu(self.hidden_nn(x))returnself.out_nn(hid)model=SimpleNetwork()x=torch.randn(16,128)out=model(x)print(out.shape)torch.Size([16, 1])There are two annoyances with this approach. First, the overall structure of the network is repeated in both the__init__andforwardmethods. This may be fine for a toy example, but it gets pretty annoying with larger networks. Every time you make a change to the architecture, you need to constantly scroll between__init__andforward.Additionally, you need to specify beforehand what the input shapes to the model will be. In this example, changing the input dimension from 128 to 256 would require changing code in two places. For models with many inputs with shapes, this can be a real pain.TerraceModulessolve both these problems by allowing you to code the entire model in theforwardmethod. Here's how the same model would be written with Terrace.importterraceasterclassBetterSimpleNetwork(ter.Module):defforward(self,x):self.start_forward()# with the LazyLinear layers, we only need to specify the output dimensionhid=F.relu(self.make(ter.LazyLinear,64)(x))out=self.make(ter.LazyLinear,1)(hid)returnoutx=torch.randn(16,128)model=BetterSimpleNetwork()out=model(x)print(out.shape)torch.Size([16, 1])The first time this model is run, themakecalls with create new linear layers, each of which lazily creates their weight matrices based on their inputs. Writing complex models is now easier, faster, and just more fun.There are some important caveats to this approach, so please make sure to check out thedocumentation.BatchesOut of the box, PyTorch can collate tensors (and tuples of tensors) into batches. But what about arbitrary classes? If your neural network is dealing with complex datatypes, structuring your data in classes is the solution. By inheriting from Terrace'sBatchableclass, you can create datasets of arbitrary objects.Let's say you're working with people who have faces and namesMAX_NAME_LEN=128IMG_SIZE=256classPerson(ter.Batchable):face:torch.Tensorname:torch.Tensordef__init__(self):# dummy person dataself.face=torch.zeros((3,IMG_SIZE,IMG_SIZE))self.name=torch.zeros((MAX_NAME_LEN,))Now you can collate multiple people them into batches of people. These batches behave like people whose member data are batchified.dave=Person()rhonda=Person()batch=ter.collate([dave,rhonda])print(batch)Batch[Person](
   face=Tensor(shape=torch.Size([2, 3, 256, 256]), dtype=torch.float32)
   name=Tensor(shape=torch.Size([2, 128]), dtype=torch.float32)
)With Terrace'sDataloaderclass, you can make your datasets return people directly.classPersonDataset(torch.utils.data.Dataset):def__len__(self):return16def__getitem__(self,index):returnPerson()batch_size=8dataset=PersonDataset()loader=ter.DataLoader(dataset,batch_size=batch_size)forbatchinloader:print(batch)Batch[Person](
   face=Tensor(shape=torch.Size([8, 3, 256, 256]), dtype=torch.float32)
   name=Tensor(shape=torch.Size([8, 128]), dtype=torch.float32)
)
Batch[Person](
   face=Tensor(shape=torch.Size([8, 3, 256, 256]), dtype=torch.float32)
   name=Tensor(shape=torch.Size([8, 128]), dtype=torch.float32)
)Terrace also has a higher-level interface for graph data, and several more features. Check out thedocumentationfor more info!Getting startedIf you're interested in using Terrace for your own work, simply install via pip.pipinstallterraceIf you find Terrace useful, or find a bug, or have an idea for further functionality, please reach out!Email meor find me onTwitter.DisclaimerAt the moment, Terrace is very much a work in progress. The API is subject to change and there are likely many bugs. In its current state, I would not recommend this package for production use.