Package: lightning-tutorial
⚡ lightning-tutorialInstallation of the partner packagepipinstalllightning-tutorialTable of contentsPyTorch Datasets and DataLoadersKey module:torch.utils.data.DatasetKey module:torch.utils.data.DataLoaderOther essential functionsSingle-cell data structures meet pytorch:torch-adataLightning basics and theLightningModuleLightningDataModulePyTorch Datasets and DataLoadersKey module:torch.utils.data.DatasetTheDatasetmodule is an overwritable python module. You can modify it at will as long as you maintain the following three class methods:__init____len____getitem__These are name-specific handles used bytorchunder the hood when passing data through a model.fromtorch.utils.dataimportDatasetclassTurtleData(Dataset):def__init__(self):"""here we should pass requisite argumentsthat enable __len__() and __getitem__()"""def__len__(self):"""Returns the length/size/# of samples in the dataset.e.g., a 20,000 cell dataset would return `20_000`."""return# lendef__getitem__(self,idx):"""Subset and return a batch of the data.`idx` is the batch index (# of idx values = batch size).Maximum `idx` passed is <= `self.__len__()`"""return# sampled dataFantastic PyTorchDatasettutorial from StanfordTry it for yourself!ColabDatasettutorial notebookKey module:torch.utils.data.DataLoaderSimilar to the usefulness ofAnnData, theDatasetmodule creates a base unit for distributing and handling data. We can then take advantage of several torch built-ins to enable not only more organized, but faster data processing.fromtorch.utils.dataimportDataLoaderdataset=TurtleData()data_size=dataset.__len__()print(data_size)20_000Other essential functionsfromtorch.utils.dataimportrandom_splittrain_dataset,val_dataset=random_split(dataset,[18_000,2_000])# this can then be fed to a DataLoader, as abovetrain_loader=DataLoader(train_dataset)val_loader=DataLoader(val_dataset)Useful tutorials and documentationParent module:torch.utils.dataDatasets and DataLoaders tutorial☝️ back to table of contentsSingle-cell data structures meet pytorch:torch-adataCreate pytorch Datasets fromAnnDataInstallationNote: This is already done for you, if you've installed this tutorials associated packagepip install torch-adataExample use of the base classThe base class,AnnDatasetis a subclass of the widely-usedtorch.utils.data.Dataset.importanndataasaimporttorch_adataadata=a.read_h5ad("/path/to/data.h5ad")dataset=torch_adata.AnnDataset(adata)Returns sampled dataX_batchas atorch.Tensor.# create a dummy indexidx=np.random.choice(range(dataset.__len__()),5)X_batch=dataset.__getitem__(idx)TimeResolvedAnnDatasetSpecialized class for time-resolved datasets. A subclass of the class,AnnDataset.importanndataasaimporttorch_adataastaadata=a.read_h5ad("/path/to/data.h5ad")dataset=torch_adata.TimeResolvedAnnDataset(adata,time_key="Time point")☝️ back to table of contentsLightning basics and theLightningModulefrompytorch_lightningimoportLightningModuleclassYourSOTAModel(LightningModule):def__init__(self,net,optimizer_kwargs={"lr":1e-3},scheduler_kwargs={},):super().__init__()self.net=netself.optimizer_kwargs=optimizer_kwargsself.scheduler_kwargs=scheduler_kwargsdefforward(self,batch):x,y=batchy_hat=self.net(x)loss=LossFunc(y_hat,y)returny_hat,lossdeftraining_step(self,batch,batch_idx):y_hat,loss=self.forward(batch)returnloss.sum()defvalidation_step(self,batch,batch_idx):y_hat,loss=self.forward(batch)returnloss.sum()deftest_step(self,batch,batch_idx):y_hat,loss=self.forward(batch)returnloss.sum()defconfigure_optimizers(self):optimizer=torch.optim.Adam(self.parameters(),**self._optim_kwargs)scheduler=torch.optim.lr_scheduler.StepLR(optimizer(),**self._scheduler_kwargs)return[optimizer,...],[scheduler,...]Additional useful documentation and standalone tutorialsLightning in 15 minutesLogging metrics at each epoch☝️ back to table of contentsLightningDataModulePurpose: Make your model independent of a given dataset, while at the same time making your dataset reproducible and perhaps just as important:easily shareable.frompytorch_lightningimportLightningDataModulefromtorch.data.utilsimportDataLoaderclassYourDataModule(LightningDataModule):def__init__(self):# define any setup computationsdefprepare_data(self):# download data if applicabledefsetup(self,stage):# assign data to `Dataset`(s)deftrain_dataloader(self):returnDataLoader(self.train_dataset,batch_size=self.batch_size,num_workers=self.num_workers)defval_dataloader(self):returnDataLoader(self.val_dataset,batch_size=self.batch_size,num_workers=self.num_workers)deftest_dataloader(self):returnDataLoader(self.test_dataset,batch_size=self.batch_size,num_workers=self.num_workers)When it comes to actually using one of these, it looks something like the following:# Init the LightningDataModule as well as the LightningModeldata=YourDataModule()model=YourLightningModel()# Define trainertrainer=Trainer(accelerator="auto",devices=1)# Ultimately, both  model and data are passed as an arg to trainer.fittrainer.fit(model,data)OfficialLightningDataModuledocumentationHere's an example of aLightningDataModuleimplemented in practice, using the LARRY single-cell dataset:link. Initial downloading and formatting occurs only once but takes several minutes so we will leave it outside the scope of this tutorial.☝️ back to table of contents