Package: fastdebug
fastdebugA helpful library for improving torch and fastai errorsInstallpip install fastdebugHow to usefastdebugis designed around improving the quality of life when dealing with Pytorch and fastai errors, while also including some new sanity checks (fastai only)PytorchPytorch now has:device_errorlayer_errorBoth can be imported with:fromfastdebug.error.torchimportdevice_error,layer_errordevice_errorprints out a much more readable error for when two tensors aren't on the same device:inp=torch.rand().cuda()model=model.cpu()try:_=model(inp)exceptExceptionase:device_error(e,'Input type','Model weights')And our new log:---------------------------------------------------------------------------
RuntimeErrorTraceback(mostrecentcalllast)<ipython-input-28-981e0ace9c38>in<module>()2model(x)3exceptExceptionase:
---->4device_error(e,'Input type','Model weights')10frames
/usr/local/lib/python3.7/dist-packages/torch/tensor.pyin__torch_function__(cls,func,types,args,kwargs)993994with_C.DisableTorchFunction():
-->995ret=func(*args,**kwargs)996return_convert(ret,cls)997RuntimeError:Mismatchbetweenweighttypes

Inputtypehastype:(torch.cuda.FloatTensor)Modelweightshavetype:(torch.FloatTensor)Bothshouldbethesame.And withlayer_error, if there is a shape mismatch it will attempt to find the right layer it was at:inp=torch.rand(5,2,3)try:m(inp)exceptExceptionase:layer_error(e,m)---------------------------------------------------------------------------RuntimeErrorTraceback(mostrecentcalllast)<ipython-input-84-d4ab91131841>in<module>()3m(inp)4exceptExceptionase:---->5layer_error(e,m)<ipython-input-83-ca2dc02cfff4>inlayer_error(e,model)8i,layer=get_layer_by_shape(model,shape)9e.args=[f'Size mismatch between input tensors and what the model expects\n\n{args}\n\tat layer{i}:{layer}']--->10raisee<ipython-input-84-d4ab91131841>in<module>()1inp=torch.rand(5,2,3)2try:---->3m(inp)4exceptExceptionase:5layer_error(e,m)/mnt/d/lib/python3.7/site-packages/torch/nn/modules/module.pyin_call_impl(self,*input,**kwargs)725result=self._slow_forward(*input,**kwargs)726else:-->727result=self.forward(*input,**kwargs)728forhookinitertools.chain(729_global_forward_hooks.values(),/mnt/d/lib/python3.7/site-packages/torch/nn/modules/container.pyinforward(self,input)115defforward(self,input):116formoduleinself:-->117input=module(input)118returninput119/mnt/d/lib/python3.7/site-packages/torch/nn/modules/module.pyin_call_impl(self,*input,**kwargs)725result=self._slow_forward(*input,**kwargs)726else:-->727result=self.forward(*input,**kwargs)728forhookinitertools.chain(729_global_forward_hooks.values(),/mnt/d/lib/python3.7/site-packages/torch/nn/modules/conv.pyinforward(self,input)421422defforward(self,input:Tensor)->Tensor:-->423returnself._conv_forward(input,self.weight)424425classConv3d(_ConvNd):/mnt/d/lib/python3.7/site-packages/torch/nn/modules/conv.pyin_conv_forward(self,input,weight)418_pair(0),self.dilation,self.groups)419returnF.conv2d(input,weight,self.bias,self.stride,-->420self.padding,self.dilation,self.groups)421422defforward(self,input:Tensor)->Tensor:RuntimeError:SizemismatchbetweeninputtensorsandwhatthemodelexpectsModelexpected4-dimensionalinputfor4-dimensionalweight[3,3,1,1],butgot3-dimensionalinputofsize[5,2,3]insteadatlayer1:Conv2d(3,3,kernel_size=(1,1),stride=(1,1))fastaiAlong with the additions above (and are used duringfit), fastai now has aLearner.sanity_checkfunction, which allows you to quickly perform a basic check to ensure that your call tofitwon't raise any exceptions. They are performed on the CPU for a partial epoch to make sure thatCUDAdevice-assist errors can be preemptively found.To use it simply do:fromfastdebug.fastaiimport*fromfastai.vision.allimport*learn=Learner(...)learn.sanity_check()This is also now an argument inLearner, set toFalseby default, so that after making yourLearnera quick check is ensured.learn=Learner(...,sanity_check=True)