32-bit float it's also stupidly easy to compute the gradient it's one or at zero right you just take that top bit and you're done so the derivatives ridiculously usually rel you have some downsides it does have those dead neurons on on the left side you can fix that with a leaky rail you there's this discontinuity in the gradient of the origin you can fix that with Gail u which burnt uses and so this brings me to a little aside about general deep learning wisdom if you're designing a new network for whatever reason don't