Fixed neural networks from blowing up