You have to decide whether you want to use a complex or a simple model and experimentally test where the error can be minimal. Let’s say you are dealing with an image processing task.Once the first layer is fixed, as you have a dataset, and use the digital version of the image as your input layer. Then you do the convolutional layer. After you are done with that, you proceed to the flatten layer, and finally, to the resulting layer. This way, you build your basic NN architecture, test the performance, and then improve it by incrementally adding layers and changing hyperparameters such as the number of kernels. If you see that the performance is not good enough, you can add more layers and see where it is going. If it is eventually getting worse, try to reduce the amount of added layers. So it is always an experimental way. At this point, we don’t have any rule of thumb as to how to choose hyperparameters. In some of our recent work, we tried analysing the sensitivity of deep learning hyperparameters.