In a deep learning neural network, hyperparameters are the variables that determine the network structure as well as how the network is trained. These hyperparameters are set before the network is trained before the weights and bias are optimized.