Question: what is the difference between cost function and activation function?
Answer: Usually, when you “train” a network, you are defining the coefficients of the activation functions. The cost function is used to determine the error your network produces on an iteration of training with the training data. This is used to direct modification of the training variables to improve performance.