Question: It seems that my program is trying to learn until a certain point, and then it's satisfied and stops improving and changing at all. With my testing it usually goes to a value of -5 at most, and then it remains there no matter how long I keep it running. The result set does not change either. Just to keep track of it I made my own kind of logging thing to see which did best. The array of ones and zeroes is referring to how often the AI made a right choice (1), and how often the AI made a wrong choice (0). My goal is to get the AI to repeat a pattern of going above 0.5 and then going below 0.5, not necessarily find the odd number. This was meant as just a little test to see if I could get an AI working properly with some basic data, before doing something a bit more advanced. But unfortunately it's not working and I am not certain why. The code: import os import neat def main(genomes, config): networks = [] ge = [] choices = [] for _, genome in genomes: network = neat.nn.FeedForwardNetwork.create(genome, config) networks.append(network) genome.fitness = 0 ge.append(genome) choices.append([]) for x in range(25): for i, genome in enumerate(ge): output = networks[i].activate([x]) # print(str(x) + " - " + str(i) + " chose " + str(output[0])) if output[0] > 0.5: if x % 2 == 0: ge[i].fitness += 1 choices[i].append(1) else: ge[i].fitness -= 5 choices[i].append(0) else: if not x % 2 == 0: ge[i].fitness += 1 choices[i].append(1) else: ge[i].fitness -= 5 choices[i].append(0) pass # Optional death function, if I use this there are no winners at any point. # if ge[i].fitness <= 20: # ge[i].fitness -= 100 # ge.pop(i) # choices.pop(i) # networks.pop(i) if len(ge) > 0: fittest = -1 fitness = -999999 for i, genome in enumerate(ge): if ge[i].fitness > fitness: fittest = i fitness = ge[i].fitness print("Best: " + str(fittest) + " with fitness " + str(fitness)) print(str(choices[fittest])) else: print("Done with no best.") def run(config_path): config = neat.config.Config(neat.DefaultGenome, neat.DefaultReproduction, neat.DefaultSpeciesSet, neat.DefaultStagnation, config_path) pop = neat.Population(config) #pop.add_reporter(neat.StdOutReporter(True)) #stats = neat.StatisticsReporter() #pop.add_reporter(stats) winner = pop.run(main, 100) if __name__ == "__main__": local_dir = os.path.dirname(__file__) config_path = os.path.join(local_dir, "config-feedforward.txt") run(config_path)  The NEAT config: [NEAT] fitness_criterion = max fitness_threshold = 100000 pop_size = 5000 reset_on_extinction = False [DefaultGenome] # node activation options activation_default = tanh activation_mutate_rate = 0.0 activation_options = tanh # node aggregation options aggregation_default = sum aggregation_mutate_rate = 0.0 aggregation_options = sum # node bias options bias_init_mean = 0.0 bias_init_stdev = 1.0 bias_max_value = 30.0 bias_min_value = -30.0 bias_mutate_power = 0.5 bias_mutate_rate = 0.7 bias_replace_rate = 0.1 # genome compatibility options compatibility_disjoint_coefficient = 1.0 compatibility_weight_coefficient = 0.5 # connection add/remove rates conn_add_prob = 0.5 conn_delete_prob = 0.5 # connection enable options enabled_default = True enabled_mutate_rate = 0.1 feed_forward = True initial_connection = full # node add/remove rates node_add_prob = 0.2 node_delete_prob = 0.2 # network parameters num_hidden = 0 num_inputs = 1 num_outputs = 1 # node response options response_init_mean = 1.0 response_init_stdev = 0.0 response_max_value = 30.0 response_min_value = -30.0 response_mutate_power = 0.0 response_mutate_rate = 0.0 response_replace_rate = 0.0 # connection weight options weight_init_mean = 0.0 weight_init_stdev = 1.0 weight_max_value = 30 weight_min_value = -30 weight_mutate_power = 0.5 weight_mutate_rate = 0.8 weight_replace_rate = 0.1 [DefaultSpeciesSet] compatibility_threshold = 3.0 [DefaultStagnation] species_fitness_func = max max_stagnation = 20 species_elitism = 2 [DefaultReproduction] elitism = 2 survival_threshold = 0.2 

Answer: Sorry to tell you that this approach just isn't going to work. Remember that neural networks are typically built out of doing a matrix multiply and then max with 0 (this is called RELU), so basically linear at each layer with a cutoff (and no, picking a different activation like sigmoid is not going to help). You want the network to produce >.5, <.5, >.5, <.5, ... 25 times. Imagine what it would take to build that out of RELU pieces. You would at the very least need a network that is ~25 layers deep, and NEAT is just not going to produce a network that large without consistent incremental progress in the evolution. You are in good company though, what you are doing is equivalent to learning the modulo operator, which has been studied for many years. Here is one post that succeeds, though not using NEAT. Keras: Making a neural network to find a number's modulus The only real progress you could make with NEAT is to give the network many more features as inputs, e.g. give it x%2 as input and it will learn quickly, though this is obviously 'cheating'.