We introduce a graduated assignment approach to neural network optimization. Operating within a deterministic annealing framework, at each temperature, we solve a series of assignment problems which are obtained by linearizing the quadratic assignment problem around the current estimate. The softassign, a new technique for constraint satisfaction, is used to solve each assignment problem. Improvements in speed, accuracy, convergence to feasible solutions and constraint satisfaction are reported for three problems, graph matching, Euclidean TSP and graph partitioning, all of which have permutation matrix like constraints. The problem of finding the size of the largest clique in an undirected graph is NP-hard, even to approximate well.
Simple algorithms work quite well however on random graphs.
The experimental results are as follows. PBIL explicitly maintains the statistics contained in a GA's population, but abstracts away the crossover operator and redefines the role of the population. PBIL is based upon simple hebbian weight updates and supervised competitive learning. Empirical results indicate that PBIL is faster and more effective than a standard GA on the problems attempted. Problems included several instances of TSP, jobshop scheduling, knapsack, binpacking, and standard numerical function optimization.