Unsupervised learning with stochastic gradient
A stochastic gradient is formulated based on deterministic gradient augmented with Cauchy simulated annealing capable to reach a global minimum with a convergence speed significantly faster when simulated annealing is used alone. In order to solve space-time variant inverse problems known as blind source separation, a novel Helmholtz free energy contrast function, H=E-T0S, H = E - T 0 S , with imposed thermodynamics constraint at a constant temperature T0 was introduced generalizing the Shannon maximum entropy S of the closed systems to the open systems having non-zero inputoutput energy exchange E. Here, only the input data vector was known while source vector and mixing matrix were unknown. A stochastic gradient was successfully applied to solve inverse space-variant imaging problems on a concurrent pixel-by-pixel basis with the unknown mixing matrix (imaging point spread function) varying from pixel to pixel.
blind source separation helmholtz free energy stochastic optimization cauchy annealing