In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

Ex Input:
Stochastic Gradient Descent or SGD is the most popular optimization algorithm for large-scale problems. SGD estimates the gradient by uniform sampling with sample size one. There have been several other works that suggest faster epoch wise convergence by using weighted non-uniform sampling for better gradient estimates. Unfortunately, the per-iteration cost of maintaining this adaptive distribution for gradient estimation is more than calculating the full gradient. As a result, the false impression of faster convergence in iterations leads to slower convergence in time, which we call a chicken-and-egg loop. In this paper, we break this barrier by providing the first demonstration of a sampling scheme, which leads to superior gradient estimation, while keeping the sampling cost per iteration similar to that of the uniform sampling. Such an algorithm is possible due to the sampling view of Locality Sensitive Hashing (LSH), which came to light recently. As a consequence of superior and fast estimation, we reduce the running time of all existing gradient descent algorithms. We demonstrate the benefits of our proposal on both SGD and AdaGrad.

Ex Output:
We improve the running of all existing gradient descent algorithms.


Ex Input:
Normalization methods are a central building block in the deep learning toolbox. They accelerate and stabilize training, while decreasing the dependence on manually tuned learning rate schedules. When learning from multi-modal distributions, the effectiveness of batch normalization (BN), arguably the most prominent normalization method, is reduced. As a remedy, we propose a more flexible approach: by extending the normalization to more than a single mean and variance, we detect modes of data on-the-fly, jointly normalizing samples that share common features. We demonstrate that our method outperforms BN and other widely used normalization techniques in several experiments, including single and multi-task datasets.

Ex Output:
We present a novel normalization method for deep neural networks that is robust to multi-modalities in intermediate feature distributions.


Ex Input:
The fabrication of semiconductor involves etching process to remove selected areas from wafers. However, the measurement of etched structure in micro-graph heavily relies on time-consuming manual routines. Traditional image processing usually demands on large number of annotated data and the performance is still poor. We treat this challenge as segmentation problem and use deep learning approach to detect masks of objects in etched structure of wafer. Then, we use simple image processing to carry out automatic measurement on the objects. We attempt Generative Adversarial Network (GAN) to generate more data to overcome the problem of very limited dataset. We download 10 SEM (Scanning Electron Microscope) images of 4 types from Internet, based on which we carry out our experiments. Our deep learning based method demonstrates superiority over image processing approach with  mean accuracy reaching over 96% for the measurements, compared with the ground truth. To the best of our knowledge, it is the first time that deep learning has been applied in semiconductor industry for automatic measurement.

Ex Output:

Using deep learning method to carry out automatic measurement of SEM images in semiconductor industry