In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
--------
Question: The significant computational costs of deploying neural networks in large-scale or resource constrained environments, such as data centers and mobile devices, has spurred interest in model compression, which can achieve a reduction in both arithmetic operations and storage memory. Several techniques have been proposed for reducing or compressing the parameters for feed-forward and convolutional neural networks, but less is understood about the effect of parameter compression on recurrent neural networks (RNN). In particular, the extent to which the recurrent parameters can be compressed and the impact on short-term memory performance, is not well understood. In this paper, we study the effect of complexity reduction, through singular value decomposition rank reduction, on RNN and minimal gated recurrent unit (MGRU) networks for several tasks. We show that considerable rank reduction is possible when compressing recurrent weights, even without fine tuning. Furthermore, we propose a perturbation model for the effect of general perturbations, such as a compression, on the recurrent parameters of RNNs. The model is tested against a noiseless memorization experiment that elucidates the short-term memory performance. In this way, we demonstrate that the effect of compression of recurrent parameters is dependent on the degree of temporal coherence present in the data and task. This work can guide on-the-fly RNN compression for novel environments or tasks, and provides insight for applying RNN compression in low-power devices, such as hearing aids. Keywordscompression; recurrent neural networks; complexity reduction; SVD; RNN; MGRU; MRU

Answer: Parameter Compression of Recurrent Neural Networks and Degredation of Short-term Memory

Question: Automated learning of patients' demographics can be seen as multilabel problem where a patient model is based on different race and gender groups. The resulting model can be further integrated into Privacy-Preserving Data Mining, where they can be used to assess risk of identification of different patient groups. Our project considers relations between diabetes and demographics of patients as a multi-labelled problem. Most research in this area has been done as binary classification, where the target class is finding if a person has diabetes or not. But very few, and maybe no work has been done in multi-labeled analysis of the demographics of patients who are likely to be diagnosed with diabetes. To identify such groups, we applied ensembles of several multilabel learning algorithms. The best performing multi label ensembles include BR/Hoeffding Tree, CC/Hoeffding Tree, BCC/Hoeffding Tree, BR/JRIP, CC/JRIP, BCC/ JRIP respectively. In the empirical part of this study, we used on the UCI Diabetics dataset of over 100,000 records, collected from 130 US hospitals. The dataset consisted of attributes that included personal demographics, diagnoses code, lab results, etc. Experiments conducted on datasets of 1000, 10000, 20000 samples, show that BR/JRip model achieves a high overall accuracy of 0.533 (1000 samples), 0.702 (10000 samples), 0.569 (20000 samples), improving over the baseline model ZeroR with accuracy of 0.526, 0.586, .562 respectively. Loss functions such as Rank Loss, One Error, Hamming Loss, and Zero One Loss are also low for BCC/JRIP model for all samples of dataset, making it the best candidate for better performance given the label dependencies.

Answer: Multi-Labeled Classification of Demographic Attributes of Patients: a case study of diabetics patients

Question: The k-fold cross-validation is commonly used to evaluate the effectiveness of SVMs with the selected hyperparameters. It is known that the SVM k-fold crossvalidation is expensive, since it requires training k SVMs. However, little work has explored reusing the h SVM for training the (h+1) SVM for improving the efficiency of k-fold cross-validation. In this paper, we propose three algorithms that reuse the h SVM for improving the efficiency of training the (h + 1) SVM. Our key idea is to efficiently identify the support vectors and to accurately estimate their associated weights (also called alpha values) of the next SVM by using the previous SVM. Our experimental results show that our algorithms are several times faster than the k-fold cross-validation which does not make use of the previously trained SVM. Moreover, our algorithms produce the same results (hence same accuracy) as the k-fold cross-validation which does not make use of the previously trained SVM.

Answer:
Improving Efficiency of SVM k-fold Cross-validation by Alpha Seeding