Query: PRACTICAL HYPERPARAMETER OPTIMIZATION

Relevant passages:
Practical Bayesian Optimization of Machine Learning Algorithms <=SEP=> The use of machine learning algorithms frequently involves careful tuning of learning parameters and model hyperparameters. Unfortunately, this tuning is often a “black art” requiring expert experience, rules of thumb, or sometimes bruteforce search. There is therefore great appeal for automatic approaches that can optimize the performance of any given learning algorithm to the problem at hand. In this work, we consider this problem through the framework of Bayesian optimization, in which a learning algorithm’s generalization performance is modeled as a sample from a Gaussian process (GP). We show that certain choices for the nature of the GP, such as the type of kernel and the treatment of its hyperparameters, can play a crucial role in obtaining a good optimizer that can achieve expertlevel performance. We describe new algorithms that take into account the variable cost (duration) of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation. We show that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms including latent Dirichlet allocation, structured SVMs and convolutional neural networks.

Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid <=SEP=> Naive Bayes induction algorithms were previously shown to be surprisingly accurate on many classi cation tasks even when the conditional independence assumption on which they are based is violated How ever most studies were done on small databases We show that in some larger databases the accuracy of Naive Bayes does not scale up as well as decision trees We then propose a new algorithm NBTree which in duces a hybrid of decision tree classi ers and Naive Bayes classi ers the decision tree nodes contain uni variate splits as regular decision trees but the leaves contain Naive Bayesian classi ers The approach re tains the interpretability of Naive Bayes and decision trees while resulting in classi ers that frequently out perform both constituents especially in the larger databases tested

Bayesian Optimization with Robust Bayesian Neural Networks <=SEP=> Bayesian optimization is a prominent method for optimizing expensive-to-evaluate black-box functions that is widely applied to tuning the hyperparameters of machine learning algorithms. Despite its successes, the prototypical Bayesian optimization approach – using Gaussian process models – does not scale well to either many hyperparameters or many function evaluations. Attacking this lack of scalability and flexibility is thus one of the key challenges of the field. We present a general approach for using flexible parametric models (neural networks) for Bayesian optimization, staying as close to a truly Bayesian treatment as possible. We obtain scalability through stochastic gradient Hamiltonian Monte Carlo, whose robustness we improve via a scale adaptation. Experiments including multi-task Bayesian optimization with 21 tasks, parallel optimization of deep neural networks and deep reinforcement learning show the power and flexibility of this approach.

Taking the Human Out of the Loop: A Review of Bayesian Optimization <=SEP=> Big Data applications are typically associated with systems involving large numbers of users, massive complex software systems, and large-scale heterogeneous computing and storage architectures. The construction of such systems involves many distributed design choices. The end products (e.g., recommendation systems, medical analysis tools, real-time game engines, speech recognizers) thus involve many tunable configuration parameters. These parameters are often specified and hard-coded into the software by various developers or teams. If optimized jointly, these parameters can result in significant improvements. Bayesian optimization is a powerful tool for the joint optimization of design choices that is gaining great popularity in recent years. It promises greater automation so as to increase both product quality and human productivity. This review paper introduces Bayesian optimization, highlights some of its methodological aspects, and showcases a wide range of applications.

Efficient and Robust Automated Machine Learning <=SEP=> The success of machine learning in a broad range of applications has led to an ever-growing demand for machine learning systems that can be used off the shelf by non-experts. To be effective in practice, such systems need to automatically choose a good algorithm and feature preprocessing steps for a new dataset at hand, and also set their respective hyperparameters. Recent work has started to tackle this automated machine learning (AutoML) problem with the help of efficient Bayesian optimization methods. Building on this, we introduce a robust new AutoML system based on scikit-learn (using 15 classifiers, 14 feature preprocessing methods, and 4 data preprocessing methods, giving rise to a structured hypothesis space with 110 hyperparameters). This system, which we dub AUTO-SKLEARN, improves on existing AutoML methods by automatically taking into account past performance on similar datasets, and by constructing ensembles from the models evaluated during the optimization. Our system won the first phase of the ongoing ChaLearn AutoML challenge, and our comprehensive analysis on over 100 diverse datasets shows that it substantially outperforms the previous state of the art in AutoML. We also demonstrate the performance gains due to each of our contributions and derive insights into the effectiveness of the individual components of AUTO-SKLEARN.

Embedded image coding using zerotrees of wavelet coefficients <=SEP=> The embedded zerotree wavelet algorithm (EZW) is a simple, yet remarkably effective, image compression algorithm, having the property that the bits in the bit stream are generated in order of importance, yielding a fully embedded code. The embedded code represents a sequence of binary decisions that distinguish a n image from the “null” image. Using a n embedded coding algorithm, a n encoder can terminate the encoding a t any point thereby allowing a target rate or target distortion metric to be met exactly. Also, given a bit stream, the decoder can cease decoding a t any point in the bit stream and still produce exactly the same image that would have been encoded a t the bit rate corresponding to the truncated bit stream. In addition to producing a fully embedded bit stream, EZW consistently produces compression results that a re competitive with virtually all known compression algorithms on standard test images. Yet this performance is achieved with a technique that requires absolutely no training, no pre-stored tables or codebooks, and requires no prior knowledge of the image source. The EZW algorithm is based on four key concepts: 1) a discrete wavelet transform or hierarchical subband decomposition, 2) prediction of the absence of significant information across scales by exploiting the self-similarity inherent in images, 3) entropy-coded successive-approximation quantization, and 4) universal lossless data compression which is achieved via adaptive arithmetic coding.

The biology of cancer: metabolic reprogramming fuels cell growth and proliferation. <=SEP=> Cell proliferation requires nutrients, energy, and biosynthetic activity to duplicate all macromolecular components during each passage through the cell cycle. It is therefore not surprising that metabolic activities in proliferating cells are fundamentally different from those in nonproliferating cells. This review examines the idea that several core fluxes, including aerobic glycolysis, de novo lipid biosynthesis, and glutamine-dependent anaplerosis, form a stereotyped platform supporting proliferation of diverse cell types. We also consider regulation of these fluxes by cellular mediators of signal transduction and gene expression, including the phosphatidylinositol 3-kinase (PI3K)/Akt/mTOR system, hypoxia-inducible factor 1 (HIF-1), and Myc, during physiologic cell proliferation and tumorigenesis.

Plantar fasciitis: evidence-based review of diagnosis and therapy. <=SEP=> Plantar fasciitis causes heel pain in active as well as sedentary adults of all ages. The condition is more likely to occur in persons who are obese or in those who are on their feet most of the day. A diagnosis of plantar fasciitis is based on the patient's history and physical findings. The accuracy of radiologic studies in diagnosing plantar heel pain is unknown. Most interventions used to manage plantar fasciitis have not been studied adequately; however, shoe inserts, stretching exercises, steroid injection, and custom-made night splints may be beneficial. Extracorporeal shock wave therapy may effectively treat runners with chronic heel pain but is ineffective in other patients. Limited evidence suggests that casting or surgery may be beneficial when conservative measures fail.

Penetration state transition analysis: A rule-based intrusion detection approach <=SEP=> | This paper presents a new approach to representing and detecting computer penetrations in real-time. The approach, called state transition analysis, models penetrations as a series of state changes that lead from an initial secure state to a target compromised state. State transition diagrams, the graphical representation of penetrations, identify precisely the requirements for and the compromise of a penetration and present only the critical events that must occur for the successful completion of the penetration. State transition diagrams are written to correspond to the states of an actual computer system, and these diagrams form the basis of a rule-based expert system for detecting penetrations, called the State Transition Analysis Tool (STAT). The design and implementation of a UNIX-speci c prototype of this expert system, called USTAT, is also presented. This prototype provides a further illustration of the overall design and functionality of this intrusion detection approach. Lastly, STAT is compared to the functionality of comparable intrusion detection tools. Keywords|Security, Intrusion Detection, Expert Systems

Detection and counting of pothole using image processing techniques <=SEP=> Pothole are the primary cause of accidents, hence identification and classification using image processing techniques is very important. In this paper image pre-processing based on difference of Gaussian-Filtering and clustering based image segmentation methods are implemented for better results. From the results the K-Means clustering based segmentation was preferred for its fastest computing time and edge detection based segmentation is preferred for its specificity. The main goal of this paper is to identify a better method which is highly efficient and accurate compared to the conventional methods. Different image pre-processing and segmentation methods for pothole detection where reviewed using performance measures.

Robust facial expression recognition using local binary patterns <=SEP=> A novel low-computation discriminative feature space is introduced for facial expression recognition capable of robust performance over a rang of image resolutions. Our approach is based on the simple local binary patterns (LBP) for representing salient micro-patterns of face images. Compared to Gabor wavelets, the LBP features can be extracted faster in a single scan through the raw image and lie in a lower dimensional space, whilst still retaining facial information efficiently. Template matching with weighted Chi square statistic and support vector machine are adopted to classify facial expressions. Extensive experiments on the Cohn-Kanade Database illustrate that the LBP features are effective and efficient for facial expression discrimination. Additionally, experiments on face images with different resolutions show that the LBP features are robust to low-resolution images, which is critical in real-world applications where only low-resolution video input is available.

1 FEEDFORWARD NEURAL NETWORKS : AN INTRODUCTION <=SEP=> 1 A neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects (Haykin 1998): 1. Knowledge is acquired by the network through a learning process. 2. Interconnection strengths known as synaptic weights are used to store the knowledge. Basically, learning is a process by which the free parameters (i.e., synaptic weights and bias levels) of a neural network are adapted through a continuing process of stimulation by the environment in which the network is embedded. The type of learning is determined by the manner in which the parameter changes take place. In a general sense, the learning process may be classified as follows:

Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention <=SEP=> Multimedia content is dominating today's Web information. The nature of multimedia user-item interactions is 1/0 binary implicit feedback (e.g., photo likes, video views, song downloads, etc.), which can be collected at a larger scale with a much lower cost than explicit feedback (e.g., product ratings). However, the majority of existing collaborative filtering (CF) systems are not well-designed for multimedia recommendation, since they ignore the implicitness in users' interactions with multimedia content. We argue that, in multimedia recommendation, there exists item- and component-level implicitness which blurs the underlying users' preferences. The item-level implicitness means that users' preferences on items (e.g. photos, videos, songs, etc.) are unknown, while the component-level implicitness means that inside each item users' preferences on different components (e.g. regions in an image, frames of a video, etc.) are unknown. For example, a 'view'' on a video does not provide any specific information about how the user likes the video (i.e.item-level) and which parts of the video the user is interested in (i.e.component-level). In this paper, we introduce a novel attention mechanism in CF to address the challenging item- and component-level implicit feedback in multimedia recommendation, dubbed Attentive Collaborative Filtering (ACF). Specifically, our attention model is a neural network that consists of two attention modules: the component-level attention module, starting from any content feature extraction network (e.g. CNN for images/videos), which learns to select informative components of multimedia items, and the item-level attention module, which learns to score the item preferences. ACF can be seamlessly incorporated into classic CF models with implicit feedback, such as BPR and SVD++, and efficiently trained using SGD. Through extensive experiments on two real-world multimedia Web services: Vine and Pinterest, we show that ACF significantly outperforms state-of-the-art CF methods.

3D Mesh Compression: Survey, Comparisons, and Emerging Trends <=SEP=> 3D meshes are commonly used to represent virtual surface and volumes. However, their raw data representations take a large amount of space. Hence, 3D mesh compression has been an active research topic since the mid 1990s. In 2005, two very good review articles describing the pioneering works were published. Yet, new technologies have emerged since then. In this article, we summarize the early works and put the focus on these novel approaches. We classify and describe the algorithms, evaluate their performance, and provide synthetic comparisons. We also outline the emerging trends for future research.

Prescribed performance control of quadruped robot <=SEP=> Due to the complex and high nonlinear structure of the quadruped robots, the dynamic control of quadruped robots has long been a big challenge for the researchers. In this paper, a guaranteed performance adaptive control algorithm is proposed for tracking control of a quadruped robot with parameter uncertainties. The controller is designed based on the sliding mode theory, and incorporates a transformed error, which includes the performance indices. Parameter uncertainties are estimated by a linear network. By choosing a proper Lyapunov function, the prescribed performance with the proposed controller is proven in the scenes of the parameter uncertainties and the environment constraint. Numerical simulations demonstrate the effectiveness and superiority of the proposed controller.

Largest inscribed rectangles in convex polygons <=SEP=> We consider approximation algorithms for the problem of computing an inscribed rectangle having largest area in a convex polygon on n vertices. If the order of the vertices of the polygon is given, we present a randomized algorithm that computes an inscribed rectangle with area at least (1− ) times the optimum with probability t in time O( 1 log n) for any constant t < 1. We further give a deterministic approximation algorithm that computes an inscribed rectangle of area at least (1− ) times the optimum in running time O( 1 2 log n) and show how this running time can be slightly improved.

Self-enucleation in a young schizophrenic patient--a case report. <=SEP=> Self-enucleation represents an extreme but fortunately rare form of deliberate self-harm. Case reports of patients who self-enucleate reveal some common features. A case of auto-enucleation in a young schizophrenic patient and a short discussion on deliberate self-harm are presented.

Random Forests and VGG-NET: An Algorithm for the ISIC 2017 Skin Lesion Classification Challenge <=SEP=> This manuscript briefly describes an algorithm developed for the ISIC 2017 Skin Lesion Classification Competition. In this task, participants are asked to complete two independent binary image classification tasks that involve three unique diagnoses of skin lesions (melanoma, nevus, and seborrheic keratosis). In the first binary classification task, participants are asked to distinguish between (a) melanoma and (b) nevus and seborrheic keratosis. In the second binary classification task, participants are asked to distinguish between (a) seborrheic keratosis and (b) nevus and melanoma. The other phases of the competition are not considered. Our proposed algorithm consists of three steps: preprocessing, classification using VGG-NET [2] and Random Forests [3], and calculation of a final score.

A theory of visual attention. <=SEP=> A unified theory of visual recognition and attentional selection is developed by integrating the biased-choice model for single-stimulus recognition (Luce, 1963; Shepard, 1957) with a choice model for selection from multielement displays (Bundesen, Pedersen, & Larsen, 1984) in a race model framework. Mathematically, the theory is tractable, and it specifies the computations necessary for selection. The theory is applied to extant findings from a broad range of experimental paradigms. The findings include effects of object integrality in selective report, number and spatial position of targets in divided-attention paradigms, selection criterion and number of distracters in focused-attention paradigms, delay of selection cue in partial report, and consistent practice in search. On the whole, the quantitative fits are encouraging.

Combining minutiae descriptors for fingerprint matching <=SEP=> A novel minutiae-based fingerprint matching algorithm is proposed. A minutiae matching algorithm has to solve two problems: correspondence and similarity computation. For the correspondence problem, we assign each minutia two descriptors: texture-based and minutiae-based descriptors, and use an alignment-based greedy matching algorithm to establish the correspondences between minutiae. For the similarity computation, we extract a 17-D feature vector from the matching result, and convert the feature vector into a matching score using support vector classifier. The proposed algorithm is tested on FVC2002 databases and compared to all participators in FVC2002. According to equal error rate, the proposed algorithm ranks 1st on DB3, the most difficult database in FVC2002, and on the average ranks 2nd on all 4 databases. 2007 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

A MATLAB-based simulation program for indoor visible light communication system <=SEP=> We report a simulation program for indoor visible light communication environment based on MATLAB and Simulink. The program considers the positions of the transmitters and the reflections at each wall. For visible light communication environment, the illumination light-emitting diode is used not only as a lighting device, but also as a communication device. Using the simulation program, the distributions of illuminance and root-mean-square delay spread are analyzed at bottom surface.

Cognitive Biases in Visualizations <=SEP=> 

Automatic creation of WordNets from parallel corpora <=SEP=> In this paper we present the evaluation results for the creation of WordNets for five languages (Spanish, French, German, Italian and Portuguese) using an approach based on parallel corpora. We have used three very large parallel corpora for our experiments: DGT-TM, EMEA and ECB. The English part of each corpus is semantically tagged using Freeling and UKB. After this step, the process of WordNet creation is converted into a word alignment problem, where we want to align WordNet synsets in the English part of the corpus with lemmata on the target language part of the corpus. The word alignment algorithm used in these experiments is a simple most frequent translation algorithm implemented into the WN-Toolkit. The obtained precision values are quite satisfactory, but the overall number of extracted synset-variant pairs is too low, leading into very poor recall values. In the conclusions, the use of more advanced word alignment algorithms, such as Giza++, Fast Align or Berkeley aligner is suggested.

Perception on the Usage of Mobile Assisted Language Learning ( MALL ) in English as a Second Language ( ESL ) Learning among Vocational College Students <=SEP=> Mobile phones are viewed as significant aids to English language learning. Numerous studies have been conducted in different contexts on mobile assisted language learning (MALL) that indicate the benefits of mobile devices for English language learning. Nevertheless, few studies on MALL had been conducted on among vocational college students in Malaysia. Therefore, this study was aimed to investigate the perception on the usage of MALL in English as a Second Language (ESL) among private vocational college students. Data were collected from a survey questionnaire adapted from Technology Acceptance Model (TAM). The results show the respondents have positive perception on the usage of MALL. Majority of the respondents showed overall agreement on both constructs perceived usefulness (PU) and perceived ease of use (PEoU) of MALL. They believed the usage of MALL will enhance the teaching and learning process. This evidence of acceptance has implication for educators and curriculum designers to exploit the mobile phone for autonomous and interactive ESL learning beyond the classroom context. It is hoped that MALL will be used as one of the teaching aids that could assist educators to teach English as a Second Language (ESL) more effectively.

Preliminary safety assessment for a sectorless ATM concept <=SEP=> In a sectorless air traffic management concept the airspace is no longer divided into sectors but regarded as one piece. A number of aircraft, not necessarily in the same airspace region, are assigned to each air traffic controller who is then responsible for these aircraft from their entry into the airspace to their exit. These individually assigned flights can be anywhere in the airspace and therefore in different traffic situations. This means that air traffic controllers will manage flights which may not be geographically connected. Such a concept change will necessitate also a significant change in the controllers' routines and the support tools. Naturally, the question of safety arises regarding new procedures and systems. This paper provides a preliminary safety assessment for a sectorless air traffic management concept. The assessment is based on the Single European Sky ATM Research (SESAR) Safety Reference Material which was originally developed for SESAR purposes. This success-based approach stresses the positive contribution of a new concept while traditional approaches mainly consider the negative effect of possible hazards and failures. Based on validation activities including realtime simulations we have developed safety acceptance criteria and safety objectives for a sectorless air traffic management (ATM) concept. Starting from these we have sketched the safety performance requirement model and deduce the first safety requirements for normal conditions, abnormal conditions and in the case of internal system failures.

Two decades of terror management theory: a meta-analysis of mortality salience research. <=SEP=> A meta-analysis was conducted on empirical trials investigating the mortality salience (MS) hypothesis of terror management theory (TMT). TMT postulates that investment in cultural worldviews and self-esteem serves to buffer the potential for death anxiety; the MS hypothesis states that, as a consequence, accessibility of death-related thought (MS) should instigate increased worldview and self-esteem defense and striving. Overall, 164 articles with 277 experiments were included. MS yielded moderate effects (r = .35) on a range of worldview- and self-esteem-related dependent variables (DVs), with effects increased for experiments using (a) American participants, (b) college students, (c) a longer delay between MS and the DV, and (d) people-related attitudes as the DV. Gender and self-esteem may moderate MS effects differently than previously thought. Results are compared to other reviews and examined with regard to alternative explanations of TMT. Finally, suggestions for future research are offered.

A Fast and Compact Saliency Score Regression Network Based on Fully Convolutional Network <=SEP=> Visual saliency detection aims at identifying the most visually distinctive parts in an image, and serves as a pre-processing step for a variety of computer vision and image processing tasks. To this end, the saliency detection procedure must be as fast and compact as possible and optimally processes input images in a real time manner. It is an essential application requirement for the saliency detection task. However, contemporary detection methods often utilize some complicated procedures to pursue feeble improvements on the detection precession, which always take hundreds of milliseconds and make them not easy to be applied practically. In this paper, we tackle this problem by proposing a fast and compact saliency score regression network which employs fully convolutional network, a special deep convolutional neural network, to estimate the saliency of objects in images. It is an extremely simplified endto-end deep neural network without any pre-processings and post-processings. When given an image, the network can directly predict a dense full-resolution saliency map (image-to-image prediction). It works like a compact pipeline which effectively simplifies the detection procedure. Our method is evaluated on six public datasets, and experimental results show that it can achieve comparable or better precision performance than the state-of-the-art methods while get a significant improvement in detection speed (35 FPS, processing in real time).

The Evolution to Modern Phased Array Architectures <=SEP=> Phased array technology has been evolving steadily with advances in solid-state microwave integrated circuits, analysis and design tools, and reliable fabrication practices. With significant government investments, the technologies have matured to a point where phased arrays are widely used in military systems. Next-generation phased arrays will employ high levels of digitization, which enables a wide range of improvements in capability and performance. Digital arrays leverage the rapid commercial evolution of digital processor technology. The cost of phased arrays can be minimized by utilizing high-volume commercial microwave manufacturing and packaging techniques. Dramatic cost reductions are achieved by employing a tile array architecture, which greatly reduces the number of printed circuit boards and connectors in the array.

Spectrum Survey in Singapore: Occupancy Measurements and Analyses <=SEP=> We study the 24-hour spectrum usage pattern in Singapore in the frequency bands ranging from 80 MHz to 5850 MHz. The objectives are to find how the scarce radio spectrum allocated to different services is utilized in Singapore and identify the bands that could be accessed for future opportunistic use due to their low or no active utilization. The results from the spectrum measurements taken over 12 weekday periods reveal that a significant amount of spectrum in Singapore has very low occupancy all the time. The occupancy is quantified as the amount of spectrum detected above a certain received power threshold. The outcome of this study suggests that Singapore has a great potential for employing emerging spectrum sharing technology such as the cognitive radio technology to accommodate enormous demands for future wireless services. However, this study of spectrum survey is preliminary in its nature and future long term studies need to be performed to determine any potential secondary usage on those channels that have low or no active utilization.

Nonlinear Dynamics of Spring Softening and Hardening in Folded-MEMS Comb Drive Resonators <=SEP=> This paper studies analytically and numerically the spring softening and hardening phenomena that occur in electrostatically actuated microelectromechanical systems comb drive resonators utilizing folded suspension beams. An analytical expression for the electrostatic force generated between the combs of the rotor and the stator is derived and takes into account both the transverse and longitudinal capacitances present. After formulating the problem, the resulting stiff differential equations are solved analytically using the method of multiple scales, and a closed-form solution is obtained. Furthermore, the nonlinear boundary value problem that describes the dynamics of inextensional spring beams is solved using straightforward perturbation to obtain the linear and nonlinear spring constants of the beam. The analytical solution is verified numerically using a Matlab/Simulink environment, and the results from both analyses exhibit excellent agreement. Stability analysis based on phase plane trajectory is also presented and fully explains previously reported empirical results that lacked sufficient theoretical description. Finally, the proposed solutions are, once again, verified with previously published measurement results. The closed-form solutions provided are easy to apply and enable predicting the actual behavior of resonators and gyroscopes with similar structures.