Query: Detecting Webshell Based on Random Forest with FastText

Relevant passages:
Bag of Tricks for Efficient Text Classification <=SEP=> This paper explores a simple and efficient baseline for text classification. Our experiments show that our fast text classifier fastText is often on par with deep learning classifiers in terms of accuracy, and many orders of magnitude faster for training and evaluation. We can train fastText on more than one billion words in less than ten minutes using a standard multicore CPU, and classify half a million sentences among 312K classes in less than a minute.

Convolutional Neural Networks for Sentence Classification <=SEP=> We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the architecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification.

A Matrix Decomposition based Webshell Detection Method <=SEP=> WebShell is a web based network backdoor. With the help of WebShells, the hacker can take any control of the web services illegally. The current method of detecting WebShells is just matching the eigenvalues or detecting the produced flow or services, which is hard to find new kinds of WebShells. To solve these problems, this paper analyzes the different features of a page and proposes a novel matrix decomposition based WebShell detection algorithm. The algorithm is a supervised machine learning algorithm. By analyzing and learning features of known existing and non-existing WebShell pages, the algorithm can make predictions on the unknown pages. The experimental results show that, compared with traditional detection methods, this algorithm spends less time, has higher accuracy and recall rate, and can detect new kinds of WebShells with a certain probability, overcoming the shortcomings of the traditional feature matching based method, improving the accuracy and recalling rate of WebShell detection.

Classification and Regression by randomForest <=SEP=> Recently there has been a lot of interest in “ensemble learning” — methods that generate many classifiers and aggregate their results. Two well-known methods are boosting (see, e.g., Shapire et al., 1998) and bagging Breiman (1996) of classification trees. In boosting, successive trees give extra weight to points incorrectly predicted by earlier predictors. In the end, a weighted vote is taken for prediction. In bagging, successive trees do not depend on earlier trees — each is independently constructed using a bootstrap sample of the data set. In the end, a simple majority vote is taken for prediction. Breiman (2001) proposed random forests, which add an additional layer of randomness to bagging. In addition to constructing each tree using a different bootstrap sample of the data, random forests change how the classification or regression trees are constructed. In standard trees, each node is split using the best split among all variables. In a random forest, each node is split using the best among a subset of predictors randomly chosen at that node. This somewhat counterintuitive strategy turns out to perform very well compared to many other classifiers, including discriminant analysis, support vector machines and neural networks, and is robust against overfitting (Breiman, 2001). In addition, it is very user-friendly in the sense that it has only two parameters (the number of variables in the random subset at each node and the number of trees in the forest), and is usually not very sensitive to their values. The randomForest package provides an R interface to the Fortran programs by Breiman and Cutler (available at http://www.stat.berkeley.edu/ users/breiman/). This article provides a brief introduction to the usage and features of the R functions.

Webshell detection techniques in web applications <=SEP=> With widely adoption of online services, malicious web sites have become a malignant tumor of the Internet. Through system vulnerabilities, attackers can upload malicious files (which are also called webshells) to web server to create a backdoor for hackers' further attacks. Therefore, finding and detecting webshell inside web application source code are crucial to secure websites. In this paper, we propose a novel method based on the optimal threshold values to identify files that contain malicious codes from web applications. Our detection system will scan and look for malicious codes inside each file of the web application, and automatically give a list of suspicious files and a detail log analysis table of each suspicious file for administrators to check further. The Experimental results show that our approach is applicable to identify webshell more efficient than some other approaches.

Countering kernel rootkits with lightweight hook protection <=SEP=> Kernel rootkits have posed serious security threats due to their stealthy manner. To hide their presence and activities, many rootkits hijack control flows by modifying control data or hooks in the kernel space. A critical step towards eliminating rootkits is to protect such hooks from being hijacked. However, it remains a challenge because there exist a large number of widely-scattered kernel hooks and many of them could be dynamically allocated from kernel heap and co-located together with other kernel data. In addition, there is a lack of flexible commodity hardware support, leading to the socalled protection granularity gap -- kernel hook protection requires byte-level granularity but commodity hardware only provides page level protection.
 To address the above challenges, in this paper, we present HookSafe, a hypervisor-based lightweight system that can protect thousands of kernel hooks in a guest OS from being hijacked. One key observation behind our approach is that a kernel hook, once initialized, may be frequently "read"-accessed, but rarely write"-accessed. As such, we can relocate those kernel hooks to a dedicated page-aligned memory space and then regulate accesses to them with hardware-based page-level protection. We have developed a prototype of HookSafe and used it to protect more than 5,900 kernel hooks in a Linux guest. Our experiments with nine real-world rootkits show that HookSafe can effectively defeat their attempts to hijack kernel hooks. We also show that HookSafe achieves such a large-scale protection with a small overhead (e.g., around 6% slowdown in performance benchmarks).

Social Collaborative Viewpoint Regression with Explainable Recommendations <=SEP=> A recommendation is called explainable if it not only predicts a numerical rating for an item, but also generates explanations for users' preferences. Most existing methods for explainable recommendation apply topic models to analyze user reviews to provide descriptions along with the recommendations they produce. So far, such methods have neglected user opinions and influences from social relations as a source of information for recommendations, even though these are known to improve the rating prediction.
 In this paper, we propose a latent variable model, called social collaborative viewpoint regression (sCVR), for predicting item ratings based on user opinions and social relations. To this end, we use so-called viewpoints, represented as tuples of a concept, topic, and a sentiment label from both user reviews and trusted social relations. In addition, such viewpoints can be used as explanations. We apply a Gibbs EM sampler to infer posterior distributions of sCVR. Experiments conducted on three large benchmark datasets show the effectiveness of our proposed method for predicting item ratings and for generating explanations.

Feature hashing for large scale multitask learning <=SEP=> Empirical evidence suggests that hashing is an effective strategy for dimensionality reduction and practical nonparametric estimation. In this paper we provide exponential tail bounds for feature hashing and show that the interaction between random subspaces is negligible with high probability. We demonstrate the feasibility of this approach with experimental results for a new use case --- multitask learning with hundreds of thousands of tasks.

PERFORMANCE ANALYSIS OF CHAOTIC CHIRP SPREAD SPECTRUM SYSTEM IN MULTIPATH ENVIRONMENT <=SEP=> Wireless channels often include multipath propagation, in which the signal has more than one path from the transmitter to the receiver. Atmospheric reflection or refraction, as well as reflection from the ground or from objects such as buildings may cause this phenomena. The aim of this research is to study the effects of multipath environment on the performance of a proposed chaotic chirp spread spectrum system (CCSSS). The benchmark performance of the proposed system is investigated in term of bit error rate (BER) under multipath environment. A Quadrature Phase Shift Keying (QPSK) modulation scheme and a matched filter based on adaptive threshold decision were used in this work. To assess the performance of this system, simulations were performed to compare the proposed system with the traditional frequency modulation differential chaotic shift keying (DCSK) spread spectrum system. The proposed system may be considered as a well qualified communication system in multipath environment.

Knowledge Technologies <=SEP=> Several technologies are emerging that provide new ways to capture, store, present and use knowledge. This book is the first to provide a comprehensive introduction to five of the most important of these technologies: Knowledge Engineering, Knowledge Based Engineering, Knowledge Webs, Ontologies and Semantic Webs. For each of these, answers are given to a number of key questions (What is it? How does it operate? How is a system developed? What can it be used for? What tools are available? What are the main issues?). The book is aimed at students, researchers and practitioners interested in Knowledge Management, Artificial Intelligence, Design Engineering and Web Technologies. During the 1990s, Nick worked at the University of Nottingham on the application of AI techniques to knowledge management and on various knowledge acquisition projects to develop expert systems for military applications. In 1999, he joined Epistemics where he worked on numerous knowledge projects and helped establish knowledge management programmes at large organisations in the engineering, technology and legal sectors. He is author of the book"Knowledge Acquisition in Practice", which describes a step-by-step procedure for acquiring and implementing expertise. He maintains strong links with leading research organisations working on knowledge technologies, such as knowledge-based engineering, ontologies and semantic technologies.

From goals to components: a combined approach to self-management <=SEP=> Autonomous or semi-autonomous systems are deployed in environments where contact with programmers or technicians is infrequent or undesirable. To operate reliably, such systems should be able to adapt to new circumstances on their own. This paper describes our combined approach for adaptable software architecture and task synthesis from high-level goals, which is based on a three-layer model. In the uppermost layer, reactive plans are generated from goals expressed in a temporal logic. The middle layer is responsible for plan execution and assembling a configuration of domain-specific software components, which reside in the lowest layer. Moreover, the middle layer is responsible for selecting alternative components when the current configuration is no longer viable for the circumstances that have arisen. The implementation demonstrates that the approach enables us to handle non-determinism in the environment and unexpected failures in software components.

Support Vector Machines and Kernel Methods: The New Generation of Learning Machines <=SEP=> algorithms, utilize techniques from optimization, statistics, and functional analysis to achieve maximal generality, flexibility, and performance. These algorithms are different from earlier techniques used in machine learning in many respects: For example, they are explicitly based on a theoretical model of learning rather than on loose analogies with natural learning systems or other heuristics. They come with theoretical guarantees about their performance and have a modular design that makes it possible to separately implement and analyze their components. They are not affected by the problem of local minima because their training amounts to convex optimization. In the last decade, a sizable community of theoreticians and practitioners has formed around these methods, and a number of practical applications have been realized. Although the research is not concluded, already now kernel methods are considered the state of the art in several machine learning tasks. Their ease of use, theoretical appeal, and remarkable performance have made them the system of choice for many learning problems. Successful applications range from text categorization to handwriting recognition to classification of geneexpression data.

A survey on unmanned aerial vehicle collision avoidance systems <=SEP=> Collision avoidance is a key factor in enabling the integration of unmanned aerial vehicle into real life use, whether it is in military or civil application. For a long time there have been a large number of works to address this problem; therefore a comparative summary of them would be desirable. This paper presents a survey on the major collision avoidance systems developed in up to date publications. Each collision avoidance system contains two main parts: sensing and detection, and collision avoidance. Based on their characteristics each part is divided into different categories; and those categories are explained, compared and discussed about advantages and disadvantages in this paper.

No-reference perceptual quality assessment of JPEG compressed images <=SEP=> Human observers can easily assess the quality of a distorted image without examining the original image as a reference. By contrast, designing objective No-Reference (NR) quality measurement algorithms is a very difficult task. Currently, NR quality assessment is feasible only when prior knowledge about the types of image distortion is available. This research aims to develop NR quality measurement algorithms for JPEG compressed images. First, we established a JPEG image database and subjective experiments were conducted on the database. We show that Peak Signalto-Noise Ratio (PSNR), which requires the reference images, is a poor indicator of subjective quality. Therefore, tuning an NR measurement model towards PSNR is not an appropriate approach in designing NR quality metrics. Furthermore, we propose a computational and memory efficient NR quality assessment model for JPEG images. Subjective test results are used to train the model, which achieves good quality prediction performance. A Matlab implementation of the proposed method is available at http://anchovy.ece.ute xas.edu/ ̃zwang/research/nr jpeg quality/index.html .

Binarization-free OCR for historical documents using LSTM networks <=SEP=> A primary preprocessing block of almost any typical OCR system is binarization, through which it is intended to remove unwanted part of the input image, and only keep a binarized and cleaned-up version for further processing. The binarization step does not, however, always perform perfectly, and it can happen that binarization artifacts result in important information loss, by for instance breaking or deforming character shapes. In historical documents, due to a more dominant presence of noise and other sources of degradations, the performance of binarization methods usually deteriorates; as a result the performance of the recognition pipeline is hindered by such preprocessing phases. In this paper, we propose to skip the binarization step by directly training a 1D Long Short Term Memory (LSTM) network on gray-level text lines. We collect a large set of historical Fraktur documents, from publicly available online sources, and form train and test sets for performing experiments on both gray-level and binarized text lines. In order to observe the impact of resolution, the experiments are carried out on two identical sets of low and high resolutions. Overall, using gray-level text lines, the 1D LSTM network can reach 24% and 19% lower error rates on the low- and high-resolution sets, respectively, compared to the case of using binarization in the recognition pipeline.

A3: a coding guideline for HCI+autism research using video annotation <=SEP=> Due to the profile of strengths and weaknesses indicative of autism spectrum disorders (ASD), technology may play a key role in ameliorating communication difficulties with this population. This paper documents coding guidelines established through cross-disciplinary work focused on facilitating communication development in children with ASD using computerized feedback. The guidelines, referred to as A3 (pronounced A-Cubed) or Annotation for ASD Analysis, define and operationalize a set of dependent variables coded via video annotation. Inter-rater reliability data are also presented from a study currently in-progress, as well as related discussion to help guide future work in this area. The design of the A3 methodology is well-suited for the examination and evaluation of the behavior of low-functioning subjects with ASD who interact with technology.

“Unexpected Item in the Bagging Area”: Anomaly Detection in X-Ray Security Images <=SEP=> The role of anomaly detection in X-ray security imaging, as a supplement to targeted threat detection, is described, and a taxonomy of anomaly types in this domain is presented. Algorithms are described for detecting appearance anomalies of shape, texture, and density, and semantic anomalies of object category presence. The anomalies are detected on the basis of representations extracted from a convolutional neural network pre-trained to identify object categories in photographs, from the final pooling layer for appearance anomalies, and from the logit layer for semantic anomalies. The distribution of representations in normal data is modeled using high-dimensional, full-covariance, Gaussians, and anomalies are scored according to their likelihood relative to those models. The algorithms are tested on X-ray parcel images using stream-of-commerce data as the normal class, and parcels with firearms present the examples of anomalies to be detected. Despite the representations being learned for photographic images and the varied contents of stream-of-commerce parcels, the system, trained on stream-of-commerce images only, is able to detect 90% of firearms as anomalies, while raising false alarms on 18% of stream-of-commerce.

Smart home design based on ZigBee wireless sensor network <=SEP=> This work is based on the Internet of Things and ZigBee wireless sensor network technology. A kind of smart home design based on ZigBee wireless sensor network was proposed in this paper. Texas Instruments MCU device LM3S9B96, which is the ARM Cortex-M3 based controllers, was used in this system. The entire system is running on the μC/OS-II embedded real-time multitasking operating system. Users can access this system by a dynamic webpage of LwIP TCP/IP protocol stack or GSM SMS. Using this system users can conveniently know the environment parameters of home, such as temperature, humidity, meter readings, light, and control the home electronic equipments, such as light, aircondition, heater, by ZigBee wireless sensor network.

Aiding in the Treatment of Low Back Pain by a Fuzzy Linguistic Web System <=SEP=> Low back pain affects a large proportion of the adult population at some point in their lives and has a major economic and social impact. To soften this impact, one possible solution is to make use of recommender systems, which have already been introduced in several health fields. In this paper, we present TPLUFIB-WEB, a novel fuzzy linguistic Web system that uses a recommender system to provide personalized exercises to patients with low back pain problems and to offer recommendations for their prevention. This system may be useful to reduce the economic impact of low back pain, help professionals to assist patients, and inform users on low back pain prevention measures. A strong part of TPLUFIB-WEB is that it satisfies the Web quality standards proposed by the Health On the Net Foundation (HON), Official College of Physicians of Barcelona, and Health Quality Agency of the Andalusian Regional Government, endorsing the health information provided and warranting the trust of users.

Revisiting Batch Normalization For Practical Domain Adaptation <=SEP=> Deep neural networks (DNN) have shown unprecedented success in various computer vision applications such as image classification and object detection. However, it is still a common annoyance during the training phase, that one has to prepare at least thousands of labeled images to fine-tune a network to a specific domain. Recent study (Tommasi et al. 2015) shows that a DNN has strong dependency towards the training dataset, and the learned features cannot be easily transferred to a different but relevant task without finetuning. In this paper, we propose a simple yet powerful remedy, called Adaptive Batch Normalization (AdaBN) to increase the generalization ability of a DNN. By modulating the statistics in all Batch Normalization layers across the network, our approach achieves deep adaptation effect for domain adaptation tasks. In contrary to other deep learning domain adaptation methods, our method does not require additional components, and is parameter-free. It archives stateof-the-art performance despite its surprising simplicity. Furthermore, we demonstrate that our method is complementary with other existing methods. Combining AdaBN with existing domain adaptation treatments may further improve model performance.

Behavior of Machine Learning Algorithms in Adversarial Environments <=SEP=> Behavior of Machine Learning Algorithms in Adversarial Environments

Connecting modalities: Semi-supervised segmentation and annotation of images using unaligned text corpora <=SEP=> We propose a semi-supervised model which segments and annotates images using very few labeled images and a large unaligned text corpus to relate image regions to text labels. Given photos of a sports event, all that is necessary to provide a pixel-level labeling of objects and background is a set of newspaper articles about this sport and one to five labeled images. Our model is motivated by the observation that words in text corpora share certain context and feature similarities with visual objects. We describe images using visual words, a new region-based representation. The proposed model is based on kernelized canonical correlation analysis which finds a mapping between visual and textual words by projecting them into a latent meaning space. Kernels are derived from context and adjective features inside the respective visual and textual domains. We apply our method to a challenging dataset and rely on articles of the New York Times for textual features. Our model outperforms the state-of-the-art in annotation. In segmentation it compares favorably with other methods that use significantly more labeled training data.

Diffusion of innovations in service organizations: systematic review and recommendations. <=SEP=> This article summarizes an extensive literature review addressing the question, How can we spread and sustain innovations in health service delivery and organization? It considers both content (defining and measuring the diffusion of innovation in organizations) and process (reviewing the literature in a systematic and reproducible way). This article discusses (1) a parsimonious and evidence-based model for considering the diffusion of innovations in health service organizations, (2) clear knowledge gaps where further research should be focused, and (3) a robust and transferable methodology for systematically reviewing health service policy and management. Both the model and the method should be tested more widely in a range of contexts.

Ultra-low on-resistance LDMOS implementation in 0.13µm CD and BiCD process technologies for analog power IC's <=SEP=> Toshiba's 5th generation BiCD/CD-0.13 is a new process platform for analog power applications based on 0.13µm CMOS technology. The process platform has six varieties of rated voltage, 5V, 6V, 18V, 25V, 40V, and 60V. 5 to 18V CD-0.13 process use P-type silicon substrate. 25 to 60V BiCD-0.13 process use N-Epi wafer with N+/P+ buried layer on P type silicon substrate. Each LDMOS recode ultra-low on-resistance compared with that of previous papers, and we will realize the highest performance analog power IC's using this technology.

Going Wild: Large-Scale Classification of Open DNS Resolvers <=SEP=> Since several years, millions of recursive DNS resolvers are-deliberately or not-open to the public. This, however, is counter-intuitive, since the operation of such openly accessible DNS resolvers is necessary in rare cases only. Furthermore, open resolvers enable both amplification DDoS and cache snooping attacks, and can be abused by attackers in multiple other ways. We thus find open recursive DNS resolvers to remain one critical phenomenon on the Internet.
 In this paper, we illuminate this phenomenon by analyzing it from two different angles. On the one hand, we study the landscape of DNS resolvers based on empirical data we collected for over a year. We analyze the changes over time and classify the resolvers according to device type and software version. On the other hand, we take the viewpoint of a client and measure the response authenticity of these resolvers. Besides legitimate redirections (e.g., to captive portals or router login pages), we find millions of resolvers to deliberately manipulate DNS resolutions (i.e., return bogus IP address information). To understand this threat in more detail, we systematically analyze non-legitimate DNS responses and reveal open DNS resolvers that manipulate DNS resolutions to censor communication channels, inject advertisements, serve malicious files, perform phishing, or redirect to other kinds of suspicious or malicious activities.

The evolutionary significance of polyploidy <=SEP=> Polyploidy, or the duplication of entire genomes, has been observed in prokaryotic and eukaryotic organisms, and in somatic and germ cells. The consequences of polyploidization are complex and variable, and they differ greatly between systems (clonal or non-clonal) and species, but the process has often been considered to be an evolutionary 'dead end'. Here, we review the accumulating evidence that correlates polyploidization with environmental change or stress, and that has led to an increased recognition of its short-term adaptive potential. In addition, we discuss how, once polyploidy has been established, the unique retention profile of duplicated genes following whole-genome duplication might explain key longer-term evolutionary transitions and a general increase in biological complexity.

Dietary energy density is associated with obesity and the metabolic syndrome in U.S. adults. <=SEP=> OBJECTIVE
Rising obesity rates have been linked to the consumption of energy-dense diets. We examined whether dietary energy density was associated with obesity and related disorders including insulin resistance and the metabolic syndrome.


RESEARCH DESIGN AND METHODS
We conducted a cross-sectional study using nationally representative data of U.S. adults > or =20 years of age from the 1999-2002 National Health and Nutrition Examination Survey (n = 9,688). Dietary energy density was calculated based on foods only. We used a series of multivariate linear regression models to determine the independent association between dietary energy density, obesity measures (BMI [in kilograms per meters squared] and waist circumference [in centimeters]), glycemia, or insulinemia. We used multivariate Poisson regression models to determine the independent association between dietary energy density and the metabolic syndrome as defined by the National Cholesterol and Education Program (Adult Treatment Panel III).


RESULTS
Dietary energy density was independently and significantly associated with higher BMI in women (beta = 0.44 [95% CI 0.14-0.73]) and trended toward a significant association in men (beta = 0.37 [-0.007 to 0.74], P = 0.054). Dietary energy density was associated with higher waist circumference in women (beta = 1.11 [0.42-1.80]) and men (beta = 1.33 [0.46-2.19]). Dietary energy density was also independently associated with elevated fasting insulin (beta = 0.65 [0.18-1.12]) and the metabolic syndrome (prevalence ratio = 1.10 [95% CI 1.03-1.17]).


CONCLUSIONS
Dietary energy density is an independent predictor of obesity, elevated fasting insulin levels, and the metabolic syndrome in U.S. adults. Intervention studies to reduce dietary energy density are warranted.

Performance Evaluation and Optimization of Communication Infrastructure for the Next Generation Air Transportation System <=SEP=> Automatic dependent surveillance-broadcast (ADS-B) is one of the fundamental surveillance technologies to improve the safety, capacity, and efficiency of the national airspace system. ADS-B shares its frequency band with current radar systems that use the same 1,090 MHz band. The coexistence of radar systems and ADS-B systems is a key issue to detect and resolve conflicts in the next generation air transportation system (NextGen). This paper focuses on the performance evaluation of ADS-B with existing radar systems and performance optimization of ADS-B systems to improve the safety and efficiency of conflict detection and resolution in NextGen. We have developed a simulation environment which models the complex interplay among the air traffic load, the radar systems, the ADS-B systems, and the wireless channel. A simple model is used to derive an analytical expression for a performance metric of ADS-B. This model is then used to design an adaptive ADS-B protocol for maximizing the information coverage while guaranteeing reliable and timely communication in air traffic surveillance networks. Simulation results show that the effect of ADS-B interference on the current radar system is negligible. The operational ability of ADS-B meets the performance requirements of conflict detection and resolution in air traffic control. However, upgrades are required in the current radar system for operation within an ADS-B environment since the current radars can significantly degrade the ADS-B performance. Numerical results indicate that the proposed adaptive protocol has the potential to improve the performance of conflict detection and resolution in air traffic control.

Cross-Validation and the Estimation of Conditional Probability Densities <=SEP=> Many practical problems, especially some connected with forecasting, require nonparametric estimation of conditional densities from mixed data. For example, given an explanatory data vector X for a prospective customer, with components that could include the customer’s salary, occupation, age, sex, marital status, and address, a company might wish to estimate the density of the expenditure, Y , that could be made by that person, basing the inference on observations of (X,Y) for previous clients. Choosing appropriate smoothing parameters for this problem can be tricky, not in the least because plug-in rules take a particularly complex form in the case of mixed data. An obvious difficulty is that there exists no general formula for the optimal smoothing parameters. More insidiously, and more seriously, it can be difficult to determine which components of X are relevant to the problem of conditional inference. For example, if the jth component of X is independent of Y , then that component is irrelevant to estimating the density of Y given X, and ideally should be dropped before conducting inference. In this article we show that cross-validation overcomes these difficulties. It automatically determines which components are relevant and which are not, through assigning large smoothing parameters to the latter and consequently shrinking them toward the uniform distribution on the respective marginals. This effectively removes irrelevant components from contention, by suppressing their contribution to estimator variance; they already have very small bias, a consequence of their independence of Y . Cross-validation also yields important information about which components are relevant; the relevant components are precisely those that cross-validation has chosen to smooth in a traditional way, by assigning them smoothing parameters of conventional size. Indeed, cross-validation produces asymptotically optimal smoothing for relevant components, while eliminating irrelevant components by oversmoothing. In the problem of nonparametric estimation of a conditional density, cross-validation comes into its own as a method with no obvious peers.

How Much Attention Do You Need? A Granular Analysis of Neural Machine Translation Architectures <=SEP=> ACL 2018 • Baseline: 6 layer Transformer, 512 hidden units • Data sets: IWSLT'16, WMT'17 • Metrics: BLEU (and METEOR in the paper) • 3 runs, reporting mean and standard deviation Encoder/Decoder Combinations • Architecture Definition Language • Allows easy experimentation of Neural Machine Translation (NMT) architecture variations • Detailed analysis of NMT architecture combinations • Multiple source attention layers and residual feedforward layer are key • Self-attention more important on the source side Introduction