Query: Factored Language Models and Generalized Parallel Backoff

Relevant passages:
Two decades of statistical language modeling: where do we go from here? <=SEP=> Statistical language models estimate the distribution of various natural language phenomena for the purpose of speech recognition and other language technologies. Since the first significant model was proposed in 1980, many attempts have been made to improve the state of the art. We review them, point to a few promising directions, and argue for a Bayesian approach to integration of linguistic theories with data.

SRILM - an extensible language modeling toolkit <=SEP=> SRILM is a collection of C++ libraries, executable programs, and helper scripts designed to allow both production of and experimentation with statistical language models for speech recognition and other applications. SRILM is freely available for noncommercial purposes. The toolkit supports creation and evaluation of a variety of language model types based on N-gram statistics, as well as several related tasks, such as statistical tagging and manipulation of N-best lists and word lattices. This paper summarizes the functionality of the toolkit and discusses its design and implementation, highlighting ease of rapid prototyping, reusability, and combinability of tools.

On structuring probabilistic dependences in stochastic language modelling <=SEP=> 

Inducing Features of Random Fields <=SEP=> We present a technique for constructing random elds from a set of training samples. The learning paradigm builds increasingly complex elds by allowing potential functions, or features, that are supported by increasingly large subgraphs. Each feature has a weight that is trained by minimizing the Kullback-Leibler divergence between the model and the empirical distribution of the training data. A greedy algorithm determines how features are incrementally added to the eld and an iterative scaling algorithm is used to estimate the optimal values of the weights. The random eld models and techniques introduced in this paper di er from those common to much of the computer vision literature in that the underlying random elds are nonMarkovian and have a large number of parameters that must be estimated. Relations to other learning approaches including decision trees and Boltzmann machines are given. As a demonstration of the method, we describe its application to the problem of automatic word classi cation in natural language processing.

Statistical language modeling using the CMU-cambridge toolkit <=SEP=> The CMU Statistical Language Modeling toolkit was re leased in in order to facilitate the construction and testing of bigram and trigram language models It is currently in use in over academic government and industrial laboratories in over countries This paper presents a new version of the toolkit We outline the con ventional language modeling technology as implemented in the toolkit and describe the extra e ciency and func tionality that the new toolkit provides as compared to previous software for this task Finally we give an exam ple of the use of the toolkit in constructing and testing a simple language model

Translational Recommender Networks <=SEP=> Representing relationships as translations in vector space lives at the heart of many neural embedding models such as word embeddings and knowledge graph embeddings. In this work, we study the connections of this translational principle with collaborative ltering algorithms. We propose Translational Recommender Networks (TransRec), a new aentive neural architecture that utilizes the translational principle to model the relationships between user and item pairs. Our model employs a neural aention mechanism over a Latent Relational Aentive Memory (LRAM) module to learn the latent relations between user-item pairs that best explains the interaction. By exploiting adaptive user-item specic translations in vector space, our model also alleviates the geometric inexibility problem of other metric learning algorithms while enabling greater modeling capability and ne-grained ing of users and items in vector space. e proposed architecture not only demonstrates the state-of-the-art performance across multiple recommendation benchmarks but also boasts of improved interpretability. alitative studies over the LRAM module shows evidence that our proposed model is able to infer and encode explicit sentiment, temporal and aribute information despite being only trained on implicit feedback. As such, this ascertains the ability of TransRec to uncover hidden relational structure within implicit datasets.

Nanoconnectomic upper bound on the variability of synaptic plasticity <=SEP=> Information in a computer is quantified by the number of bits that can be stored and recovered. An important question about the brain is how much information can be stored at a synapse through synaptic plasticity, which depends on the history of probabilistic synaptic activity. The strong correlation between size and efficacy of a synapse allowed us to estimate the variability of synaptic plasticity. In an EM reconstruction of hippocampal neuropil we found single axons making two or more synaptic contacts onto the same dendrites, having shared histories of presynaptic and postsynaptic activity. The spine heads and neck diameters, but not neck lengths, of these pairs were nearly identical in size. We found that there is a minimum of 26 distinguishable synaptic strengths, corresponding to storing 4.7 bits of information at each synapse. Because of stochastic variability of synaptic activation the observed precision requires averaging activity over several minutes.

How STEM Education Improves Student Learning <=SEP=> The intent of this article is to examine current taxonomies used to design and deliver Science, Technology, Engineering, and Mathematics (i.e., STEM) curriculum in an effort to attract a greater variety of students to the STEM field of study in the K-12 public school environment. I analyzed specific aspects of STEM-based programs to compare STEM education components to traditional college-preparatory methods of instruction, including looking for environmental practices that may attract female and first-generation college attendees toward developing a positive attitude toward attaining a STEM education. I also made connections between current instructional trends and their impact on student mastery.

Health Information Management System for Elderly Health Sector: A Qualitative Study in Iran. <=SEP=> BACKGROUND
There are increasing change and development of information in healthcare systems. Given the increase in aging population, managers are in need of true and timely information when making decision.


OBJECTIVES
The aim of this study was to investigate the current status of the health information management system for the elderly health sector in Iran.


MATERIALS AND METHODS
This qualitative study was conducted in two steps. In the first step, required documents for administrative managers were collected using the data gathering form and observed and reviewed by the researcher. In the second step, using an interview guide, the required information was gathered through interviewing experts and faculty members. The convenience, purposeful and snowball sampling methods were applied to select interviewees and the sampling continued until reaching the data saturation point. Finally, notes and interviews were transcribed and content analysis was used to analyze them.


RESULTS
The results of the study showed that there was a health information management system for the elderly health sector in Iran. However, in all primary health care centers the documentation of data was done manually; the data flow was not automated; and the analysis and reporting of data are also manually. Eventually, decision makers are provided with delayed information.


CONCLUSIONS
It is suggested that the steward of health in Iran, the ministry of health, develops an appropriate infrastructure and finally puts a high priority on the implementation of the health information management system for elderly health sector in Iran.

A 0.18μm CMOS fully integrated RFDAC and VGA for WCDMA transmitters <=SEP=> A digital IF to RF converter architecture with an RF VGA targeted for WCDMA transmitters is presented. The RFDAC consists of a 1.5-bits current steering DAC with an embedded semi-digital FIR filter and a mixer with an LC load. The VGA controls the output power in 10 dB steps. The prototype is fabricated in 0.18 mum CMOS technology and achieves 0 dBm output power at 2 GHz and 33 dB image rejection. The measured rms EVM and ACPR show the system fulfills the standard specifications for the WCDMA. The chip draws 123.9 mA from 1.8 V supply. The die area is 0.8 times 1.6 mm2.

Personalized Multitask Learning for Predicting Tomorrow ’ s Mood , Stress , and Health <=SEP=> While accurately predicting mood and wellbeing could have a number of important clinical benefits, traditional machine learning (ML) methods frequently yield low performance in this domain. We posit that this is because a one-size-fits-all machine learning model is inherently ill-suited to predicting outcomes like mood and stress, which vary greatly due to individual differences. Therefore, we employ Multitask Learning (MTL) techniques to train personalized ML models which are customized to the needs of each individual, but still leverage data from across the population. Three formulations of MTL are compared: i) MTL deep neural networks, which share several hidden layers but have final layers unique to each task; ii) Multi-task Multi-Kernel learning, which feeds information across tasks through kernel weights on feature types; and iii) a Hierarchical Bayesian model in which tasks share a common Dirichlet Process prior. We offer the code for this work in open source. These techniques are investigated in the context of predicting future mood, stress, and health using data collected from surveys, wearable sensors, smartphone logs, and the weather. Empirical results demonstrate that using MTL to account for individual differences provides large performance improvements over traditional machine learning methods and provides personalized, actionable insights.

Automatically extracting cancer disease characteristics from pathology reports into a Disease Knowledge Representation Model <=SEP=> We introduce an extensible and modifiable knowledge representation model to represent cancer disease characteristics in a comparable and consistent fashion. We describe a system, MedTAS/P which automatically instantiates the knowledge representation model from free-text pathology reports. MedTAS/P is based on an open-source framework and its components use natural language processing principles, machine learning and rules to discover and populate elements of the model. To validate the model and measure the accuracy of MedTAS/P, we developed a gold-standard corpus of manually annotated colon cancer pathology reports. MedTAS/P achieves F1-scores of 0.97-1.0 for instantiating classes in the knowledge representation model such as histologies or anatomical sites, and F1-scores of 0.82-0.93 for primary tumors or lymph nodes, which require the extractions of relations. An F1-score of 0.65 is reported for metastatic tumors, a lower score predominantly due to a very small number of instances in the training and test sets.

Reduced Number of Hypocretin Neurons in Human Narcolepsy <=SEP=> Murine and canine narcolepsy can be caused by mutations of the hypocretin (Hcrt) (orexin) precursor or Hcrt receptor genes. In contrast to these animal models, most human narcolepsy is not familial, is discordant in identical twins, and has not been linked to mutations of the Hcrt system. Thus, the cause of human narcolepsy remains unknown. Here we show that human narcoleptics have an 85%-95% reduction in the number of Hcrt neurons. Melanin-concentrating hormone (MCH) neurons, which are intermixed with Hcrt cells in the normal brain, are not reduced in number, indicating that cell loss is relatively specific for Hcrt neurons. The presence of gliosis in the hypocretin cell region is consistent with a degenerative process being the cause of the Hcrt cell loss in narcolepsy.

A taxonomy and rating system to measure situation awareness in resuscitation teams <=SEP=> Team SA involves a common perspective between two or more individuals regarding current environmental events, their meaning, and projected future status. Team SA has been theorized to be important for resuscitation team effectiveness. Accordingly, multidimensional frameworks of observable behaviors relevant to resuscitation teams are needed to understand more deeply the nature of team SA, its implications for team effectiveness, and whether it can be trained. A seven-dimension team resuscitation SA framework was developed following a literature review and consensus process using a modified Delphi approach with a group of content experts. We applied a pre-post design within a day-long team training program involving four video-recorded simulated resuscitation events and 42 teams across Canada. The first and fourth events represented "pre" and "post" training events, respectively. Teams were scored on SA five times within each 15-minute event. Distractions were introduced to investigate whether SA scores would be affected. The current study provides initial construct validity evidence for a new measure of SA and explicates SA's role in resuscitation teams.

Compact Aspect Embedding for Diversified Query Expansions <=SEP=> Diversified query expansion (DQE) based approaches aim to select a set of expansion terms with less redundancy among them while covering as many query aspects as possible. Recently they have experimentally demonstrate their effectiveness for the task of search result diversification. One challenge faced by existing DQE approaches is to ensure the aspect coverage. In this paper, we propose a novel method for DQE, called compact aspect embedding, which exploits trace norm regularization to learn a low rank vector space for the query, with each eigenvector of the learnt vector space representing an aspect, and the absolute value of its corresponding eigenvalue representing the association strength of that aspect to the query. Meanwhile, each expansion term is mapped into the vector space as well. Based on this novel representation of the query aspects and expansion terms, we design a greedy selection strategy to choose a set of expansion terms to explicitly cover all possible aspects of the query. We test our method on several TREC diversification data sets, and show that our method significantly outperforms the state-of-the-art search result diversification approaches.

LATIONS FOR KNOWLEDGE BASE COMPLETION <=SEP=> In this work we present a novel approach for the utilization of observed relations between entity pairs in the task of triple argument prediction. The approach is based on representing observations in a shared, continuous vector space of structured relations and text. Results on a recent benchmark dataset demonstrate that the new model is superior to existing sparse feature models. In combination with state-of-the-art models, we achieve substantial improvements when observed relations are available.

What's in an Embedding? Analyzing Word Embeddings through Multilingual Evaluation <=SEP=> In the last two years, there has been a surge of word embedding algorithms and research on them. However, evaluation has mostly been carried out on a narrow set of tasks, mainly word similarity/relatedness and word relation similarity and on a single language, namely English. We propose an approach to evaluate embeddings on a variety of languages that also yields insights into the structure of the embedding space by investigating how well word embeddings cluster along different syntactic features. We show that all embedding approaches behave similarly in this task, with dependency-based embeddings performing best. This effect is even more pronounced when generating low dimensional embed-

Integration of a Wireless I/O Interface for PROFIBUS and PROFINET for Factory Automation <=SEP=> Cost related to the wiring of sensors and actuators in the manufacturing industry is rapidly increasing with the number of field devices. Complex engineering as well as cable wear and tear are important factors. For over a decade, fieldbuses have been the dominating approach to simplify installation, reduce engineering efforts, and increase reliability at the field level. The next step in this process is to introduce wireless communication in certain problematic parts of the field network. In order to do this, a reliable real-time communication system, designed to cope with the harsh environments where factory automation systems are employed, is required. The wireless interface for sensors and actuators (WISA), with its two subconcepts WISA-power and WISA-com, is such a system. This paper discusses how the WISA concept can be efficiently integrated into wired field networks for factory automation-both state-of-the-art [process fieldbus decentral peripherals (PROFIBUS DP)] and new (PROFINET IO) networks. We propose amendments to WISA, which will improve the 802.11b/g coexistence and harmonize the integration of WISA for sensor/actuator communication in, e.g., PROFINET IO systems. The suggested improvements relate to both the radio protocol in order to improve coexistence with other wireless technologies as well as concepts for the integration of WISA into the field network of PROFINET/PROFIBUS.

More than Just Two Sexes: The Neural Correlates of Voice Gender Perception in Gender Dysphoria <=SEP=> Gender dysphoria (also known as "transsexualism") is characterized as a discrepancy between anatomical sex and gender identity. Research points towards neurobiological influences. Due to the sexually dimorphic characteristics of the human voice, voice gender perception provides a biologically relevant function, e.g. in the context of mating selection. There is evidence for a better recognition of voices of the opposite sex and a differentiation of the sexes in its underlying functional cerebral correlates, namely the prefrontal and middle temporal areas. This fMRI study investigated the neural correlates of voice gender perception in 32 male-to-female gender dysphoric individuals (MtFs) compared to 20 non-gender dysphoric men and 19 non-gender dysphoric women. Participants indicated the sex of 240 voice stimuli modified in semitone steps in the direction to the other gender. Compared to men and women, MtFs showed differences in a neural network including the medial prefrontal gyrus, the insula, and the precuneus when responding to male vs. female voices. With increased voice morphing men recruited more prefrontal areas compared to women and MtFs, while MtFs revealed a pattern more similar to women. On a behavioral and neuronal level, our results support the feeling of MtFs reporting they cannot identify with their assigned sex.

A Practical Security Architecture for In-Vehicle CAN-FD <=SEP=> The controller area network with flexible data rate (CAN-FD) is attracting attention as the next generation of in-vehicle network technology. However, security issues have not been completely taken into account when designing CAN-FD, although every bit of information transmitted could be critical to driver safety. If we fail to solve the security vulnerabilities of CAN-FD, we cannot expect Vehicle-Information and Communications Technology (Vehicle-ICT) convergence to continue to develop. Fortunately, secure in-vehicle CAN-FD communication environments can be constructed using the larger data payload of CAN-FD. In this paper, we propose a security architecture for in-vehicle CAN-FD as a countermeasure (designed in accordance with CAN-FD specifications). We considered the characteristics of the International Organization for Standardization (ISO) 26262 Automotive Safety Integrity Level and the in-vehicle subnetwork to design a practical security architecture. We also evaluated the feasibility of the proposed security architecture using three kinds of microcontroller unit and the CANoe software. Our evaluation findings may be used as an indicator of the performance level of electronic control units for manufacturing next-generation vehicles.

Attributing physical and biological impacts to anthropogenic climate change <=SEP=> Significant changes in physical and biological systems are occurring on all continents and in most oceans, with a concentration of available data in Europe and North America. Most of these changes are in the direction expected with warming temperature. Here we show that these changes in natural systems since at least 1970 are occurring in regions of observed temperature increases, and that these temperature increases at continental scales cannot be explained by natural climate variations alone. Given the conclusions from the Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment Report that most of the observed increase in global average temperatures since the mid-twentieth century is very likely to be due to the observed increase in anthropogenic greenhouse gas concentrations, and furthermore that it is likely that there has been significant anthropogenic warming over the past 50 years averaged over each continent except Antarctica, we conclude that anthropogenic climate change is having a significant impact on physical and biological systems globally and in some continents.

Design, measurement and equivalent circuit synthesis of high power HF transformer for three-phase composite dual active bridge topology <=SEP=> High voltage high frequency (HF) transformer provides the isolation between high and low dc link voltages in dual active bridge (DAB) converters. Such DAB converters are finding wide applications as an intermediate DC-DC converter in transformerless intelligent power substation (TIPS), which is proposed as an alternative for conventional distribution-transformer connecting 13.8 kV and 480 V grids. The design of HF transformer used in DAB stage of such application is very challenging considering the required isolation, size and cost. In this paper, the specification generation, design, characterization, test and measurement results on a 10kHz HF transformer are presented, highlighting the above challenges.

Analysis of the weighted Chinese air transportation multilayer network <=SEP=> Air transportation system is a crucial infrastructure of modern world. In this paper, we analyze the weighted Chinese air transportation network with the framework of multilayer network, in which each layer is defined by a commercial airline (company) and the weight of links is set respectively by the number of flights, the number of seats and the geographical distance between pairs of airports. It is shown that the size of the airlines can be characterized approximatively by the number of air routes they operated. The numeric results also indicate that the CCA (air China) is of a considerable higher value of maximal degree and betweenness than other top airlines. Our work will be helpful to better understand and model the Chinese air transportation network.

Domain Transfer SVM for video concept detection <=SEP=> Cross-domain learning methods have shown promising results by leveraging labeled patterns from auxiliary domains to learn a robust classifier for target domain, which has a limited number of labeled samples. To cope with the tremendous change of feature distribution between different domains in video concept detection, we propose a new cross-domain kernel learning method. Our method, referred to as Domain Transfer SVM (DTSVM), simultaneously learns a kernel function and a robust SVM classifier by minimizing both the structural risk functional of SVM and the distribution mismatch of labeled and unlabeled samples between the auxiliary and target domains. Comprehensive experiments on the challenging TRECVID corpus demonstrate that DTSVM outperforms existing cross-domain learning and multiple kernel learning methods.

Characterization of a Novel Nine-Transistor SRAM Cell <=SEP=> Data stability of SRAM cells has become an important issue with the scaling of CMOS technology. Memory banks are also important sources of leakage since the majority of transistors are utilized for on-chip caches in today's high performance microprocessors. A new nine-transistor (9T) SRAM cell is proposed in this paper for simultaneously reducing leakage power and enhancing data stability. The proposed 9T SRAM cell completely isolates the data from the bit lines during a read operation. The read static-noise-margin of the proposed circuit is thereby enhanced by 2 X as compared to a conventional six-transistor (6T) SRAM cell. The idle 9T SRAM cells are placed into a super cutoff sleep mode, thereby reducing the leakage power consumption by 22.9% as compared to the standard 6T SRAM cells in a 65-nm CMOS technology. The leakage power reduction and read stability enhancement provided with the new circuit technique are also verified under process parameter variations.

Labelled data collection for anomaly detection in wireless sensor networks <=SEP=> Security of wireless sensor networks (WSN) is an important research area in computer and communications sciences. Anomaly detection is a key challenge in ensuring the security of WSN. Several anomaly detection algorithms have been proposed and validated recently using labeled datasets that are not publicly available. Our group proposed an ellipsoid-based anomaly detection algorithm but demonstrated its performance using synthetic datasets and real Intel Berkeley Research Laboratory and Grand St. Bernard datasets which are not labeled with anomalies. This approach requires manual assignment of the anomalies' positions based on visual estimates for performance evaluation. In this paper, we have implemented a single-hop and multi-hop sensor-data collection network. In both scenarios we generated real labeled data for anomaly detection and identified different types of anomalies. These labeled sensor data and types of anomalies are useful for research, such as machine learning, and this information will be disseminated to the research community.

Performance of LoRa-Based IoT Applications on Campus <=SEP=> To promote smart campus, National Chiao Tung University is deploying several location-based IoT applications based on wireless communications technologies including LoRa and NB-IoT. The applications are indoor/outdoor environment conditions monitoring (such as PM2.5, temperature, and humidity), emergency buttons, and so on. This paper uses PM2.5 measurement application as an example to present the performance of LoRa when it is used on campus. We investigated the LoRa transmission performance from LoRa end-devices to the LoRa gateway. We show how packet losses were affected by the distance between the end-device and the gateway, the transmit power, the payload length, the antenna angle, the time of day, and the weather conditions. The pattern of LoRa packet losses were also measured and we found that in our long-term PM2.5 monitoring application, more than 99\% of LoRa packet losses occurred with three or less consecutive packet losses.

Large-Scale Video Classification with Convolutional Neural Networks <=SEP=> Convolutional Neural Networks (CNNs) have been established as a powerful class of models for image recognition problems. Encouraged by these results, we provide an extensive empirical evaluation of CNNs on large-scale video classification using a new dataset of 1 million YouTube videos belonging to 487 classes. We study multiple approaches for extending the connectivity of a CNN in time domain to take advantage of local spatio-temporal information and suggest a multiresolution, foveated architecture as a promising way of speeding up the training. Our best spatio-temporal networks display significant performance improvements compared to strong feature-based baselines (55.3% to 63.9%), but only a surprisingly modest improvement compared to single-frame models (59.3% to 60.9%). We further study the generalization performance of our best model by retraining the top layers on the UCF-101 Action Recognition dataset and observe significant performance improvements compared to the UCF-101 baseline model (63.3% up from 43.9%).

Mechanisms governing empowerment effects: a self-efficacy analysis. <=SEP=> This experiment tested the hypotheses that perceived coping and cognitive control self-efficacy govern the effects of personal empowerment over physical threats. Women participated in a mastery modeling program in which they mastered the physical skills to defend themselves successfully against unarmed sexual assailants. Multifaceted measures of theoretically relevant variables were administered within a staggered intragroup control design to test the immediate and long-term effects of the empowerment program and the mechanisms through which it produced its effects. Mastery modeling enhanced perceived coping and cognitive control efficacy, decreased perceived vulnerability to assault, and reduced the incidence of intrusive negative thinking and anxiety arousal. These changes were accompanied by increased freedom of action and decreased avoidant behavior. Path analyses of causal structures revealed a dual path of regulation of behavior by perceived coping self-efficacy, one mediated through perceived vulnerability and risk discernment and the other through perceived cognitive control self-efficacy and intrusive negative thinking.

Computationally efficient low-pass FIR filter design using Cuckoo Search with adaptive Levy step size <=SEP=> This paper looks into efficient implementation of one dimensional low pass Finite Impulse Response filters using certain commonly used and state-of-the-art optimization techniques. Methods like Parks-McClellan (PM) equiripple design, Quantum-behaved Particle Swarm Optimization (QPSO) and Cuckoo Search Algorithm (CSA) with Levy Flight are employed and overall performance is further improved by hybridization and adaptive step size update. Various performance metrics are analyzed with a focus on increasing convergence speed to reach global optima faster. It is seen that the improved search methods used in this work, i.e., Simulated Annealing based Weighted Mean Best QPSO (SAWQPSO) and Adaptive CSA (ACSA) effect significant reductions in convergence time with ACSA proving to be the faster one. The designed filter is used in the receiver stage of a Frequency Modulated Radio Transmission model using a Quadrature-Phase Shift Keyed (QPSK) Modulator and Demodulator. Its efficiency is validated by obtaining near perfect correlation between the message and recovered signals.