Query: Collaborative Learning for Deep Neural Networks

Relevant passages:
Temporal Ensembling for Semi-Supervised Learning <=SEP=> In this paper, we present a simple and efficient method for training deep neural networks in a semi-supervised setting where only a small portion of training data is labeled. We introduce self-ensembling, where we form a consensus prediction of the unknown labels using the outputs of the network-in-training on different epochs, and most importantly, under different regularization and input augmentation conditions. This ensemble prediction can be expected to be a better predictor for the unknown labels than the output of the network at the most recent training epoch, and can thus be used as a target for training. Using our method, we set new records for two standard semi-supervised learning benchmarks, reducing the (non-augmented) classification error rate from 18.44% to 7.05% in SVHN with 500 labels and from 18.63% to 16.55% in CIFAR-10 with 4000 labels, and further to 5.12% and 12.16% by enabling the standard augmentations. We additionally obtain a clear improvement in CIFAR-100 classification accuracy by using random images from the Tiny Images dataset as unlabeled extra inputs during training. Finally, we demonstrate good tolerance to incorrect labels.

Gradient descent GAN optimization is locally stable <=SEP=> Despite the growing prominence of generative adversarial networks (GANs), optimization in GANs is still a poorly understood topic. In this paper, we analyze the “gradient descent” form of GAN optimization, i.e., the natural setting where we simultaneously take small gradient steps in both generator and discriminator parameters. We show that even though GAN optimization does not correspond to a convex-concave game (even for simple parameterizations), under proper conditions, equilibrium points of this optimization procedure are still locally asymptotically stable for the traditional GAN formulation. On the other hand, we show that the recently proposed Wasserstein GAN can have non-convergent limit cycles near equilibrium. Motivated by this stability analysis, we propose an additional regularization term for gradient descent GAN updates, which is able to guarantee local stability for both the WGAN and the traditional GAN, and also shows practical promise in speeding up convergence and addressing mode collapse.

Born Again Neural Networks <=SEP=> Knowledge Distillation (KD) consists of transferring “knowledge” from one machine learning model (the teacher) to another (the student). Commonly, the teacher is a high-capacity model with formidable performance, while the student is more compact. By transferring knowledge, one hopes to benefit from the student’s compactness, without sacrificing too much performance. We study KD from a new perspective: rather than compressing models, we train students parameterized identically to their teachers. Surprisingly, these Born-Again Networks (BANs), outperform their teachers significantly, both on computer vision and language modeling tasks. Our experiments with BANs based on DenseNets demonstrate state-of-the-art performance on the CIFAR-10 (3.5%) and CIFAR-100 (15.5%) datasets, by validation error. Additional experiments explore two distillation objectives: (i) Confidence-Weighted by Teacher Max (CWTM) and (ii) Dark Knowledge with Permuted Predictions (DKPP). Both methods elucidate the essential components of KD, demonstrating the effect of the teacher outputs on both predicted and nonpredicted classes.

Learning Internal Representations <=SEP=> Probably the most important, problem in machine learning is the preliminary biasing of a learner’s hypothesis space so that it is small enough to ensure good generalisation from reasonable training sets, yet large enough that it contains a good solution to the problem being learnt, In this paper a mechanism for automailcall,y learning or biasing the learner’s hypothesis space is introduced. It works by first learning an appropriate [nterna! represenlaiton for a learning en~-ironment and then using that representation to hiss the learner’s hypothesis space for the learning of future tasks drawn from the same environment. Jln internal represent at ion must be learnt by sampling from many s2mtiar tasks, not just a single task as occurs in ordinary machine learning. It is proved that the number of examples m per task required to ensure good generalisation from a representation learner obeys }n = O(a + b/n) where n is the number of tasks being learnt, and a and b are constants. If the tasks are learnt independently (/. e. without a common representation ) then m = 0( a + b). It is argued that for learning environments such as speech and character recognition b >> a and hence representation learning in these environments can potentially yield a drastic reduction in the number of examples required per task. It is also proved that if n = O(b) (with m = O(a+b/n ) ) then the representation learnt will be good for learning novel tasks from the same environment, and that the number of examples required to generalise well on a novel task will be reduced to O(a) (as opposed to {)( a + b) if no represent ation is used). It, is shown that gradient descent can be used to train neural network representations and the results of an experiment are reported in which a neural network representation was learnt for an environment consisting of transJaf ion ail,y in uarlanf Boolean functions. The experiment, provides strong qualitative support for the theoretical results.

The Numerics of GANs <=SEP=> In this paper, we analyze the numerics of common algorithms for training Generative Adversarial Networks (GANs). Using the formalism of smooth two-player games we analyze the associated gradient vector field of GAN training objectives. Our findings suggest that the convergence of current algorithms suffers due to two factors: i) presence of eigenvalues of the Jacobian of the gradient vector field with zero real-part, and ii) eigenvalues with big imaginary part. Using these findings, we design a new algorithm that overcomes some of these limitations and has better convergence properties. Experimentally, we demonstrate its superiority on training common GAN architectures and show convergence on GAN architectures that are known to be notoriously hard to train.

Self-Censorship on Facebook <=SEP=> We report results from an exploratory analysis examining “last-minute” self-censorship, or content that is filtered after being written, on Facebook. We collected data from 3.9 million users over 17 days and associate self-censorship behavior with features describing users, their social graph, and the interactions between them. Our results indicate that 71% of users exhibited some level of last-minute self-censorship in the time period, and provide specific evidence supporting the theory that a user’s “perceived audience” lies at the heart of the issue: posts are censored more frequently than comments, with status updates and posts directed at groups censored most frequently of all sharing use cases investigated. Furthermore, we find that: people with more boundaries to regulate censor more; males censor more posts than females and censor even more posts with mostly male friends than do females, but censor no more comments than females; people who exercise more control over their audience censor more content; and, users with more politically and age diverse friends censor less, in general.

On code reuse from StackOverflow: An exploratory study on Android apps <=SEP=> Context: Source code reuse has been widely accepted as a fundamental activity in software development. Recent studies showed that StackOverflow has emerged as one of the most popular resources for code reuse. Therefore, a plethora of work proposed ways to optimally ask questions, search for answers and find relevant code on StackOverflow. However, little work studies the impact of code reuse from Stack-

On the rate of channel polarization <=SEP=> A bound is given on the rate of channel polarization. As a corollary, an earlier bound on the probability of error for polar coding is improved. Specifically, it is shown that, for any binary-input discrete memoryless channel W with symmetric capacity I(W) and any rate R ≪ I(W), the polar-coding block-error probability under successive cancellation decoding satisfies P<inf>e</inf>(N, R) ≤ 2<sup>−Nβ</sup> for any β ≪ 1/2 when the block-length N is large enough.

Natural Language Generation in Interactive Systems <=SEP=> An informative and comprehensive overview of the state-of-the-art in natural language generation (NLG) for interactive systems, this guide introduces graduate students and new researchers to the field of natural language processing and artificial intelligence, while inspiring them with ideas for future research. Detailing the techniques and challenges of NLG for interactive applications, it focuses on research into systems that model collaborativity and uncertainty, are capable of being scaled incrementally, and can engage with the user effectively. A range of real-world case studies is also included. The book and the accompanying website feature a comprehensive bibliography, and refer the reader to corpora, data, software, and other resources for pursuing research on natural language generation and interactive systems, including dialogue systems, multimodal interfaces, and assistive technologies. It is an ideal resource for students and researchers in computational linguistics, natural language processing, and related fields.

Stretching Agile to fit CMMI Level 3 - the story of creating MSF for CMMI Process Improvement at Microsoft Corporation <=SEP=> Agile practitioners pride themselves on highly productive, responsive, low ceremony, lightweight, tacit knowledge processes with little waste, adaptive planning and frequent iterative delivery of value. It is often assumed that CMMI compliant processes need to be heavyweight, bureaucratic, slow moving, high ceremony and plan driven. Agile developers often skeptically perceive formal process improvement initiatives as management generated inefficiency that gets in the way of productivity. At Microsoft, we’ve adopted the teachings of W. Edwards Deming and stretched our MSF for Agile Software Development method to fit the requirements for CMMI Level 3. The resultant MSF for CMMI Process Improvement is a highly iterative, adaptive planning method, light on documentation, and heavily automated through tooling. It enables management and organization of software engineering through use of agile metrics such as velocity and cumulative flow but with an added dimension of an understanding of variation – adapted from Deming’s teachings. This is the story of how mixing Deming with Agile produced a lightweight CMMI solution for .Net developers everywhere.

Face Recognition in a Meeting Room <=SEP=> In this paper, weinvestigaterecognition of humanfaces in a meetingroom. The major challenges of identifying humanfacesin this environmentincludelow quality of input images,poor illumination,unrestrictedheadposesand continuouslychangingfacial expressionsandocclusion.In order to addresstheseproblemswe proposea novel algorithm, DynamicSpaceWarping (DSW).Thebasic idea of the algorithm is to combinelocal features under certain spatial constraints. We compare DSWwith the eigenface approachondatacollectedfromvariousmeetings.Wehave testedboth front and profile face imagesand imageswith two stagesof occlusion.Theexperimentalresultsindicate thattheDSWapproachoutperformstheeigenfaceapproach in bothcases.

Anomaly Detection in Text: The Value of Domain Knowledge <=SEP=> We consider the problem of detecting anomalies from text data. Our hypothesis is that as with classical anomaly detection algorithms, domain-specific features are more important than the linguistic features. We employ the use of first-order logic and demonstrate the effectiveness of useful domain knowledge in two domains. Our results show that the domain-specific features are more predictive and that the relational learning methods exhibit superior performance.

Accuracy of automatic number plate recognition (ANPR) and real world UK number plate problems <=SEP=> This paper considers real world UK number plates and relates these to ANPR. It considers aspects of the relevant legislation and standards when applying them to real world number plates. The varied manufacturing techniques and varied specifications of component parts are also noted. The varied fixing methodologies and fixing locations are discussed as well as the impact on image capture.

Full 3-D Printed Microwave Termination: A Simple and Low-Cost Solution <=SEP=> This paper describes the realization and characterization of microwave 3-D printed loads in rectangular waveguide technology. Several commercial materials were characterized at X-band (8-12 GHz). Their dielectric properties were extracted through the use of a cavity-perturbation method and a transmission/reflection rectangular waveguide method. A lossy carbon-loaded Acrylonitrile Butadiene Styrene (ABS) polymer was selected to realize a matched load between 8 and 12 GHz. Two different types of terminations were realized by fused deposition modeling: a hybrid 3-D printed termination (metallic waveguide + pyramidal polymer absorber + metallic short circuit) and a full 3-D printed termination (self-consistent matched load). Voltage standing wave ratio of less than 1.075 and 1.025 were measured over X-band for the hybrid and full 3-D printed terminations, respectively. Power behavior of the full 3-D printed termination was investigated. A very linear evolution of reflected power as a function of incident power amplitude was observed at 10 GHz up to 11.5 W. These 3-D printed devices appear as a very low cost solution for the realization of microwave matched loads in rectangular waveguide technology.

Captology: A Critical Review <=SEP=> This critical review of B.J. Fogg’s book Persuasive Technology regards captology as an eclectic and formative work. It summarises two other reviewers’ work and identifies several new strengths. It scrutinises Fogg’s functional triad computers functioning as tools, media and social actors and some categorical changes are recommended. It investigates further Johnson’s concerns about specific ethical omissions, nominating a new term, compusuasion, for the resultant but unintended, exogenous behaviour/attitude change effects of captological design. The review commences to more carefully define what constitutes persuasion and draws attention to the distinction between persuasion techniques in general and the behavioural changes that result from advocacy and education. The reviewer concludes that a fundamental ethic be that the designer’s intent be exposed at the commencement of the user’s engagement with the program and proffers the idea of persuasion resulting in a new conviction, induced by others, as a helpful definition of

Identifying the Scan and Attack Infrastructures Behind Amplification DDoS Attacks <=SEP=> Amplification DDoS attacks have gained popularity and become a serious threat to Internet participants. However, little is known about where these attacks originate, and revealing the attack sources is a non-trivial problem due to the spoofed nature of the traffic.
 In this paper, we present novel techniques to uncover the infrastructures behind amplification DDoS attacks. We follow a two-step approach to tackle this challenge: First, we develop a methodology to impose a fingerprint on scanners that perform the reconnaissance for amplification attacks that allows us to link subsequent attacks back to the scanner. Our methodology attributes over 58% of attacks to a scanner with a confidence of over 99.9%. Second, we use Time-to-Live-based trilateration techniques to map scanners to the actual infrastructures launching the attacks. Using this technique, we identify 34 networks as being the source for amplification attacks at 98\% certainty.

Finger Sleeve : A Wearable Navigation Device <=SEP=> In this paper we present a novel wearable navigation system along with implicit HCI (iHCI) model, where interaction with technology is dissolved into a day-today activity. In this type of HCI model a computer takes the input and tries to output an action that is a proactive anticipation of next action of a user. Usually, in urban areas people use voice assisted navigation systems or navigation guidelines displayed on a mobile phone. Some navigation systems are already installed on car dashboard, which needs explicit attention in order to make driving decisions. A navigation system using haptic perception to guide a user throughout a journey is the key contribution of this paper. It does not ask for explicit user attention and demonstrates the indolent form of technological interaction. This wearable device is an index finger sleeve, which consists of vibrator modules, Bluetooth communication module and Microcontroller Unit (MCU). A working prototype has been built and tested.

Learning to Detect Rooftops in Aerial Images <=SEP=> In this paper, we examine the use of machine learning to improve the robustness of systems for image analysis on the task of roof detection. We review the problem of analyzing aerial photographs, and describe an existing vision system that attempts to automate the identiication of buildings in aerial images. After this, we brieey review several well-known learning algorithms that represent a wide variety of inductive biases. We report three experiments designed to illuminate facets of applying machine learning methods to the image analysis task; one experiment focuses on within-image learning, another deals with the cost of diierent errors, and a third addresses between-image learning. Experimental results demonstrate that machine-learned classiiers meet or exceed the accuracy of handcrafted solutions and that useful generalization occurs when training and testing on data derived from diierent images.

Question Answering on Freebase via Relation Extraction and Textual Evidence <=SEP=> Existing knowledge-based question answering systems often rely on small annotated training data. While shallow methods like relation extraction are robust to data scarcity, they are less expressive than the deep meaning representation methods like semantic parsing, thereby failing at answering questions involving multiple constraints. Here we alleviate this problem by empowering a relation extraction method with additional evidence from Wikipedia. We first present a neural network based relation extractor to retrieve the candidate answers from Freebase, and then infer over Wikipedia to validate these answers. Experiments on the WebQuestions question answering dataset show that our method achieves an F1 of 53.3%, a substantial improvement over the state-of-the-art.

SlangSD: Building and Using a Sentiment Dictionary of Slang Words for Short-Text Sentiment Classification <=SEP=> Sentiment in social media is increasingly considered as an important resource for customer segmentation, market understanding, and tackling other socio-economic issues. However, sentiment in social media is difficult to measure since user-generated content is usually short and informal. Although many traditional sentiment analysis methods have been proposed, identifying slang sentiment words remains untackled. One of the reasons is that slang sentiment words are not available in existing dictionaries or sentiment lexicons. To this end, we propose to build the first sentiment dictionary of slang words to aid sentiment analysis of social media content. It is laborious and time-consuming to collect and label the sentiment polarity of a comprehensive list of slang words. We present an approach to leverage web resources to construct an extensive Slang Sentiment word Dictionary (SlangSD) that is easy to maintain and extend. SlangSD is publicly available1 for research purposes. We empirically show the advantages of using SlangSD, the newly-built slang sentiment word dictionary for sentiment classification, and provide examples demonstrating its ease of use with an existing sentiment system.

Vehicle control system for automatic valet parking with infrastructure sensors <=SEP=> This paper presents vehicle control system for automatic valet parking with infrastructure sensors. First, we describe the automatic valet parking service system. In the service, vehicle moves autonomously based on sensing data generated by infrastructure sensors. Second, we implement vehicle control system for automatic valet parking. We design hardware and software components focusing on minimizing the add-on devices and maximizing data integration. Finally, we do a feasibility test for verification of the system.

Comparisons of centerline extraction methods for liver blood vessels in ImageJ and 3D slicer <=SEP=> In this paper, we introduce the centerline extraction processes of the liver vessel systems by using the plug-ins in ImageJ and 3D Slicer. The Skeletonize (2D/3D) plug-in in ImageJ is easy to use, and only based on the volumetric data. It performs fast and require no user interactions in terms of seed points selection. The VMTK toolkit in 3D slicer is more powerful. It uses the surface model and can provide more smooth and consistent results. Due to its complexities, it requires more computational resources and user specified seed points. Some possible improvements for these two methods are discussed.

Implantable Myoelectric Sensors (IMESs) for Intramuscular Electromyogram Recording <=SEP=> We have developed a multichannel electrogmyography sensor system capable of receiving and processing signals from up to 32 implanted myoelectric sensors (IMES). The appeal of implanted sensors for myoelectric control is that electromyography (EMG) signals can be measured at their source providing relatively cross-talk-free signals that can be treated as independent control sites. An external telemetry controller receives telemetry sent over a transcutaneous magnetic link by the implanted electrodes. The same link provides power and commands to the implanted electrodes. Wireless telemetry of EMG signals from sensors implanted in the residual musculature eliminates the problems associated with percutaneous wires, such as infection, breakage, and marsupialization. Each implantable sensor consists of a custom-designed application-specified integrated circuit that is packaged into a biocompatible RF BION capsule from the Alfred E. Mann Foundation. Implants are designed for permanent long-term implantation with no servicing requirements. We have a fully operational system. The system has been tested in animals. Implants have been chronically implanted in the legs of three cats and are still completely operational four months after implantation.

Comparative Metagenomics Reveals Host Specific Metavirulomes and Horizontal Gene Transfer Elements in the Chicken Cecum Microbiome <=SEP=> BACKGROUND
The complex microbiome of the ceca of chickens plays an important role in nutrient utilization, growth and well-being of these animals. Since we have a very limited understanding of the capabilities of most species present in the cecum, we investigated the role of the microbiome by comparative analyses of both the microbial community structure and functional gene content using random sample pyrosequencing. The overall goal of this study was to characterize the chicken cecal microbiome using a pathogen-free chicken and one that had been challenged with Campylobacter jejuni.


METHODOLOGY/PRINCIPAL FINDINGS
Comparative metagenomic pyrosequencing was used to generate 55,364,266 bases of random sampled pyrosequence data from two chicken cecal samples. SSU rDNA gene tags and environmental gene tags (EGTs) were identified using SEED subsystems-based annotations. The distribution of phylotypes and EGTs detected within each cecal sample were primarily from the Firmicutes, Bacteroidetes and Proteobacteria, consistent with previous SSU rDNA libraries of the chicken cecum. Carbohydrate metabolism and virulence genes are major components of the EGT content of both of these microbiomes. A comparison of the twelve major pathways in the SEED Virulence Subsystem (metavirulome) represented in the chicken cecum, mouse cecum and human fecal microbiomes showed that the metavirulomes differed between these microbiomes and the metavirulomes clustered by host environment. The chicken cecum microbiomes had the broadest range of EGTs within the SEED Conjugative Transposon Subsystem, however the mouse cecum microbiomes showed a greater abundance of EGTs in this subsystem. Gene assemblies (32 contigs) from one microbiome sample were predominately from the Bacteroidetes, and seven of these showed sequence similarity to transposases, whereas the remaining sequences were most similar to those from catabolic gene families.


CONCLUSION/SIGNIFICANCE
This analysis has demonstrated that mobile DNA elements are a major functional component of cecal microbiomes, thus contributing to horizontal gene transfer and functional microbiome evolution. Moreover, the metavirulomes of these microbiomes appear to associate by host environment. These data have implications for defining core and variable microbiome content in a host species. Furthermore, this suggests that the evolution of host specific metavirulomes is a contributing factor in disease resistance to zoonotic pathogens.

A Survey and Analysis of Various Health-Related Knowledge Mining Techniques in Social Media <=SEP=> Smart extraction of knowledge from social media has received the recent interest of the Biomedical and Health Informatics community for the simultaneous improvement of healthcare outcomes and lessen the expenses making use of consumergenerated reviews. Social media provides chances for patients and doctors to share their views and experiences without any obtrusion through online communities that might generate information, which is much beyond what is known by the domain experts. Nonetheless, for conventional public health surveillance systems, it is difficult to detect and then monitor the concerns related to health and the changes seen in attitudes of the public towards health-related problems. To solve this problem, several studies have shown the usage of information in social media for the discovery of biomedical and healthrelated information. Several disease-specific knowledge exchanges are now available on Face book and other portals of online social networking. These kind of new sources of information, support, and engagement have gone to become significant for patients who are suffering with the disease, and still the quality and the content of the knowledge contributed in these digital areas are not properly comprehended. The existing research methodologies are discussed with their merits and demerits, so that the further research works can be concentrated more. The experimental tests conducted were on all the research works in mat lab simulation environment and it is compared against each other to find the better approach under various performance measures such as Accuracy, Precision and Recall.

Approaching the Limit of Predictability in Human Mobility <=SEP=> In this study we analyze the travel patterns of 500,000 individuals in Cote d'Ivoire using mobile phone call data records. By measuring the uncertainties of movements using entropy, considering both the frequencies and temporal correlations of individual trajectories, we find that the theoretical maximum predictability is as high as 88%. To verify whether such a theoretical limit can be approached, we implement a series of Markov chain (MC) based models to predict the actual locations visited by each user. Results show that MC models can produce a prediction accuracy of 87% for stationary trajectories and 95% for non-stationary trajectories. Our findings indicate that human mobility is highly dependent on historical behaviors, and that the maximum predictability is not only a fundamental theoretical limit for potential predictive power, but also an approachable target for actual prediction accuracy.

Simple Customization of Recursive Neural Networks for Semantic Relation Classification <=SEP=> In this paper, we present a recursive neural network (RNN) model that works on a syntactic tree. Our model differs from previous RNN models in that the model allows for an explicit weighting of important phrases for the target task. We also propose to average parameters in training. Our experimental results on semantic relation classification show that both phrase categories and task-specific weighting significantly improve the prediction accuracy of the model. We also show that averaging the model parameters is effective in stabilizing the learning and improves generalization capacity. The proposed model marks scores competitive with state-of-the-art RNN-based models.

Effects of communication styles on acceptance of recommendations in intercultural collaboration <=SEP=> The objective of this study is to investigate the impact of culture and communication style (explicit versus implicit) on people’s reactions on recommendations in intercultural collaboration. The experimental results from three intercultural collaboration teams were studied: Chinese-American, Chinese-German, and Chinese-Korean. The results indicate that Chinese participants showed more positive evaluations (i.e., higher trust, higher satisfaction, and more future collaboration intention) on the implicit advisor than American and German participants. Compared with Chinese participants, Korean participants accepted explicit recommendations more often and showed more positive evaluations on the explicit advisor. The results also show that when Chinese express recommendations in an explicit way, their recommendations were accepted more often and were more positively evaluated by cross-cultural partners.

R2O, an extensible and semantically based database-to-ontology mapping language <=SEP=> We present R2O, an extensible and declarative language to describe mappings between relational DB schemas and ontologies implemented in RDF(S) or OWL. R2O provides an extensible set of primitives with welldefined semantics. This language has been conceived expressive enough to cope with complex mapping cases arisen from situations of low similarity between the ontology and the DB models.

Low-Dimensional Representation of Spectral Envelope Without Deterioration for Full-Band Speech Analysis/Synthesis System <=SEP=> A speech coding for a full-band speech analysis/synthesis system is described. In this work, full-band speech is defined as speech with a sampling frequency above 40 kHz, whose Nyquist frequency covers the audible frequency range. In prior works, speech coding has generally focused on the narrowband speech with a sampling frequency below 16 kHz. On the other hand, statistical parametric speech synthesis currently uses the full-band speech, and low-dimensional representation of speech parameters is being used. The purpose of this study is to achieve speech coding without deterioration for full-band speech. We focus on a high-quality speech analysis/synthesis system and mel-cepstral analysis using frequency warping. In the frequency warping function, we directly use three auditory scales. We carried out a subjective evaluation using the WORLD vocoder and found that the optimum number of dimensions was around 50. The kind of frequency warping did not significantly affect the sound quality in the dimensions.