Miracle-dz/UniFAR-FTUs huggingface dataset:
Knowledge distillation for small-footprint highway networks

Deep learning has significantly advanced state-of-the-art of speech recognition in the past few years. However, compared to conventional Gaussian mixture acoustic models, neural network models are usually much larger, and are therefore not very deployable in embedded devices. Previously, we investigated a compact highway deep neural network (HDNN) for acoustic modelling, which is a type of depth-gated feedforward neural network. We have shown that HDNN-based acoustic models can achieve comparable recognition accuracy with much smaller number of model parameters compared to plain deep neural network (DNN) acoustic models. In this paper, we push the boundary further by leveraging on the knowledge distillation technique that is also known as teacher-student training, i.e., we train the compact HDNN model with the supervision of a high accuracy cumbersome model. Furthermore, we also investigate sequence training and adaptation in the context of teacher-student training. Our experiments were performed on the AMI meeting speech recognition corpus. With this technique, we significantly improved the recognition accuracy of the HDNN acoustic model with less than 0.8 million parameters, and narrowed the gap between this model and the plain DNN with 30 million parameters.
Heavy-Traffic Analysis of a Multiple-Phase Network with Discriminatory Processor Sharing

We analyze a generalization of the discriminatory processor-sharing (DPS) queue in a heavy-traffic setting. Customers present in the system are served simultaneously at rates controlled by a vector of weights. We assume that customers have phase-type distributed service requirements and allow that customers have different weights in various phases of their service. In our main result we establish a state-space collapse for the queue-length vector in heavy traffic. The result shows that in the limit, the queue-length vector is the product of an exponentially distributed random variable and a deterministic vector. This generalizes a previous result by Rege and Sengupta [Rege, K. M. , B. Sengupta. 1996. Queue length distribution for the discriminatory processor-sharing queue. Oper. Res.44(4) 653--657], who considered a DPS queue with exponentially distributed service requirements. Their analysis was based on obtaining all moments of the queue-length distributions by solving systems of linear equations. We undertake a more direct approach by showing that the probability-generating function satisfies a partial differential equation that allows a closed-form solution after passing to the heavy-traffic limit. Making use of the state-space collapse result, we derive interesting properties in heavy traffic: (i) For the DPS queue, we obtain that, conditioned on the number of customers in the system, the residual service requirements are asymptotically independent and distributed according to the forward recurrence times. (ii) We then investigate how the choice for the weights influences the asymptotic performance of the system. In particular, for the DPS queue we show that the scaled holding cost reduces as classes with a higher value for dk/E(Bkfwd) obtain a larger share of the capacity, where dk is the cost associated to class k, and E(Bkfwd) is the forward recurrence time of the class-k service requirement. The applicability of this result for a moderately loaded system is investigated by numerical experiments.
Componentes de variância e interação entre genótipo e ambiente no incremento anual do caule em seringueira

Com o objetivo de determinar a natureza da grandeza e controle genetico e ambiental do vigor atraves do incremento anual do caule em seringueira [ Hevea brasiliensis (Willd. ex Adr. de Juss.) Muell. Arg. ], sete clones foram estudados em quatro diferentes locais do Estado de Sao Paulo. Os clones em estudo, na sua maioria, sao de origem asiatica, e os locais selecionados representam as diferentes regioes da heveicultura do Estado. O incremento anual da circunferencia das arvores de cada local foi obtido por um periodo de cinco anos antes do inicio da sangria para obtencao do latex. Os resultados das analises mostraram uma reducao consistente no componente genetico, no periodo de cinco anos, de 10,0 a 0,5 por cento, com um aumento correspondente no efeito do ambiente de 19,0 a 93,0 por cento. Os efeitos de genotipo x ambiente da analise de variância agrupada foram significantes para analise de variância de tres fatores (genotipos x locais x anos). Correlacoes geneticas entre locais para o incremento anual do caule mostraram valores positivos entre Mococa e Matao no quinto ano (r g = 0,7128), Ribeirao Preto e Matao no quinto ano (r g = 0,7113), e Ribeirao Preto e Presidente Prudente no quarto ano (r g = 0,9367). Discutem-se as implicacoes destes resultados em relacao a selecao de clones com base no incremento anual do caule.
Connect Everywhere : A Glimpse at the Internet Ecosystem through the Lens of the Netflix CDN

Netflix has become a worldwide video-streaming platform and the source of a large amount of the Internet traffic. It has achieved this without building its own datacentres, by controlling a network of servers deployed at Internet eXchange Points (IXPs) and within Internet Service Providers (ISPs). Despite its wide success and novel approach to infrastructure deployment, we have little understanding of its deployments and operations. Through extensive measurements, we reveal Netflix’s footprint, its traffic patterns and volumes. Our analysis of Netflix’s server deployment exposes the diversity of the Internet ecosystem world-wide. Our findings also suggest that the specifics of each regional ecosystem, coupled with the demand of each local market, explain the different deployment strategies.
Radiological Diagnosis of Adult Cases Presenting with Symptomatic Lumbar Spondylolisthesis: A Retrospective Study in a Tertiary Care Hospital

Background: The term spondylolisthesis is derived from two Greek word : Spondo (Vertebrate) and disthenein (to slip) . The first written description is given by Herbiniaur, a Belgian obstetneiae. Aim:- To Radiological diagnosis of Symptomatic Lumbar Spondylolisthesis in adults . Methods: 422 patients each of spondylolisthesis in age group 18 – 60 year were included in this study. This study conducted in department of orthopedics over a period of one year. Results: In this present study , patients were included who have back ache. 422 (8.3%) were with spondylolisthesis and 4698 (91.7%) cases were without spondylolisthesis. From the 422 patients 24 were male and 174 were female. From the 422 patients with spondylolisthesis, (24.6% ) most of the cases were belong to 46-52 age group in male and 39-45 in case of female. Conclusion: Both Surgical and conservative management have benefit of cases showing with symptomatic lumbar spondylolisthesis. In most of the cases treat by non-operative.
Performance of Zalawadi goats under range conditions.

Data recorded from 166 flocks of dual-purpose Zalawadi goats of Surendranagar district of Gujarat, was analyzed to evaluate performance and production environment. Sizable flocks (25–50%) resort to periodic migration (5–8 months) towards North and South Gujarat during summer and spring. Males for breeding were selected initially from high yielding (2.5–3, kg peak yield) dam’s subsequently on their growth and finally on sex libido, mostly (67%) from their own flock. Females, initially selected on the basis of morphology, followed by milk yield, udder and teat size, body height, lactation length and body weight. Bucks were observed with 48.23% flocks with proportions of 1:71 of buck and breedable females. Maximum goats (87%) were bred in summer on account of better biomass availability for pregnant does with onset of rain in July. The averages of age at first kidding, lactation length, kidding interval and herd life were 22.17±0.16, 6.57±0.31, 11.28±0.87 and 91±7 months, respectively. The multiple birth rates observed were 57%. The average milk yield recorded at fortnightly interval ranged from 1.09 (first parity) to 1.42 litre (fourth parity) with an average of 1.17±0.09 litre in 120 days. Average mortality in kids and adults was about 10 and 5%, respectively. Grass root infrastructure available in the state for sheep development may be utilized for goat development by state government to motivate goat keepers for the adoption of strategic management inputs, proper selection and judicious utilization of bucks, marketing of goats and goat products.
DTLS performance in duty-cycled networks

The Datagram Transport Layer Security (DTLS) protocol is the IETF standard for securing the Internet of Things. The Constrained Application Protocol, ZigBee IP, and Lightweight Machine-to-Machine (LWM2M) mandate its use for securing application traffic. There has been much debate in both the standardization and research communities on the applicability of DTLS to constrained environments. The main concerns are the communication overhead and latency of the DTLS handshake, and the memory footprint of a DTLS implementation. This paper provides a thorough performance evaluation of DTLS in different duty-cycled networks through real-world experimentation, emulation and analysis. In particular, we measure the duration of the DTLS handshake when using three duty cycling link-layer protocols: preamble-sampling, the IEEE 802.15. 4 beacon-enabled mode and the IEEE 802.15. 4e Time Slotted Channel Hopping mode. The reported results demonstrate surprisingly poor performance of DTLS in radio duty-cycled networks. Because a DTLS client and a server exchange more than 10 signaling packets, the DTLS handshake takes between a handful of seconds and several tens of seconds, with similar results for different duty cycling protocols. Moreover, because of their limited memory, typical constrained nodes can only maintain 3-5 simultaneous DTLS sessions, which highlights the need for using DTLS parsimoniously.
Enabling Spike-based Backpropagation in State-of-the-art Deep Neural Network Architectures

Spiking Neural Networks (SNNs) has recently emerged as a prominent neural computing paradigm. However, the typical shallow spiking network architectures have limited capacity for expressing complex representations, while training a very deep spiking network have not been successful so far. Diverse methods have been proposed to get around this issue such as converting off-line trained deep Artificial Neural Networks (ANNs) to SNNs. However, ANN-to-SNN conversion scheme fails to capture the temporal dynamics of a spiking system. On the other hand, it is still a difficult problem to directly train deep SNNs using input spike events due to the discontinuous and non-differentiable nature of the spike signals. To overcome this problem, we propose using differentiable (but approximate) activation for Leaky Integrate-and-Fire (LIF) spiking neurons to train deep convolutional SNNs with input spike events using spike-based backpropagation algorithm. Our experiments show the effectiveness of the proposed spike-based learning strategy on state-of-the-art deep networks (VGG and Residual architectures) by achieving the best classification accuracies in MNIST, SVHN and CIFAR-10 datasets compared to other SNNs trained with spike-based learning. Moreover, we analyze sparse event-driven computations to demonstrate the efficacy of proposed SNN training method for inference operation in the spiking domain.
A Generative Parser with a Discriminative Recognition Algorithm

Generative models defining joint distributions over parse trees and sentences are useful for parsing and language modeling, but impose restrictions on the scope of features and are often outperformed by discriminative models. We propose a framework for parsing and language modeling which marries a generative model with a discriminative recognition model in an encoder-decoder setting. We provide interpretations of the framework based on expectation maximization and variational inference, and show that it enables parsing and language modeling within a single implementation. On the English Penn Treen-bank, our framework obtains competitive performance on constituency parsing while matching the state-of-the-art single-model language modeling score.
Dialogue act modeling for automatic tagging and recognition of conversational speech

Abstract We describe a statistical approach for modeling dialogue acts in conversational speech, i.e., speech-act-like units such as STATEMENT, Question, BACKCHANNEL, Agreement, Disagreement, and Apology. Our model detects and predicts dialogue acts based on lexical, collocational, and prosodic cues, as well as on the discourse coherence of the dialogue act sequence. The dialogue model is based on treating the discourse structure of a conversation as a hidden Markov model and the individual dialogue acts as observations emanating from the model states. Constraints on the likely sequence of dialogue acts are modeled via a dialogue act n-gram. The statistical dialogue grammar is combined with word n-grams, decision trees, and neural networks modeling the idiosyncratic lexical and prosodic manifestations of each dialogue act. We develop a probabilistic integration of speech recognition with dialogue modeling, to improve both speech recognition and dialogue act classification accuracy. Models are trained and evaluated using a large hand-labeled database of 1,155 conversations from the Switchboard corpus of spontaneous human-to-human telephone speech. We achieved good dialogue act labeling accuracy (65% based on errorful, automatically recognized words and prosody, and 71% based on word transcripts, compared to a chance baseline accuracy of 35% and human accuracy of 84%) and a small reduction in word recognition error.
Bats of Hoosier National Forest

We summarize results of bat surveys in Hoosier National Forest 1981-2003. Mist net captures included the northern myotis, Myotis septentrionalis (137); red bat, Lasiurus borealis (132); eastern pipistrelle, Pipistrellus subflavus (131); big brown bat, Eptesicus fuscus (43); little brown myotis, Myotis lucifugus (40), Indiana myotis, Myotis sodalis (6); hoary bat, Lasiurus cinereus (4); silver-haired bat, Lasionycteris noctivagans (1); evening bat, Nycticeius humeralis (1); and gray myotis, Myotis grises- cens (1). Evidence of reproduction was found for northern and little brown myotis, big brown, red, and hoary bats, and the eastern pipistrelle, but not for Indiana myotis, despite an abundance of wooded habitats. Radio-transmitters were placed on five male Indiana myotis, and 14 roost trees of five species were found in riparian and upland habitats; six trees were 38.1 cm (15 inches) dbh (43% each). Some species of trees are more likely to provide roosts when live (in this study, shagbark hickory, Carya ovata, and white oak, Quercus alba), while others are more likely to provide roosts when dead (pine, Pinus sp., American elm, Ulmus americana, and northern red oak, Q. rubra). Activity areas (x = 95.1 ha minimum convex polygon) included wooded and open habitats, and roads ranging in size from forest trails to a divided four-lane interstate highway. Twenty-one visits to 18 caves produced eastern pipistrelles (15 caves, n = 449), little brown myotis (9 caves, n = 205), Indiana myotis (2 caves, n = 134-250), big brown bats (8 caves, n = 40), and northern myotis (3 caves, n = 4). Spring and autumn trapping at a kaolinite mine yielded 38 northern myotis, 10 eastern pipistrelles, and 4 little brown myotis.
Predicting Target Language CCG Supertags Improves Neural Machine Translation

Neural machine translation (NMT) models are able to partially learn syntactic information from sequential lexical information. Still, some complex syntactic phenomena such as prepositional phrase attachment are poorly modeled. This work aims to answer two questions: 1) Does explicitly modeling target language syntax help NMT? 2) Is tight integration of words and syntax better than multitask training? We introduce syntactic information in the form of CCG supertags in the decoder, by interleaving the target supertags with the word sequence. Our results on WMT data show that explicitly modeling target-syntax improves machine translation quality for German->English, a high-resource pair, and for Romanian->English, a low-resource pair and also several syntactic phenomena including prepositional phrase attachment. Furthermore, a tight coupling of words and syntax improves translation quality more than multitask training. By combining target-syntax with adding source-side dependency labels in the embedding layer, we obtain a total improvement of 0.9 BLEU for German->English and 1.2 BLEU for Romanian->English.
Optimizing wind farm control through wake steering using surrogate models based on high-fidelity simulations

Abstract. This paper aims to develop fast and reliable surrogate models for yaw-based wind farm control. The surrogates, based on polynomial chaos expansion (PCE), are built using high-fidelity flow simulations coupled with aeroelastic simulations of the turbine performance and loads. Developing a model for wind farm control is a challenging control problem due to the time-varying dynamics of the wake. The wind farm control strategy is optimized for both the power output and the loading of the turbines. The optimization performed using two Vestas V27 turbines in a row for a specific atmospheric condition suggests that a power gain of almost 3%±1% can be achieved at close spacing by yawing the upstream turbine more than 15∘ . At larger spacing the optimization shows that yawing is not beneficial as the optimization reverts to normal operation. Furthermore, it was also identified that a reduction in the equivalent loads was obtained at the cost of power production. The total power gains are discussed in relation to the associated model errors and the uncertainty of the surrogate models used in the optimization, as well as the implications for wind farm control.
LocNet: Improving Localization Accuracy for Object Detection

We propose a novel object localization methodology with the purpose of boosting the localization accuracy of stateof-the-art object detection systems. Our model, given a search region, aims at returning the bounding box of an object of interest inside this region. To accomplish its goal, it relies on assigning conditional probabilities to each row and column of this region, where these probabilities provide useful information regarding the location of the boundaries of the object inside the search region and allow the accurate inference of the object bounding box under a simple probabilistic framework. For implementing our localization model, we make use of a convolutional neural network architecture that is properly adapted for this task, called LocNet. We show experimentally that LocNet achieves a very significant improvement on the mAP for high IoU thresholds on PASCAL VOC2007 test set and that it can be very easily coupled with recent stateof-the-art object detection systems, helping them to boost their performance. Finally, we demonstrate that our detection approach can achieve high detection accuracy even when it is given as input a set of sliding windows, thus proving that it is independent of box proposal methods.
An Incremental Parser for Abstract Meaning Representation

Abstract Meaning Representation (AMR) is a semantic representation for natural language that embeds annotations related to traditional tasks such as named entity recognition, semantic role labeling, word sense disambiguation and co-reference resolution. We describe a transition-based parser for AMR that parses sentences left-to-right, in linear time. We further propose a test-suite that assesses specific subtasks that are helpful in comparing AMR parsers, and show that our parser is competitive with the state of the art on the LDC2015E86 dataset and that it outperforms state-of-the-art parsers for recovering named entities and handling polarity.
On the Expressive Power of Deep Learning: A Tensor Analysis

It has long been conjectured that hypotheses spaces suitable for data that is compositional in nature, such as text or images, may be more efficiently represented with deep hierarchical networks than with shallow ones. Despite the vast empirical evidence supporting this belief, theoretical justifications to date are limited. In particular, they do not account for the locality, sharing and pooling constructs of convolutional networks, the most successful deep learning architecture to date. In this work we derive a deep network architecture based on arithmetic circuits that inherently employs locality, sharing and pooling. An equivalence between the networks and hierarchical tensor factorizations is established. We show that a shallow network corresponds to CP (rank-1) decomposition, whereas a deep network corresponds to Hierarchical Tucker decomposition. Using tools from measure theory and matrix algebra, we prove that besides a negligible set, all functions that can be implemented by a deep network of polynomial size, require exponential size in order to be realized (or even approximated) by a shallow network. Since log-space computation transforms our networks into SimNets, the result applies directly to a deep learning architecture demonstrating promising empirical performance. The construction and theory developed in this paper shed new light on various practices and ideas employed by the deep learning community.
Multiplicative LSTM for sequence modelling

We introduce multiplicative LSTM (mLSTM), a recurrent neural network architecture for sequence modelling that combines the long short-term memory (LSTM) and multiplicative recurrent neural network architectures. mLSTM is characterised by its ability to have different recurrent transition functions for each possible input, which we argue makes it more expressive for autoregressive density estimation. We demonstrate empirically that mLSTM outperforms standard LSTM and its deep variants for a range of character level language modelling tasks. In this version of the paper, we regularise mLSTM to achieve 1.27 bits/char on text8 and 1.24 bits/char on Hutter Prize. We also apply a purely byte-level mLSTM on the WikiText-2 dataset to achieve a character level entropy of 1.26 bits/char, corresponding to a word level perplexity of 88.8, which is comparable to word level LSTMs regularised in similar ways on the same task.