pwc-archive/papers-with-abstracts huggingface dataset:
The Samsung and University of Edinburgh’s submission to IWSLT17
Authors: Pawel Przybysz, Marcin Chochowski, Rico Sennrich, Barry Haddow, Alexandra Birch
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Decoder, de-en, Domain Adaptation, Translation

This paper describes the joint submission of Samsung Research and Development, Warsaw, Poland and the University of Edinburgh team to the IWSLT MT task for TED talks. We took part in two translation directions, en-de and de-en. We also participated in the en-de and de-en lectures SLT task. The models have been trained with an attentional encoder-decoder model using the BiDeep model in Nematus. We filtered the training data to reduce the problem of noisy data, and we use back-translated monolingual data for domain-adaptation. We demonstrate the effectiveness of the different techniques that we applied via ablation studies. Our submission system outperforms our baseline, and last year’s University of Edinburgh submission to IWSLT, by more than 5 BLEU.
The RWTH Aachen Machine Translation Systems for IWSLT 2017
Authors: Parnia Bahar, Jan Rosendahl, Nick Rossenbach, Hermann Ney
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Domain Adaptation, Machine Translation, NMT, Translation

This work describes the Neural Machine Translation (NMT) system of the RWTH Aachen University developed for the English$German tracks of the evaluation campaign of the International Workshop on Spoken Language Translation (IWSLT) 2017. We use NMT systems which are augmented by state-of-the-art extensions. Furthermore, we experiment with techniques that include data filtering, a larger vocabulary, two extensions to the attention mechanism and domain adaptation. Using these methods, we can show considerable improvements over the respective baseline systems and our IWSLT 2016 submission.
FBK’s Multilingual Neural Machine Translation System for IWSLT 2017
Authors: Surafel M. Lakew, Quintino F. Lotito, Marco Turchi, Matteo Negri, Marcello Federico
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, Transfer Learning, Translation

Neural Machine Translation has been shown to enable inference and cross-lingual knowledge transfer across multiple language directions using a single multilingual model. Focusing on this multilingual translation scenario, this work summarizes FBK’s participation in the IWSLT 2017 shared task. Our submissions rely on two multilingual systems trained on five languages (English, Dutch, German, Italian, and Romanian). The first one is a 20 language direction model, which handles all possible combinations of the five languages. The second multilingual system is trained only on 16 directions, leaving the others as zero-shot translation directions (i.e representing a more complex inference task on language pairs not seen at training time). More specifically, our zero-shot directions are Dutch$German and Italian$Romanian (resulting in four language combinations). Despite the small amount of parallel data used for training these systems, the resulting multilingual models are effective, even in comparison with models trained separately for every language pair (i.e. in more favorable conditions). We compare and show the results of the two multilingual models against a baseline single language pair systems. Particularly, we focus on the four zero-shot directions and show how a multilingual model trained with small data can provide reasonable results. Furthermore, we investigate how pivoting (i.e using a bridge/pivot language for inference in a source!pivot!target translations) using a multilingual model can be an alternative to enable zero-shot translation in a low resource setting.
KIT’s Multilingual Neural Machine Translation systems for IWSLT 2017
Authors: Ngoc-Quan Pham, Matthias Sperber, Elizabeth Salesky, Thanh-Le Ha, Jan Niehues, Alexander Waibel
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, NMT, Translation

In this paper, we present KIT’s multilingual neural machine translation (NMT) systems for the IWSLT 2017 evaluation campaign machine translation (MT) and spoken language translation (SLT) tasks. For our MT task submissions, we used our multi-task system, modified from a standard attentional neural machine translation framework, instead of building 20 individual NMT systems. We investigated different architectures as well as different data corpora in training such a multilingual system. We also suggested an effective adaptation scheme for multilingual systems which brings great improvements compared to monolingual systems. For the SLT track, in addition to a monolingual neural translation system used to generate correct punctuations and true cases of the data prior to training our multilingual system, we introduced a noise model in order to make our system more robust. Results show that our novel modifications improved our systems considerably on all tasks.
Towards better translation performance on spoken language
Authors: Chao Bei, Hao Zong
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Decoder, Machine Translation, NMT, Translation

In this paper, we describe GTCOM’s neural machine translation(NMT) systems for the International Workshop on Spoken Language Translation(IWSLT) 2017. We participated in the English-to-Chinese and Chinese-to-English tracks in the small data condition of the bilingual task and the zero-shot condition of the multilingual task. Our systems are based on the encoder-decoder architecture with attention mechanism. We build byte pair encoding (BPE) models in parallel data and back-translated monolingual training data provided in the small data condition. Other techniques we explored in our system include two deep architectures, layer nomalization, weight normalization and training models with annealing Adam, etc. The official scores of English-to-Chinese, Chinese-to-English are 28.13 and 21.35 on test set 2016 and 28.30 and 22.16 on test set 2017. The official scores on German-to-Dutch, Dutch-to-German, Italian-to-Romanian and Romanian-to-Italian are 19.59, 17.95, 18.62 and 20.39 respectively.
Kyoto University MT System Description for IWSLT 2017
Authors: Raj Dabre, Fabien Cromieres, Sadao Kurohashi
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, NMT, Translation

We describe here our Machine Translation (MT) model and the results we obtained for the IWSLT 2017 Multilingual Shared Task. Motivated by Zero Shot NMT [1] we trained a Multilingual Neural Machine Translation by combining all the training data into one single collection by appending the tokens to the source sentences in order to indicate the target language they should be translated to. We observed that even in a low resource situation we were able to get translations whose quality surpass the quality of those obtained by Phrase Based Statistical Machine Translation by several BLEU points. The most surprising result we obtained was in the zero shot setting for Dutch-German and Italian-Romanian where we observed that despite using no parallel corpora between these language pairs, the NMT model was able to translate between these languages and the translations were either as good as or better (in terms of BLEU) than the non zero resource setting. We also verify that the NMT models that use feed forward layers and self attention instead of recurrent layers are extremely fast in terms of training which is useful in a NMT experimental setting.
The 2017 KIT IWSLT Speech-to-Text Systems for English and German
Authors: Thai-Son Nguyen, Markus Müller, Matthias Sperber, Thomas Zenkel, Sebastian Stüker, Alex Waibel
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Speech-to-Text

This paper describes our German and English Speech-to-Text (STT) systems for the 2017 IWSLT evaluation campaign. The campaign focuses on the transcription of unsegmented lecture talks. Our setup includes systems using both the Janus and Kaldi frameworks. We combined the outputs using both ROVER [1] and confusion network combination (CNC) [2] to achieve a good overall performance. The individual subsystems are built by using different speaker-adaptive feature combination (e.g., lMEL with i-vector or bottleneck speaker vector), acoustic models (GMM or DNN) and speaker adaptation (MLLR or fMLLR). Decoding is performed in two stages, where the GMM and DNN systems are adapted on the combination of the first stage outputs using MLLR, and fMLLR. The combination setup produces a final hypothesis that has a significantly lower WER than any of the individual sub-systems. For the English lecture task, our best combination system has a WER of 8.3% on the tst2015 development set while our other combinations gained 25.7% WER for German lecture tasks.
Domain-independent Punctuation and Segmentation Insertion
Authors: Eunah Cho, Jan Niehues, Alex Waibel
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, POS, Segmentation, Translation

Punctuation and segmentation is crucial in spoken language translation, as it has a strong impact to translation performance. However, the impact of rare or unknown words in the performance of punctuation and segmentation insertion has not been thoroughly studied. In this work, we simulate various degrees of domain-match in testing scenario and investigate their impact to the punctuation insertion task. We explore three rare word generalizing schemes using part-of-speech (POS) tokens. Experiments show that generalizing rare and unknown words greatly improves the punctuation insertion performance, reaching up to 8.8 points of improvement in F-score when applied to the out-of-domain test scenario. We show that this improvement in punctuation quality has a positive impact on a following machine translation (MT) performance, improving it by 2 BLEU points.
Toward Robust Neural Machine Translation for Noisy Input Sequences
Authors: Matthias Sperber, Jan Niehues, Alex Waibel
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, Translation

Translating noisy inputs, such as the output of a speech recognizer, is a difficult but important challenge for neural machine translation. One way to increase robustness of neural models is by introducing artificial noise to the training data. In this paper, we experiment with appropriate forms of such noise, exploring a middle ground between general-purpose regularizers and highly task-specific forms of noise induction. We show that with a simple generative noise model, moderate gains can be achieved in translating erroneous speech transcripts, provided that type and amount of noise are properly calibrated. The optimal amount of noise at training time is much smaller than the amount of noise in our test data, indicating limitations due to trainability issues. We note that unlike our baseline model, models trained on noisy data are able to generate outputs of proper length even for noisy inputs, while gradually reducing output length for higher amount of noise, as might also be expected from a human translator. We discuss these findings in details and give suggestions for future work.
Monolingual Embeddings for Low Resourced Neural Machine Translation
Authors: Mattia Antonino Di Gangi, Marcello Federico
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, NMT, Translation, Word Embeddings

Neural machine translation (NMT) is the state of the art for machine translation, and it shows the best performance when there is a considerable amount of data available. When only little data exist for a language pair, the model cannot produce good representations for words, particularly for rare words. One common solution consists in reducing data sparsity by segmenting words into sub-words, in order to allow rare words to have shared representations with other words. Taking a different approach, in this paper we present a method to feed an NMT network with word embeddings trained on monolingual data, which are combined with the task-specific embeddings learned at training time. This method can leverage an embedding matrix with a huge number of words, which can therefore extend the word-level vocabulary. Our experiments on two language pairs show good results for the typical low-resourced data scenario (IWSLT in-domain dataset). Our consistent improvements over the baselines represent a positive proof about the possibility to leverage models pre-trained on monolingual data in NMT.
Evolution Strategy Based Automatic Tuning of Neural Machine Translation Systems
Authors: Hao Qin, Takahiro Shinozaki, Kevin Duh
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) systems have demonstrated promising results in recent years. However, non-trivial amounts of manual effort are required for tuning network architectures, training configurations, and pre-processing settings such as byte pair encoding (BPE). In this study, we propose an evolution strategy based automatic tuning method for NMT. In particular, we apply the covariance matrix adaptation-evolution strategy (CMA-ES), and investigate a Pareto-based multi-objective CMA-ES to optimize the translation performance and computational time jointly. Experimental results show that the proposed method automatically finds NMT systems that outperform the initial manual setting.
CHARCUT: Human-Targeted Character-Based MT Evaluation with Loose Differences
Authors: Adrien Lardilleux, Yves Lepage
Date: 2017-12-01 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, Translation

We present CHARCUT, a character-based machine translation evaluation metric derived from a human-targeted segment difference visualisation algorithm. It combines an iterative search for longest common substrings between the candidate and the reference translation with a simple length-based threshold, enabling loose differences that limit noisy character matches. Its main advantage is to produce scores that directly reflect human-readable string differences, making it a useful support tool for the manual analysis of MT output and its display to end users. Experiments on WMT16 metrics task data show that it is on par with the best “un-trained” metrics in terms of correlation with human judgement, well above BLEU and TER baselines, on both system and segment tasks.
Global Object Proposals for Improving Multi-Sentence Video Descriptions
Authors: Chandresh S. Kanani, Sriparna Saha, Pushpak Bhattacharyya
Date: 2021-07-18 00:00:00
Proceeding: International Joint Conference on Neural Networks (IJCNN) 2021 7
Tasks: Caption Generation, Dense Video Captioning, Image Captioning, Object, object-detection, Object Detection, Sentence

There has been significant progress in image captioning in recent years. The generation of video descriptions is still in its early stages; this is due to the complex nature of videos in comparison to images. Generating paragraph descriptions of a video is even more challenging. Amongst the main issues are temporal object dependencies and complex object-object relationships. Recently, many works are proposed on the generation of multi-sentence video descriptions. The majority of the proposed works are based on a two-step approach: 1) event proposals and 2) caption generation. While these approaches produce good results, they miss out on globally available information. Here we propose the use of global object proposals while generating the video captions. Experimental results on ActivityNet dataset illustrate that the use of global object proposals can produce more informative and correct captions. We also propose three scores to evaluate the object detection capacity of the generator. A qualitative comparison of captions generated by the proposed method and the state-of-the-art techniques proves the efficacy of the proposed method.
Shallow over Deep Neural Networks: A empirical analysis for human emotion classification using audio data
Authors: Chandresh S. Kanani, Karanjit Singh Gill, Sourajit Behera, Anurag Choubey, Rohit Kumar Gupta, and Rajiv Misra
Date: 2020-07-03 00:00:00
Proceeding: International Conference on Internet of Things and Connected Technologies 2020 7
Tasks: Emotion Classification, Emotion Recognition, Speech Emotion Recognition

Human emotions can be identified in numerous ways, ranging from analyzing the tonal properties of speech to the facial expressions created before speech delivery and even the body gestures that can suggest various emotions without saying anything. Knowing the correct emotions of an individual can help is understand the situation and even react to it. This phenomena is even true for many feedback system used for day-to-day communication with humans, specifically the ones used for smart home solutions. The field of automated emotion recognition involves use-cases in different fields of research from computer vision, physiology to even artificial intelligence. This work focuses on classifying emotions into eight categories which are neutral, happy, sad, angry, calm, fearful, disgust and surprised based on the way those sentences have been spoken, using the “Ryerson Audio-Visual Database of Emotional Speech and Song” (RAVDESS). We propose a novel approach for emotion classification of audio conversations based on speech signals. Acoustic properties based emotion classification is independent of any spoken language and it can be used for cross-language emotion classification. The aim of the contribution was to develop a system capable of automatically recognising emotions for real-time speech. We performed several simulations and were able to achieve the highest accuracy of 82.99% with our shallow CNN model.
Boosting Deep Ensemble Performance with Hierarchical Pruning
Authors: Yanzhao Wu, Ling Liu
Date: 2021-12-07 00:00:00
Proceeding: IEEE International Conference on Data Mining (ICDM) 2021 12
Tasks: Decision Making, Diversity, Ensemble Pruning, Image Classification

Deep neural network ensembles have become attractive learning techniques with better generalizability over individual models. Some mission critical applications may require a large number of deep neural networks to achieve desirable accuracy and generalizability, making the ensemble execution costly with respect to runtime and space. This paper proposes a novel hierarchical ensemble pruning approach, which can effectively examine a given pool of M base models and identify smaller high quality deep ensembles of size S (<< M) with higher ensemble accuracy than the entire deep ensemble of all M models. Our hierarchical pruning approach, coined as HQ, combines three novel techniques. First, we show that the focal diversity metrics is innovative and can accurately capture the negative correlation among the member models of an ensemble, and the use of focal diversity metrics can boost ensemble accuracy. Second, we introduce a focal-diversity based hierarchical pruning algorithm to progressively identify low-cost ensembles with high ensemble diversity and accuracy. Third, we design a focal diversity consensus method to find smaller deep ensembles with low negative correlation. We demonstrate such ensembles offer high accuracy and high robustness while being more time and space efficient in ensemble decision making. Evaluated using two benchmark datasets, we show that the proposed focal diversity powered hierarchical pruning can find significantly smaller ensembles of deep neural network models while achieving the same or better classification generalizability.
Volumetric landmark detection with a multi-scale translation equivariant neural network
Authors: Tianyu Ma, Ajay Gupta, Mert R. Sabuncu
Date: 2020-04-03 00:00:00
Proceeding: IEEE 17th International Symposium on Biomedical Imaging (ISBI) 2020 4
Tasks: Anatomical Landmark Detection, GPU, Translation

Deep neural networks yield promising results in a wide range of computer vision applications, including landmark detection. A major challenge for accurate anatomical landmark detection in volumetric images such as clinical CT scans is that large-scale data often constrain the capacity of the employed neural network architecture due to GPU memory limitations, which in turn can limit the precision of the output. We propose a multi-scale, end-to-end deep learning method that achieves fast and memory-efficient landmark detection in 3D images. Our architecture consists of blocks of shift-equivariant networks, each of which performs landmark detection at a different spatial scale. These blocks are connected from coarse to fine-scale, with differentiable resampling layers, so that all levels can be trained together. We also present a noise injection strategy that increases the robustness of the model and allows us to quantify uncertainty at test time. We evaluate our method for carotid artery bifurcations detection on 263 CT volumes and achieve a better than state-of-the-art accuracy with a mean Euclidean distance error of 2.81mm.
Acoustic To Articulatory Speech Inversion Using Multi-Resolution Spectro-Temporal Representations Of Speech Signals
Authors: Rahil Parikh, Nadee Seneviratne, Ganesh Sivaraman, Shihab Shamma, Carol Espy-Wilson
Date: 2022-03-11 00:00:00

Multi-resolution spectro-temporal features of a speech signal represent how the brain perceives sounds by tuning cortical cells to different spectral and temporal modulations. These features produce a higher dimensional representation of the speech signals. The purpose of this paper is to evaluate how well the auditory cortex representation of speech signals contribute to estimate articulatory features of those corresponding signals. Since obtaining articulatory features from acoustic features of speech signals has been a challenging topic of interest for different speech communities, we investigate the possibility of using this multi-resolution representation of speech signals as acoustic features. We used U. of Wisconsin X-ray Microbeam (XRMB) database of clean speech signals to train a feed-forward deep neural network (DNN) to estimate articulatory trajectories of six tract variables. The optimal set of multi-resolution spectro-temporal features to train the model were chosen using appropriate scale and rate vector parameters to obtain the best performing model. Experiments achieved a correlation of 0.675 with ground-truth tract variables. We compared the performance of this speech inversion system with prior experiments conducted using Mel Frequency Cepstral Coefficients (MFCCs).
Unitary rotation of pixellated polychromatic images
Authors: Alejandro R. Urzúa, Kurt Bernardo Wolf
Date: 2022-03-11 00:00:00

Unitary rotations of polychromatic images on finite two-dimensional pixellated screens provide invertibility, group composition, and thus conservation of information. Rotations have been applied on monochromatic image data sets, where we now examine closer the Gibbs-like oscillations that appear due to discrete "discontinuities" of the input images under unitary transformations. Extended to three-color images we examine here the display of color at the pixels where, due to the oscillations, some pixel color values may fall outside their required common numerical range [0, 1], between absence and saturation of the red, green, and blue formant color images.
Game Dynamics Structure Control by Design: an Example from Experimental Economics
Authors: Wang Zhijian
Date: 2022-03-11 00:00:00

Game dynamics structure (e.g., endogenous cycle motion) in human subjects game experiments can be predicted by game dynamics theory. However, whether the structure can be controlled by mechanism design to a desired goal is not known. Here, using the pole assignment approach in modern control theory, we demonstrate how to control the structure in two steps: (1) Illustrate an theoretical workflow on how to design a state-depended feedback controller for desired structure; (2) Evaluate the controller by laboratory human subject game experiments and by agent-based evolutionary dynamics simulation. To our knowledge, this is the first realisation of the control of the human social game dynamics structure in theory and experiment.