Reviewerly/exHarmony huggingface dataset:
Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts

Dialogue participants often refer to entities or situations repeatedly within a conversation, which contributes to its cohesiveness.Subsequent references exploit the common ground accumulated by the interlocutors and hence have several interesting properties, namely, they tend to be shorter and reuse expressions that were effective in previous mentions.In this paper, we tackle the generation of first and subsequent references in visually grounded dialogue.We propose a generation model that produces referring utterances grounded in both the visual and the conversational context.To assess the referring effectiveness of its output, we also implement a reference resolution system.Our experiments and analyses show that the model produces better, more effective referring utterances than a model not grounded in the dialogue context, and generates subsequent references that exhibit linguistic patterns akin to humans.Referring utterances extracted from dialogue 1 A: a white fuzzy dog with a wine glass up to his face ; B: I see the wine glass dog ; A: no I don't have the wine glass dog Referring utterances extracted from dialogue 2 C: white dog sitting on something red ; D: yes I have the dog on the red chair ; C: white dog on the red chair
Explainable Clinical Decision Support from Text

Clinical prediction models often use structured variables and provide outcomes that are not readily interpretable by clinicians.Further, free-text medical notes may contain information not immediately available in structured variables.We propose a hierarchical CNNtransformer model with explicit attention as an interpretable, multi-task clinical language model, which achieves an AUROC of 0.75 and 0.78 on sepsis and mortality prediction on the English MIMIC-III dataset, respectively.We also explore the relationships between learned features from structured and unstructured variables using projection-weighted canonical correlation analysis.Finally, we outline a protocol to evaluate model usability in a clinical decision support context.From domain-expert evaluations, our model generates informative rationales that have promising real-life applications.
Temporal Smoothing for 3D Human Pose Estimation and Localization for Occluded People

In multi-person pose estimation actors can be heavily occluded, even become fully invisible behind another person. While temporal methods can still predict a reasonable estimation for a temporarily disappeared pose using past and future frames, they exhibit large errors nevertheless. We present an energy minimization approach to generate smooth, valid trajectories in time, bridging gaps in visibility. We show that it is better than other interpolation based approaches and achieves state of the art results. In addition, we present the synthetic MuCo-Temp dataset, a temporal extension of the MuCo-3DHP dataset. Our code is made publicly available. ( https://github.com/vegesm/pose_refinement ).
Comparing feedforward and recurrent neural network architectures with human behavior in artificial grammar learning

Abstract In recent years artificial neural networks achieved performance close to or better than humans in several domains: tasks that were previously human prerogatives, such as language processing, have witnessed remarkable improvements in state of the art models. One advantage of this technological boost is to facilitate comparison between different neural networks and human performance, in order to deepen our understanding of human cognition. Here, we investigate which neural network architecture (feedforward vs. recurrent) matches human behavior in artificial grammar learning, a crucial aspect of language acquisition. Prior experimental studies proved that artificial grammars can be learnt by human subjects after little exposure and often without explicit knowledge of the underlying rules. We tested four grammars with different complexity levels both in humans and in feedforward and recurrent networks. Our results show that both architectures can “learn” (via error back-propagation) the grammars after the same number of training sequences as humans do, but recurrent networks perform closer to humans than feedforward ones, irrespective of the grammar complexity level. Moreover, similar to visual processing, in which feedforward and recurrent architectures have been related to unconscious and conscious processes, the difference in performance between architectures over ten regular grammars shows that simpler and more explicit grammars are better learnt by recurrent architectures, supporting the hypothesis that explicit learning is best modeled by recurrent networks, whereas feedforward networks supposedly capture the dynamics involved in implicit learning.
Cryptanalysis of LowMC instances using single plaintext/ciphertext pair

Arguably one of the main applications of the LowMC family ciphers is in the post-quantum signature scheme PICNIC. Although LowMC family ciphers have been studied from a cryptanalytic point of view before, none of these studies were directly concerned with the actual use case of this cipher in PICNIC signature scheme. Due to the design paradigm of PICNIC, an adversary trying to perform a forgery attack on the signature scheme instantiated with LowMC would have access to only a single given plaintext/ciphertext pair, i.e. an adversary would only be able to perform attacks with data complexity 1 in a known-plaintext attack scenario. This restriction makes it impossible to employ classical cryptanalysis methodologies such as differential and linear cryptanalysis. In this paper we introduce two key-recovery attacks, both in known-plaintext model and of data complexity 1 for two variants of LowMC, both instances of the LowMC cryptanalysis challenge.
Novel fractional wavelet transform: Principles, MRA and application

Wavelet transform (WT) can be viewed as a differently scaled bandpass filter in the frequency domain, so WT is not the optimal time-frequency representation method for those signals which are not band-limited in the frequency domain. A novel fractional wavelet transform (FRWT) is proposed to break the limitation of WT, it displays the time and fractional frequency information jointly in the time-fractional-frequency (TFF) plane. The definition and basic properties of FRWT are studied firstly. Furthermore, the multiresolution analysis and orthogonal fractional wavelets associated with FRWT are explored. Finally, the application of FRWT in the LFM signal TFF analysis is discussed and verified by simulations. The experimental results show that the energy concentration of LFM signal representation by proposed FRWT is better than that of some existing methods. The better energy concentration makes it can be further applied to the denoising, detection, parameter estimation and separation of LFM signal.
LSTM based Ensemble Network to enhance the learning of long-term dependencies in chatbot

A chatbot is a software that can reproduce a discussion portraying a specific dimension of articulation among people and machines utilizing Natural Human Language. With the advent of AI, chatbots have developed from being minor guideline-based models to progressively modern models. A striking highlight of the current chatbot frameworks is their capacity to maintain and support explicit highlights and settings of the discussions empowering them to have human interaction in real-time surroundings. The paper presents a detailed database concerning the models utilized to deal with the learning of long haul conditions in a chatbot. The paper proposes a novel crossbreed Long Short Term Memory based Ensemble model to retain the information in specific situations. The proposed model uses a characterized number of Long Short Term Memory Networks as a significant aspect of its working as one to create the aggregate forecast class for the information inquiry and conversation. We found that both of the ensemble methods LSTM and GRU work well in different dataset environments and the ensemble technique is an effective one in chatbot applications.
Modeling Multi-Targets Sentiment Classification Via Graph Convolutional Networks and Auxiliary Relation

Existing solutions do not work well when multi-targets coexist in a sentence. The reason is that the existing solution is usually to separate multiple targets and process them separately. If the original sentence has N target, the original sentence will be repeated for N times, and only one target will be processed each time. To some extent, this approach degenerates the fine-grained sentiment classification task into the sentencelevel sentiment classification task, and the research method of processing the target separately ignores the internal relation and interaction between the targets. Based on the above considerations, we proposes to use Graph Convolutional Network (GCN) to model and process multi-targets appearing in sentences at the same time based on the positional relationship, and then to construct a graph of the sentiment relationship between targets based on the difference of the sentiment polarity between target words. In addition to the standard target-dependent sentiment classification task, an auxiliary node relation classification task is constructed. Experiments demonstrate that our model achieves new comparable performance on the benchmark datasets: SemEval-2014 Task 4, i.e., reviews for restaurants and laptops. Furthermore, the method of dividing the target words into isolated individuals has disadvantages, and the multi-task learning model is beneficial to enhance the feature extraction ability and expression ability of the model.
LT@Helsinki at SemEval-2020 Task 12: Multilingual or Language-specific BERT?

This paper presents the different models submitted by the LT@Helsinki team for the SemEval 2020 Shared Task 12.Our team participated in sub-tasks A and C; titled offensive language identification and offense target identification, respectively.In both cases we used the so-called Bidirectional Encoder Representation from Transformer (BERT), a model pre-trained by Google and fine-tuned by us on the OLID and SOLID datasets.The results show that offensive tweet classification is one of several language-based tasks where BERT can achieve state-of-the-art results.
UPB at SemEval-2020 Task 8: Joint Textual and Visual Modeling in a Multi-Task Learning Architecture for Memotion Analysis

Users from the online environment can create different ways of expressing their thoughts, opinions, or conception of amusement.Internet memes were created specifically for these situations.Their main purpose is to transmit ideas by using combinations of images and texts such that they will create a certain state for the receptor, depending on the message the meme has to send.These posts can be related to various situations or events, thus adding a funny side to any circumstance our world is situated in.In this paper, we describe the system developed by our team for SemEval-2020 Task 8: Memotion Analysis.More specifically, we introduce a novel system to analyze these posts, a multimodal multi-task learning architecture that combines ALBERT for text encoding with VGG-16 for image representation.In this manner, we show that the information behind them can be properly revealed.Our approach achieves good performance on each of the three subtasks of the current competition, ranking 11 th for Subtask A (0.3453 macro F1-score), 1 st for Subtask B (0.5183 macro F1-score), and 3 rd for Subtask C (0.
Retrieving Skills from Job Descriptions: A Language Model Based Extreme Multi-label Classification Framework

We introduce a deep learning model to learn the set of enumerated job skills associated with a job description.In our analysis of a large-scale government job portal mycareersfuture.sg,we observe that as much as 65% of job descriptions miss describing a significant number of relevant skills.Our model addresses this task from the perspective of an extreme multi-label classification (XMLC) problem, where descriptions are the evidence for the binary relevance of thousands of individual skills.Building upon the current state-of-the-art language modeling approaches such as BERT, we show our XMLC method improves on an existing baseline solution by over 9% and 7% absolute improvements in terms of recall and normalized discounted cumulative gain.We further show that our approach effectively addresses the missing skills problem, and helps in recovering relevant skills that were missed out in the job postings by taking into account the structured semantic representation of skills and their co-occurrences through a Correlation Aware Bootstrapping process.To facilitate future research and replication of our work, we have made the dataset and the implementation of our model publicly available.
A Color Image Encryption Scheme Based on a Novel 3D Chaotic Mapping

Low-dimensional chaotic mappings are simple functions that have low computation cost and are easy to realize, but applying them in a cryptographic algorithm will lead to security vulnerabilities. To overcome this shortcoming, this paper proposes the coupled chaotic system, which coupled the piecewise and Henon mapping. Simulation results indicate that the novel mapping has better complexity and initial sensitivity and larger key space compared with the original mapping. Then, a new color image encryption algorithm is proposed based on the new chaotic mapping. The algorithm has two processes: diffusion and confusion. In this scheme, the key is more than <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <msup> <mrow> <mn>2</mn> </mrow> <mrow> <mn>216</mn> </mrow> </msup> </math> , and SSIM and PSNR are 0.009675 and 8.6767, respectively. The secret key is applied in the shuffling and diffusion. Security analysis indicates that the proposed scheme can resist cryptanalytic attacks. It has superior performance and has high security.
Adoption of Machine Learning Algorithm-Based Intelligent Basketball Training Robot in Athlete Injury Prevention

In order to effectively prevent sports injuries caused by collisions in basketball training, realize efficient shooting, and reduce collisions, the machine learning algorithm was applied to intelligent robot for path planning in this study. First of all, combined with the basketball motion trajectory model, the sport recognition in basketball training was analyzed. Second, the mathematical model of the basketball motion trajectory of the shooting motion was established, and the factors affecting the shooting were analyzed. Thirdly, on this basis, the machine learning-based improved Q-Learning algorithm was proposed, the path planning of the moving robot was realized, and the obstacle avoidance behavior was accomplished effectively. In the path planning, the principle of fuzzy controller was applied, and the obstacle ultrasonic signals acquired around the robot were taken as input to effectively avoid obstacles. Finally, the robot was able to approach the target point while avoiding obstacles. The results of simulation experiment show that the obstacle avoidance path obtained by the improved Q-Learning algorithm is flatter, indicating that the algorithm is more suitable for the obstacle avoidance of the robot. Besides, it only takes about 250 s for the robot to find the obstacle avoidance path to the target state for the first time, which is far lower than the 700 s of the previous original algorithm. As a result, the fuzzy controller applied to the basketball robot can effectively avoid the obstacles in the robot movement process, and the motion trajectory curve obtained is relatively smooth. Therefore, the proposed machine learning algorithm has favorable obstacle avoidance effect when it is applied to path planning in basketball training, and can effectively prevent sports injuries in basketball activities.
A Comparative Study of Swarm Intelligence Algorithms for UCAV Path-Planning Problems

Path-planning for uninhabited combat air vehicles (UCAV) is a typically complicated global optimization problem. It seeks a superior flight path in a complex battlefield environment, taking into various constraints. Many swarm intelligence (SI) algorithms have recently gained remarkable attention due to their capability to address complex optimization problems. However, different SI algorithms present various performances for UCAV path-planning since each algorithm has its own strengths and weaknesses. Therefore, this study provides an overview of different SI algorithms for UCAV path-planning research. In the experiment, twelve algorithms that published in major journals and conference proceedings are surveyed and then applied to UCAV path-planning. Moreover, to demonstrate the performance of different algorithms in further, we design different scales of problem cases for those comparative algorithms. The experimental results show that UCAV can find the safe path to avoid the threats efficiently based on most SI algorithms. In particular, the Spider Monkey Optimization is more effective and robust than other algorithms in handling the UCAV path-planning problem. The analysis from different perspectives contributes to highlight trends and open issues in the field of UCAVs.
Computer-aided Instruction in College English Teaching under the Network Environment

The mining and application of data in sports training through intelligent computer-aided research, this paper proposes a system development and design method to improve the development and design efficiency of the data visualization system and reduce the development cost by using the Python language to handle the high efficiency of system programming and the convenience of charts.Thus, it further improves the efficiency and interactivity of the information management of training data and the convenience and stability of managing training data.This system is a preliminary exploration of Python language and Echarte technology in the management of training data of sports training programs.Because of the integration of training programs with computer application technology and data visualization technology, the system still needs to be strengthened in terms of data processing methods and the diversity of chart styles generated.In the future, the use of artificial intelligence, big data analysis, and other technologies in sports training data management and analysis has more room for development.The model only needs to measure 14 indicators of bipedal closed-eye standing and single-leg closed-eye standing movements to achieve the ability to predict the balance of motor training, and its accuracy meets the clinical requirements.This method has a shorter measurement time and fewer metrics than traditional methods.
Efficient Object-Level Visual Context Modeling for Multimodal Machine Translation: Masking Irrelevant Objects Helps Grounding

Visual context provides grounding information for multimodal machine translation (MMT). However, previous MMT models and probing studies on visual features suggest that visual information is less explored in MMT as it is often redundant to textual information. In this paper, we propose an Object-level Visual Context modeling framework (OVC) to efficiently capture and explore visual information for multimodal machine translation. With detected objects, the proposed OVC encourages MMT to ground translation on desirable visual objects by masking irrelevant objects in the visual modality. We equip the proposed with an additional object-masking loss to achieve this goal. The object-masking loss is estimated according to the similarity between masked objects and the source texts so as to encourage masking source-irrelevant objects. Additionally, in order to generate vision-consistent target words, we further propose a vision-weighted translation loss for OVC. Experiments on MMT datasets demonstrate that the proposed OVC model outperforms state-of-the-art MMT models and analyses show that masking irrelevant objects helps grounding in MMT.
Robust Blind Speech Watermarking via FFT-Based Perceptual Vector Norm Modulation With Frame Self-Synchronization

Watermarking is an important measure for protecting proprietary digital multimedia data. This paper presents a novel approach to achieving robust and imperceptible blind speech watermarking on a frame-by-frame basis. The proposed method employs two modules operating in the fast Fourier transform (FFT) domain. The first module is referred to as downward progressive quantization index modulation. It modulates the vector norms drawn from FFT coefficients according to a guideline deduced from human auditory masking properties. The second module is referred to as boundary-constrained iterative adjustment. It provides a smooth transition across frames in the resulting speech waveform. Experiment results confirm the imperceptibility of the proposed modulation scheme in terms of the mean opinion score - listening quality objective (MOS-LQO) based on the perceptual evaluation of speech quality (PESQ) metric. The proposed watermarking method matched and exceeded the performance of five state-of-the-art methods in terms of robustness against common speech processing attacks.
WebSRC: A Dataset for Web-Based Structural Reading Comprehension

Web search is an essential way for humans to obtain information, but it's still a great challenge for machines to understand the contents of web pages.In this paper, we introduce the task of structural reading comprehension (SRC) on web.Given a web page and a question about it, the task is to find the answer from the web page.This task requires a system not only to understand the semantics of texts but also the structure of the web page.Moreover, we proposed Web-SRC, a novel Web-based Structural Reading Comprehension dataset.WebSRC consists of 400K question-answer pairs, which are collected from 6.4K web pages.Along with the QA pairs, corresponding HTML source code, screenshots, and metadata are also provided in our dataset.Each question in WebSRC requires a certain structural understanding of a web page to answer, and the answer is either a text span on the web page or yes/no.We evaluate various baselines on our dataset to show the difficulty of our task.We also investigate the usefulness of structural information and visual features.Our dataset and baselines have been publicly available 1 .
Recognizing Human Races through Machine Learning—A Multi-Network, Multi-Features Study

The human face holds a privileged position in multi-disciplinary research as it conveys much information—demographical attributes (age, race, gender, ethnicity), social signals, emotion expression, and so forth. Studies have shown that due to the distribution of ethnicity/race in training datasets, biometric algorithms suffer from “cross race effect”—their performance is better on subjects closer to the “country of origin” of the algorithm. The contributions of this paper are two-fold: (a) first, we gathered, annotated and made public a large-scale database of (over 175,000) facial images by automatically crawling the Internet for celebrities’ images belonging to various ethnicity/races, and (b) we trained and compared four state of the art convolutional neural networks on the problem of race and ethnicity classification. To the best of our knowledge, this is the largest, data-balanced, publicly-available face database annotated with race and ethnicity information. We also studied the impact of various face traits and image characteristics on the race/ethnicity deep learning classification methods and compared the obtained results with the ones extracted from psychological studies and anthropomorphic studies. Extensive tests were performed in order to determine the facial features to which the networks are sensitive to. These tests and a recognition rate of 96.64% on the problem of human race classification demonstrate the effectiveness of the proposed solution.