Miracle-dz/UniFAR-FTUs huggingface dataset:
Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generation

Visual dialogue is a challenging task since it needs to answer a series of coherent questions on the basis of understanding the visual environment. Previous studies focus on the implicit exploration of multimodal co-reference by implicitly attending to spatial image features or object-level image features but neglect the importance of locating the objects explicitly in the visual content, which is associated with entities in the textual content. Therefore, in this paper we propose a {\bf M}ultimodal {\bf I}ncremental {\bf T}ransformer with {\bf V}isual {\bf G}rounding, named MITVG, which consists of two key parts: visual grounding and multimodal incremental transformer. Visual grounding aims to explicitly locate related objects in the image guided by textual entities, which helps the model exclude the visual content that does not need attention. On the basis of visual grounding, the multimodal incremental transformer encodes the multi-turn dialogue history combined with visual scene step by step according to the order of the dialogue and then generates a contextually and visually coherent response. Experimental results on the VisDial v0.9 and v1.0 datasets demonstrate the superiority of the proposed model, which achieves comparable performance.
Paths to Relation Extraction through Semantic Structure

Syntactic and semantic structure directly re-ﬂect relations expressed by the text at hand and are thus very useful for the relation extraction (RE) task. Their symbolic nature allows increased interpretability for end-users and developers, which is particularly appealing in RE. Although they have been somewhat overshad-owed recently by the use of end-to-end neural network models and contextualized word embeddings, we show that they may be lever-aged as input for neural networks to positive effect. We present two methods for integrating broad-coverage semantic structure (specifically, UCCA) into supervised neural RE models, demonstrating beneﬁts over the use of exclusively syntactic integrations. The ﬁrst method involves reduction of UCCA into a bilexical structure, while the second leverages a novel technique for encoding semantic DAG structures. Our approach is general and can be used for integrating a wide range of graph-based semantic structures. 1
Towards Efficient Hashing in Ethereum Smart Contracts

Ethereum is a popular public blockchain platform and currently the most significant featuring smart contract functionality. Smart contracts are small programs that are executed on the blockchain nodes, which can be used to implement complex transaction logic. Several high-level programming languages are available for writing Ethereum smart contracts, the most used being Solidity. The high-level code is further translated into a bytecode executed by a dedicated runtime environment, the Ethereum Virtual Machine (EVM). A few operations are, however, externalized as precompiled contracts, and run by the native implementation of the Ethereum node. These are typically computationally intensive operations such as cryptographic hash functions. Various smart contract patterns require hash computations. In such contexts, the current hash functions supported by Ethereum have a direct impact in both the performance and cost inflicted on the blockchain users. In this paper we investigate the available options for hashing in smart contracts, we discuss the implications regarding some patterns and we evaluate possible improvements. In particular we focus on the recent Blake family of cryptographic hash functions, which show promising performance results, but has yet limited support in the
Relation Extraction with Type-aware Map Memories of Word Dependencies

Relation extraction is an important task in information extraction and retrieval that aims to extract relations among the given entities from running texts. To achieve a good performance for this task, previous studies have shown that a good modeling of the contextual information is required, where the dependency tree of the input sentence can be a beneﬁcial source among different types of contextual information. However, most of these studies focus on the dependency connections between words with limited attention paid to exploiting dependency types. In addition, they often treat different dependency connections equally in modeling so that suffer from the noise (inaccurate dependency parses) in the auto-generated dependency tree. In this paper, we propose a neural approach for relation extraction, with type-aware map memories (TaMM) for encoding dependency types obtained from an off-the-shelf dependency parser for the input sentence. Speciﬁcally, for each word in an entity, TaMM maps all associated words along with the dependencies among them to memory slots and then assigns a weight to each slot according to its contribution to relation extraction. Our approach not only leverages dependency connections and types between words, but also distinguishes reliable dependency information from noisy ones and appropriately model them. The effectiveness of our approach is demonstrated by the experiments on two English benchmark datasets, where our approach achieves state-of-the-art performance on both datasets. 1
Investigating Text Simplification Evaluation

Modern text simplification (TS) heavily relies on the availability of gold standard data to build machine learning models. However, existing studies show that parallel TS corpora contain inaccurate simplifications and incorrect alignments. Additionally, evaluation is usually performed by using metrics such as BLEU or SARI to compare system output to the gold standard. A major limitation is that these metrics do not match human judgements and the performance on different datasets and linguistic phenomena vary greatly. Furthermore, our research shows that the test and training subsets of parallel datasets differ significantly. In this work, we investigate existing TS corpora, providing new insights that will motivate the improvement of existing state-of-the-art TS evaluation methods. Our contributions include the analysis of TS corpora based on existing modifications used for simplification and an empirical study on TS models performance by using better-distributed datasets. We demonstrate that by improving the distribution of TS datasets, we can build more robust TS models.
Reduction-assisted Fault Localization: Don't Throw Away the By-products!

Spectrum-based fault localization (SBFL) is a popular idea for automated software debugging. SBFL techniques use information about the execution of program elements, recorded on a suite of test cases, and derive statistics from them, which are then used to determine the suspiciousness of program elements, thus guiding the debugging efforts. However, even the best techniques can face problems when the statistics are unbalanced. If only one test case causes a program failure and all other inputs execute correctly, as is typical for fuzz testing, then it may be hard to differentiate between the program elements suspiciousness-wise. In this paper, we propose to utilize test case reduction, a technique to minimize unnecessarily large test cases often generated with fuzzing, to assist SBFL in such scenarios. As the intermediate results, or by-products, of the reduction are additional test cases to the program, we use these by-products when applying SBFL. We have evaluated this idea, and our results show that it can help SBFL precision by up to 49% on a real-world use-case.
Automatic Text Simplification for Social Good: Progress and Challenges

Since the late 1990s, automatic text simpliﬁca-tion (ATS) was promoted as a natural language processing (NLP) task with great potential to make texts more accessible to people with various reading or cognitive disabilities, and enable their better social inclusion. Large multidisciplinary projects showed promising steps in that direction. Since 2010, the ﬁeld started attracting more attention but at the cost of major shifts in system architecture, target audience, and evaluation strategies. Somewhere along the way, the focus has shifted from ATS for social good towards building complex end-to-end neural architectures that are not aimed at any particular target population. This study presents the trajectory of ATS for social good, the main issues in current ATS trends, and the ways forward that could bring the ﬁeld back to its initial goals.
Multimodal Fusion with Co-Attention Networks for Fake News Detection

Fake news with textual and visual contents has a better story-telling ability than text-only contents, and can be spread quickly with social media. People can be easily deceived by such fake news, and traditional expert identiﬁcation is labor-intensive. Therefore, automatic detection of multimodal fake news has become a new hot-spot issue. A shortcoming of existing approaches is their inability to fuse multi-modality features effectively. They simply concatenate unimodal features without considering inter-modality relations. Inspired by the way people read news with image and text, we propose a novel Multimodal Co-Attention Networks (MCAN) to better fuse textual and visual features for fake news detection. Extensive experiments conducted on two real-world datasets demonstrate that MCAN can learn inter-dependencies among multimodal features and outperforms state-of-the-art meth-ods.
Deep vs. Deep Bayesian: Faster Reinforcement Learning on a Multi-robot Competitive Experiment

Deep Learning experiments commonly require hundreds of trials to properly train neural networks, often labeled as Big Data, while Bayesian learning leverages scarce data points to infer next iterations, also known as Micro Data. Deep Bayesian Learning combines the complexity from multi-layered neural networks to probabilistic inferences, and it allows a robot to learn good policies within few trials in the real world. In here we propose, for the first time, an application of Deep Bayesian Reinforcement Learning (RL) on a realworld multi-robot confrontation game, and compare the algorithm with a model-free Deep RL algorithm, Deep Q-Learning. Our experiments show that DBRL significantly outperforms DRL in learning efficiency and scalability. The results of this work point to the advantages of Deep Bayesian approaches in bypassing the Reality Gap and sim-to-real implementations, as the time taken for real-world learning can quickly outperform data-intensive Deep alternatives.
Can Cognate Prediction Be Modelled as a Low-Resource Machine Translation Task?

Cognate prediction is the task of generating, in a given language, the likely cognates of words in a related language, where cognates are words in related languages that have evolved from a common ancestor word. It is a task for which little data exists and which can aid linguists in the discovery of previously undis-covered relations. Previous work has applied machine translation (MT) techniques to this task, based on the tasks’ similarities, without, however, studying their numerous differences or optimising architectural choices and hyper-parameters. In this paper, we investigate whether cognate prediction can bene-ﬁt from insights from low-resource MT. We ﬁrst compare statistical MT (SMT) and neural MT (NMT) architectures in a bilingual setup. We then study the impact of employing data augmentation techniques commonly seen to give gains in low-resource MT: monolingual pretraining, backtranslation and multilingual-ity. Our experiments on several Romance languages show that cognate prediction behaves only to a certain extent like a standard low-resource MT task. In particular, MT architectures, both statistical and neural, can be successfully used for the task, but using supplementary monolingual data is not always as ben-eﬁcial as using additional language data, con-trarily to what is observed for MT.
Survival text regression for time-to-event prediction in conversations

Time-to-event prediction tasks are common in conversation modelling, for applications such as predicting the length of a conversation or when a user will stop contributing to a platform. Despite the fact that it is natural to frame such predictions as regression tasks, recent work has modelled them as classiﬁcation tasks, determining whether the time-to-event is greater than a pre-determined cut-off point. While this allows for the application of classi-ﬁcation models which are well studied in NLP, it imposes a formulation that is contrived, as well as less informative. In this paper, we explore how to handle time-to-event forecasting in conversations as regression tasks. We focus on a family of regression techniques known as survival regression, which are commonly used in the context of healthcare and reliability engineering. We adapt these models to time-to-event prediction in conversations, us-ing linguistic markers as features. On three datasets, we demonstrate that they outperform commonly considered text regression methods and comparable classiﬁcation models.
Aspect Based Sentiment Analysis on Online Review Data to Predict Corporate Reputation

Corporate reputation is an intangible resource that is closely tied to an organization’s success but measuring it and to derive actions that can improve the reputations can be a long and expensive journey for an organization. In the available literature, corporate reputation is primarily measured through surveys, which can be time and cost intensive. This paper uses online reviews on the web as the source for a machine-learning driven aspect-based sentiment analysis that can enable organizations to evaluate their corporate reputation on a fine-grained level. The analysis is done unsupervised without organizations needing to manually label datasets. Using the insights generated through the analysis, on one hand, organizations can save costs and time to measure corporate reputation, and, on the other hand, it provides an in-depth analysis that splits the overall reputation into multiple aspects, with which organizations can identify weaknesses and in turn improve their corporate reputation. Therefore, this research is relevant for organizations aiming to understand and improve their corporate reputation to achieve success, for example, in form of financial performance, or for organizations that help and consult other organizations on their journeys to increased success. Our approach is validated, evaluated and illustrated with Trustpilot review data.
CNN-based robust sidewalk identiﬁcation for autonomous drone applications

As a society, our needs have evolved with advancements in science and technology. Today, we stand at the juncture where we feel the necessity of instant services, such as shipment delivery and emergency support. Among our current systems, autonomous drone is only viable option that can meet these requirements cost-effectively. And some companies, like Amazon.com, are already testing these services for shipment delivery within an hour. Unfortunately, our current city infrastructures are not flexible enough to easily accommodate drone navigation: flying too high in air, they’d pose security threat to city, and flying over roads, they pose a risk to the traffic. In this scenario, a practical solution would be to fly drones over sidewalks. The advantage of using sidewalks is two-fold: they facilitate planned routes to almost all locations in cities, but without posing any risk to the traffic flow.
HIJaX: Human Intent JavaScript XSS Generator

: Websites remain popular targets for web-based attacks such as Cross-Site Scripting (XSS). As a remedy, new research is needed to preemptively secure applications with the use of Automated Exploit Generation (AEG), whereby probing and patching of system vulnerabilities occurs autonomously. In this paper, we present HIJaX, a novel Natural Language-to-JavaScript generator prototype, that creates workable XSS exploit code from English sentences using neural machine translation. We train and test the HIJaX model with a variety of datasets containing benign and malicious intents along with differing numbers of baseline code entries to demonstrate how to best create datasets for XSS code generation. We also examine part-of-speech tagging algorithms and automated dataset expansion scripts to aid the dataset creation and code generation processes. Finally, we demonstrate the feasibility of deploying auto-generated XSS attacks against real-world websites.
A Double Auction Mechanism for Coded Distributed Computing in Smart Vehicles

The development of smart vehicles and rich cloud services have led to the emergence of vehicular edge computing. To perform the distributed computation tasks efficiently, Coded Distributed Computing (CDC) was proposed to reduce communication costs and mitigate the straggler effects through the use of coding techniques. In this paper, we propose a double auction mechanism to allocate the resources of the edge servers to the vehicles in order to complete the CDC tasks. Specifically, the vehicles use the PolyDot codes to manage the tradeoff between communication costs and recovery threshold. Given the requirements of various vehicles, the double auction mechanism matches the edge servers with the required resources to the vehicles. Besides, the double auction mechanism also determines the prices that the vehicles need to pay for the resources of the edge servers. The double auction mechanism satisfies the properties of individual rationality, incentive compatibility and budget-balance.
Goal and Threat Modelling for Driving Automotive Cybersecurity Risk Analysis Conforming to ISO/SAE 21434

As cars are increasingly connected and autonomous, they also become more exposed to cyber security threats. Providing strong protection and reactive response to such threats in a large industry involving many tiers and complex safety critical systems is challenging and required the development of the new ISO 21434 standard. Along with ISO 2626 dedicated to safety, it provides solid grounds for safety-security co-engineering. This paper focuses on how to provide effective and efficient support to the risk assessment phase based on a model-based approach. A rich goal-oriented meta-model is proposed to capture automotive assets and system properties, to estimate the impact of damage scenarios, to identify threats and to assess their feasibility. The approach is implemented as proof-of-concept through the meta-model adaptation of a generic co-engineering platform and is illustrated on the car light control sub-system.
Dialogue in the Wild: Learning from a Deployed Role-Playing Game with Humans and Bots

Much of NLP research has focused on crowd-sourced static datasets and the supervised learning paradigm of training once and then evaluating test performance. As argued in de Vries et al. (2020), crowdsourced data has the issues of lack of naturalness and relevance to real-world use cases, while the static dataset paradigm does not allow for a model to learn from its experiences of using language (Silver et al., 2013). In contrast, one might hope for machine learning systems that become more useful as they interact with people. In this work, we build and deploy a role-playing game, whereby human players converse with learning agents situated in an open-domain fantasy world. We show that by training models on the conversations they have with humans in the game the models progressively improve, as measured by automatic metrics and online engagement scores. This learning is shown to be more efﬁcient than crowdsourced data when applied to conversations with real users, as well as being far cheaper to collect.
Designing Operational Safety Procedures for UAV According to NATO Architecture Framework

The article presents the principles of designing unmanned aerial platforms, which belong to the group of near real-time systems. The correct and legally compliant design process of such systems requires adherence to the principles of designing operational safety procedures for UAVs in accordance with the NATO Architecture Framework. The NAF-compliant approach presented in the article enables meeting the requirements for the certification of flying systems in accordance with the guidelines DO-178 and DO-254, which are the basic documents on the basis of which the airworthiness of the system is assessed. The article presents the most important stages of designing unmanned systems that were used in a military project. An example of a system modeling method in UML and its extension, which is SysML, was also presented.
Multi-robot Decentralized Exploration using Weighted Random Selection

The exploration tasks using multi-robot systems require efficient coordination and information sharing between robots. The map creation is usually done through allocating frontiers,i.e., the boundary between explored and unknown regions of the map, to each robot. This paper introduces an efficient frontier allocation method based on weighted random selection for a decentralized multi-robot system. The weights are calculated based on the size of the frontiers and the distance between a robot and the frontiers. In this strategy, each robot identifies the available frontiers in a shared map and select the goal for exploration through a random selection. Even though a robot is randomly picking a frontier without coordination with other robots, a collective intelligence is developed at the swarm level. The proposed method is computationally efficient and uses minimal communication between the robots in a decentralized multi-robot system. As a result, the robots get allocated to frontier according to the calculated weight. A comparison with the nearest frontier exploration approach in computerized simulation demonstrates the efficiency of the proposed algorithm.
Deep Reinforcement Learning for Dynamic Power Allocation in Cell-free mmWave Massive MIMO

Numerical optimization has been investigated for decades to solve complex problems in wireless communication systems. This has resulted in many effective methods, e.g., the weighted minimum mean square error (WMMSE) algorithm. However, these methods often incur a high computational cost, making their application to time-constrained problems difficult. Recently data-driven methods have attracted a lot of attention due to their near-optimal performance with affordable computational cost. Deep reinforcement learning (DRL) is one of the most promising optimization methods for future wireless communication systems. In this paper, we investigate the DRL method, using a deep Q-network (DQN), to allocate the downlink transmission power in cell-free (CF) mmWave massive multiple-input multiple-output (MIMO) systems. We consider the sum spectral efficiency (SE) optimization for systems with mobile user equipment (UEs). The DQN is trained by the rewards of trial-and-error interactions with the environment over time. It takes as input the long-term fading information and it outputs the downlink transmission power values. The numerical results, obtained for a particular 3GPP scenario, show that DQN outperforms WMMSE in terms of sum-SE and has a much lower computational complexity.