Reviewerly/exHarmony huggingface dataset:
See, hear, read: Leveraging multimodality with guided attention for abstractive text summarization

In recent years, abstractive text summarization with multimodal inputs has started drawing attention due to its ability to accumulate information from different source modalities and generate a fluent textual summary. However, existing methods use short videos as the visual modality and short summary as the ground-truth, therefore, perform poorly on lengthy videos and long ground-truth summary. Additionally, there exists no benchmark dataset to generalize this task on videos of varying lengths. In this paper, we introduce AVIATE, the first large-scale dataset for abstractive text summarization with videos of diverse duration, compiled from presentations in well-known academic conferences like NDSS, ICML, NeurIPS, etc. We use the abstract of corresponding research papers as the reference summaries, which ensure adequate quality and uniformity of the ground-truth. We then propose FLORAL , a factorized multi-modal Transformer based decoder-only language model, which inherently captures the intra-modal and inter-modal dynamics within various input modalities for the text summarization task. FLORAL utilizes an increasing number of self-attentions to capture multimodality and performs significantly better than traditional encoder–decoder based networks. Extensive experiments illustrate that FLORAL achieves significant improvement over the baselines in both qualitative and quantitative evaluations on the existing How2 dataset for short videos and newly introduced AVIATE dataset for videos with diverse duration, beating the best baseline on the two datasets by 1.39 and 2.74 ROUGE-L points respectively.
Countermeasures Against Adversarial Examples in Radio Signal Classification

Deep learning algorithms have been shown to be powerful in many communication network design problems, including that in automatic modulation classification. However, they are vulnerable to carefully crafted attacks called adversarial examples. Hence, the reliance of wireless networks on deep learning algorithms poses a serious threat to the security and operation of wireless networks. In this letter, we propose for the first time a countermeasure against adversarial examples in modulation classification. Our countermeasure is based on a neural rejection technique, augmented by label smoothing and Gaussian noise injection, that allows to detect and reject adversarial examples with high accuracy. Our results demonstrate that the proposed countermeasure can protect deep-learning based modulation classification systems against adversarial examples.
Assembly sequence and path planning for monotone and nonmonotone assemblies with rigid and flexible parts

The Assembly Sequence and Path Planning (ASPP) problem deals with finding a proper sequence of parts to be assembled into a finished product and short assembly paths for each part. The problem is a combination of Assembly Sequence Planning (ASP) and Assembly Path Planning (APP) subproblems, which are both NP-complete and therefore intractable at large sizes. Nearly in all works on ASPP, it is assumed that planning is monotone (i.e., parts are moved only once, without considering intermediate placements) and each part is completely rigid. These are simplifying, yet limiting assumptions, since most assembled products like ships, aircraft, and automobiles are composed of rigid and flexible parts, and the generation of assembly sequence and path plans for most real-world products requires intermediate placement of parts to be taken into account. None of the existing works in the literature, however, have handled nonmonotone ASPP problems for rigid and flexible parts, and this issue remains largely untouched. In this paper, we present a new method called SPP-Flex for solving monotone and nonmonotone ASPP for rigid and flexible parts. SPP-Flex first utilizes a Directional Assembly Stress Matrix (DASM) for describing interference relations between all pairs of parts and the amounts of compressive stresses needed for assembling flexible parts and then obtains an initial tentative assembly sequence using a simple new greedy heuristic. Next, short assembly paths are iteratively computed and planned from initial to goal configurations of all parts using a novel sampling-based path planner called BXXT. If finding a free path for an active part fails due to obstruction of a previously assembled part, then such a part is identified, relocated, and its path replanned until the active part is moved to its final position. In case of failure again, if the part is flexible, through finite element analysis, it is determined if the part can still be assembled by undergoing elastic deformation. To evaluate the performance of the SPP-Flex and its components, two new products were designed and solved by four combinations of ASP and APP methods 20 times each, and the means and standard deviations of five performance criteria (total path length, total number of generated nodes and edges in the search tree, total number of collision (interference) checks, and total runtime) were calculated. Analysis of the computational results showed that the proposed greedy heuristic sequence planner together with the BXXT path planner/replanner outperformed other variations with at most 4.6% average gap in path length and 2.1% average gap in runtime compared to the best-found solution in all runs.
Defending Against Adversarial Attacks On Medical Imaging Ai System, Classification Or Detection?

Medical imaging AI systems such as disease classification and segmentation are increasingly inspired and transformed from computer vision based AI systems. Although an array of defense techniques have been developed and proved to be effective in computer vision, defending against adversarial attacks on medical images remains largely an uncharted territory due to their unique challenges: 1) label scarcity limits adversarial generalizability; 2) vastly similar and dominant fore- and background make it difficult for learning the discriminating features; and 3) crafted adversarial noises added to a highly standardized medical image can make it a hard sample for model to predict. In this paper, we propose a novel robust medical imaging AI framework based on Semi-Supervised Adversarial Training (SSAT) and Unsupervised Adversarial Detection (UAD), followed by a new measure for assessing systems adversarial risk. We systematically demonstrate the advantages of our robust medical imaging AI system over the existing adversarial defense techniques under diverse real-world settings of adversarial attacks using a benchmark OCT imaging data set.
Tracking Counterfeit Cryptocurrency End-to-end

The production of counterfeit money has a long history. It refers to the creation of imitation currency that is produced without the legal sanction of government. With the growth of the cryptocurrency ecosystem, there is expanding evidence that counterfeit cryptocurrency has also appeared. In this paper, we empirically explore the presence of counterfeit cryptocurrencies on Ethereum and measure their impact. By analyzing over 190K ERC-20 tokens (or cryptocurrencies) on Ethereum, we have identified $2,117$ counterfeit tokens that target 94 of the 100 most popular cryptocurrencies. We perform an end-to-end characterization of the counterfeit token ecosystem, including their popularity, creators and holders, fraudulent behaviors and advertising channels. Through this, we have identified two types of scams related to counterfeit tokens and devised techniques to identify such scams. We observe that over 7,104 victims were deceived in these scams, and the overall financial loss sums to a minimum of \$ 17 million (74,271.7 ETH). Our findings demonstrate the urgency to identify counterfeit cryptocurrencies and mitigate this threat.
Online Hate Speech and the Radical Right in Times of Pandemic: The Italian and English Cases

Social media is considered a particularly conducive arena for hate speech, a form of communication often linked to the radical right. The goal of this study is to offer an empirical contribution that comparatively explores the presence and features of hate speech in the social media discourse of the radical right (leaders and parties) in Italy and the UK during the first year of the pandemic. This mixed-methods study analyses 21,360 tweets using wordcloud analysis (to conceptually map the social media discourse of the radical right and mainstream parties), topic modelling (to identify the main topics of the radical right’s tweets and how they relate to Covid-19) and formalised content analysis (to better understand how hate speech is related to the virus). We find that radical right leaders have managed to bring exclusion-oriented issues to the agenda at this time of crisis, albeit in different ways, by emphasising different understandings of in-groups and out-groups in relation to Covid-19.
Learning sentiment sentence representation with multiview attention model

Self-attention mechanisms in deep neural networks, such as CNN, GRU and LSTM, have been proven to be effective for sentiment analysis. However, existing attention models tend to focus on individual tokens or aspect meanings in an expression. If a text contains information on multiple sentiments from different perspectives, the existing models will fail to extract the most critical and comprehensive features of the whole text. In the present study, a multiview attention model was proposed for learning sentence representation. Instead of using a single attention, multiple view vectors were used to map the attentions from different perspectives. Then, a fusion gate was adopted to combine these multiview attentions to draw a conclusion. To ensure the differences between multiview attentions, a regularization item was introduced to add a penalty to the loss function. In addition, the proposed model can be extended to other text tasks, such as questions and topics, to provide a comprehensive representation for the classification. Comparative experiments were conducted on both multiclass and multilabel classification datasets. The results revealed that the proposed method improves the performance of several previously proposed attention models.
At-bit estimation of rock density from real-time drilling data using deep learning with online calibration

We present a novel streaming learning approach, utilizing a deep neural network (DNN) to learn from data available during operation to estimate at-bit density using drilling parameters. Since every wellbore is different, the relationship between drilling parameters and at-bit density varies. Equipment used, well trajectory, friction and bit wear are examples of conditions that affect this relationship and makes a pre-trained model unable to represent an accurate input/output mapping applicable to all wells. However, using delayed density log measurements, continuously supervising updates to the model is possible during operation. The algorithm has been tested on drilling data from wells on a field operated by Equinor and compared to a standard deep learning approach, where results show that a streaming learning approach outperforms the traditional method. Statistical analyses have been performed to verify the statistical significance and effect size on the data sets. Data visualizations using a t-distributed Stochastic Neighbor Embedding (t-SNE) indicate that the relationship between drilling parameters and density log indeed vary between wellbores, making generalizability an issue for a traditional supervised learning approach to this problem, and motivating a streaming learning approach. Using the proposed method, more accurate at-bit estimates can be made, providing preliminary indications ahead of the tool placed 20–30 m behind the bit, which, dependent on rate of penetration (ROP), will be available 20–120min later.
Understanding the performance of knowledge graph embeddings in drug discovery

Knowledge Graphs (KG) and associated Knowledge Graph Embedding (KGE) models have recently begun to be explored in the context of drug discovery and have the potential to assist in key challenges such as target identification. In the drug discovery domain, KGs can be employed as part of a process which can result in lab-based experiments being performed, or impact on other decisions, incurring significant time and financial costs and most importantly, ultimately influencing patient healthcare. For KGE models to have impact in this domain, a better understanding of not only of performance, but also the various factors which determine it, is required. In this study we investigate, over the course of many thousands of experiments, the predictive performance of five KGE models on two public drug discovery-oriented KGs. Our goal is not to focus on the best overall model or configuration, instead we take a deeper look at how performance can be affected by changes in the training setup, choice of hyperparameters, model parameter initialisation seed and different splits of the datasets. Our results highlight that these factors have significant impact on performance and can even affect the ranking of models. Indeed these factors should be reported along with model architectures to ensure complete reproducibility and fair comparisons of future work, and we argue this is critical for the acceptance of use, and impact of KGEs in a biomedical setting.
Eye Tracking Interaction on Unmodified Mobile VR Headsets Using the Selfie Camera

Input methods for interaction in smartphone-based virtual and mixed reality (VR/MR) are currently based on uncomfortable head tracking controlling a pointer on the screen. User fixations are a fast and natural input method for VR/MR interaction. Previously, eye tracking in mobile VR suffered from low accuracy, long processing time, and the need for hardware add-ons such as anti-reflective lens coating and infrared emitters. We present an innovative mobile VR eye tracking methodology utilizing only the eye images from the front-facing (selfie) camera through the headset’s lens, without any modifications. Our system first enhances the low-contrast, poorly lit eye images by applying a pipeline of customised low-level image enhancements suppressing obtrusive lens reflections. We then propose an iris region-of-interest detection algorithm that is run only once. This increases the iris tracking speed by reducing the iris search space in mobile devices. We iteratively fit a customised geometric model to the iris to refine its coordinates. We display a thin bezel of light at the top edge of the screen for constant illumination. A confidence metric calculates the probability of successful iris detection. Calibration and linear gaze mapping between the estimated iris centroid and physical pixels on the screen results in low latency, real-time iris tracking. A formal study confirmed that our system’s accuracy is similar to eye trackers in commercial VR headsets in the central part of the headset’s field-of-view. In a VR game, gaze-driven user completion time was as fast as with head-tracked interaction, without the need for consecutive head motions. In a VR panorama viewer, users could successfully switch between panoramas using gaze.
EHR-Oriented Knowledge Graph System: Toward Efficient Utilization of Non-Used Information Buried in Routine Clinical Practice

Non-used clinical information has negative implications on healthcare quality. Clinicians pay priority attention to clinical information relevant to their specialties during routine clinical practices but may be insensitive or less concerned about information showing disease risks beyond their specialties, resulting in delayed and missed diagnoses or improper management. In this study, we introduced an electronic health record (EHR)-oriented knowledge graph system to efficiently utilize non-used information buried in EHRs. EHR data were transformed into a semantic patient-centralized information model under the ontology structure of a knowledge graph. The knowledge graph then creates an EHR data trajectory and performs reasoning through semantic rules to identify important clinical findings within EHR data. A graphical reasoning pathway illustrates the reasoning footage and explains the clinical significance for clinicians to better understand the neglected information. An application study was performed to evaluate unconsidered chronic kidney disease (CKD) reminding for non-nephrology clinicians to identify important neglected information. The study covered 71,679 patients in non-nephrology departments. The system identified 2,774 patients meeting CKD diagnosis criteria and 10,377 patients requiring high attention. A follow-up study of 5,439 patients showed that 82.1% of patients who met the diagnosis criteria and 61.4% of patients requiring high attention were confirmed to be CKD positive during follow-up research. The application demonstrated that the proposed approach is feasible and effective in clinical information utilization. Additionally, it's valuable as an explainable artificial intelligence to provide interpretable recommendations for specialist physicians to understand the importance of non-used data and make comprehensive decisions.
Harmonizing and publishing heterogeneous premodern manuscript metadata as Linked Open Data

Abstract Manuscripts are a crucial form of evidence for research into all aspects of premodern European history and culture, and there are numerous databases devoted to describing them in detail. This descriptive information, however, is typically available only in separate data silos based on incompatible data models and user interfaces. As a result, it has been difficult to study manuscripts comprehensively across these various platforms. To address this challenge, a team of manuscript scholars and computer scientists worked to create “Mapping Manuscript Migrations” (MMM), a semantic portal, and a Linked Open Data service. MMM stands as a successful proof of concept for integrating distinct manuscript datasets into a shared platform for research and discovery with the potential for future expansion. This paper will discuss the major products of the MMM project: a unified data model, a repeatable data transformation pipeline, a Linked Open Data knowledge graph, and a Semantic Web portal. It will also examine the crucial importance of an iterative process of multidisciplinary collaboration embedded throughout the project, enabling humanities researchers to shape the development of a digital platform and tools, while also enabling the same researchers to ask more sophisticated and comprehensive research questions of the aggregated data.
Automatic design of quantum feature maps

Abstract We propose a new technique for the automatic generation of optimal ad-hoc ansätze for classification by using quantum support vector machine. This efficient method is based on non-sorted genetic algorithm II multiobjective genetic algorithms which allow both maximize the accuracy and minimize the ansatz size. It is demonstrated the validity of the technique by a practical example with a non-linear dataset, interpreting the resulting circuit and its outputs. We also show other application fields of the technique that reinforce the validity of the method, and a comparison with classical classifiers in order to understand the advantages of using quantum machine learning.
Bandit algorithms to personalize educational chatbots

To emulate the interactivity of in-person math instruction, we developed MathBot, a rule-based chatbot that explains math concepts, provides practice questions, and offers tailored feedback. We evaluated MathBot through three Amazon Mechanical Turk studies in which participants learned about arithmetic sequences. In the first study, we found that more than 40% of our participants indicated a preference for learning with MathBot over videos and written tutorials from Khan Academy. The second study measured learning gains, and found that MathBot produced comparable gains to Khan Academy videos and tutorials. We solicited feedback from users in those two studies to emulate a real-world development cycle, with some users finding the lesson too slow and others finding it too fast. We addressed these concerns in the third and main study by integrating a contextual bandit algorithm into MathBot to personalize the pace of the conversation, allowing the bandit to either insert extra practice problems or skip explanations. We randomized participants between two conditions in which actions were chosen uniformly at random (i.e., a randomized A/B experiment) or by the contextual bandit. We found that the bandit learned a similarly effective pedagogical policy to that learned by the randomized A/B experiment while incurring a lower cost of experimentation. Our findings suggest that personalized conversational agents are promising tools to complement existing online resources for math education, and that data-driven approaches such as contextual bandits are valuable tools for learning effective personalization.
Consortium Blockchain for Cooperative Location Privacy Preservation in 5G-Enabled Vehicular Fog Computing

Privacy is a key requirement for connected vehicles. Cooperation between vehicles is mandatory for achieving location privacy preservation. However, non-cooperative vehicles can be a big issue to achieve this objective. To this end, we propose a novel monetary incentive scheme for cooperative location privacy preservation in 5G-enabled Vehicular Fog Computing. This scheme leverages a consortium blockchain-enabled fog layer and smart contracts to ensure a trusted and secure cooperative Pseudonym Changing Processes (PCPs). We also propose optimized smart contracts to reduce the monetary costs of vehicles while providing more location privacy preservation. Moreover, a resilient and lightweight Utility-based Delegated Byzantine Fault Tolerance (U-DBFT) consensus protocol is proposed to ensure fast and reliable block mining and validation. The performance analysis shows that our scheme has effective incentive techniques to stimulate non-cooperative vehicles and provides optimal monetary cost management and secure, private, fast validation of blocks.
Application of machine learning approaches to administrative claims data to predict clinical outcomes in medical and surgical patient populations

This study aimed to develop and validate a claims-based, machine learning algorithm to predict clinical outcomes across both medical and surgical patient populations.This retrospective, observational cohort study, used a random 5% sample of 770,777 fee-for-service Medicare beneficiaries with an inpatient hospitalization between 2009-2011. The machine learning algorithms tested included: support vector machine, random forest, multilayer perceptron, extreme gradient boosted tree, and logistic regression. The extreme gradient boosted tree algorithm outperformed the alternatives and was the machine learning method used for the final risk model. Primary outcome was 30-day mortality. Secondary outcomes were: rehospitalization, and any of 23 adverse clinical events occurring within 30 days of the index admission date.The machine learning algorithm performance was evaluated by both the area under the receiver operating curve (AUROC) and Brier Score. The risk model demonstrated high performance for prediction of: 30-day mortality (AUROC = 0.88; Brier Score = 0.06), and 17 of the 23 adverse events (AUROC range: 0.80-0.86; Brier Score range: 0.01-0.05). The risk model demonstrated moderate performance for prediction of: rehospitalization within 30 days (AUROC = 0.73; Brier Score: = 0.07) and six of the 23 adverse events (AUROC range: 0.74-0.79; Brier Score range: 0.01-0.02). The machine learning risk model performed comparably on a second, independent validation dataset, confirming that the risk model was not overfit.We have developed and validated a robust, claims-based, machine learning risk model that is applicable to both medical and surgical patient populations and demonstrates comparable predictive accuracy to existing risk models.