OpenRaiser/Intern-Atlas huggingface dataset:
TSGN: Transaction Subgraph Networks for Identifying Ethereum Phishing Accounts
Venue: arXiv cs.CR 2021
Year: 2021

Blockchain technology and, in particular, blockchain-based transaction offers us information that has never been seen before in the financial world. In contrast to fiat currencies, transactions through virtual currencies like Bitcoin are completely public. And these transactions of cryptocurrencies are permanently recorded on Blockchain and are available at any time. Therefore, this allows us to build transaction networks (TN) to analyze illegal phenomenons such as phishing scams in blockchain from a network perspective. In this paper, we propose a Transaction SubGraph Network (TSGN) based classification model to identify phishing accounts in Ethereum. Firstly we extract transaction subgraphs for each address and then expand these subgraphs into corresponding TSGNs based on the different mapping mechanisms. We find that TSGNs can provide more potential information to benefit the identification of phishing accounts. Moreover, Directed-TSGNs, by introducing direction attributes, can retain the transaction flow information that captures the significant topological pattern of phishing scams. By comparing with the TSGN, Directed-TSGN indeed has much lower time complexity, benefiting the graph representation learning. Experimental results demonstrate that, combined with network representation algorithms, the TSGN model can capture more features to enhance the classification algorithm and improve phishing nodes' identification accuracy in the Ethereum networks.
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation
Venue: arXiv cs.CL 2021
Year: 2021

We study the power of cross-attention in the Transformer architecture within the context of transfer learning for machine translation, and extend the findings of studies into cross-attention when training from scratch. We conduct a series of experiments through fine-tuning a translation model on data where either the source or target language has changed. These experiments reveal that fine-tuning only the cross-attention parameters is nearly as effective as fine-tuning all parameters (i.e., the entire translation model). We provide insights into why this is the case and observe that limiting fine-tuning in this manner yields cross-lingually aligned embeddings. The implications of this finding for researchers and practitioners include a mitigation of catastrophic forgetting, the potential for zero-shot translation, and the ability to extend machine translation models to several new language pairs with reduced parameter storage overhead.
Monte Carlo Elites: Quality-Diversity Selection as a Multi-Armed Bandit Problem
Venue: arXiv cs.NE 2021
Year: 2021

A core challenge of evolutionary search is the need to balance between exploration of the search space and exploitation of highly fit regions. Quality-diversity search has explicitly walked this tightrope between a population's diversity and its quality. This paper extends a popular quality-diversity search algorithm, MAP-Elites, by treating the selection of parents as a multi-armed bandit problem. Using variations of the upper-confidence bound to select parents from under-explored but potentially rewarding areas of the search space can accelerate the discovery of new regions as well as improve its archive's total quality. The paper tests an indirect measure of quality for parent selection: the survival rate of a parent's offspring. Results show that maintaining a balance between exploration and exploitation leads to the most diverse and high-quality set of solutions in three different testbeds.
An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth
Venue: arXiv cs.CV 2021
Year: 2021

Early detection and quantification of tumour growth would help clinicians to prescribe more accurate treatments and provide better surgical planning. However, the multifactorial and heterogeneous nature of lung tumour progression hampers identification of growth patterns. In this study, we present a novel method based on a deep hierarchical generative and probabilistic framework that, according to radiological guidelines, predicts tumour growth, quantifies its size and provides a semantic appearance of the future nodule. Unlike previous deterministic solutions, the generative characteristic of our approach also allows us to estimate the uncertainty in the predictions, especially important for complex and doubtful cases. Results of evaluating this method on an independent test set reported a tumour growth balanced accuracy of 74%, a tumour growth size MAE of 1.77 mm and a tumour segmentation Dice score of 78%. These surpassed the performances of equivalent deterministic and alternative generative solutions (i.e. probabilistic U-Net, Bayesian test dropout and Pix2Pix GAN) confirming the suitability of our approach.
Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning
Venue: arXiv cs.CL 2021
Year: 2021

Aiming to generate a set of keyphrases, Keyphrase Generation (KG) is a classical task for capturing the central idea from a given document. Based on Seq2Seq models, the previous reinforcement learning framework on KG tasks utilizes the evaluation metrics to further improve the well-trained neural models. However, these KG evaluation metrics such as $F_1@5$ and $F_1@M$ are only aware of the exact correctness of predictions on phrase-level and ignore the semantic similarities between similar predictions and targets, which inhibits the model from learning deep linguistic patterns. In response to this problem, we propose a new fine-grained evaluation metric to improve the RL framework, which considers different granularities: token-level $F_1$ score, edit distance, duplication, and prediction quantities. On the whole, the new framework includes two reward functions: the fine-grained evaluation score and the vanilla $F_1$ score. This framework helps the model identifying some partial match phrases which can be further optimized as the exact match ones. Experiments on KG benchmarks show that our proposed training framework outperforms the previous RL training frameworks among all evaluation scores. In addition, our method can effectively ease the synonym problem and generate a higher quality prediction. The source code is available at \url{https://github.com/xuyige/FGRL4KG}.
Multilingual Knowledge Graph Completion with Joint Relation and Entity Alignment
Venue: arXiv cs.AI 2021
Year: 2021

Knowledge Graph Completion (KGC) predicts missing facts in an incomplete Knowledge Graph. Almost all of existing KGC research is applicable to only one KG at a time, and in one language only. However, different language speakers may maintain separate KGs in their language and no individual KG is expected to be complete. Moreover, common entities or relations in these KGs have different surface forms and IDs, leading to ID proliferation. Entity alignment (EA) and relation alignment (RA) tasks resolve this by recognizing pairs of entity (relation) IDs in different KGs that represent the same entity (relation). This can further help prediction of missing facts, since knowledge from one KG is likely to benefit completion of another. High confidence predictions may also add valuable information for the alignment tasks. In response, we study the novel task of jointly training multilingual KGC, relation alignment and entity alignment models. We present ALIGNKGC, which uses some seed alignments to jointly optimize all three of KGC, EA and RA losses. A key component of ALIGNKGC is an embedding based soft notion of asymmetric overlap defined on the (subject, object) set signatures of relations this aids in better predicting relations that are equivalent to or implied by other relations. Extensive experiments with DBPedia in five languages establish the benefits of joint training for all tasks, achieving 10-32 MRR improvements of ALIGNKGC over a strong state-of-the-art single-KGC system completion model over each monolingual KG . Further, ALIGNKGC achieves reasonable gains in EA and RA tasks over a vanilla completion model over a KG that combines all facts without alignment, underscoring the value of joint training for these tasks.
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

Video-text retrieval plays an essential role in multi-modal research and has been widely used in many real-world web applications. The CLIP (Contrastive Language-Image Pre-training), an image-language pre-training model, has demonstrated the power of visual concepts learning from web collected image-text datasets. In this paper, we propose a CLIP4Clip model to transfer the knowledge of the CLIP model to video-language retrieval in an end-to-end manner. Several questions are investigated via empirical studies: 1) Whether image feature is enough for video-text retrieval? 2) How a post-pretraining on a large-scale video-text dataset based on the CLIP affect the performance? 3) What is the practical mechanism to model temporal dependency between video frames? And 4) The Hyper-parameters sensitivity of the model on video-text retrieval task. Extensive experimental results present that the CLIP4Clip model transferred from the CLIP can achieve SOTA results on various video-text retrieval datasets, including MSR-VTT, MSVC, LSMDC, ActivityNet, and DiDeMo. We release our code at https://github.com/ArrowLuo/CLIP4Clip.
A Simulated Experiment to Explore Robotic Dialogue Strategies for People with Dementia
Venue: arXiv cs.RO 2021
Year: 2021

People with Alzheimer's disease and related dementias (ADRD) often show the problem of repetitive questioning, which brings a great burden on persons with ADRD (PwDs) and their caregivers. Conversational robots hold promise of coping with this problem and hence alleviating the burdens on caregivers. In this paper, we proposed a partially observable markov decision process (POMDP) model for the PwD-robot interaction in the context of repetitive questioning, and used Q-learning to learn an adaptive conversation strategy (i.e., rate of follow-up question and difficulty of follow-up question) towards PwDs with different cognitive capabilities and different engagement levels. The results indicated that Q-learning was helpful for action selection for the robot. This may be a useful step towards the application of conversational social robots to cope with repetitive questioning in PwDs.
Carrot and Stick approaches revisited when managing Technical Debt in an educational context
Venue: arXiv cs.SE 2021
Year: 2021

Technical Debt management is an important aspect in the training of Software Engineering students. In this paper we study the effect of two assessment strategies in an educational context: One based on penalisation, the other based on rewards. Both are applied to assignments where the students develop a project focusing on keeping a low technical debt level, and obtaining a high quality code. We describe the design, tools and context of the strategies applied. SonarQube, a tool commonly used in production environments, is used for measuring the metrics. The penalisation strategy is based on a SonarQube quality gate. The reward strategy is based on a contest, where an automatic judge tool is devised to provide an online leaderboard with a classification based on the SonarQube metrics. An empirical study is conducted to determine which of the strategies works better to help the students/trainees keep the Technical Debt low. Statistically significant results are obtained in 5 of the 8 analysed metrics, showing that the reward strategy works much better. The effect size of the executed statistical tests is analysed, resulting in medium and large effect size in the majority of the analysed metrics.
Sentiment Classification in Swahili Language Using Multilingual BERT
Venue: arXiv cs.CL 2021
Year: 2021

The evolution of the Internet has increased the amount of information that is expressed by people on different platforms. This information can be product reviews, discussions on forums, or social media platforms. Accessibility of these opinions and peoples feelings open the door to opinion mining and sentiment analysis. As language and speech technologies become more advanced, many languages have been used and the best models have been obtained. However, due to linguistic diversity and lack of datasets, African languages have been left behind. In this study, by using the current state-of-the-art model, multilingual BERT, we perform sentiment classification on Swahili datasets. The data was created by extracting and annotating 8.2k reviews and comments on different social media platforms and the ISEAR emotion dataset. The data were classified as either positive or negative. The model was fine-tuned and achieve the best accuracy of 87.59%.
The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors
Venue: arXiv cs.RO 2021
Year: 2021

Demonstrating acrobatic behavior of a humanoid robot such as flips and spinning jumps requires systematic approaches across hardware design, motion planning, and control. In this paper, we present a new humanoid robot design, an actuator-aware kino-dynamic motion planner, and a landing controller as part of a practical system design for highly dynamic motion control of the humanoid robot. To achieve the impulsive motions, we develop two new proprioceptive actuators and experimentally evaluate their performance using our custom-designed dynamometer. The actuator's torque, velocity, and power limits are reflected in our kino-dynamic motion planner by approximating the configuration-dependent reaction force limits and in our dynamics simulator by including actuator dynamics along with the robot's full-body dynamics. For the landing control, we effectively integrate model-predictive control and whole-body impulse control by connecting them in a dynamically consistent way to accomplish both the long-time horizon optimal control and high-bandwidth full-body dynamics-based feedback. Actuators' torque output over the entire motion are validated based on the velocity-torque model including battery voltage droop and back-EMF voltage. With the carefully designed hardware and control framework, we successfully demonstrate dynamic behaviors such as back flips, front flips, and spinning jumps in our realistic dynamics simulation.
RingCNN: Exploiting Algebraically-Sparse Ring Tensors for Energy-Efficient CNN-Based Computational Imaging
Venue: arXiv cs.NE 2021
Year: 2021

In the era of artificial intelligence, convolutional neural networks (CNNs) are emerging as a powerful technique for computational imaging. They have shown superior quality for reconstructing fine textures from badly-distorted images and have potential to bring next-generation cameras and displays to our daily life. However, CNNs demand intensive computing power for generating high-resolution videos and defy conventional sparsity techniques when rendering dense details. Therefore, finding new possibilities in regular sparsity is crucial to enable large-scale deployment of CNN-based computational imaging. In this paper, we consider a fundamental but yet well-explored approach -- algebraic sparsity -- for energy-efficient CNN acceleration. We propose to build CNN models based on ring algebra that defines multiplication, addition, and non-linearity for n-tuples properly. Then the essential sparsity will immediately follow, e.g. n-times reduction for the number of real-valued weights. We define and unify several variants of ring algebras into a modeling framework, RingCNN, and make comparisons in terms of image quality and hardware complexity. On top of that, we further devise a novel ring algebra which minimizes complexity with component-wise product and achieves the best quality using directional ReLU. Finally, we implement an accelerator, eRingCNN, in two settings, n=2 and 4 (50% and 75% sparsity), with 40 nm technology to support advanced denoising and super-resolution at up to 4K UHD 30 fps. Layout results show that they can deliver equivalent 41 TOPS using 3.76 W and 2.22 W, respectively. Compared to the real-valued counterpart, our ring convolution engines for n=2 achieve 2.00x energy efficiency and 2.08x area efficiency with similar or even better image quality. With n=4, the efficiency gains of energy and area are further increased to 3.84x and 3.77x with 0.11 dB drop of PSNR.
A Negation Quantum Decision Model to Predict the Interference Effect in Categorization
Venue: arXiv cs.AI 2021
Year: 2021

Categorization is a significant task in decision-making, which is a key part of human behavior. An interference effect is caused by categorization in some cases, which breaks the total probability principle. A negation quantum model (NQ model) is developed in this article to predict the interference. Taking the advantage of negation to bring more information in the distribution from a different perspective, the proposed model is a combination of the negation of a probability distribution and the quantum decision model. Information of the phase contained in quantum probability and the special calculation method to it can easily represented the interference effect. The results of the proposed NQ model is closely to the real experiment data and has less error than the existed models.
An Oracle for Guiding Large-Scale Model/Hybrid Parallel Training of Convolutional Neural Networks
Venue: arXiv cs.DC 2021
Year: 2021

Deep Neural Network (DNN) frameworks use distributed training to enable faster time to convergence and alleviate memory capacity limitations when training large models and/or using high dimension inputs. With the steady increase in datasets and model sizes, model/hybrid parallelism is deemed to have an important role in the future of distributed training of DNNs. We analyze the compute, communication, and memory requirements of Convolutional Neural Networks (CNNs) to understand the trade-offs between different parallelism approaches on performance and scalability. We leverage our model-driven analysis to be the basis for an oracle utility which can help in detecting the limitations and bottlenecks of different parallelism approaches at scale. We evaluate the oracle on six parallelization strategies, with four CNN models and multiple datasets (2D and 3D), on up to 1024 GPUs. The results demonstrate that the oracle has an average accuracy of about 86.74% when compared to empirical results, and as high as 97.57% for data parallelism.
Digital twins with distributed particle simulation for mine-to-mill material tracking
Venue: arXiv cs.CE 2021
Year: 2021

Systems for transport and processing of granular media are challenging to analyse, operate and optimise. In the mining and mineral processing industries these systems are chains of processes with complex interplay between the equipment, control, and the processed material. The material properties have natural variations that are usually only known at certain locations. Therefore, we explore a material-oriented approach to digital twins with a particle representation of the granular media. In digital form, the material is treated as pseudo-particles, each representing a large collection of real particles of various sizes, shapes and, mineral properties. Movements and changes in the state of the material are determined by the combined data from control systems, sensors, vehicle telematics, and simulation models at locations where no real sensors can see. The particle-based representation enables material tracking along the chain of processes. Each digital particle can act as a carrier of observational data generated by the equipment as it interacts with the real material. This makes it possible to better learn material properties from process observations, and to predict the effect on downstream processes. We test the technique on a mining simulator and demonstrate analysis that can be performed using data from cross-system material tracking.
No comments: Addressing commentary sections in websites' analyses
Venue: arXiv cs.CL 2021
Year: 2021

Removing or extracting the commentary sections from a series of websites is a tedious task, as no standard way to code them is widely adopted. This operation is thus very rarely performed. In this paper, we show that these commentary sections can induce significant biases in the analyses, especially in the case of controversial Highlights $\bullet$ Commentary sections can induce biases in the analysis of websites' contents $\bullet$ Analyzing these sections can be interesting per se. $\bullet$ We illustrate these points using a corpus of anti-vaccine websites. $\bullet$ We provide guidelines to remove or extract these sections.
Bribery as a Measure of Candidate Success: Complexity Results for Approval-Based Multiwinner Rules
Venue: arXiv cs.GT 2021
Year: 2021

We study the problem of bribery in multiwinner elections, for the case where the voters cast approval ballots (i.e., sets of candidates they approve) and the bribery actions are limited to: adding an approval to a vote, deleting an approval from a vote, or moving an approval within a vote from one candidate to the other. We consider a number of approval-based multiwinner rules (AV, SAV, GAV, RAV, approval-based Chamberlin--Courant, and PAV). We find the landscape of complexity results quite rich, going from polynomial-time algorithms through NP-hardness with constant-factor approximations, to outright inapproximability. Moreover, in general, our problems tend to be easier when we limit out bribery actions on increasing the number of approvals of the candidate that we want to be in a winning committee (i.e., adding approvals only for this preferred candidate, or moving approvals only to him or her). We also study parameterized complexity of our problems, with a focus on parameterizations by the numbers of voters or candidates.
Cures, Treatments and Vaccines for Covid-19: International differences in interest on Twitter
Venue: arXiv cs.DL 2021
Year: 2021

Since the Covid-19 pandemic is a global threat to health that few can fully escape, it has given a unique opportunity to study international reactions to a common problem. Such reactions can be partly obtained from public posts to Twitter, allowing investigations of changes in interest over time. This study analysed English-language Covid-19 tweets mentioning cures, treatments, or vaccines from 1 January 2020 to 8 April 2021, seeking trends and international differences. The results have methodological limitations but show a tendency for countries with a lower human development index score to tweet more about cures, although they were a minor topic for all countries. Vaccines were discussed about as much as treatments until July 2020, when they generated more interest because of developments in Russia. The November 2020 Pfizer-BioNTech preliminary Phase 3 trials results generated an immediate and sustained sharp increase, however, followed by a continuing roughly linear increase in interest for vaccines until at least April 2021. Against this background, national deviations from the average were triggered by country-specific news about cures, treatments or vaccines. Nevertheless, interest in vaccines in all countries increased in parallel to some extent, despite substantial international differences in national regulatory approval and availability. The results also highlight that unsubstantiated claims about alternative medicine remedies gained traction in several countries, apparently posing a threat to public health.