Reviewerly/exHarmony huggingface dataset:
Aligning and linking entity mentions in image, text, and knowledge base

A picture is worth a thousand words, the adage reads. However, pictures cannot replace words in terms of their ability to efficiently convey clear (mostly) unambiguous and concise knowledge. Images and text, indeed reveal different and complementary information that, if combined will result in more information than the sum of that contained in a single media. The combination of visual and textual information can be obtained by linking the entities mentioned in the text with those shown in the pictures. To further integrate this with the agent’s background knowledge, an additional step is necessary. That is, either finding the entities in the agent knowledge base that correspond to those mentioned in the text or shown in the picture or, extending the knowledge base with the newly discovered entities. We call this complex task Visual-Textual-Knowledge Entity Linking (VTKEL). In this article, after providing a precise definition of the VTKEL task, we present two datasets called VTKEL1k* and VTKEL30k. These datasets consisting of images and corresponding captions, in which the image and textual mentions are both annotated with the corresponding entities typed according to the YAGO ontology. The datasets can be used for training and evaluating algorithms of the VTKEL task. Successively, we introduce a baseline algorithm called VT-LinKEr (Visual-Textual-Knowledge Entity Linker) for the solution of the VTKEL task. We evaluate the performances of VT-LinKEr on both datasets. We then contribute a supervised algorithm called ViTKan (Visual-Textual-Knowledge Alignment Network). We trained the ViTKan algorithm using features data of the VTKEL1k* dataset. The experimental results on VTKEL1k* and VTKEL30k datasets show that ViTKan substantially outperforms the baseline algorithm.
Neural Network Pruning Based on Improved Constrained Particle Swarm Optimization

Pruning has recently become ever-important research to compress deep neural networks. Previous pruning methods focus on removing filters, channels, or weights to reduce the large layer width of the models. Therefore, constrained by the dense model’s depth, these methods often cannot achieve a high latency reduction. Another line of work focuses on reducing the model’s depth by removing the irrelevant layers/blocks to guarantee a reduction in prediction time and resource usage. The weakness of these methods is that they used handcrafted rules to select the irrelevant layers/blocks that can lead to sub-optimal pruning. In this work, a novel method named IPSOPruner is proposed to automatically search the optimal pruned model that satisfies the resource constraint. IPSOPruner is based on improved particle swarm optimization (PSO). The main two improvements adopted into the original PSO to solve the layer/block pruning problem are: (1) the use of chaotic map and opposition based learning to speed up the convergence and avoid being trapped in a locally optimal solution and (2) the incorporation with the constrained handling method to lead the search to the feasible region. Finally, to recover the accuracy, the obtained pruned model will be fine-tuned. Experimental results on CIFAR-10 demonstrate that our IPSOPruner achieves superior results in depth and FLOPs reduction and also in an accuracy improvement.
VASA: Vector AES Instructions for Security Applications

Due to standardization, AES is today’s most widely used block cipher. Its security is well-studied and hardware acceleration is available on a variety of platforms. Following the success of the Intel AES New Instructions (AES-NI), support for Vectorized AES (VAES) has been added in 2018 and already shown to be useful to accelerate many implementations of AES-based algorithms where the order of AES evaluations is fixed a priori.
Performance Comparison of Automated Essay Graders Based on Various Language Models

Manual grading of essays is a time-consuming task. Students in large classes may therefore get delayed feedback and/or subpar feedback on their essay submissions. This has a negative impact on the students' learning experience. Many automated graders have been developed to relieve instructors from time-consuming grading tasks and providing timely feedback to students. In recent years, research in deep learning has resulted in several language models. This paper compares the performance of several automated graders that are built on top of three language models: BERT, RoBERTa and DeBERTa.
Identification and Detection of Cyberbullying on Facebook Using Machine Learning Algorithms

The use of social media platforms such as Facebook, Twitter, Instagram, WhatsApp, etc. have enabled a lot of people to communicate effectively and frequently with each other and this has enabled cyberbullying to occur more frequently while using these networks. Cyberbullying is known to be the cause of some serious health issues among social media users and creating a way to identify and detect this holds significant importance. This paper takes a look at unique features gotten from the Facebook dataset and develops a model that identifies and detect cyberbullying posts by applying machine learning algorithms (Naïve Bayes Algorithm and K-Nearest Neighbor). The project also uses a feature selection algorithm namely x2 test (Chi-Square test) to select important features which can improve the performance of the classifiers and decrease classification time. The result of this paper tends to detect cyberbullying in Facebook with a high degree of accuracy and also improve the performance of the machine learning classifiers.
Prediction of energy consumption of machine tools using multi-gene genetic programming

In the past, researchers have applied different analytical, numerical, and empirical modelling techniques to analyze energy consumption. In the present study, computational artificial intelligence-based Multi-Gene Genetic Programming is used to model the energy consumption of machine tool. The experiments were performed on a heavy-duty HMT lathe machine tool under a dry environment in the interest of sustainable machining. The Taguchi full factorial orthogonal array L27 was used to develop the experimental plan. The power consumption of the machine tool was measured using a Fluke 435 power analyzer. The dataset was split into training and testing data based on the 80–20 ratio. Further, 99.77% goodness of fit was achieved in training and 98.60% for testing the model. The adequacy of the model was tested by determining four error indices i.e. root means square error, mean absolute error, sum of squared error, and mean square error. The model is validated by conducting two hypothesis tests, t-test and f-test on predicted data. The hypothesis results confirm the model’s goodness of fit statistically, indicating that the proposed model can be easily applied in the manufacturing industry to predict energy consumption.
Efficient virtual data search for annotation‐free vehicle reidentification

Vehicle reidentification (re-ID) is the task of retrieving the same vehicle across nonoverlapping cameras, which has made significant progress with the help of abundant manually annotated real images. To avoid the time-consuming and tedious labeling of real images, virtual data sets with large-scale synthetic images have recently been constructed to perform annotation-free model training. However, current methods fail to exploit the potential of virtual data search, that is, searching valuable and representative virtual subdata set for efficient training. This paper presents a novel data sampling strategy from both semantic and feature levels to perform an effective data search. The semantic level determines the sample number of each vehicle identity via the consistency constraint of attribute distribution for source domain and target domain; while the feature level searches valuable and representative samples of each vehicle identity. To our knowledge, we are among the first attempts to search effective virtual data to perform annotation-free vehicle re-ID. Extensive cross-domain experiments from virtual vehicle re-ID data sets to real vehicle re-ID data sets show that our data sampling strategy can significantly reduce the training data volume and even boost the re-ID performance.
A high speed processor for elliptic curve cryptography over NIST prime field

Elliptic curve cryptography (ECC), as one of the public key cryptography systems, has been widely applied to many security applications. It is challenging to implement a scalar multiplication (SM) operation which has the highest computational complexity in ECC. In this study, we propose a hardware processor which achieves high speed and high security for ECC. We first present a three-clock cycle, divide-and-conquer multiplication algorithm which greatly reduces the number of execution cycles of multiplication. We then propose a dedicated multiplication hardware structure which reuses the multiplier and optimizes data path delay. To keep multiplication running in non-idle status and executing in parallel with other modular operations, the operation scheduling of point addition and point doubling has been re-designed and optimized based on an effective segmentation and pipeline strategy. Finally, under the premise of similar computing and hardware overhead, we propose an improved high-security SM algorithm which involves random points to resist side-channel attacks. On a 55 nm complementary metal oxide semiconductor application specific integrated circuit platform, the processor costs 463k gates and requires 0.028 ms for one SM. Our results indicate that the ECC processor is superior to other state-of-the-art designs reported in the literature in terms of speed and area-time product metrics.
Multi-view self-attention networks

Self-attention networks (SANs) have attracted an amount of research attention for their outstanding performance under the machine translation community. Recent studies proved that SANs can be further improved by exploiting different inductive biases, each of which guides SANs to learn a specific view of the input sentence, e.g., short-term dependencies, forward and backward views, as well as phrasal patterns. However, less studies investigate how these inductive techniques complementarily improve the capability of SANs and this would be an interesting question to be answered. In this paper we selected five inductive biases which are simple and not over parameterized to investigate their complementarily. We further propose multi-view self-attention networks, which jointly learn different linguistic aspects of the input sentence under a unified framework. Specifically, we propose and exploit a variety of inductive biases to regularize the conventional attention distribution. Different views are then aggregated by a hybrid attention mechanism to quantify and leverage the specific views and their associated representation conveniently. Experiments on various translation tasks demonstrate that different views are able to progressively improve the performance of SANs, and the proposed approach outperforms both the strong Transformer baseline and related models on Transformer-base and Transformer-big settings. Extensive analyses on 10 linguistic probing tasks verify that different views indeed tend to extract distinct linguistic features and our method gives highly effective improvements in their integration.1
Social dual-effect driven group modeling for neural group recommendation

Frequent group activities of human beings have become an indispensable part of human daily life. Group recommendation aims to recommend preferred items to a group of users in recommender systems. The existing solutions on group recommendation are to explore group modeling with or without the help of auxiliary information on social networks. However, we observe that the social factors can be explored directly in the group without employing social information on social networks. Towards this end, we study the social effect-based design guideline to drive group modeling. In this work, we propose a novel Social dual-Effect driven Attentive Group Recommendation method (SEAGR) that well utilizes social selection effect and social influence effect from sociology to explore group representation learning for neural group recommendation. Specifically, we construct the social selection-driven group inherent modeling from interaction-level and user-level. To mimic interaction-based dynamic group decision-making, we also design a social influence-driven attentive influence mining model in terms of users’ influence distinction in different groups. Based on these two components, an aggregative group representation is obtained. Moreover, neural recommendation for groups and users could be intensified reciprocally considering the impact of groups on users. The experimental results validate the effectiveness of the proposed method on three real-world datasets, and demonstrate its advantages over state-of-the-art methods in accuracy through extensive experiments.
Explanation guided cross-modal social image clustering

The integration of visual and semantic information has been found to play a role in increasing the accuracy of social image clustering methods. However, existing approaches are limited by the heterogeneity gap between the visual and semantic modalities, and their performances significantly degrade due to the commonly sparse and incomplete tags in semantic modality. To address these problems, we propose a novel clustering framework to discover reasonable categories in unlabeled social images under the guidance of human explanations. First of all, a novel Explanation Generation Model (EGM) is proposed to automatically boost textual information for the sparse and incomplete tags based on an extra lexical database with human knowledge. Then, a novel clustering algorithm called Group Constrained Information Maximization (GCIM) is proposed to learn image categories. In this algorithm, a new type of constraint named group level side information is unprecedentedly defined to bridge the well-known heterogeneity gap between visual and textual modalities. Finally, an interactive draw-and-merge optimization method is proposed to ensure an optimal solution. Extensive experiments on several social image datasets including NUS-Wide, IAPRTC, MIRFlickr, ESP-Game and COCO demonstrate the superiority of the proposed approach to state-of-the-art baselines.
Thinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code

Neural models of code are successfully tackling various prediction tasks, complementing and sometimes even outperforming traditional program analyses. While most work focuses on end-to-end evaluations of such models, it often remains unclear what the models actually learn, and to what extent their reasoning about code matches that of skilled humans. A poor understanding of the model reasoning risks deploying models that are right for the wrong reason, and taking decisions based on spurious correlations in the training dataset. This paper investigates to what extent the attention weights of effective neural models match the reasoning of skilled humans. To this end, we present a methodology for recording human attention and use it to gather 1,508 human attention maps from 91 participants, which is the largest such dataset we are aware of. Computing human-model correlations shows that the copy attention of neural models often matches the way humans reason about code (Spearman rank coefficients of 0.49 and 0.47), which gives an empirical justification for the intuition behind copy attention. In contrast, the regular attention of models is mostly uncorrelated with human attention. We find that models and humans sometimes focus on different kinds of tokens, e.g., strings are important to humans but mostly ignored by models. The results also show that human-model agreement positively correlates with accurate predictions by a model, which calls for neural models that even more closely mimic human reasoning. Beyond the insights from our study, we envision the release of our dataset of human attention maps to help understand future neural models of code and to foster work on human-inspired models.
Estimating ferric iron content in clinopyroxene using machine learning models

Abstract Clinopyroxene ferric iron content is an important consideration for garnet-clinopyroxene geothermometry and estimations of water storage in the Earth’s interior but remains difficult and expensive to measure. Here, we develop seven classic algorithms and machine learning methods to estimate Fe3+/ΣFe in clinopyroxene using major element data from electron microprobe analyses. The models were first trained using a large data set of clinopyroxene Fe3+/ΣFe values determined by Mössbauer spectroscopy and spanning a wide compositional range, with major uncertainties ranging from 0.25 to 0.3 and root-mean-square errors on the test data set ranging from 0.071 to 0.089. After dividing the entire data set into three compositional sub-data sets, the machine learning models were trained and compared for each sub-data set. Our results suggest that ensemble learning algorithms (random forest and Extra-Trees) perform better than principal component analysis-based elastic net polynomial, artificial neural network, artificial neural network ensemble, decision trees, and linear regressions. Using a sub-data set excluding clinopyroxene in spinel peridotite and omphacite in eclogite, the new models achieved uncertainties of 0.15 to 0.2 and root-mean-square errors on the test data set ranging from 0.051 to 0.078, decreasing prediction errors by 30–40%. By incorporating compositional data on coexisting spinel, new models for clinopyroxene in spinel peridotite show improved performance, indicating the interaction between spinel and clinopyroxene in spinel peridotite. Feature importance analysis shows Na+, Ca2+, and Mg2+ to be the most important for predicting Fe3+ content, supporting the coupled substitution between Ca2+-M2+ and Na+-M3+ in natural clinopyroxenes. The application of our models to garnet-clinopyroxene geothermometry greatly improves temperature estimates, achieving uncertainties of ±50 °C, compared with uncertainties of ±250 °C using previous models assuming all Fe as Fe2+ or calculating Fe3+ by charge conservation. Differences in the ferric iron contents, as calculated using the machine learning models, of clinopyroxenes that did or did not experience hydrogen diffusion during their crystallization from basaltic magma support a redox-driven mechanism for hydrogen diffusion in clinopyroxene.
Identifying solutions to meet unmet needs of family caregivers using human-centered design

Given the rapidly aging society, shrinking workforce, and reducing dependency ratio, there is an increasing challenge for family members to provide care for older adults. While a broad understanding of caregiver burden and its consequences have been studied across various contexts, there is a need to better understand this challenge among family caregivers in Asian societies.This study is a cross-sectional observational study. A total of 20 dyads of community-based older adults, who required assistance with at least one activities of daily living, and family caregivers in Thailand participated in the study. We used the first three stages out of five stages of human-centered design: empathize, define, and ideate.On average caregivers were 59.2 years old, with 43% still employed. Of the older adult participants, 10 were interviewed, the others had moderate-to-severe cognitive impairment. Based on the analysis, six caregiver personas (i.e. semi-fictional characters) are identified. Caregiver personas of "The 2-Jober" and "My Life Purpose" has the highest caregiver burden score whereas "The Spouse" has the lowest. Based on the specific needs of the caregiver persona "My Life Purpose", the team brainstormed more than 80 potential solutions which were classified into three categories of solutions that satisfied the metrics of desirability, feasibility and viability: distributed medical care system, technology-charged care network, and community gathering for rest and recuperation.These solutions are culturally sensitive given that they are built around established behavioral patterns. This is an illustration of a method of innovation that can be applied to bring a culturally specific understanding, and to develop products and services to enable further independent aging.
On-Device Neural Speech Synthesis

Recent advances in text-to-speech (TTS) synthesis, such as Tacotron and WaveRNN, have made it possible to construct a fully neural network based TTS system, by coupling the two components together. Such a system is conceptually sim-ple as it only takes grapheme or phoneme input, uses Mel-spectrogram as an intermediate feature, and directly generates speech samples. The system achieves quality equal or close to natural speech. However, the high computational cost of the system and issues with robustness have limited their usage in real-world speech synthesis applications and products. In this paper, we present key modeling improvements and optimization strategies that enable deploying these models, not only on GPU servers, but also on mobile devices. The proposed system can generate high-quality 24 kHz speech at 5x faster than real time on server and 3x faster than real time on mobile devices.
Dialoguing with Tangible Traces: A Method to Elicit Autoethnographic Narratives

Through this pictorial, I propose a generative method for the elicitation of autoethnographic themes for design research. This dialogue with tangible traces uses photography in conversation with tangible body maps (TBMs) towards harvesting evocative content for exploration. This dialogical tool functions as a way of generating conceptual knowledge inspired by the self and environment, including other subjectivities. This dialogue with tangible traces contributes to first-person design research, where traceability and communicability of outcomes become necessary preconditions for rigour. Although systematic, this method also respects the non-prescriptive and drifting nature of design.