Reviewerly/exHarmony huggingface dataset:
Dynamic Pruning of Regions for Image–Sentence Matching

Image–sentence matching is becoming increasingly essential in the integrated understanding of vision and language. Prior approaches apply a pre-trained detection model to extract region features and explore fine-grained relationships between image and sentence by aggregating the similarities of all region–word pairs. However, all images are represented by the same number of regions, regardless of their respective semantic complexity, which results in a large number of redundant regions interfering with semantic inference and bringing additional computational burden. To address the lack of flexibility in image representation and information redundancy, a novel method named Dynamic Pruning of Regions for Image–Sentence Matching (DPRM) is proposed to efficiently capture relationships between text and image. In particular, a dynamic region pruning module is presented to dynamically select the appropriate number of regions according to the semantic complexity of each image, thus pruning redundant regions and reducing superfluous computations. Moreover, an inter-modality refinement module is designed to refine the fine-grained relationships of region–word pairs by retaining meaningful interaction features and suppressing interference from redundant alignments, which learns the more accurate semantic correspondences. Extensive experiments on MSCOCO and Flickr30K datasets prove the superiority of DPRM compared with previous approaches.
Carvings in stone: design research for public health investigations in the age of COVID-19

The COVID-19 pandemic presented complex challenges to public health research involving human subjects, necessitating creative thinking to maintain safe and productive qualitative data collection. In this paper, we describe how an interdisciplinary team overcame these challenges by translating in-person workshops into at-home design probes to explore connections between university students’ climate change attitudes and their health. Probes were purposively designed to facilitate and focus participation during lockdown with two key interactional components: (1) participant ⇔ probe, and (2) participant ⇔ physical environment. Thematic analysis of 18 completed probes (replete with elicitive drawing, etched stone, mapping, and writing activities) revealed a rich spectrum of climate change perceptions, eco-emotions, and key areas of impact. This case study provides insight into how pandemic-imposed restrictions presented a novel opportunity to reconceptualize how we collect qualitative data in public health research and summarizes the unique benefits of integrating such design-based approaches into the field moving forward.
Magnifying multimodal forgery clues for Deepfake detection

Advancements in computer vision and deep learning have led to difficulty in distinguishing the generated Deepfake media. In addition, recent forgery techniques also modify the audio information based on the forged video, which brings new challenges. However, due to the cross-modal bias, recent multimodal detection methods do not well explore the intra-modal and cross-modal forgery clues, which leads to limited detection performance. In this paper, we propose a novel audio-visual aware multimodal Deepfake detection framework to magnify intra-modal and cross-modal forgery clues. Firstly, to capture temporal intra-modal defects, Forgery Clues Magnification Transformer (FCMT) module is proposed to magnify forgery clues based on sequence-level relationships. Then, the Distribution Difference based Inconsistency Computing (DDIC) module based on Jensen–Shannon divergence is designed to adaptively align multimodal information for further magnifying the cross-modal inconsistency. Next, we further explore spatial artifacts by connecting multi-scale feature representation to provide comprehensive information. Finally, a feature fusion module is designed to adaptively fuse features to generate a more discriminative feature. Experiments demonstrate that the proposed framework outperforms independently trained models, and at the same time, yields superior generalization capability on unseen types of Deepfake.
Multi-label adversarial fine-grained cross-modal retrieval

Most supervised cross-modal approaches transform features into a common representation space in which semantic similarity can be measured directly. However, there exist modal specific features in the common semantic space and most methods cannot fully eliminate them. In order to bridge the semantic gap and eliminate modal specific features, we propose a novel Multi-label Adversarial Fine-grained Cross-modal Retrieval Based on Transformer (MLAT). MLAT constructs a semantic consistency enhanced module (SCE) which includes the semantic mask attention module and a fine-grained feature generator based on transformer. It learns fine-grained semantic information to preserve the high-level semantic relevance and eliminate modal specific features. In order to narrow the distance between common representations and further eliminate modal specific features, we construct a multi-stage adversarial learning module to optimize feature representations. Furthermore, we design a label graph network based on graph attention network (GAT) to better explore the semantic correlations between labels and learn a classifier. Three benchmark datasets are synthesized to demonstrate the superiority of MLAT method.
Unpacking Preservice Elementary Education Majors’ Pre-Covid Experiences with Instructional Technology: Implications for Post-Covid Technology Use

Virtual learning during the Covid-19 pandemic has made the effective use of technology, as a learning tool, more important than it has ever been. One of the challenges within instructional technology courses for preservice elementary education majors was whether or not to train students to use technology as a means to an end or to focus on technology skills. Instructional technology courses could reinforce traditional approaches or encourage higher order thinking or acquisition of twenty first century skills for formal or informal settings. This study explored the pre-Covid inclinations and experiences of undergraduate preservice teachers who engaged in various types of technology that facilitated both the learning of content and the building of technological skills to varying degrees. This basic qualitative exploratory study looked at preservice teachers’ perceptions about their engagement with the technologies and about their own capabilities. The findings show connections that were most salient to the preservice teachers. These pre-Covid pandemic findings have implications for the current state of instructional technology and learning using technology in the post-Covid pandemic era.
Giottus story—A case study of India’s highly rated crypto exchange

Giottus is one of India’s most preferred cryptocurrency exchanges, which saw a user growth rate of more than 400% during the pandemic. The platform offers unique and innovative products and secured transactions, making its application interface very user-friendly. This case study elucidates the various factors that enable the immense growth of Giottus exchange in the cryptocurrency market. The case also discusses cryptocurrency’s existing regulatory environment and its challenges. This teaching case also addresses the significant issues that deter the growth of cryptocurrency exchanges in India. It also addresses the future pathway for crypto investments in India and how exchanges like Giottus play an essential role in regulating their market vulnerability.
A study on cloud data access through browser credential migration in Windows environment

Various user data stored in cloud services for data continuity and efficiency is one of the main collection targets in digital forensic investigation. Some forensic tools collect cloud data based on user account and password, or provide data collection functions based on user credentials stored in the web browser. However, because many web services require additional authentication using user devices to protect user data, access using only the account and password is becoming difficult. In the case of credentials generated by auto-login, it does not work or requires re-authentication when moved to the investigator's device. This is so that other devices cannot utilize the credentials that are kept on the device due to security measures. In this paper, we propose a new method to migrate the credentials stored by the web browser to other devices and effectively utilize them, unlike the forensic method that involves using local credentials. Our analysis revealed that the majority of browsers encrypt and store credentials, so we researched credential decryption methods. We proceeded with the migration; move and encrypt the decrypted credentials to the investigator's device, or move the not encrypted credentials simply. As a result, we conducted credentials migration experiments on a total of 28 browsers, among which we have clarified that migration is possible in all browsers except three that do not store data, such as Tor. We verified that it is possible to log in and collect data on 20 types of web services that are frequently used using migrated credentials. Although the approach we propose is straightforward, it allows for effective and efficient cloud data collection in digital forensic investigation.
Relevance meets calibration: Triple calibration distance design for neighbour-based recommender systems

Calibrated recommendations are devoted to revealing the various preferences of users with the appropriate proportions in the recommendation list. Most of the existing calibrated-oriented recommendations take an extra postprocessing step to rerank the initial outputs. However, applying this postprocessing strategy may decrease the recommendation relevance, since the origin accurate outputs have been scattered, and they usually ignore the calibration between pairwise users/items. Instead of reranking the recommendation outputs, this article is dedicated to modifying the criterion of neighbour users’ selection, where we look forward to strengthening the recommendation relevance by calibrating the neighbourhood. We propose the first-order, second-order and the third-order calibration distance based on the motivation that if a user has a similar genre distribution or genre rating schema towards the target user, then his or her suggestions will be more useful for rating prediction. We also provide an equivalent transformation for the original method to speed up the algorithm with solid theoretical proof. Experimental analysis on two publicly available data sets empirically shows that our approaches are better than some of the state-of-the-art methods in terms of recommendation relevance, calibration and efficiency.
Efficient Density-peaks Clustering Algorithms on Static and Dynamic Data in Euclidean Space

Clustering multi-dimensional points is a fundamental task in many fields, and density-based clustering supports many applications because it can discover clusters of arbitrary shapes. This article addresses the problem of Density-Peaks Clustering (DPC) in Euclidean space. DPC already has many applications, but its straightforward implementation incurs O ( n 2 ) time, where n is the number of points, thereby does not scale to large datasets. To enable DPC on large datasets, we first propose empirically efficient exact DPC algorithm, Ex-DPC. Although this algorithm is much faster than the straightforward implementation, it still suffers from O ( n 2 ) time theoretically. We hence propose a new exact algorithm, Ex-DPC++, that runs in o ( n 2 ) time. We accelerate their efficiencies by leveraging multi-threading. Moreover, real-world datasets may have arbitrary updates (point insertions and deletions). It is hence important to support efficient cluster updates. To this end, we propose D-DPC for fully dynamic DPC. We conduct extensive experiments using real datasets, and our experimental results demonstrate that our algorithms are efficient and scalable.
Artificial Intelligence Approaches and Mechanisms for Big Data Analytics: A Systematic Study

Recent advances in sensor networks and Internet of Things (IoT) technology have led to the collection of large amounts of data. Searching for this much information requires a more efficient and accurate analysis method. Artificial intelligence (AI) techniques such as machine learning and evolutionary algorithms can provide better, faster and more efficient results in big data. Despite this interest, to the best of our knowledge, there has not been a thorough study of the various artificial intelligence techniques for analyzing big data. Most products in the literature focus on the extraordinary ability of intelligence. Recently, challenges such as security, security, integrity, sustainability and utility have emerged during the development of intelligent systems. As the use of smart technology increases, so do new challenges. Obviously, understanding of issues like security will change as we move from narrow AI to super AI. Also, recent advances in human intelligence cannot take place without considering all the problems in creating intelligent machines.
SCOPE: predicting future diagnoses in office visits using electronic health records

We propose an interpretable and scalable model to predict likely diagnoses at an encounter based on past diagnoses and lab results. This model is intended to aid physicians in their interaction with the electronic health records (EHR). To accomplish this, we retrospectively collected and de-identified EHR data of 2,701,522 patients at Stanford Healthcare over a time period from January 2008 to December 2016. A population-based sample of patients comprising 524,198 individuals (44% M, 56% F) with multiple encounters with at least one frequently occurring diagnosis codes were chosen. A calibrated model was developed to predict ICD-10 diagnosis codes at an encounter based on the past diagnoses and lab results, using a binary relevance based multi-label modeling strategy. Logistic regression and random forests were tested as the base classifier, and several time windows were tested for aggregating the past diagnoses and labs. This modeling approach was compared to a recurrent neural network based deep learning method. The best model used random forest as the base classifier and integrated demographic features, diagnosis codes, and lab results. The best model was calibrated and its performance was comparable or better than existing methods in terms of various metrics, including a median AUROC of 0.904 (IQR [0.838, 0.954]) over 583 diseases. When predicting the first occurrence of a disease label for a patient, the median AUROC with the best model was 0.796 (IQR [0.737, 0.868]). Our modeling approach performed comparably as the tested deep learning method, outperforming it in terms of AUROC (p < 0.001) but underperforming in terms of AUPRC (p < 0.001). Interpreting the model showed that the model uses meaningful features and highlights many interesting associations among diagnoses and lab results. We conclude that the multi-label model performs comparably with RNN based deep learning model while offering simplicity and potentially superior interpretability. While the model was trained and validated on data obtained from a single institution, its simplicity, interpretability and performance makes it a promising candidate for deployment.
Probability-Distribution-Guided Adversarial Sample Attacks for Boosting Transferability and Interpretability

In recent years, with the rapid development of technology, artificial intelligence (AI) security issues represented by adversarial sample attack have aroused widespread concern in society. Adversarial samples are often generated by surrogate models and then transfer to attack the target model, and most AI models in real-world scenarios belong to black boxes; thus, transferability becomes a key factor to measure the quality of adversarial samples. The traditional method relies on the decision boundary of the classifier and takes the boundary crossing as the only judgment metric without considering the probability distribution of the sample itself, which results in an irregular way of adding perturbations to the adversarial sample, an unclear path of generation, and a lack of transferability and interpretability. In the probabilistic generative model, after learning the probability distribution of the samples, a random term can be added to the sampling to gradually transform the noise into a new independent and identically distributed sample. Inspired by this idea, we believe that by removing the random term, the adversarial sample generation process can be regarded as the static sampling of the probabilistic generative model, which guides the adversarial samples out of the original probability distribution and into the target probability distribution and helps to boost transferability and interpretability. Therefore, we proposed a score-matching-based attack (SMBA) method to perform adversarial sample attacks by manipulating the probability distribution of the samples, which showed good transferability in the face of different datasets and models and provided reasonable explanations from the perspective of mathematical theory and feature space. Compared with the current best methods based on the decision boundary of the classifier, our method increased the attack success rate by 51.36% and 30.54% to the maximum extent in non-targeted and targeted attack scenarios, respectively. In conclusion, our research established a bridge between probabilistic generative models and adversarial samples, provided a new entry angle for the study of adversarial samples, and brought new thinking to AI security.
Exploring the Educational Value and Impact of Vision-Impairment Simulations on Sympathy and Empathy with XREye

To create a truly accessible and inclusive society, we need to take the more than 2.2 billion people with vision impairments worldwide into account when we design our cities, buildings, and everyday objects. This requires sympathy and empathy, as well as a certain level of understanding of the impact of vision impairments on perception. In this study, we explore the potential of an extended version of our vision-impairment simulation system XREye to increase sympathy and empathy and evaluate its educational value in an expert study with 56 educators and education students. We include data from a previous study in related work on sympathy and empathy as a baseline for comparison with our data. Our results show increased sympathy and empathy after experiencing XREye and positive feedback regarding its educational value. Hence, we believe that vision-impairment simulations, such as XREye, have merit to be used for educational purposes in order to increase awareness for the challenges people with vision impairments face in their everyday lives.
Сравнение и отбор ситуаций в системах вывода решений на прецедентах для «умной» фермы

The trend of development of smart farms is aimed at their becoming fully autonomous, robotic enterprises. The prospects for the intellectualization of agricultural production and smart farms, in particular, today are associated with the development of technology systems used to detect, recognize complex production situations and search for effective solutions in these situations. The article presents the concept of such a decision support system on smart farms using the method of decision support based on case-based reasoning - CBR system. Its implementation requires a number of non-trivial tasks, which include, first of all, the tasks of formalizing the presentation of situations and creating methods for comparing and retrieving situations from the KB on this basis. In this study, a smart farm is presented as a complex technological object consisting of interrelated components, which are the technological subsystems of a smart farm, the products produced, the objects of the operational environment, as well as the relationships between them. To implement algorithms for situational decision-making based on precedents, a formalized representation of the situation in the form of a multivector is proposed. This allowed us to develop a number of models of the trained similarity function between situations. The conducted experiments have shown the operability of the proposed models, on the basis of which ensemble architecture of a neural network has been developed for comparing situations and selecting them from the knowledge base in decision-making processes. Of practical interest is monitoring the condition of plants by their video and photo images, which allows detecting undesirable plant conditions (diseases), which can serve as a signal to activate the process of searching for solutions in the knowledge base.
Learning graph structures with transformer for weakly supervised semantic segmentation

Abstract Weakly supervised semantic segmentation (WSSS) is a challenging task of computer vision. The state-of-the-art semantic segmentation methods are usually based on the convolutional neural network (CNN), which mainly have the drawbacks of inability to explore the global information correctly and failure to activate potential object regions. To avoid such drawbacks, the transformer approach is explored in the WSSS task, but no effective semantic association between different patch tokens can be determined in the transformer. To address this issue, inspired by the graph convolutional network (GCN), this paper proposes a graph structure to learn the semantic category relationships between different blocks in the vector sequence. To verify the effectiveness of the proposed method in this paper, a large number of experiments were conducted on the publicly available PASCAL VOC2012 dataset. The experimental results show that our proposed method achieves significant performance improvement in the WSSS task and outperforms other state-of-the-art transformer-based methods.
A post-quantum signature scheme from the secant variety of the Grassmannian

We propose a new digital signature scheme. This scheme is based on a new problem: determine a point in a shifted Grassmannian variety or on its secant variety using the knowledge of its defining polynomials. An advantage of using the secant variety of the Grassmannian is that it is defined by cubic equations, which are, in general, more difficult to solve than quadratic ones. Moreover, these equations are sparse, therefore they can be represented efficiently.
Prioritizing Use Cases: A Systematic Literature Review

The prioritization of software requirements is necessary for successful software development. A use case is a useful approach to represent and prioritize user-centric requirements. Use-case-based prioritization is used to rank use cases to attain a business value based on identified criteria. The research community has started engaging use case modeling for emerging technologies such as the IoT, mobile development, and big data. A systematic literature review was conducted to understand the approaches reported in the last two decades. For each of the 40 identified approaches, a review is presented with respect to consideration of scenarios, the extent of formality, and the size of requirements. Only 32.5% of the reviewed studies considered scenario-based approaches, and the majority of reported approaches were semiformally developed (53.8%). The reported result opens prospects for the development of new approaches to fill a gap regarding the inclusive of strategic goals and respective business processes that support scenario representation. This study reveals that existing approaches fail to consider necessary criteria such as risks, goals, and some quality-related requirements. The findings reported herein are useful for researchers and practitioners aiming to improve current prioritization practices using the use case approach.