Reviewerly/exHarmony huggingface dataset:
LIDER

Passage retrieval has been studied for decades, and many recent approaches of passage retrieval are using dense embeddings generated from deep neural models, called "dense passage retrieval". The state-of-the-art end-to-end dense passage retrieval systems normally deploy a deep neural model followed by an approximate nearest neighbor (ANN) search module. The model generates embeddings of the corpus and queries, which are then indexed and searched by the high-performance ANN module. With the increasing data scale, the ANN module unavoidably becomes the bottleneck on efficiency. An alternative is the learned index, which achieves significantly high search efficiency by learning the data distribution and predicting the target data location. But most of the existing learned indexes are designed for low dimensional data, which are not suitable for dense passage retrieval with high-dimensional dense embeddings. In this paper, we propose LIDER , an efficient high-dimensional L earned I ndex for large-scale DE nse passage R etrieval. LIDER has a clustering-based hierarchical architecture formed by two layers of core models. As the basic unit of LIDER to index and search data, a core model includes an adapted recursive model index (RMI) and a dimension reduction component which consists of an extended SortingKeys-LSH (SK-LSH) and a key re-scaling module. The dimension reduction component reduces the high-dimensional dense embeddings into one-dimensional keys and sorts them in a specific order, which are then used by the RMI to make fast prediction. Experiments show that LIDER has a higher search speed with high retrieval quality comparing to the state-of-the-art ANN indexes on passage retrieval tasks, e.g., on large-scale data it achieves 1.2x search speed and significantly higher retrieval quality than the fastest baseline in our evaluation. Furthermore, LIDER has a better capability of speed-quality trade-off.
Graph-Based Soft-Balanced Fuzzy Clustering

Spectral clustering have attracted more and more attention due to their well-defined mathematical frameworks and superior performance. However, there still exist two limitations to be solved: 1) most spectral clustering methods consist of two independent stages, which may cause unpredictable deviation of obtained clustering results from the genuine ones and lead to severe information loss and performance degradation; 2) spectral clustering methods employ the hard clustering mode, which lacks interpretability for data points in the boundary area belonging to multiple clusters. To simultaneously address these challenging issues for spectral clustering, we propose a graph-based soft-balanced fuzzy clustering (GBFC) model. Specifically, we explicitly preserve the nonnegative property of the clustering indicator matrix to enhance the interpretability of clustering results. Moreover, row normalization is imposed on the cluster indicator matrix to show the membership of each data point to different clusters. Additionally, a novel balanced constraint is designed to regularize the clustering results and constrain the size of clusters. We can directly obtain the clustering assignments without any post-processing, and the limitations of the previous two-stage clustering framework can be effectively addressed. Extensive experiments performed on both synthetic datasets and real world datasets demonstrate the superiority and effectiveness of the proposed algorithm compared with several state-of-the-art methods.
Privacy-Preserving and Publicly Verifiable Matrix Multiplication

Outsourcing computations allows the resource-constrained clients (such as, IoT devices) to offload heavy computations to powerful cloud servers. At the same time, it brings many challenges such as data privacy, result verification and fair payment. In this paper, we propose a privacy-preserving multi-function verifiable computation (MFVC) model and construct an MFVC scheme for outsourcing matrix multiplication computations (MMC), which have many real-life applications, such as machine learning. Our scheme keeps one of the matrices in MMC <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">semantically secure</i> and allows a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">public verification</i> of the server's work. In particular, the client's work is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">faster</i> than the native MMC for square matrices of order <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\geq 7000$</tex-math></inline-formula> , while the best existing scheme requires matrices of order <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\geq 270000$</tex-math></inline-formula> . We also propose a smart contract-based framework that equips the proposed scheme with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fair payment</i> property. By offloading the verification to the blockchain, the client's work is faster than the native MMC for square matrices of order <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\geq 6000$</tex-math></inline-formula> .
Temporal Action Segmentation With High-Level Complex Activity Labels

The temporal action segmentation task segments videos temporally and predicts action labels for all frames. Fully supervising such a segmentation model requires dense frame-wise action annotations, which are expensive and tedious to collect. This work is the first to propose a Constituent Action Discovery (CAD) framework that only requires the video-wise high-level complex activity label as supervision for temporal action segmentation. The proposed approach automatically discovers constituent video actions using an activity classification task. Specifically, we define a finite number of latent action prototypes to construct video-level dual representations with which these prototypes are learned collectively through the activity classification training. This setting endows our approach with the capability to discover potentially shared actions across multiple complex activities. Due to the lack of action-level supervision, we adopt the Hungarian matching algorithm to relate latent action prototypes to ground truth semantic classes for evaluation. We show that with the high-level supervision, the Hungarian matching can be extended from the existing video and activity levels to the global level. The global-level matching allows for action sharing across activities, which has never been considered in the literature before. Extensive experiments demonstrate that our discovered actions can help perform temporal action segmentation and activity recognition tasks.
Strategy and Feasibility Study for the Construction of High Resolution Images Adversarial Against Convolutional Neural Networks

Convolutional Neural Networks, that perform image recognition, assess images by first resizing them to their fitting input size. In particular, high resolution images are scaled down, say to $$224 \times 224$$ for CNNs trained on ImageNet. So far, existing attacks, that aim at creating an adversarial image that a CNN would misclassify while a human would not notice any difference between the modified and the unmodified image, actually work in the $$224 \times 224$$ resized domain and not in the high resolution domain. Indeed, attacking high resolution images directly leads to complex challenges in terms of speed, adversity and visual quality, that make these attacks infeasible in practice. We design an indirect strategy that addresses effectively this issue. It lifts to the high resolution domain any existing attack that works in the CNN's input size domain. The adversarial noise is of the same size as the original image. We apply this strategy to construct efficiently high resolution adversarial images of good visual quality that fool VGG-16 trained on ImageNet.
A Distance Learning VR Technology Tool for Science Labs

Distance learning technology and online learning are here to stay. The pandemic crisis made that very clear but the increasing number of the students who are grown up as “digital natives” and want to communicate and learn mostly through technology, make institutions to rethink their instructional approaches in ways that are innovative and technologically modern. It now seems quite feasible to use digital technology in order to organize a virtual classroom and teach a topic that is traditionally taught through lectures in a physical classroom. But, what about the distance teaching of a laboratory course where physical presence in the lab was, until recently, necessary and sometimes mandatory? In this work we propose a solution to this new demand. We present an educational interactive activity that uses desktop immersive virtual reality (VR) technology to familiarize students with the use of laboratory instruments, specifically of a photonic microscope. To that end we also record and analyze students’ actions in an Analytics Platform. The presented VR activity is an example of how instructors in Natural and Physical Sciences can expand their options of engaging their students in the laboratory courses. This approach can also be adopted in a flipped or blended classroom, in pandemic or normal situation, to prepare students for their experiments in a stimulating, safe, low-budget and time-flexible way. Additionally, the instructors may have immediate access to valuable information on their students’ engagement with the subject matter, as the learning analytics back-end offers them a clear view of the most difficult steps of an experiment, thus allowing them to improve their instructional approaches in their on-site labs.
Privacy Preserving Computation in Cloud Using Reusable Garbled Oblivious RAMs

When users store encrypted data in a cloud environment, it is important for users to ask cloud to carry out some computation on the remote data remotely. ORAM is a good potential approach to carry out this kind of remote operation. In order to use ORAM for this purpose, we still need to have garbled programs to run on ORAM. Goldwasser et al. and Lu-Ostrovsky initiated the study of garbled RAM machines in their 2013 Crypto papers. Goldwasser et al's scheme is based on fully homomorphic encryption schemes and attribute based encryption schemes for general RAM machines. Lu and Ostrovsky's scheme is based on one-time garbled circuits and for each input, one has to design as many one-time garbled circuits as ORAM CPU running steps. That is, for each execution of the program, the data owner needs to upload a new program to the cloud to run on ORAM. Using recent results on indistinguishability obfuscation, this paper designs alternative reusable garbled ORAM programs. The reusable garbled ORAM CPU constructed in this paper is of constant size while the size of the garbled ORAM CPUs by Lu and Ostrovsky depends on the number of ORAM CPU running steps.
Measuring Compositional Consistency for Video Question Answering

Recent video question answering benchmarks indicate that state-of-the-art models struggle to answer compositional questions. However, it remains unclear which types of compositional reasoning cause models to mispredict. Furthermore, it is difficult to discern whether models arrive at answers using compositional reasoning or by leveraging data biases. In this paper, we develop a question decomposition engine that programmatically deconstructs a compositional question into a directed acyclic graph of sub-questions. The graph is designed such that each parent question is a composition of its children. We present AGQA-Decomp, a benchmark containing 2.3M question graphs, with an average of 11.49 sub-questions per graph, and 4.55M total new sub-questions. Using question graphs, we evaluate three state-of-the-art models with a suite of novel compositional consistency metrics. We find that models either cannot reason correctly through most compositions or are reliant on incorrect reasoning to reach answers, frequently contradicting themselves or achieving high accuracies when failing at intermediate reasoning steps.
Aspect-based Sentiment Analysis with Graph Convolutional Networks over Dependency Awareness

Aspect-based sentiment analysis (ABSA) aims to predict the sentiment polarities of specific aspects in a comment sentence. Nowadays, models based on graph neural networks enhance semantic perception by using dependency relations on dependency graphs to analyze context and aspect words. However, these models ignore the importance of the dependency type information contained in word relations and do not utilize the dependency types to pay attention to semantic information and the noise problem caused by dependency tree parsing error. To solve the above problems, we propose a novel deep dependency-aware graph convolutional networks (DA-GCN) model in this paper. Among them, the DA-GCN establishes interactive relations with multi-head attention and makes use of the grammar information of dependency perception jointly to effectively learn related information from the generated graphs. We introduce multiple conditional random fields fusing structured attention to better capture specific aspect opinion words. Experimental results on five datasets prove the effectiveness and advancement of our proposed model.
Learning Video Representations of Human Motion from Synthetic Data

In this paper, we take an early step towards video representation learning of human actions with the help of large-scale synthetic videos, particularly for human motion representation enhancement. Specifically, we first introduce an automatic action-related video synthesis pipeline based on a photorealistic video game. A large-scale human action dataset named GATA (GTA Animation Transformed Actions) is then built by the proposed pipeline, which includes 8.1 million action clips spanning over 28K action classes. Based on the presented dataset, we design a contrastive learning framework for human motion representation learning, which shows significant performance improvements on several typical video datasets for action recognition, e.g., Charades, HAA 500 and NTU-RGB. Besides, we further explore a domain adaptation method based on cross-domain positive pairs mining to alleviate the domain gap between synthetic and realistic data. Extensive properties analyses of learned representation are conducted to demonstrate the effectiveness of the proposed dataset for enhancing human motion representation learning.
Double Recursive Sparse Self-attention Based Crowd Counting in the Cluttered Background

Crowd counting algorithms play an important role in the field of public safety management. Most of the current mainstream crowd counting methods are based on deep convolutional neural networks (CNNs), which use multi-column or multi-scale convolutional structures to obtain contextual information in images to compensate for the impact of perspective distortion on counting results. However, due to the locally connected nature of convolution, this method cannot obtain enough global context, which often leads to misidentification in complex background regions, which affects the accuracy of counting. To solve this problem. First, we design a double recursive sparse self-attention module, which can better obtain long-distance dependency information and improve the problem of background false detection on the basis of reducing the amount of computation and parameters. Secondly, we design a Transformer structure based on feature pyramid as the feature extraction module of the crowd counting algorithm, which effectively improves the algorithm’s ability to extract global information. The experimental results on public datasets show that our proposed algorithm outperforms the current mainstream crowd counting methods, and effectively improves the background false detection problem of complex scene images.
Scalable M+1st-Price Auction with Infinite Bidding Price

$$M+1$$ st-price auction, also called Vickrey auction, is a type of sealed-bid auction to sell M identical goods. B bidders secretly choose a bid. The top M bidders can buy the goods at the $$M+1$$ st bidding price. In previous research, a trusted manager is commonly used to decide the $$M+1$$ st bidding price from these sealed ones and the top M bidders. In addition, there’s an upper bound to the bidding price. We construct a scheme that removes all trusted parties such as managers and Mix servers in such a way that winning bidders themselves just prove that they are winners. By adopting a compact bit-slice design, the upper bound is removed, and the compact bit-slice design can also reduce the complexity of the number of bidders to sublinear. Our implementation shows that the gas usage reduces by $$95\%$$ after we use zero-knowledge proof to replace Mix and Match. The overall gas usage is also reduced by $$83\%$$ . This protocol reached the ultimate goal of decentralized apps (DApps): Decentralized: no TTP or manager is used. Scalable: unlimited bidding price and sublinear complexity to the number of bidders. Robustness, the auction does not necessarily need to restart if there are some malicious bidders at the first time.
Analysis and Development of E-Commerce Web Application

E-commerce sites to a system or methodology that incorporates the use of computers and electronics to render opportunities to inculcate and reinforce business and commercial vulnerabilities. It encompasses things like carrying-out transactions, procuring groceries and household things, booking plane tickets and other monetary services like stock investment and on-line banking. In this study, we have tried to build such a web application that expedites all these things. To keep the study focused and adept, we have confined the development to a simple web application that powers people to get household things with the comfort of their home. For development, we have used client-server architecture, in which the user interface is developed by the dint of ‘Bootstrap 4.0’ together with some vanilla ‘JS’ and ‘CSS’. For the backend operations and APIs we used ‘Django’ framework. The working principle is simple and sound, user connects with the interface and fires requests to the corresponding APIs to purchase any product. The API in turn adds the products to the user’s shopping cart and makes provisions for performing check-out operation. It uses a test server of ‘PayPal’ payment gateway to facilitate a secure and legit payment mechanism. Further, it returns an ‘order-id’ to check the delivery details of the shipment. It employs a ‘tracker’ as a component which serves the function of pushing updates about the shipment arrival. Furthermore, it employs a JS library ‘TimeMejs’ to track the amount that a user spent on the application and then uses it to award user with some credit points which he can use during checkout, doing so would lead to the value creation of business and assist the business to withstand amid colossal organizations. So, this study throws light on the development of an e-commerce web application from scratch and as a result, we were able to successfully develop the same.
FDGATII: Fast Dynamic Graph Attention with Initial Residual and Identity

Despite their recent popularity, deep and efficient Graph Neural Networks remain a major challenge due to (a) over-smoothing, (b) noisy neighbours (heterophily), and (c) the suspended animation problem. Inspired by the attention mechanism's ability to focus on selective information, and prior work on feature preserving mechanisms, we propose FDGATII, a dynamic deep-capable model that addresses all these challenges simultaneously and efficiently. Specifically, by combining Initial Residuals and Identity with the more expressive dynamic self-attention, FDGATII effectively handles noise in heterophilic graphs and is capable of depths over 32 with no over-smoothing, overcoming two main limitations of many prior GNN techniques. By using edge-lists, FDGTII avoids computationally intensive matrix operations, is parallelizable and does not require knowing the graph structure upfront. Experiments on 7 standard datasets show that FDGATII outperforms the GAT and GCN based benchmarks in accuracy and performance on fully supervised tasks. We obtain State-of-the-art (SOTA) on the highly heterophilic Chameleon and Cornell datasets with 1 layer, and come only 0.1% short of Cora SOTA with zero graph pre processing. https://github.com/gayanku/FDGATII
Accelerating Hyperparameter Tuning of a Deep Learning Model for Remote Sensing Image Classification

Deep Learning models have proven necessary in dealing with the challenges posed by the continuous growth of data volume acquired from satellites and the increasing complexity of new Remote Sensing applications. To obtain the best performance from such models, it is necessary to fine-tune their hyperparameters. Since the models might have massive amounts of parameters that need to be tuned, this process requires many computational resources. In this work, a method to accelerate hyperparameter optimization on a High-Performance Computing system is proposed. The data batch size is increased during the training, leading to a more efficient execution on Graphics Processing Units. The experimental results confirm that this method reduces the runtime of the hyperparameter optimization step by a factor of 3 while achieving the same validation accuracy as a standard training procedure with a fixed batch size.
Sculpting Enhanced Dependencies for Belarusian

Enhanced Universal Dependencies (EUD) are enhanced graphs expressed on top of basic dependency trees. EUD support representation of deeper syntactic relations in constructions such as coordination, gapping, relative clauses, argument sharing through control and raising. The paper presents experiments on the EUD parsing of the low-resource Belarusian language, for which no corpus with enhanced annotations was available. Models trained on the Universal Dependencies treebanks of two closely related Slavic languages, Russian and Ukrainian, were used to parse sentences translated from Belarusian. After that, enhanced dependencies were projected to the original sentences, which gave us ELAS (Enhanced Labeled Attachment Score) 78.1% for both Russian and Ukrainian in evaluation. We also trained the model of one of the IWPT 2020 Shared Task participants on obtained annotations in Belarusian and achieved ELAS 83.4%. Analysis shows that the most common mistakes of cross-lingual parsing root in different theoretical perspectives and practice approaches to the annotation of particular types of clauses in the three Slavic treebanks. Both Russian and Ukrainian EUD transfer models tend to make mistakes when dealing with the predicate argument relations, which are hard to identify without understanding the semantics of the sentence. The alignment method decreases the quality of the annotation by confusing tokens that occur in a sentence more than once.