Reviewerly/exHarmony huggingface dataset:
Multi-Hyperedge Hypergraph for Group Activity Recognition

Group activity recognition aims to identify group activities from the videos. Most of the previous methods focus on modeling between individuals (one-to-one), which ignores the fact that a single individual's behavior may be jointly determined by multiple individual behaviors (many-to-one). For this reason, we propose a Multi-Hyperedge Hypergraph (MHH) to capture high-order relationships between multiple people. Specifically, we build three different types of hyperedges on the hypergraph structure. Each hyperedge can accommodate the characteristics of multiple nodes to capture different types of high-order relationships between nodes. Then, we use the late fusion method to fuse the three features to further enhance the overall behavioral representation. Finally, we perform a series of experiments on two of the most widely used benchmarks in group activity recognition, which have proved the effectiveness of MHH. More importantly, as far as we know, this is the first case of using a hypergraph structure for group activity recognition.
ViewFormer: NeRF-Free Neural Rendering from Few Images Using Transformers

Novel view synthesis is a long-standing problem. In this work, we consider a variant of the problem where we are given only a few context views sparsely covering a scene or an object. The goal is to predict novel viewpoints in the scene, which requires learning priors. The current state of the art is based on Neural Radiance Field (NeRF), and while achieving impressive results, the methods suffer from long training times as they require evaluating millions of 3D point samples via a neural network for each image. We propose a 2D-only method that maps multiple context views and a query pose to a new image in a single pass of a neural network. Our model uses a two-stage architecture consisting of a codebook and a transformer model. The codebook is used to embed individual images into a smaller latent space, and the transformer solves the view synthesis task in this more compact space. To train our model efficiently, we introduce a novel branching attention mechanism that allows us to use the same model not only for neural rendering but also for camera pose estimation. Experimental results on real-world scenes show that our approach is competitive compared to NeRF-based methods while not reasoning explicitly in 3D, and it is faster to train.
Distant Object Manipulation with Adaptive Gains in Virtual Reality

Object Manipulation is a fundamental interaction in virtual reality (VR). The efficiency and accuracy of object manipulation are important to provide immersion to users. We propose a manipulation method with adaptive gains to improve the efficiency and accuracy of object manipulation in VR applications. First, we introduce manipulation gains. We then design an experiment to collect user behavior during manipulation to determine fitting functions for calculating manipulation gain. At last, we design a user study to evaluate the performance of our distant object manipulation method with adaptive gains. The results show that, compared with the state of the art methods, our method has a significant improvement in the completion time, and the manipulation accuracy of the tasks. Moreover, our method significantly increases usability and reduces task load.
Adopting DevOps: Capabilities, practices, and challenges faced by organizations

DevOps- a portmanteau of development and operations that come together to improve the software quality as well as to speed up the software development with its continuous delivery to its clients and customers. DevOps, emerging as an attractive software development movement, appears to be well regarded, but it is proving difficult to adopt fully in reality. Therefore, this paper contributes to understanding DevOps and addressing its challenges and practices while considering the real-world scenarios where organizations implement DevOps to pace their productivity. This paper also proposes a model which shows the key contributors in DevOps and how they are used to achieve the desired outcomes. We have carried out an exploratory case study by understanding how businesses adopt DevOps in real life. A case study is considered an effective form of analysis for studying current trends within their real-life context. DevOps is commonly described as collaborative work done between development and operations team by continuous communication and cooperation as well as featuring a cross-functional team having a common goal to achieve.
A Deep Learning-Based Approach for Mammographic Architectural Distortion Classification

Breast cancer is the most deadly cancer in females globally. Architectural distortion is the third most often reported irregularity on digital mammograms among the masses and microcalcification. Physically identifying architectural distortion for radiologists is problematic because of its subtle appearance on the dense breast. Automatic early identification of breast cancer using computer algorithms from a mammogram may assist doctors in eliminating unwanted biopsies. This research presents a novel diagnostic method to identify AD ROIs from mammograms using computer vision-based depth-wise CNN. The proposed methodology is examined on private PINUM 2885 and public DDSM 3568 images and achieved a 0.99 and 0.95 sensitivity, respectively. The experimental findings revealed that the proposed scheme outperformed SVM, KNN, and previous studies.
Design of Novel Hardware Architecture for Fully Homomorphic Encryption Algorithms in FPGA for Real-Time Data in Cloud Computing

In the cloud computing environment, the data is encrypted using a variety of cryptographic techniques before being sent to the cloud. But in this method, if someone wants to do some operation on the ciphertext, then the client needs to share the secret key to decrypt the ciphertext. So, there is a chance of data leakage. Now more research is being carried out on a fully homomorphic encryption scheme (FHE) to maintain the privacy and secrecy of user’s data in the cloud environment. Calculations can be performed on the ciphertext in this encryption technique without the need for decryption. The user will use a private key to decode the result into plain text format. But this method takes more time if it is implemented in software. So, it is not feasible to implement FHE on real-time data. To overcome these difficulties, we have developed a new hardware architecture for Brakerski, Vaikuntanathan (BV) fully homomorphic encryption scheme using Field Programmable Gate Array (FPGA). In this work, we have carried out an extensive simulation of the proposed design in MATLAB, and Python to validate the encryption and decryption algorithm. Further, we have simulated the design architecture of the BV scheme in the Questasim simulator and implemented it in an Intel FPGA using the Quartus tool. In this research work, we found that the proposed architecture takes 0.33 sec for key generation, 0.851 sec for encryption, and 72us for decryption when it is implemented using 32 butter-fly-based Number Theoretic Transfer (NTT) and consumes less FPGA hardware resources.
Chest X-Ray Caption Generation with CheXNet

Chest X-rays are provided with descriptive captions that summarize the crucial radiology findings in them in natural language. Although chest X-Ray image captioning is currently done manually by radiologists, automating it has received growing research interest in the medical domain because it is a tedious task and the high number of medical reports that are to be generated daily. In this paper, we propose an automatic chest X-ray captioning system consisting of two main components: an image feature extractor and a sentence generator. We did our experiment in two approaches. First, we tried using LXMERT, which is originally designed for question answering, as the sentence generator in our model combined with the Faster RCNN model. Second, we used CheXNet and a memory-driven transformer as the feature extractor and the sentence generator respectively. We trained and tested our model using the IU chest X-ray dataset. We evaluated the model using the BLUE, ROUGE-L and METEOR metrics which shows the CheXNet based approach outperforms the latter models.
CRNet: Modeling Concurrent Events over Temporal Knowledge Graph

Temporal knowledge graph (TKG) reasoning, which aims to extrapolate missing facts in TKGs, is vital for many significant applications, such as event prediction. Previous studies have attempted to equip entities and relations with temporal information in historical timestamps and have achieved promising performance. While ignoring the likelihood that future occurrences would occur simultaneously, they independently forecast the missing data. However, there are complicated connections between future concurrent events that might correlate with and influence one another. Therefore, we propose our Concurrent Reasoning Network (CRNet) to leverage event concurrency in both historical and future timestamps for TKG reasoning. Specifically, we select the top-k candidate events for each missing event and construct a candidate graph based on the candidate events of all missing events at the future timestamp. The candidate graph connects missing facts by sharing the same entities. Furthermore, we employ a novel relational graph attention network to represent the interactions of candidate events. We evaluate our proposal by the entity prediction task on three well-known public event-based TKG datasets. Extensive experimental results show that our CRNet complete future missing facts with a 15–20% improvement over MRR. (The source code is available at https://github.com/shichao-wang/CRNet-ISWC2022 .)
Summarizing Source Code from Structure and Context

Modern software developers tend to engage in social coding platforms to reuse code snippets to expedite the development process, while the codes on such platforms are often suffering from comments being mismatched, missing or outdated. This puts the code search and comprehension in difficulty, and increases the burden of maintenance for software building upon these codes. As summarizing code is beneficial yet it is very expensive for manual operation, in this paper, we elaborate an automatic and effective code summarization paradigm to address this laborious challenge. We represent a given code snippet as an abstract syntax tree (AST), and generate a set of compositional root-to-leaf paths to make the AST accessible regarding code context and structure in a less complex yet expressive way. Accordingly, we design a tree-based transformer model, called TreeXFMR, on these paths to summarize source code in a hierarchical attention operation. This yields two advantages on code representation learning: (1) attention mechanisms at token-and path-level attend the semantics and interactions of source code from different aspects; (2) bi-level positional encodings introduced reveal the intra- and inter-path structure of AST and improve the unambiguity of the representations. During decoding, TreeXFMR attends such learned representations to produce each output of natural language word. We further pre-train the transformer to achieve faster and better training convergence results. Extensive experiments on the code collection from GitHub demonstrate the effectiveness of TreeXFMR, which significantly outperforms state-of-the-art baselines.
LSGC: An Interactive Text Matching Model Combined with Enhanced Encoding

Text matching which measures the degree of similarity between text pair plays a vital role in a variety of applications. However, a single process of encoding such as BiLSTM may lose textual information. In this paper, we propose a new interactive text matching model named LSGC, where we design an enhanced encoding layer that uses not only BiLSTM but also gated convolutional neural network to enhance the ability to capture both key word and sequence information. Additionally, for the purpose of avoiding long-term dependency and fusing more important information in the fusion layer, we take advantage of BiLSTM and attention to enhance the inference fusion. Finally, experiment results demonstrate that our model outperforms other neural network models and achieve a remarkable performance on both Chinese and English paraphrase identification datasets.
Type Inference for Rank-2 Intersection Types Using Set Unification

Several type inference approaches for rank-2 idempotent and commutative intersection types have been presented in the literature. Type inference relies on two stages: type constraint generation and solving. Defining constraint generation rules is rather straightforward, with one exception. To infer the type of an application, several derivations of the argument are required, one for each instance of the domain type of the function. The types of these derivations are then constrained against the instances. Noting that these derivations are isomorphic, by renaming of type variables, they can be obtained via a duplication operation on a single derivation of the argument. The application rule then constrains the intersection type resulting from duplication against the domain type of the function, resulting in an equality constraint between intersections. By treating intersections as sets, these constraints can be solved by solving a set unification problem, thus ensuring the types of the argument unify with the domain type of the function. Here we present a new type inference algorithm for rank-2 intersection types, which relies on set unification to solve equality constraints between intersections, and show it is both sound and complete.
Multi-Robot Security System based on Robot Operating System and Hybridized Blockchain Model

Multi robot systems are defined as a collection of two or more robots that are capable of working autonomously while coordinating with each other. Three challenges emerge while designing any multi robot system. The robots have to coordinate their path planning or trajectory planning in order to avoid collision during the course of navigation, while collaborating tasks with other robots to achieve a specific end goal for the system. The other challenge, which is the focus of this paper, is the security of the entire multi robot system. Since robots have to coordinate with each other, any one of them being malicious due to any kind of security threat, can lead to a chain reaction that may compromise the entire system. Such security threats can be fatal if not dealt with immediately. This paper proposes the use of a Hybridized Blockchain Model (HBM) to identify such security threats and take necessary actions in real time so that the system does not encounter any catastrophic failure. The proposed security architecture uses ROS (Robot operating system) to decentralize the information collected by robot clients and HBM to monitor the clients and take necessary real time actions.
Assembly of Echo State Networks Driven by Segregated Low Dimensional Signals

An echo state network (ESN), consisting of an input layer, reservoir, and output layer, provides a higher learning-efficient approach than other recurrent neural networks (RNNs). In the design of ESNs, a sufficiently large number of reservoir neurons is required compared to the dimension of the input signal. Thus, the number of neurons must be increased for high-dimensional input to achieve good performance. However, an increase in the number of neurons increases the computational load. To solve this problem, we propose an assembly ESN (AESN) architecture comprising a feature extraction part that uses multiple sub-ESNs with segregated components of high-dimensional input and a feature integration part. To validate the effectiveness of the proposed AESN, we investigated and compared the conventional ESN with the AESN under high-dimensional input. The results show that the AESN is possibly superior to the conventional ESN in accuracy, memory performance, and computational load. We believe that the AESN also has a correct integration function. Therefore, the proposed method is expected to solve high-dimensional problems with improved accuracy.
Multi-Domain Active Learning: Literature Review and Comparative Study

Multi-domain learning (MDL) refers to learning a set of models simultaneously, where each model is specialized to perform a task in a particular domain. Generally, a high labeling effort is required in MDL, as data needs to be labeled by human experts for every domain. Active learning (AL) can be utilized in MDL to reduce the labeling effort by only using the most informative data. The resultant paradigm is termed multi-domain active learning (MDAL). In this work, we provide an exhaustive literature review for MDAL on the relevant fields, including AL, cross-domain information sharing schemes, and cross-domain instance evaluation approaches. It is found that the few studies which have been directly conducted on MDAL cannot serve as off-the-shelf solutions on more general MDAL tasks. To fill this gap, we construct a pipeline of MDAL and present a comprehensive comparative study of thirty different algorithms, which are established by combining six representative MDL models and five commonly used AL strategies. We evaluate the algorithms on six datasets involving textual and visual classification tasks. In most cases, AL brings notable improvements to MDL, and the naive BvSB (best vs. second best) Uncertainty strategy can perform competitively with the state-of-the-art AL strategies. Besides, BvSB with the MAN (multinomial adversarial networks) model can consistently achieve top or above-average performance on all the datasets. Furthermore, we qualitatively analyze the behaviors of the well-performed strategies and models, shedding light on their superior performance in the comparison. Finally, we recommend using BvSB with the MAN model in the application of MDAL due to their good performance in the experiments.
Spectral-Spatial-Aware Transformer Fusion Network For Hyperspectral Object Tracking

Single object tracking (SOT) based on deep learning methods has been developed by leaps and bounds. Nevertheless, these methods seldom take spectral information into consideration, which does not utilize the spectral properties of objects. Since that spectral imaging can probably distinguish a particular object more discriminatively, a more effective model is expected to be proposed for the exploration of spectral information. To study the advantages of the hyperspectral image in extracting spectral features of the target in a tracking task, this work proposed a novel Spectral-spatial-aware Transformer Fusion Network (SSATFN) for hyperspectral single object tracking, which efficiently combines spectral and spatial features with the template and search region branches. Specifically, the method focuses on multiband feature fusion by Hyperspectral Transformer Self-attention (HTSA) and Hyperspectral Transformer Cross-attention (HTCA). Meanwhile, a multi-scale spectral feature fusion auxiliary branch is utilized for template enhancement. Finally, we present the online tracking learning networks to fine-tune the last two layers of the feature extraction network. Quantitative experiments are conducted on a close-up hyperspectral video dataset, and verified that the proposed SSATFN achieves promising tracking performances, compared with the other state-of-the-art trackers.
DIFM: An Effective Deep Interaction and Fusion Model for Sentence Matching

Natural language sentence matching is the task of comparing two sentences and identifying the relationship between them. It has a wide range of applications in natural language processing tasks such as reading comprehension, question and answer systems. The main approach is to compute the interaction between text representations and sentence pairs through an attention mechanism, which can extract the semantic information between sentence pairs well. However, this kind of methods fail to capture deep semantic information and effectively fuse the semantic information of the sentence. To solve this problem, we propose a sentence matching method based on deep interaction and fusion. We first use pre-trained word vectors Glove and character-level word vectors to obtain word embedding representations of the two sentences. In the encoding layer, we use bidirectional LSTM to encode the sentence pairs. In the interaction layer, we initially fuse the information of the sentence pairs to obtain low-level semantic information; at the same time, we use the bi-directional attention in the machine reading comprehension model and self-attention to obtain the high-level semantic information. We use a heuristic fusion function to fuse the low-level semantic information and the high-level semantic information to obtain the final semantic information, and finally we use the neural network to predict the answer. We evaluate our model on two tasks: text implication recognition and paraphrase recognition. We conducted experiments on the SNLI datasets for the recognizing textual entailment task, the Quora dataset for the paraphrase recognition task. The experimental results show that the proposed algorithm can effectively fuse different semantic information that verify the effectiveness of the algorithm on sentence matching tasks.
Speeding up Genetic Programming Based Symbolic Regression Using GPUs

Symbolic regression has multiple applications in data mining and scientific computing. Genetic Programming (GP) is the mainstream method of solving symbolic regression problems, but its execution speed under large datasets has always been a bottleneck. This paper describes a CUDA-based parallel symbolic regression algorithm that leverages the parallelism of the GPU to speed up the fitness evaluation process in symbolic regression. We make the fitness evaluation step fully performed on the GPU and make use of various GPU hardware resources. We compare training time and regression accuracy between the proposed approach and existing symbolic regression frameworks including gplearn, TensorGP, and KarooGP. The proposed approach is the fastest among all the tested frameworks in both synthetic benchmarks and large-scale benchmarks.
Behavioral Analytics of Consumer Complaints

In the current study, the author implemented cohort analysis methodology of machine learning (ML) to assess the rate of consumers' complaint retention behavior toward firms. The author obtained a three-year range of data on customer complaints from the Consumer Financial Protection Bureau's website. After removing the missing data from the customer complaints dataset, the current study also uses the cohort analysis approach of ML to assess the rate of consumer complaints retained by businesses. According to the findings, organizations retain a significant portion of complaints. This study adds to the body of knowledge on consumer complaining behavior (CCB), especially by creating and deploying unsupervised machine learning-based technique to conceptualize consumers' complaint behavior in the financial service providers industry. Moreover, professionals will benefit greatly from this research.