Reviewerly/exHarmony huggingface dataset:
Deconfounded Video Moment Retrieval with Causal Intervention

We tackle the task of video moment retrieval (VMR), which aims to localize a specific moment in a video according to a textual query. Existing methods primarily model the matching relationship between query and moment by complex cross-modal interactions. Despite their effectiveness, current models mostly exploit dataset biases while ignoring the video content, thus leading to poor generalizability. We argue that the issue is caused by the hidden confounder in VMR, i.e., temporal location of moments, that spuriously correlates the model input and prediction. How to design robust matching models against the temporal location biases is crucial but, as far as we know, has not been studied yet for VMR.
Review on self-supervised image recognition using deep neural networks

Deep learning has brought significant developments in image understanding tasks such as object detection, image classification, and image segmentation. But the success of image recognition largely relies on supervised learning that requires huge number of human-annotated labels. To avoid costly collection of labeled data and the domains where very few standard pre-trained models exist, self-supervised learning comes to our rescue. Self-supervised learning is a form of unsupervised learning that allows the network to learn rich visual features that help in performing downstream computer vision tasks such as image classification, object detection, and image segmentation. This paper provides a thorough review of self-supervised learning which has the potential to revolutionize the computer vision field using unlabeled data. First, the motivation of self-supervised learning is discussed, and other annotation efficient learning schemes. Then, the general pipeline for supervised learning and self-supervised learning is illustrated. Next, various handcrafted pretext tasks are explained that enable learning of visual features using unlabeled image dataset. The paper also highlights the recent breakthroughs in self-supervised learning using contrastive learning and clustering methods that are outperforming supervised learning. Finally, we have performance comparisons of self-supervised techniques on evaluation tasks such as image classification and detection. In the end, the paper is concluded with practical considerations and open challenges of image recognition tasks in self-supervised learning regime. From the onset of the review paper, the core focus is on visual feature learning from images using the self-supervised approaches.
Knowledge-enabled BERT for aspect-based sentiment analysis

To provide explainable and accurate aspect terms and the corresponding aspect–sentiment detection, it is often useful to take external domain-specific knowledge into consideration. In this work, we propose a knowledge-enabled language representation model BERT for aspect-based sentiment analysis. Specifically, our proposal leverages the additional information from a sentiment knowledge graph by injecting sentiment domain knowledge into the language representation model, which obtains the embedding vectors of entities in the sentiment knowledge graph and words in the text in a consistent vector space. In addition, the model is capable of achieving better performance with a small amount of training data by incorporating external domain knowledge into the language representation model to compensate for the limited training data. As a result, our model is able to provide explainable and detailed results for aspect-based sentiment analysis. Experimental results demonstrate the effectiveness of the proposed method, showing that the knowledge-enabled BERT is an excellent choice for solving aspect-based sentiment analysis problems.
Open-Vocabulary Object Detection Using Captions

Despite the remarkable accuracy of deep neural networks in object detection, they are costly to train and scale due to supervision requirements. Particularly, learning more object categories typically requires proportionally more bounding box annotations. Weakly supervised and zero-shot learning techniques have been explored to scale object detectors to more categories with less supervision, but they have not been as successful and widely adopted as supervised models. In this paper, we put forth a novel formulation of the object detection problem, namely open-vocabulary object detection, which is more general, more practical, and more effective than weakly supervised and zero-shot approaches. We propose a new method to train object detectors using bounding box annotations for a limited set of object categories, as well as image-caption pairs that cover a larger variety of objects at a significantly lower cost. We show that the proposed method can detect and localize objects for which no bounding box annotation is provided during training, at a significantly higher accuracy than zero-shot approaches. Meanwhile, objects with bounding box annotation can be detected almost as accurately as supervised methods, which is significantly better than weakly supervised baselines. Accordingly, we establish a new state of the art for scalable object detection.
Learning Placeholders for Open-Set Recognition

Traditional classifiers are deployed under closed-set setting, with both training and test classes belong to the same set. However, real-world applications probably face the input of unknown categories, and the model will recognize them as known ones. Under such circumstances, open-set recognition is proposed to maintain classification performance on known classes and reject unknowns. The closed-set models make overconfident predictions over familiar known class instances, so that calibration and thresholding across categories become essential issues when extending to an open-set environment. To this end, we proposed to learn PlaceholdeRs for Open-SEt Recognition (Proser), which prepares for the unknown classes by allocating placeholders for both data and classifier. In detail, learning data placeholders tries to anticipate open-set class data, thus transforms closed-set training into open-set training. Besides, to learn the invariant information between target and non-target classes, we reserve classifier placeholders as the class-specific boundary between known and unknown. The proposed Proser efficiently generates novel class by manifold mixup, and adaptively sets the value of reserved open-set classifier during training. Experiments on various datasets validate the effectiveness of our proposed method.
Data augmentation approaches in natural language processing: A survey

As an effective strategy, data augmentation (DA) alleviates data scarcity scenarios where deep learning techniques may fail. It is widely applied in computer vision then introduced to natural language processing and achieves improvements in many tasks. One of the main focuses of the DA methods is to improve the diversity of training data, thereby helping the model to better generalize to unseen testing data. In this survey, we frame DA methods into three categories based on the diversity of augmented data, including paraphrasing, noising, and sampling. Our paper sets out to analyze DA methods in detail according to the above categories. Further, we also introduce their applications in NLP tasks as well as the challenges. Some helpful resources are provided in the appendix.
Blockchain for Edge of Things: Applications, Opportunities, and Challenges

In recent years, blockchain networks have attracted significant attention in many research areas beyond cryptocurrency, one of them being the Edge of Things (EoT) that is enabled by the combination of edge computing and the Internet of Things (IoT). In this context, blockchain networks enabled with unique features, such as decentralization, immutability, and traceability, have the potential to reshape and transform the conventional EoT systems with higher security levels. Particularly, the convergence of blockchain and EoT leads to a new paradigm, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BEoT</i> that has been regarded as a promising enabler for future services and applications. In this article, we present a state-of-the-art review of recent developments in the BEoT technology and discover its great opportunities in many application domains. We start our survey by providing an updated introduction to blockchain and EoT along with their recent advances. Subsequently, we discuss the use of BEoT in a wide range of industrial applications, from smart transportation, smart city, smart healthcare to smart home, and smart grid. Security challenges in the BEoT paradigm are also discussed and analyzed, with some key services, such as access authentication, data privacy preservation, attack detection, and trust management. Finally, some key research challenges and future directions are also highlighted to instigate further research in this promising area.
Spatiotemporal Analysis of Haze in Beijing Based on the Multi-Convolution Model

As a kind of air pollution, haze has complex temporal and spatial characteristics. From the perspective of time, haze has different causes and levels of pollution in different seasons. From the perspective of space, the concentration of haze in adjacent areas will affect each other, showing some correlation. In this paper, we construct a multi-convolution haze-level prediction model for predicting haze levels in different areas of Beijing, which uses the remote sensing satellite image of the Beijing divided into nine regions as input and the haze pollution level as output. We categorize the predictions into four seasons in chronological order and use frequency histograms to analyze haze levels in different regions in different seasons. The results show that the haze pollution in the southern regions is significantly different from that in the northern regions. In addition, the haze tends to be clustered in adjacent areas. We use Global Moran’s I to analyze the predictions and find that haze is related to the geographical location in summer and autumn. We also use Local Moran’s I, Moran scatter plot, and Local Indicators of Spatial Association (LISA) to study the spatial characteristics of haze in adjacent areas. The results show, for the spatial distribution of haze in Beijing, that the southern regions present a high-high agglomeration, while the northern regions exhibit a ‘low-low agglomeration. The temporal evolution of haze on the seasonal scale, according to the chronological order of winter, spring, and summer to autumn, shows that the haze gradually becomes agglomerated. The main finding is that the haze pollution in southern Beijing is significantly different from that of northern regions, and haze tends to be clustered in adjacent areas.
Space-efficient optical computing with an integrated chip diffractive neural network

Large-scale, highly integrated and low-power-consuming hardware is becoming progressively more important for realizing optical neural networks (ONNs) capable of advanced optical computing. Traditional experimental implementations need N2 units such as Mach-Zehnder interferometers (MZIs) for an input dimension N to realize typical computing operations (convolutions and matrix multiplication), resulting in limited scalability and consuming excessive power. Here, we propose the integrated diffractive optical network for implementing parallel Fourier transforms, convolution operations and application-specific optical computing using two ultracompact diffractive cells (Fourier transform operation) and only N MZIs. The footprint and energy consumption scales linearly with the input data dimension, instead of the quadratic scaling in the traditional ONN framework. A ~10-fold reduction in both footprint and energy consumption, as well as equal high accuracy with previous MZI-based ONNs was experimentally achieved for computations performed on the MNIST and Fashion-MNIST datasets. The integrated diffractive optical network (IDNN) chip demonstrates a promising avenue towards scalable and low-power-consumption optical computational chips for optical-artificial-intelligence.
A Correlation Graph based Approach for Personalized and Compatible Web APIs Recommendation in Mobile APP Development

Using Web APIs registered in service sharing communities for mobile APP development can not only reduce development period and cost, but also fully reuse state-of-the-art research outcomes in broad domain so as to ensure up-to-date APP development and applications. However, the big volume of available APIs in Web communities as well as their differences make it difficult for APIs selection considering compatibility, preferred partial APIs and expected APIs functions which are often of high variety. Accordingly, how to recommend a set of functional-satisfactory and compatibility-optimal APIs based on the APP developer's multiple function expectation and pre-chosen partial APIs is on demand as a significant challenge for successful APP development. To address this challenge, we first construct a Web APIs correlation graph that incorporates functional descriptions and compatibility information of Web APIs, and then propose a correlation graph-based approach for personalized and compatible Web APIs recommendation in mobile APP development. Finally, through extensive experiments on a real dataset crawled from Web APIs websites, we prove the feasibility of our proposed recommendation approach.
The actualization of meta affordances: Conceptualizing affordance actualization in the metaverse games

Drawing on the theory of affordance, we conceptualize affordance actualization for the metaverse games (MG) relative to the affordances of internalized and embodied experiences by users. Focusing on MG players' affordances, we examine how they affect the user experience by exploring how affordances are realized and enacted in an extended environment. Based on mixed methods of empirical analysis, we identify relevant affordances, theorize affordance actualization, and characterize the duality of affordance in the metaverse. A heuristic process of immersion and selection of affordances through underlying cues together actualize a player's sensory representations of affective affordances. By identifying how extended reality mediates interactions with users, we contribute to prescriptive knowledge in the form of theoretical considerations and practical implications intended for academics and practitioners working in the context of the extended environment. We propose that affordance actualization helps to theorize the duality of affordance in the metaverse that users shape their metaverse based on their actualized affordance, and at the same time, the metaverse becomes a part of the structure shaping and constraining user actions.
Cryptocurrency reaction to FOMC Announcements: Evidence of heterogeneity based on blockchain stack position

We examine the response of a broad set of digital assets to US Federal Fund interest rate and quantitative easing announcements, specifically examining associated volatility spillover and feedback effects. We classify each digital asset into one of three categories: Currencies; Protocols; and Decentralised Applications (dApps). Currency-based digital assets experience idiosyncratic spillovers in the period immediately after US monetary policy announcements, while application or protocol-based digital assets are largely immune to policy volatility spillover and feedback. Mineable digital assets are found to be more susceptible to monetary policy volatility spillovers and feedback than non-mineable. Responses indicate a diverse market within which, not all assets are comparable to Bitcoin.
GraphH: A Processing-in-Memory Architecture for Large-Scale Graph Processing

Large-scale graph processing requires the high bandwidth of data access. However, as graph computing continues to scale, it becomes increasingly challenging to achieve a high bandwidth on generic computing architectures. The primary reasons include: the random access pattern causing local bandwidth degradation, the poor locality leading to unpredictable global data access, heavy conflicts on updating the same vertex, and unbalanced workloads across processing units. Processing-in-memory (PIM) has been explored as a promising solution to providing high bandwidth, yet open questions of graph processing on PIM devices remain in: 1) how to design hardware specializations and the interconnection scheme to fully utilize bandwidth of PIM devices and ensure locality and 2) how to allocate data and schedule processing flow to avoid conflicts and balance workloads. In this paper, we propose GraphH, a PIM architecture for graph processing on the hybrid memory cube array, to tackle all four problems mentioned above. From the architecture perspective, we integrate SRAM-based on-chip vertex buffers to eliminate local bandwidth degradation. We also introduce reconfigurable double-mesh connection to provide high global bandwidth. From the algorithm perspective, partitioning and scheduling methods like index mapping interval-block and round interval pair are introduced to GraphH, thus workloads are balanced and conflicts are avoided. Two optimization methods are further introduced to reduce synchronization overhead and reuse on-chip data. The experimental results on graphs with billions of edges demonstrate that GraphH outperforms DDR-based graph processing systems by up to two orders of magnitude and $5.12 {\times }$ speedup against the previous PIM design.
Learning for Video Compression

One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network.In this paper we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks.PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network.On the basis of PMCNN, we further explore a learning-based framework for video compression with additional components of iterative analysis/synthesis, binarization, etc. Experimental results demonstrate the effectiveness of the proposed scheme.Although entropy coding and complex configurations are not employed in this paper, we still demonstrate superior performance compared with MPEG-2 and achieve comparable results with H.264 codec.The proposed learning-based scheme provides a possible new direction to further improve compression efficiency and functionalities of future video coding.
Performance of hybrid quantum-classical variational heuristics for combinatorial optimization

The recent literature on near-term applications for quantum computers contains several examples of the applications of hybrid quantum/classical variational approaches. This methodology can be applied to a variety of optimization problems, but its practical performance is not well studied yet. This paper moves some steps in the direction of characterizing the practical performance of the methodology, in the context of finding solutions to classical combinatorial optimization problems. Our study is based on numerical results obtained applying several classical nonlinear optimization algorithms to Hamiltonians for six combinatorial optimization problems; the experiments are conducted via noise-free classical simulation of the quantum circuits implemented in Qiskit. We empirically verify that: (1) finding the ground state is harder for Hamiltonians with many Pauli terms; (2) classical global optimization methods are more successful than local methods due to their ability of avoiding the numerous local optima; (3) there does not seem to be a clear advantage in introducing entanglement in the variational form.
Influences of augmented reality head-worn display type and user interface design on performance and usability in simulated warehouse order picking

Limited information is available regarding the effective use of workplace head-worn displays (HWD), especially the choices of HWD types and user interface (UI) designs. We explored how different HWD types and UI designs affect perceived workload, usability, visual discomfort, and job performance during a simulated warehouse job involving order picking and part assembly. Sixteen gender-balanced participants completed the simulated job in all combinations of two HWD types (binocular vs. monocular) and four UIs, the latter of which manipulated information mode (text-vs. graphic-based) and information availability (always-on vs. on-demand); a baseline condition was also completed (paper pick list). Job performance, workload, and usability were more affected by UI designs than HWD type. For example, the graphic-based UI reduced job completion time and number of errors by ∼13% and ∼59%, respectively. Participants had no strong preference for either of the HWD types, suggesting that the physical HWD designs tested are suboptimal.
Bi-objective Elite Differential Evolution Algorithm for Multivalued Logic Networks

In this paper, a novel algorithm called bi-objective elite differential evolution (BOEDE) is proposed to optimize multivalued logic (MVL) networks. It is a multiobjective algorithm completely different from all previous single-objective optimization ones. The two objective functions, error and optimality, are put into evaluating the fitness of individuals in evolution simultaneously. BOEDE innovatively uses an archive population with different ranks to store elite individuals and offsprings. Moreover, a characteristic updating method based on this archive structure is designed to produce the parent population. Because of the particularity of MVL network problems, the performance of BOEDE to solve them is further improved by strictly distinguishing elite solutions and Pareto optimal solutions, and by modifying the method of dealing with illegal variables. The simulations show that BOEDE can collect a great number of solutions to provide decision support for a variety of applications. The comparison results also indicate that BOEDE is significantly better than the existing algorithms.
Automatic cell nuclei segmentation and classification of cervical Pap smear images

Pathological examination of microscopic image of Pap smear slide remains the main method for cervical cancer diagnosis. The accurate segmentation and classification of images are two important phases of the analysis. Firstly, the Mean-Shift clustering algorithm is applied to obtain regions of interest (ROI) for cell nuclei segmentation. Then the flexible mathematical morphology is applied to split overlapped cell nuclei for better accuracy and robustness. For classification of the images, features based on shape, textural features based on color space and Gabor features are extracted and put together to obtain better classification performance. The optimal feature set is obtained by chain-like agent genetic algorithm (CAGA), P-value and maximum relevance-minimum multicollinearity (MRmMC). The proposed segmentation and classification methods were tested on 362 cervical Pap smear images. Experimental results showed that the cervical cell nuclei can be segmented by the proposed segmentation method with high effective segmentation results (Sensitivity: 94.25%±1.03% and Specificity 93.45%±1.14%). The feature selection method based on CAGA with Gabor features has the highest classification performance for normal, uninvolved and abnormal images (more than 96% accuracy). The proposed method can automatically and effectively segment cell nuclei of microscopic images. From the experimental results, Gabor features and feature selection based on CAGA are apparently helpful for improving the performance of classification.