Reviewerly/exHarmony huggingface dataset:
MVS-T: A Coarse-to-Fine Multi-View Stereo Network with Transformer for Low-Resolution Images 3D Reconstruction

A coarse-to-fine multi-view stereo network with Transformer (MVS-T) is proposed to solve the problems of sparse point clouds and low accuracy in reconstructing 3D scenes from low-resolution multi-view images. The network uses a coarse-to-fine strategy to estimate the depth of the image progressively and reconstruct the 3D point cloud. First, pyramids of image features are constructed to transfer the semantic and spatial information among features at different scales. Then, the Transformer module is employed to aggregate the image's global context information and capture the internal correlation of the feature map. Finally, the image depth is inferred by constructing a cost volume and iterating through the various stages. For 3D reconstruction of low-resolution images, experiment results show that the 3D point cloud obtained by the network is more accurate and complete, which outperforms other advanced algorithms in terms of objective metrics and subjective visualization.
An efficient approach for mining weighted uncertain interesting patterns

The mining of weighted uncertain interesting patterns (WUIPs) is an interesting problem with many practical applications. In this study, we propose an effective method for mining WUIPs from uncertain databases. We first introduce the TPN-list structure, an extended version of the N-list structure, which is used to represent and discover the WUIPs. A TPN-list intersection algorithm is then developed, which has linear complexity and the ability to self-reduce its size. Several theorems are also proposed for the fast calculation and determination of a WUIP based on its TPN-list. Finally, we propose the HWUIPM algorithm, based on the above proposals, for mining WUIPs from uncertain databases. Our experimental results demonstrate that the proposed algorithm outperforms other state-of-the-art algorithms for mining WUIPs in terms of running time, resource usage and scalability.
Are cryptocurrencies homogeneous?

Abstract This article investigates if cryptocurrencies returns' are similarly affected by a selection of demand‐ and supply‐side determinants. Homogeneity among cryptocurrencies is tested via a least absolute shrinkage and selection operator (LASSO) model where determinants of Bitcoin returns are applied to a sample of 12 cryptocurrencies. The analysis goes beyond existing research by simultaneously covering different periods and design choices of cryptocurrencies. The results show that cryptocurrencies are heterogeneous, apart from some similarities in the impact of technical determinants and cybercrime. The cryptocurrency market displays evidence of substitution effects, and design choices related explain the impact of the determinants of return.
Assessing the seriousness of cybercrime: The case of computer misuse crime in the United Kingdom and the victims’ perspective

The reform of the Crime Survey of England and Wales (a national victim survey) has exposed a very high number of individuals who fall victim to computer misuse cybercrimes such as hacking, computer viruses and ransomware. These crimes receive very little attention from the criminal justice system and very few are brought to justice, partly because of the nature of them (global crimes), but also because of a lack of capability among the police. This paper draws on official statistics, an empirical survey and interview research with computer misuse victims. The paper juxtaposes the low priority and lack of resources given to this crime by political and police leaders against many victims’ perceptions and experiences of the crime as equivalent if not more serious than physical counterparts such as burglary, where there is greater interest. The increasing prominence of the virtual world in human life and the impacts of these crimes call for a reappraisal of the official assessment of seriousness in order to raise the priority and increase the capacity of criminal justice towards such offences.
Using quantitative data on postural activity to develop methods to predict and prevent cybersickness

In this article, we discuss general approaches to the design of interventions that are intended to overcome the problem of cybersickness among users of head-mounted display (HMD) systems. We note that existing approaches have had limited success, and we suggest that this may be due, in part, to the traditional focus on the design of HMD hardware and content. As an alternative, we argue that cybersickness may have its origins in the user’s ability (or inability) to stabilize their own bodies during HMD use. We argue that HMD systems often promote unstable postural control, and that existing approaches to cybersickness intervention are not likely to promote improved stability. We argue that successful cybersickness interventions will be designed to promote stability in the control of the body during HMD use. Our approach motivates new types of interventions; we describe several possible directions for the development of such interventions. We conclude with a discussion of new research that will be required to permit our approach to lead to interventions that can be implemented by HMD designers.
Unified QA-aware Knowledge Graph Generation Based on Multi-modal Modeling

Understanding the long duration videos' storyline is often considered a major challenge in the field of video understanding. To promote research on understanding longer videos in the community, the deep video understanding (DVU) task is suggested for recognizing interactions at the scene level and relationships at the movie level, as well as answering questions at these two levels. In this work, we propose a unified QA-aware knowledge graph generation approach, which consists of the relation-centric graph and interaction-centric graph and demonstrates the powerful performance of multimodal pre-training models in solving such problems. Extensive validations on the HLVU dataset demonstrate the effectiveness of our proposed method.
A reinforcement learning approach for single redundant view co-training text classification

We tackle the problem of learning classification models with very small amounts of labeled data (e.g., less than 10% of the dataset) by introducing a novel Single View Co-Training strategy supported by Reinforcement Learning (CoRL). CoRL is a novel semi-supervised learning framework that can be used with a single view (representation). Differently from traditional co-training that requires at least two sufficient and independent data views (e.g., modes), our solution is applicable to any kind of data. Our approach exploits a reinforcement learning (RL) paradigm as a strategy to relax the view independence assumption, using a stronger iterative agent that builds more precise combined decision class boundaries. Our experimental evaluation with four popular textual benchmarks demonstrates that CoRL can produce better classifiers than confidence-based co-training methods, while producing high effectiveness in comparison with the state-of-the-art in semi-supervised learning. In our experiments, CoRL reduced the labeling effort by more than 80% with no losses in classification effectiveness, outperforming state-of-the-art baselines, including methods based on neural networks, with gains of up to 96% against some of the best competitors.
Loss-tolerant quantum multi-party key agreement without quantum storage

Quantum key distribution (QKD) and quantum key agreement (QKA) are two main branches of key establishment in quantum cryptography. However, the research of QKA falls far behind that of QKD, especially in practicability. The main reason is that QKA needs to resist not only the outside eavesdropping but also the participant cheating. Resisting dishonest participant is more difficult than resisting outside eavesdropping, especially when the apparatuses are imperfect. Actually, existing QKA protocols cannot tolerate the channel loss and have to rely on stable quantum storage. To solve this problem, we give a new quantum multi-party key agreement protocol based on the error-correcting code. Our protocol is loss tolerant, and the participants can measure the received qubits immediately in one of two conjugate bases, without storage, so our protocol can eliminate the requirement of quantum storage. Besides, our protocol is more fair because it can partially discriminate dishonest participants' cheating from outside eavesdropping (previously, these two attacks are generally checked simultaneously via decoy states but cannot be discriminated), as a result, dishonest participants generally will not cheat at the cost of losing good reputation.
Aspect opinion routing network with interactive attention for aspect-based sentiment classification

Aspect-based sentiment classification task aims at discovering the sentiment polarity on a specific aspect term. Since a sentence often contains several sentiments for different aspects, it is very important to capture the correspondences between aspects and opinion words in a sentence. To this end, we propose an aspect opinion interactive attention routing network model, which adopts an interactive attention mechanism between specific aspect embeddings learned from graph convolutional network and opinion semantic embeddings learned from three bi-directional long short-term memory networks. Furthermore, dynamic routing is applied to the neural network so that its output values are constantly regulated during training to generate feature representations of specific aspects related to opinion word. Based on the above, our approach identifies opinion words feature information related to specific aspect words in context. Experimental results demonstrate that the model outperforms 16 existing methods in terms of accuracy and macro-average F1 score on most of the benchmark datasets.
Sentiment-aware Classifier for Out-of-Context Caption Detection

In this work we propose additions to the COSMOS and COSMOS on Steroids pipelines for the detection of Cheapfakes for Task 1 of the ACM Grand Challenge for Detecting Cheapfakes. We compute sentiment features, namely polarity and subjectivity, using the news image captions. Multiple logistic regression results show that these sentiment features are significant in prediction of the outcome. We then combine the sentiment features with the four image-text features obtained in the aforementioned previous works to train an MLP. This classifies sets of inputs into being out-of-context (OOC) or not-out-of-context (NOOC). On a test set of 400 samples, the MLP with all features achieved a score of 87.25%, and that with only the image-text features a score of 88%. In addition to the challenge requirements, we also propose a separate pipeline to automatically construct caption pairs and annotations using the images and captions provided in the large, un-annotated training dataset. We hope that this endeavor will open the door for improvements, since hand-annotating cheapfake labels is time-consuming. To evaluate the performance on the test set, the Docker image with the models is available at: https://hub.docker.com/repository/docker/malkaddour/mmsys22cheapfakes. The open-source code for the project is accessible at: https://github.com/malkaddour/ACMM-22-Cheapfake-Detection-Sentiment-aware-Classifier-for-Out-of-Context-Caption-Detection.
DS-MVSNet: Unsupervised Multi-view Stereo via Depth Synthesis

In recent years, supervised or unsupervised learning-based MVS methods achieved excellent performance compared with traditional methods. However, these methods only use the probability volume computed by cost volume regularization to predict reference depths and this manner cannot mine enough information from the probability volume. Furthermore, the unsupervised methods usually try to use two-step or additional inputs for training which make the procedure more complicated. In this paper, we propose the DS-MVSNet, an end-to-end unsupervised MVS structure with the source depths synthesis. To mine the information in probability volume, we creatively synthesize the source depths by splattering the probability volume and depth hypotheses to source views. Meanwhile, we propose the adaptive Gaussian sampling and improved adaptive bins sampling approach that improve the depths hypotheses accuracy. On the other hand, we utilize the source depths to render the reference images and propose depth consistency loss and depth smoothness loss. These can provide additional guidance according to photometric and geometric consistency in different views without additional inputs. Finally, we conduct a series of experiments on the DTU dataset and Tanks $&$ Temples dataset that demonstrate the efficiency and robustness of our DS-MVSNet compared with the state-of-the-art methods.
Dynamic Graph Reasoning for Multi-person 3D Pose Estimation

Multi-person 3D pose estimation is a challenging task because of occlusion and depth ambiguity, especially in the cases of crowd scenes. To solve these problems, most existing methods explore modeling body context cues by enhancing feature representation with graph neural networks or adding structural constraints. However, these methods are not robust for their single-root formulation that decoding 3D poses from a root node with a pre-defined graph. In this paper, we propose GR-M3D, which models the Multi-person 3D pose estimation with dynamic Graph Reasoning. The decoding graph in GR-M3D is predicted instead of pre-defined. In particular, It firstly generates several data maps and enhances them with a scale and depth aware refinement module (SDAR). Then multiple root keypoints and dense decoding paths for each person are estimated from these data maps. Based on them, dynamic decoding graphs are built by assigning path weights to the decoding paths, while the path weights are inferred from those enhanced data maps. And this process is named dynamic graph reasoning (DGR). Finally, the 3D poses are decoded according to dynamic decoding graphs for each detected person. GR-M3D can adjust the structure of the decoding graph implicitly by adopting soft path weights according to input data, which makes the decoding graphs be adaptive to different input persons to the best extent and more capable of handling occlusion and depth ambiguity than previous methods. We empirically show that the proposed bottom-up approach even outperforms top-down methods and achieves state-of-the-art results on three 3D pose datasets.
PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

We present PVSeRF, a learning framework that reconstructs neural radiance fields from single-view RGB images, for novel view synthesis. Previous solutions, such as pixelNeRF, rely only on pixel-aligned features and suffer from feature ambiguity issues. As a result, they struggle with the disentanglement of geometry and appearance, leading to implausible geometries and blurry results. To address this challenge, we propose to incorporate explicit geometry reasoning and combine it with pixel-aligned features for radiance field prediction. Specifically, in addition to pixel-aligned features, we further constrain the radiance field learning to be conditioned on i) voxel-aligned features learned from a coarse volumetric grid and ii) fine surface-aligned features extracted from a regressed point cloud. We show that the introduction of such geometry-aware features helps to achieve a better disentanglement between appearance and geometry, i.e. recovering more accurate geometries and synthesizing higher quality images of novel views. Extensive experiments against state-of-the-art methods on ShapeNet benchmarks demonstrate the superiority of our approach for single-image novel view synthesis.
EliMRec: Eliminating Single-modal Bias in Multimedia Recommendation

The main idea of multimedia recommendation is to introduce the profile content of multimedia documents as an auxiliary, so as to endow recommenders with generalization ability and gain better performance. However, recent studies using non-uniform datasets roughly fuse single-modal features into multi-modal features and adopt the strategy of directly maximizing the likelihood of user preference scores, leading to the single-modal bias. Owing to the defect in architecture, there is still room for improvement for recent multimedia recommendation.
Evidential Reasoning for Video Anomaly Detection

Video anomaly detection aims to discriminate events that deviate from normal patterns in a video. Modeling the decision boundaries of anomalies is challenging, due to the uncertainty in the probability of deviating from normal patterns. In this paper, we propose a deep evidential reasoning method that explicitly learns the uncertainty to model the boundaries. Our method encodes various visual cues as evidences representing potential deviations, assigns beliefs to the predicted probability of deviating from normal patterns based on the evidences, and estimates the uncertainty from the remained beliefs to model the boundaries. To do this, we build a deep evidential reasoning network to encode evidence vectors and estimate uncertainty by learning evidence distributions and deriving beliefs from the distributions. We introduce an unsupervised strategy to train our network by minimizing an energy function of the deep Gaussian mixed model (GMM). Experimental results show that our uncertainty score is beneficial for modeling the boundaries of video anomalies on three benchmark datasets.
Domain-Specific Conditional Jigsaw Adaptation for Enhancing transferability and Discriminability

Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a label-rich source domain to a target domain where the label is unavailable. Existing approaches tend to reduce the distribution discrepancy between the source and target domains or assign the pseudo target labels to implement a self-training strategy. However, the transferability or discriminability lackage of the traditional methods results in the limited ability to generalize on the target domain. To remedy this issue, a novel unsupervised domain adaptation framework called Domain-specific Conditional Jigsaw Adaptation Network (DCJAN) is proposed for UDA, which simultaneously encourages the network to extract transferable and discriminative features. To improve the discriminability, a conditional jigsaw module is presented to reconstruct class-aware features of the original images by reconstructing that of corresponding shuffled images. Moreover, in order to enhance the transferability, a domain-specific jigsaw adaptation is proposed to deal with the domain gaps, which utilizes the prior knowledge of jigsaw puzzles to reduce mismatching. It trains conditional jigsaw modules for each domain and updates the shared feature extractor to make the domain-specific conditional jigsaw modules could perform well not only on the corresponding domain but also on the other domain. A consistent conditioning strategy is proposed to ensure the safe training of conditional jigsaw. Experiments conducted on the widely-used Office-31, Office-Home, VisDA-2017, and DomainNet datasets demonstrate the effectiveness of the proposed approach, which outperforms the state-of-the-art methods.
A Knowledge Augmented and Multimodal-Based Framework for Video Summarization

Video summarization aims to generate a compact version of a lengthy video that retains its primary content. In general, humans are gifted with producing a high-quality video summary, because they acquire crucial content through multiple dimensional information and own abundant background knowledge about the original video. However, existing methods rarely consider multichannel information and ignore the impact of external knowledge, resulting in the limited quality of the generated summaries. This paper proposes a knowledge augmented and multimodal-based video summarization method, termed KAMV, to address the problem above. Specifically, we design a knowledge encoder with a hybrid method consisting of generation and retrieval, to capture descriptive content and latent connections between events and entities based on the external knowledge base, which can provide rich implicit knowledge for better comprehending the video viewed. Furthermore, for the sake of exploring the interactions among visual, audio, implicit knowledge and emphasizing the content that is most relevant to the desired summary, we present a fusion module under the supervision of these multimodal information. By conducting extensive experiments on four public datasets, the results demonstrate the superior performance yielded by the proposed KAMV compared to the state-of-the-art video summarization approaches.
CVNets

We introduce CVNets, a high-performance open-source library for training deep neural networks for visual recognition tasks, including classification, detection, and segmentation. CVNets supports image and video understanding tools, including data loading, data transformations, novel data sampling methods, and implementations of several standard networks with similar or better performance than previous studies. Our source code is available at: https://github.com/apple/ml-cvnets.
MVPTR

Previous vision-language pre-training models mainly construct multi-modal inputs with tokens and objects (pixels) followed by performing cross-modality interaction between them. We argue that the input of only tokens and object features limits high-level semantic alignment like phrase-to-region grounding. Meanwhile, multi-level alignments are inherently consistent and able to facilitate the representation learning synergistically. Therefore, in this paper, we propose to learn Multi-level semantic alignment for Vision-language Pre-TRaining (MVPTR). In MVPTR, we follow the nested structure of both modalities to introduce concepts as high-level semantics. To ease the learning from multi-modal multi-level inputs, our framework is split into two stages, the first stage focuses on intra-modality multi-level representation learning, the second enforces interactions across modalities via both coarse-grained and fine-grained semantic alignment tasks. In addition to the commonly used image-text matching and masked language model tasks, we introduce a masked concept recovering task in the first stage to enhance the concept representation learning, and two more tasks in the second stage to explicitly encourage multi-level alignments across modalities. Our model achieves state-of-the-art results on several vision and language tasks.