Reviewerly/exHarmony huggingface dataset:
Spatial Commonsense Graph for Object Localisation in Partial Scenes

We solve object localisation in partial scenes, a new problem of estimating the unknown position of an object (e.g. where is the bag?) given a partial 3D scan of a scene. The proposed solution is based on a novel scene graph model, the Spatial Commonsense Graph (SCG), where objects are the nodes and edges define pairwise distances between them, enriched by concept nodes and relationships from a commonsense knowledge base. This allows SCG to better generalise its spatial inference to unknown 3D scenes. The SCG is used to estimate the unknown position of the target object in two steps: first, we feed the SCG into a novel Proximity Prediction Network, a graph neural network that uses attention to perform distance prediction between the node representing the target object and the nodes representing the observed objects in the SCG; second, we propose a Localisation Module based on circular intersection to estimate the object position using all the predicted pairwise distances in order to be independent of any reference system. We create a new dataset of partially reconstructed scenes to benchmark our method and baselines for object localisation in partial scenes, where our proposed method achieves the best localisation performance. Code and Dataset are available here: https://github.com/IIT-PAVIS/SpatialCommonsenseGraph
Dynamically Transformed Instance Normalization Network for Generalizable Person Re-Identification

Existing person re-identification methods often suffer significant performance degradation on unseen domains, which fuels interest in domain generalizable person re-identification (DG-PReID). As an effective technology to alleviate domain variance, the Instance Normalization (IN) has been widely employed in many existing works. However, IN also suffers from the limitation of eliminating discriminative patterns that might be useful for a particular domain or instance. In this work, we propose a new normalization scheme called Dynamically Transformed Instance Normalization (DTIN) to alleviate the drawback of IN. Our idea is to employ dynamic convolution to allow the unnormalized feature to control the transformation of the normalized features into new representations. In this way, we can ensure the network has sufficient flexibility to strike the right balance between eliminating irrelevant domain-specific features and adapting to individual domains or instances. We further utilize a multi-task learning strategy to train the model, ensuring it can adaptively produce discriminative feature representations for an arbitrary domain. Our results show a great domain generalization capability and achieve state-of-the-art performance on three mainstream DG-PReID settings.
A Parallel Consensus Mechanism Using PBFT Based on DAG-Lattice Structure in the Internet of Vehicles

The blockchain technology in the Internet of Vehicles (IoV) can be applied to solve security problems. Due to the high dynamics and resource constraints of the IoV, the traditional single-chain blockchain system is not suitable for the practical scenarios. In our previous work, we proposed a lightweight blockchain architecture based on the directed acyclic graph lattice (DAG-lattice) structure for the IoV. However, the current consensus algorithms are not suitable for the previous architecture mentioned above. Therefore, this article proposes a parallel consensus mechanism based on the DAG-lattice structure in the IoV using practical Byzantine fault tolerance (PBFT) as consensus algorithm. This mechanism solves the problem of low efficiency of PBFT caused by too many consensus nodes and the mobility of nodes by adopting the method of network sharding, and meanwhile to ensure that transactions can be reached on consensus parallelly. To avoid the selection of primary nodes from being predicted in advance, this article proposes a primary node selection method based on verifiable random function (VRF). Moreover, a method is proposed of selecting transactions based on weighted round robin (WRR) scheduling to motivate nodes for the purpose of maintaining a high reputation score. Finally, the performance of the proposed scheme is verified by experiments whose results show that the proposed consensus mechanism shows better performance than other comparable schemes in terms of the time added to the blockchain, transaction per second (TPS), consensus success rate, and the time of obtaining transaction. And the proposed mechanism is less dependent on roadside unit (RSU).
DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos

Convolutional neural network inference on video data requires powerful hardware for real-time processing. Given the inherent coherence across consecutive frames, large parts of a video typically change little. By skipping identical image regions and truncating insignificant pixel updates, computational redundancy can in theory be reduced significantly. However, these theoretical savings have been difficult to translate into practice, as sparse updates hamper computational consistency and memory access coherence; which are key for efficiency on real hardware. With DeltaCNN, we present a sparse convolutional neural network framework that enables sparse frame-by-frame updates to accelerate video inference in practice. We provide sparse implementations for all typical CNN layers and propagate sparse feature updates end-to-end – without accumulating errors over time. DeltaCNN is applicable to all convolutional neural networks without retraining. To the best of our knowledge, we are the first to significantly outperform the dense reference, cuDNN, in practical settings, achieving speedups of up to 7x with only marginal differences in accuracy. Our CUDA kernels and PyTorch extensions can be found at https://github.com/facebookresearch/DeltaCNN.
A Brief Journey through Collision Models for Multipartite Open Quantum Dynamics

The quantum collision models are a useful method to describe the dynamics of an open quantum system by means of repeated interactions between the system and some particles of the environment, which are usually termed “ancillas”. In this paper, we review the main collision models for the dynamics of multipartite open quantum systems, which are composed of several subsystems. In particular, we are interested in models that are based on elementary collisions between the subsystems and the ancillas, and that simulate global and/or local Markovian master equations in the limit of infinitesimal timestep. After discussing the mathematical details of the derivation of a generic collision-based master equation, we provide the general ideas at the basis of the collision models for multipartite systems, we discuss their strengths and limitations, and we show how they may be simulated on a quantum computer. Moreover, we analyze some properties of a collision model based on entangled ancillas, derive the master equation it generates for small timesteps, and prove that the coefficients of this master equation are subject to a constraint that limits their generality. Finally, we present an example of such collision model with two bosonic ancillas entangled in a two-mode squeezed thermal state.
BAFN: Bi-Direction Attention Based Fusion Network for Multimodal Sentiment Analysis

Attention-based networks currently identify their effectiveness in multimodal sentiment analysis. However, existing methods ignore the redundancy of auxiliary modalities. More importantly, existing methods only attend to top-down attention (static process) or down-top attention (implicit process), leading to the coarse-grained multimodal sentiment context. In this paper, during the preprocessing period, we first propose the multimodal dynamic enhanced block to capture the intra-modality sentiment context. This can effectively decrease the intra-modality redundancy of auxiliary modalities. Furthermore, the bi-direction attention block is proposed to capture fine-grained multimodal sentiment context via the novel bi-direction multimodal dynamic routing mechanism. Specifically, the bi-direction attention block first highlights the explicit and low-level multimodal sentiment context. Then, the low-level multimodal context is transmitted to a carefully designed bi-direction multimodal dynamic routing procedure. This allows us to dynamically update and investigate high-level and much more fine-grained multimodal sentiment contexts. The experiments demonstrate that our fusion network can achieve state-of-the-art performance. Notably, our model outperforms the best baseline on the metric ‘Acc-7’ with an improvement of 6.9%.
Hide &amp; Seek: Privacy-Preserving Rebalancing on Payment Channel Networks

Payment channels effectively move the transaction load off-chain thereby successfully addressing the inherent scalability problem most cryptocurrencies face. A major drawback of payment channels is the need to “top up” funds on-chain when a channel is depleted. Rebalancing was proposed to alleviate this issue, where parties with depleting channels move their funds along a cycle to replenish their channels off-chain. Protocols for rebalancing so far either introduce local solutions or compromise privacy. In this work, we present an opt-in rebalancing protocol that is both private and globally optimal, meaning our protocol maximizes the total amount of rebalanced funds. We study rebalancing from the framework of linear programming. To obtain full privacy guarantees, we leverage multi-party computation in solving the linear program, which is executed by selected participants to maintain efficiency. Finally, we efficiently decompose the rebalancing solution into incentive-compatible cycles which conserve user balances when executed atomically.
Navigation Among Movable Obstacles with Object Localization using Photorealistic Simulation

While mobile navigation has been focused on obstacle avoidance, Navigation Among Movable Obstacles (NAMO) via interaction with the environment, is a problem that is still open and challenging. This paper, presents a novel system integration to handle NAMO using visual feedback. In order to explore the capabilities of our introduced system, we explore the solution of the problem via graph-based path planning in a photorealistic simulator (NVIDIA Isaac Sim), in order to identify if the simulation-to-reality (sim2real) problem in robot navigation can be resolved. We consider the case where a wheeled robot navigates in a warehouse, in which movable boxes are common obstacles. We enable online real-time object localization and obstacle movability detection, to either avoid objects or, if it is not possible, to clear them out from the robot planned path by using pushing actions. We firstly test the integrated system in photorealistic environments, and we then validate the method on a real-world mobile wheeled robot (UCL MPPL) and its on-board sensory and computing system.
PT4AL: Using Self-supervised Pretext Tasks for Active Learning

Labeling a large set of data is expensive. Active learning aims to tackle this problem by asking to annotate only the most informative data from the unlabeled set. We propose a novel active learning approach that utilizes self-supervised pretext tasks and a unique data sampler to select data that are both difficult and representative. We discover that the loss of a simple self-supervised pretext task, such as rotation prediction, is closely correlated to the downstream task loss. Before the active learning iterations, the pretext task learner is trained on the unlabeled set, and the unlabeled data are sorted and split into batches by their pretext task losses. In each active learning iteration, the main task model is used to sample the most uncertain data in a batch to be annotated. We evaluate our method on various image classification and segmentation benchmarks and achieve compelling performances on CIFAR10, Caltech-101, ImageNet, and Cityscapes. We further show that our method performs well on imbalanced datasets, and can be an effective solution to the cold-start problem where active learning performance is affected by the randomly sampled initial labeled set. Code is available at https://github.com/johnsk95/PT4AL
StrongFusionMOT: A Multi-Object Tracking Method Based on LiDAR-Camera Fusion

This article proposes a multi-object tracking (MOT) method called StrongFusionMOT, which fuses the information of light detection and ranging (LiDAR) and camera sensors. The major contributions of the proposed StrongFusionMOT are in three aspects. First, in the detection fusion stage, the depth information extracted by means of absolute difference (AD)-census is supplemented to 2-D detections to facilitate the fusion of 2-D and 3-D detections. This detection fusion pattern enhances fusion robustness and provides accurate fusion performance. Second, a new cost function design named shape-distance intersection over union (SDIoU) is proposed by taking into account not only the intersection between the two bounding boxes but also their shapes and relative distances. This cost function eliminates the shortcomings of the existing IoU designs and greatly enhances association precision. Third, a multiframe matching mechanism that involves tracks in the past <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${n}$ </tex-math></inline-formula> frames is proposed for reappeared tracks, which effectively suppresses cumulative errors resulting from consecutive frames of track predictions and greatly enhances association robustness. The effectiveness of the proposed StrongFusionMOT is evaluated by means of comparative experiments with the state-of-the-art MOT solutions using the KITTI dataset and the nuScenes dataset. Both quantitative and qualitative results demonstrate the superiorities of the proposed method in terms of various performance metrics.
Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences

We propose Probabilistic Warp Consistency, a weakly-supervised learning objective for semantic matching. Our approach directly supervises the dense matching scores predicted by the network, encoded as a conditional probability distribution. We first construct an image triplet by applying a known warp to one of the images in a pair depicting different instances of the same object class. Our probabilistic learning objectives are then derived using the constraints arising from the resulting image triplet. We further account for occlusion and background clutter present in real image pairs by extending our probabilistic output space with a learnable unmatched state. To supervise it, we design an objective between image pairs depicting different object classes. We validate our method by applying it to four recent semantic matching architectures. Our weakly-supervised approach sets a new state-of-the-art on four challenging semantic matching benchmarks. Lastly, we demonstrate that our objective also brings substantial improvements in the strongly-supervised regime, when combined with keypoint annotations.
Cycle-Consistent Counterfactuals by Latent Transformations

CounterFactual (CF) visual explanations try to find images similar to the query image that change the decision of a vision system to a specified outcome. Existing methods either require inference-time optimization or joint training with a generative adversarial model which makes them time-consuming and difficult to use in practice. We propose a novel approach, Cycle-Consistent Counterfactuals by Latent Transformations (C3LT), which learns a latent transformation that automatically generates visual CFs by steering in the latent space of generative models. Our method uses cycle consistency between the query and CF latent representations which helps our training to find better solutions. C3LT can be easily plugged into any state-of-the-art pretrained generative network. This enables our method to generate high-quality and interpretable CF images at high resolution such as those in ImageNet. In addition to several established metrics for evaluating CF explanations, we introduce a novel metric tailored to assess the quality of the generated CF examples and validate the effectiveness of our method on an extensive set of experiments.
Artifacts-Disentangled Adversarial Learning for Deepfake Detection

Due to the development of facial manipulation technologies, the generated deepfake videos cause a severe trust crisis in society. Existing methods prove that effective extraction of the artifacts introduced during the forgery process is essential for deepfake detection. However, since the features extracted by supervised binary classification contain a lot of artifact-irrelevant information, existing algorithms suffer severe performance degradation in the case of the mismatch between training and testing datasets. To overcome this issue, we propose an Artifacts-Disentangled Adversarial Learning (ADAL) framework to achieve accurate deepfake detection by disentangling the artifacts from irrelevant information. Furthermore, the proposed algorithm provides visual evidence by effectively estimating artifacts. Specifically, Multi-scale Feature Separator (MFS) in the disentanglement generator is designed to precisely transmit the artifact features and optimize the connection between the encoder and decoder. In addition, we design an Artifacts Cycle Consistency Loss (ACCL) which uses the disentangled artifacts to construct new samples and enables pixel-level supervised training for the generator to estimate more accurate artifacts. The symmetric discriminators are paralleled to differentiate the constructed samples from the original images in both fake and real domains, making the adversarial training process more stable. Extensive experiments on existing benchmarks demonstrate that the proposed method outperforms the state-of-the-art approaches.
Unpaired Image Translation via Vector Symbolic Architectures

Image-to-image translation has played an important role in enabling synthetic data for computer vision. However, if the source and target domains have a large semantic mismatch, existing techniques often suffer from source content corruption aka semantic flipping. To address this problem, we propose a new paradigm for image-to-image translation using Vector Symbolic Architectures (VSA), a theoretical framework which defines algebraic operations in a high-dimensional vector (hypervector) space. We introduce VSA-based constraints on adversarial learning for source-to-target translations by learning a hypervector mapping that inverts the translation to ensure consistency with source content. We show both qualitatively and quantitatively that our method improves over other state-of-the-art techniques.
Quality Requirement Change Management’s Challenges: An Exploratory Study Using SLR

Requirement change management plays an important role in the business world where management in the business scenario is a hard-hitting assignment because of continuously changing customer choices in respect of requirements. Giving no attention to requirement change management challenges result in consumers’ discontent. Main cause of business products is that there is no planning regarding requirement change management. Furthermore, it also affects market value. Dealing in requirement change management, it’s far important to cope with those challenges to undertake the requirement of the business consumers. This article focuses on documentation and control of quality requirement challenges by using an approach of systematic literature review. The main goal of this article is to classify critical challenges being faced by vendor companies in global software development. A total of fourteen challenges have been documented which have a severe effect on the management of quality requirements. Challenges documented by an author like ‘Incomplete requirements’, ‘Lack of Communication & Coordination, etc. are the key challenges harming managing quality requirement changes. Among these fourteen challenges, nine challenges are marked as critical challenges whose ratios are above 25%. The identified challenges were analyzed decade wise where we categorized them into three decades i.e. first decade (1992-2002), the second decade (2003-2012), and the third decade (2013-2021).
Intelligent Decision-Making and Human Language Communication Based on Deep Reinforcement Learning in a Wargame Environment

The application of artificial intelligence (AI) in games has been significantly developed and attracted much attention over the past few years. This article not only leverages the reinforcement learning multiagent deep deterministic policy gradient algorithm to realize the dynamic decision-making of game AI but also creatively incorporates deep learning and natural language processing technologies in the wargame field to transform game context situation maps into textual suggestions in wargame confrontation. In this article, we effectively integrate reinforcement learning technologies, deep learning technologies, and natural language processing technologies to generalize the semantic text output at state-of-the-art accuracy, which plays an important role in human understanding of game AI behavior. The experimental results are promising and can be used to verify the feasibility, accuracy, and performance of our proposed model in extensive simulations against benchmarking methods.
Plotly-Resampler: Effective Visual Analytics for Large Time Series

Visual analytics is arguably the most important step in getting acquainted with your data. This is especially the case for time series, as this data type is hard to describe and cannot be fully understood when using for example summary statistics. To realize effective time series visualization, four requirements have to be met; a tool should be (1) interactive, (2) scalable to millions of data points, (3) integrable in conventional data science environments, and (4) highly configurable. We observe that open source Python visualization toolkits empower data scientists in most visual analytics tasks, but lack the combination of scalability and interactivity to realize effective time series visualization. As a means to facilitate these requirements, we created Plotly-Resampler, an open source Python library. Plotly-Resampler is an add-on for Plotly's Python bindings, enhancing line chart scalability on top of an interactive toolkit by aggregating the underlying data depending on the current graph view. Plotly-Resampler is built to be snappy, as the reactivity of a tool qualitatively affects how analysts visually explore and analyze data. A benchmark task highlights how our toolkit scales better than alternatives in terms of number of samples and time series. Additionally, Plotly-Resampler's flexible data aggregation functionality paves the path towards researching novel aggregation techniques. Plotly-Resampler's integrability, together with its configurability, convenience, and high scalability, allows to effectively analyze high-frequency data in your day-to-day Python environment.