OpenRaiser/Intern-Atlas huggingface dataset:
Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild
Venue: arXiv cs.CV 2022
Year: 2022

In this work, we tackle the challenging task of jointly tracking hand object pose and reconstructing their shapes from depth point cloud sequences in the wild, given the initial poses at frame 0. We for the first time propose a point cloud based hand joint tracking network, HandTrackNet, to estimate the inter-frame hand joint motion. Our HandTrackNet proposes a novel hand pose canonicalization module to ease the tracking task, yielding accurate and robust hand joint tracking. Our pipeline then reconstructs the full hand via converting the predicted hand joints into a template-based parametric hand model MANO. For object tracking, we devise a simple yet effective module that estimates the object SDF from the first frame and performs optimization-based tracking. Finally, a joint optimization step is adopted to perform joint hand and object reasoning, which alleviates the occlusion-induced ambiguity and further refines the hand pose. During training, the whole pipeline only sees purely synthetic data, which are synthesized with sufficient variations and by depth simulation for the ease of generalization. The whole pipeline is pertinent to the generalization gaps and thus directly transferable to real in-the-wild data. We evaluate our method on two real hand object interaction datasets, e.g. HO3D and DexYCB, without any finetuning. Our experiments demonstrate that the proposed method significantly outperforms the previous state-of-the-art depth-based hand and object pose estimation and tracking methods, running at a frame rate of 9 FPS.
Facilitating Self-monitored Physical Rehabilitation with Virtual Reality and Haptic feedback
Venue: arXiv cs.HC 2022
Year: 2022

Physical rehabilitation is essential to recovery from joint replacement operations. As a representation, total knee arthroplasty (TKA) requires patients to conduct intensive physical exercises to regain the knee's range of motion and muscle strength. However, current joint replacement physical rehabilitation methods rely highly on therapists for supervision, and existing computer-assisted systems lack consideration for enabling self-monitoring, making at-home physical rehabilitation difficult. In this paper, we investigated design recommendations that would enable self-monitored rehabilitation through clinical observations and focus group interviews with doctors and therapists. With this knowledge, we further explored Virtual Reality(VR)-based visual presentation and supplemental haptic motion guidance features in our implementation VReHab, a self-monitored and multimodal physical rehabilitation system with VR and vibrotactile and pneumatic feedback in a TKA rehabilitation context. We found that the third point of view real-time reconstructed motion on a virtual avatar overlaid with the target pose effectively provides motion awareness and guidance while haptic feedback helps enhance users' motion accuracy and stability. Finally, we implemented \systemname to facilitate self-monitored post-operative exercises and validated its effectiveness through a clinical study with 10 patients.
Twin-width V: linear minors, modular counting, and matrix multiplication
Venue: arXiv cs.DS 2022
Year: 2022

We continue developing the theory around the twin-width of totally ordered binary structures, initiated in the previous paper of the series. We first introduce the notion of parity and linear minors of a matrix, which consists of iteratively replacing consecutive rows or consecutive columns with a linear combination of them. We show that a matrix class has bounded twin-width if and only if its linear-minor closure does not contain all matrices. We observe that the fixed-parameter tractable algorithm for first-order model checking on structures given with an $O(1)$-sequence (certificate of bounded twin-width) and the fact that first-order transductions of bounded twin-width classes have bounded twin-width, both established in Twin-width I, extend to first-order logic with modular counting quantifiers. We make explicit a win-win argument obtained as a by-product of Twin-width IV, and somewhat similar to bidimensionality, that we call rank-bidimensionality. Armed with the above-mentioned extension to modular counting, we show that the twin-width of the product of two conformal matrices $A, B$ over a finite field is bounded by a function of the twin-width of $A$, of $B$, and of the size of the field. Furthermore, if $A$ and $B$ are $n \times n$ matrices of twin-width $d$ over $\mathbb F_q$, we show that $AB$ can be computed in time $O_{d,q}(n^2 \log n)$. We finally present an ad hoc algorithm to efficiently multiply two matrices of bounded twin-width, with a single-exponential dependence in the twin-width bound: If the inputs are given in a compact tree-like form, called twin-decomposition (of width $d$), then two $n \times n$ matrices $A, B$ over $\mathbb F_2$, a twin-decomposition of $AB$ with width $2^{d+o(d)}$ can be computed in time $4^{d+o(d)}n$ (resp. $4^{d+o(d)}n^{1+\varepsilon}$), and entries queried in doubly-logarithmic (resp. constant) time.
Blockchain technologies in the design of Industrial Control Systems for Smart Cities
Venue: arXiv cs.CR 2022
Year: 2022

The proliferation of sensor technologies in Industrial Control Systems (ICS) helped to transform the environment towards better automation, process control and monitoring. However, sensor technologies expose the smart cities of the future to complex security challenges. Luckily, the sensing capabilities also create opportunities to capture various data types, which apart from operational use can add substantial value to developing mechanisms to protect ICS and critical infrastructure. We discuss Blockchain (BC), a disruptive technology with applications ranging from cryptocurrency to smart contracts and the value of integrating BC technologies into the design of ICS to support modern digital forensic readiness.
Face Super-Resolution Using Stochastic Differential Equations
Venue: arXiv cs.CV 2022
Year: 2022

Diffusion models have proven effective for various applications such as images, audio and graph generation. Other important applications are image super-resolution and the solution of inverse problems. More recently, some works have used stochastic differential equations (SDEs) to generalize diffusion models to continuous time. In this work, we introduce SDEs to generate super-resolution face images. To the best of our knowledge, this is the first time SDEs have been used for such an application. The proposed method provides an improved peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and consistency than the existing super-resolution methods based on diffusion models. In particular, we also assess the potential application of this method for the face recognition task. A generic facial feature extractor is used to compare the super-resolution images with the ground truth and superior results were obtained compared with other methods. Our code is publicly available at https://github.com/marcelowds/sr-sde
Impact of a non-deterministic rekey interval on the performance of arc4random
Venue: arXiv cs.CR 2022
Year: 2022

This paper is intended to provide the readers with a clear understanding of the modification done to the pseudo-random number generator (PRNG) arc4random. The arc4random PRNG was modified to implement a non-deterministic rekey interval. A comparison was made with the unpatched version regarding performance and the rekey interval's randomization. All measurements were done when the rekey was performed. A slight increase in performance was observed, and the rekey interval was randomized as expected.
Bayesian Inference of Tissue Heterogeneity for Individualized Prediction of Glioma Growth
Venue: arXiv cs.CE 2022
Year: 2022

Reliably predicting the future spread of brain tumors using imaging data and on a subject-specific basis requires quantifying uncertainties in data, biophysical models of tumor growth, and spatial heterogeneity of tumor and host tissue. This work introduces a Bayesian framework to calibrate the spatial distribution of the parameters within a tumor growth model to quantitative magnetic resonance imaging (MRI) data and demonstrates its implementation in a pre-clinical model of glioma. The framework leverages an atlas-based brain segmentation of grey and white matter to establish subject-specific priors and tunable spatial dependencies of the model parameters in each region. Using this framework, the tumor-specific parameters are calibrated from quantitative MRI measurements early in the course of tumor development in four rats and used to predict the spatial development of the tumor at later times. The results suggest that the tumor model, calibrated by animal-specific imaging data at one time point, can accurately predict tumor shapes with a Dice coefficient > 0.89. However, the reliability of the predicted volume and shape of tumors strongly relies on the number of earlier imaging time points used for calibrating the model. This study demonstrates, for the first time, the ability to determine the uncertainty in the inferred tissue heterogeneity and the model predicted tumor shape.
3D Reconstruction using Structured Light from off-the-shelf components
Venue: arXiv cs.CV 2022
Year: 2022

The coordinate measuring machine(CMM) has been the benchmark of accuracy in measuring solid objects from nearly past 50 years or more. However with the advent of 3D scanning technology, the accuracy and the density of point cloud generated has taken over. In this project we not only compare the different algorithms that can be used in a 3D scanning software, but also create our own 3D scanner from off-the-shelf components like camera and projector. Our objective has been : 1. To develop a prototype for 3D scanner to achieve a system that performs at optimal accuracy over a wide typology of objects. 2. To minimise the cost using off-the-shelf components. 3. To reach very close to the accuracy of CMM.
Vision-based Perimeter Defense via Multiview Pose Estimation
Venue: arXiv cs.CV 2022
Year: 2022

Previous studies in the perimeter defense game have largely focused on the fully observable setting where the true player states are known to all players. However, this is unrealistic for practical implementation since defenders may have to perceive the intruders and estimate their states. In this work, we study the perimeter defense game in a photo-realistic simulator and the real world, requiring defenders to estimate intruder states from vision. We train a deep machine learning-based system for intruder pose detection with domain randomization that aggregates multiple views to reduce state estimation errors and adapt the defensive strategy to account for this. We newly introduce performance metrics to evaluate the vision-based perimeter defense. Through extensive experiments, we show that our approach improves state estimation, and eventually, perimeter defense performance in both 1-defender-vs-1-intruder games, and 2-defenders-vs-1-intruder games.
Towards Stable Co-saliency Detection and Object Co-segmentation
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we present a novel model for simultaneous stable co-saliency detection (CoSOD) and object co-segmentation (CoSEG). To detect co-saliency (segmentation) accurately, the core problem is to well model inter-image relations between an image group. Some methods design sophisticated modules, such as recurrent neural network (RNN), to address this problem. However, order-sensitive problem is the major drawback of RNN, which heavily affects the stability of proposed CoSOD (CoSEG) model. In this paper, inspired by RNN-based model, we first propose a multi-path stable recurrent unit (MSRU), containing dummy orders mechanisms (DOM) and recurrent unit (RU). Our proposed MSRU not only helps CoSOD (CoSEG) model captures robust inter-image relations, but also reduces order-sensitivity, resulting in a more stable inference and training process. { Moreover, we design a cross-order contrastive loss (COCL) that can further address order-sensitive problem by pulling close the feature embedding generated from different input orders.} We validate our model on five widely used CoSOD datasets (CoCA, CoSOD3k, Cosal2015, iCoseg and MSRC), and three widely used datasets (Internet, iCoseg and PASCAL-VOC) for object co-segmentation, the performance demonstrates the superiority of the proposed approach as compared to the state-of-the-art (SOTA) methods.
Hierarchical Cyclic Pursuit: Algebraic Curves Containing the Laplacian Spectra
Venue: arXiv cs.MA 2022
Year: 2022

The paper addresses the problem of multi-agent communication in networks with regular directed ring structure. These can be viewed as hierarchical extensions of the classical cyclic pursuit topology. We show that the spectra of the corresponding Laplacian matrices allow exact localization on the complex plane. Furthermore, we derive a general form of the characteristic polynomial of such matrices, analyze the algebraic curves its roots belong to, and propose a way to obtain their closed-form equations. In combination with frequency domain consensus criteria for high-order SISO linear agents, these curves enable one to analyze the feasibility of consensus in networks with varying number of agents.
High-Resolution Satellite Imagery for Modeling the Impact of Aridification on Crop Production
Venue: arXiv cs.CV 2022
Year: 2022

The availability of well-curated datasets has driven the success of Machine Learning (ML) models. Despite the increased access to earth observation data for agriculture, there is a scarcity of curated, labelled datasets, which limits the potential of its use in training ML models for remote sensing (RS) in agriculture. To this end, we introduce a first-of-its-kind dataset, SICKLE, having time-series images at different spatial resolutions from 3 different satellites, annotated with multiple key cropping parameters for paddy cultivation for the Cauvery Delta region in Tamil Nadu, India. The dataset comprises of 2,398 season-wise samples from 388 unique plots distributed across 4 districts of the Delta. The dataset covers multi-spectral, thermal and microwave data between the time period January 2018-March 2021. The paddy samples are annotated with 4 key cropping parameters, i.e. sowing date, transplanting date, harvesting date and crop yield. This is one of the first studies to consider the growing season (using sowing and harvesting dates) as part of a dataset. We also propose a yield prediction strategy that uses time-series data generated based on the observed growing season and the standard seasonal information obtained from Tamil Nadu Agricultural University for the region. The consequent performance improvement highlights the impact of ML techniques that leverage domain knowledge that are consistent with standard practices followed by farmers in a specific region. We benchmark the dataset on 3 separate tasks, namely crop type, phenology date (sowing, transplanting, harvesting) and yield prediction, and develop an end-to-end framework for predicting key crop parameters in a real-world setting.
A Tightly Coupled LiDAR-IMU Odometry through Iterated Point-Level Undistortion
Venue: arXiv cs.RO 2022
Year: 2022

Scan undistortion is a key module for LiDAR odometry in high dynamic environment with high rotation and translation speed. The existing line of studies mostly focuses on one pass undistortion, which means undistortion for each point is conducted only once in the whole LiDAR-IMU odometry pipeline. In this paper, we propose an optimization based tightly coupled LiDAR-IMU odometry addressing iterated point-level undistortion. By jointly minimizing the cost derived from LiDAR and IMU measurements, our LiDAR-IMU odometry method performs more accurate and robust in high dynamic environment. Besides, the method characters good computation efficiency by limiting the quantity of parameters.
From One to Many: Dynamic Cross Attention Networks for LiDAR and Camera Fusion
Venue: arXiv cs.CV 2022
Year: 2022

LiDAR and cameras are two complementary sensors for 3D perception in autonomous driving. LiDAR point clouds have accurate spatial and geometry information, while RGB images provide textural and color data for context reasoning. To exploit LiDAR and cameras jointly, existing fusion methods tend to align each 3D point to only one projected image pixel based on calibration, namely one-to-one mapping. However, the performance of these approaches highly relies on the calibration quality, which is sensitive to the temporal and spatial synchronization of sensors. Therefore, we propose a Dynamic Cross Attention (DCA) module with a novel one-to-many cross-modality mapping that learns multiple offsets from the initial projection towards the neighborhood and thus develops tolerance to calibration error. Moreover, a \textit{dynamic query enhancement} is proposed to perceive the model-independent calibration, which further strengthens DCA's tolerance to the initial misalignment. The whole fusion architecture named Dynamic Cross Attention Network (DCAN) exploits multi-level image features and adapts to multiple representations of point clouds, which allows DCA to serve as a plug-in fusion module. Extensive experiments on nuScenes and KITTI prove DCA's effectiveness. The proposed DCAN outperforms state-of-the-art methods on the nuScenes detection challenge.
Towards Efficient RRAM-based Quantized Neural Networks Hardware: State-of-the-art and Open Issues
Venue: arXiv cs.ET 2022
Year: 2022

The increasing amount of data processed on edge and the demand for reducing the energy consumption for large neural network architectures have initiated the transition from traditional von Neumann architectures towards in-memory computing paradigms. Quantization is one of the methods to reduce power and computation requirements for neural networks by limiting bit precision. Resistive Random Access Memory (RRAM) devices are great candidates for Quantized Neural Networks (QNN) implementations. As the number of possible conductive states in RRAMs is limited, a certain level of quantization is always considered when designing RRAM-based neural networks. In this work, we provide a comprehensive analysis of state-of-the-art RRAM-based QNN implementations, showing where RRAMs stand in terms of satisfying the criteria of efficient QNN hardware. We cover hardware and device challenges related to QNNs and show the main unsolved issues and possible future research directions.
WARC-DL: Scalable Web Archive Processing for Deep Learning
Venue: arXiv cs.DL 2022
Year: 2022

Web archives have grown to petabytes. In addition to providing invaluable background knowledge on many social and cultural developments over the last 30 years, they also provide vast amounts of training data for machine learning. To benefit from recent developments in Deep Learning, the use of web archives requires a scalable solution for their processing that supports inference with and training of neural networks. To date, there is no publicly available library for processing web archives in this way, and some existing applications use workarounds. This paper presents WARC-DL, a deep learning-enabled pipeline for web archive processing that scales to petabytes.
Tree decompositions with bounded independence number: beyond independent sets
Venue: arXiv cs.DS 2022
Year: 2022

We continue the study of graph classes in which the treewidth can only be large due to the presence of a large clique, and, more specifically, of graph classes with bounded tree-independence number. In [Dallard, Milanič, and Štorgel, Treewidth versus clique number. {II}. Tree-independence number, 2022], it was shown that the Maximum Weight Independent Packing problem, which is a common generalization of the Independent Set and Induced Matching problems, can be solved in polynomial time provided that the input graph is given along with a tree decomposition with bounded independence number. We provide further examples of algorithmic problems that can be solved in polynomial time under this assumption. This includes, for all even positive integers $d$, the problem of packing subgraphs at distance at least $d$ (generalizing the Maximum Weight Independent Packing problem) and the problem of finding a large induced sparse subgraph satisfying an arbitrary but fixed property expressible in counting monadic second-order logic. As part of our approach, we generalize some classical results on powers of chordal graphs to the context of general graphs and their tree-independence numbers.
An Empirical Study on Cross-X Transfer for Legal Judgment Prediction
Venue: arXiv cs.CL 2022
Year: 2022

Cross-lingual transfer learning has proven useful in a variety of Natural Language Processing (NLP) tasks, but it is understudied in the context of legal NLP, and not at all in Legal Judgment Prediction (LJP). We explore transfer learning techniques on LJP using the trilingual Swiss-Judgment-Prediction dataset, including cases written in three languages. We find that cross-lingual transfer improves the overall results across languages, especially when we use adapter-based fine-tuning. Finally, we further improve the model's performance by augmenting the training dataset with machine-translated versions of the original documents, using a 3x larger training corpus. Further on, we perform an analysis exploring the effect of cross-domain and cross-regional transfer, i.e., train a model across domains (legal areas), or regions. We find that in both settings (legal areas, origin regions), models trained across all groups perform overall better, while they also have improved results in the worst-case scenarios. Finally, we report improved results when we ambitiously apply cross-jurisdiction transfer, where we further augment our dataset with Indian legal cases.
Inferring Strings from Position Heaps in Linear Time
Venue: arXiv cs.DS 2022
Year: 2022

Position heaps are index structures of text strings used for the string matching problem. They are rooted trees whose edges and nodes are labeled and numbered, respectively. This paper is concerned with variants of the inverse problem of position heap construction and gives linear-time algorithms for those problems. The basic problem is to restore a text string from a rooted tree with labeled edges and numbered nodes. In the variant problems, the input trees may miss edge labels or node numbers which we must restore as well.