OpenRaiser/Intern-Atlas huggingface dataset:
IRShield: A Countermeasure Against Adversarial Physical-Layer Wireless Sensing
Venue: arXiv cs.CR 2022
Year: 2022

Wireless radio channels are known to contain information about the surrounding propagation environment, which can be extracted using established wireless sensing methods. Thus, today's ubiquitous wireless devices are attractive targets for passive eavesdroppers to launch reconnaissance attacks. In particular, by overhearing standard communication signals, eavesdroppers obtain estimations of wireless channels which can give away sensitive information about indoor environments. For instance, by applying simple statistical methods, adversaries can infer human motion from wireless channel observations, allowing to remotely monitor premises of victims. In this work, building on the advent of intelligent reflecting surfaces (IRSs), we propose IRShield as a novel countermeasure against adversarial wireless sensing. IRShield is designed as a plug-and-play privacy-preserving extension to existing wireless networks. At the core of IRShield, we design an IRS configuration algorithm to obfuscate wireless channels. We validate the effectiveness with extensive experimental evaluations. In a state-of-the-art human motion detection attack using off-the-shelf Wi-Fi devices, IRShield lowered detection rates to 5% or less.
Transferring Unconditional to Conditional GANs with Hyper-Modulation
Venue: arXiv cs.CV 2022
Year: 2022

GANs have matured in recent years and are able to generate high-resolution, realistic images. However, the computational resources and the data required for the training of high-quality GANs are enormous, and the study of transfer learning of these models is therefore an urgent topic. Many of the available high-quality pretrained GANs are unconditional (like StyleGAN). For many applications, however, conditional GANs are preferable, because they provide more control over the generation process, despite often suffering more training difficulties. Therefore, in this paper, we focus on transferring from high-quality pretrained unconditional GANs to conditional GANs. This requires architectural adaptation of the pretrained GAN to perform the conditioning. To this end, we propose hyper-modulated generative networks that allow for shared and complementary supervision. To prevent the additional weights of the hypernetwork to overfit, with subsequent mode collapse on small target domains, we introduce a self-initialization procedure that does not require any real data to initialize the hypernetwork parameters. To further improve the sample efficiency of the transfer, we apply contrastive learning in the discriminator, which effectively works on very limited batch sizes. In extensive experiments, we validate the efficiency of the hypernetworks, self-initialization and contrastive loss for knowledge transfer on standard benchmarks.
"Stop Asian Hate!" : Refining Detection of Anti-Asian Hate Speech During the COVID-19 Pandemic
Venue: arXiv cs.CL 2022
Year: 2022

Content warning: This work displays examples of explicit and/or strongly offensive language. Fueled by a surge of anti-Asian xenophobia and prejudice during the COVID-19 pandemic, many have taken to social media to express these negative sentiments. Identifying these posts is crucial for moderation and understanding the nature of hate in online spaces. In this paper, we create and annotate a corpus of tweets to explore anti-Asian hate speech with a finer level of granularity. Our analysis reveals that this emergent form of hate speech often eludes established approaches. To address this challenge, we develop a model and an accompanied efficient training regimen that incorporates agreement between annotators. Our approach produces up to 8.8% improvement in macro F1 scores over a strong established baseline, indicating its effectiveness even in settings where consensus among annotators is low. We demonstrate that we are able to identify hate speech that is systematically missed by established hate speech detectors.
SITA: Single Image Test-time Adaptation
Venue: arXiv cs.CV 2022
Year: 2022

In Test-time Adaptation (TTA), given a source model, the goal is to adapt it to make better predictions for test instances from a different distribution than the source. Crucially, TTA assumes no access to the source data or even any additional labeled/unlabeled samples from the target distribution to finetune the source model. In this work, we consider TTA in a more pragmatic setting which we refer to as SITA (Single Image Test-time Adaptation). Here, when making a prediction, the model has access only to the given single test instance, rather than a batch of instances, as typically been considered in the literature. This is motivated by the realistic scenarios where inference is needed on-demand instead of delaying for an incoming batch or the inference is happening on an edge device (like mobile phone) where there is no scope for batching. The entire adaptation process in SITA should be extremely fast as it happens at inference time. To address this, we propose a novel approach AugBN that requires only a single forward pass. It can be used on any off-the-shelf trained model to test single instances for both classification and segmentation tasks. AugBN estimates normalization statistics of the unseen test distribution from the given test image using only one forward pass with label-preserving transformations. Since AugBN does not involve any back-propagation, it is significantly faster compared to recent test time adaptation methods. We further extend AugBN to make the algorithm hyperparameter-free. Rigorous experimentation show that our simple algorithm is able to achieve significant performance gains for a variety of datasets, tasks, and network architectures.
Joint Symmetry Detection and Shape Matching for Non-Rigid Point Cloud
Venue: arXiv cs.CV 2022
Year: 2022

Despite the success of deep functional maps in non-rigid 3D shape matching, there exists no learning framework that models both self-symmetry and shape matching simultaneously. This is despite the fact that errors due to symmetry mismatch are a major challenge in non-rigid shape matching. In this paper, we propose a novel framework that simultaneously learns both self symmetry as well as a pairwise map between a pair of shapes. Our key idea is to couple a self symmetry map and a pairwise map through a regularization term that provides a joint constraint on both of them, thereby, leading to more accurate maps. We validate our method on several benchmarks where it outperforms many competitive baselines on both tasks.
A Survey on Deep learning based Document Image Enhancement
Venue: arXiv cs.CV 2022
Year: 2022

Digitized documents such as scientific articles, tax forms, invoices, contract papers, historic texts are widely used nowadays. These document images could be degraded or damaged due to various reasons including poor lighting conditions, shadow, distortions like noise and blur, aging, ink stain, bleed-through, watermark, stamp, etc. Document image enhancement plays a crucial role as a pre-processing step in many automated document analysis and recognition tasks such as character recognition. With recent advances in deep learning, many methods are proposed to enhance the quality of these document images. In this paper, we review deep learning-based methods, datasets, and metrics for six main document image enhancement tasks, including binarization, debluring, denoising, defading, watermark removal, and shadow removal. We summarize the recent works for each task and discuss their features, challenges, and limitations. We introduce multiple document image enhancement tasks that have received little to no attention, including over and under exposure correction, super resolution, and bleed-through removal. We identify several promising research directions and opportunities for future research.
Spatio-Temporal meets Wavelet: Disentangled Traffic Flow Forecasting via Efficient Spectral Graph Attention Network
Venue: arXiv cs.LG 2022
Year: 2022

Traffic forecasting is crucial for public safety and resource optimization, yet is very challenging due to three aspects: i) current existing works mostly exploit intricate temporal patterns (e.g., the short-term thunderstorm and long-term daily trends) within a single method, which fail to accurately capture spatio-temporal dependencies under different schemas; ii) the under-exploration of the graph positional encoding limit the extraction of spatial information in the commonly used full graph attention network; iii) the quadratic complexity of the full graph attention introduces heavy computational needs. To achieve the effective traffic flow forecasting, we propose an efficient spectral graph attention network with disentangled traffic sequences. Specifically, the discrete wavelet transform is leveraged to obtain the low- and high-frequency components of traffic sequences, and a dual-channel encoder is elaborately designed to accurately capture the spatio-temporal dependencies under long- and short-term schemas of the low- and high-frequency components. Moreover, a novel wavelet-based graph positional encoding and a query sampling strategy are introduced in our spectral graph attention to effectively guide message passing and efficiently calculate the attention. Extensive experiments on four real-world datasets show the superiority of our model, i.e., the higher traffic forecasting precision with lower computational cost.
Adjusting the Ground Truth Annotations for Connectivity-Based Learning to Delineate
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning-based approaches to delineating 3D structure depend on accurate annotations to train the networks. Yet, in practice, people, no matter how conscientious, have trouble precisely delineating in 3D and on a large scale, in part because the data is often hard to interpret visually and in part because the 3D interfaces are awkward to use. In this paper, we introduce a method that explicitly accounts for annotation inaccuracies. To this end, we treat the annotations as active contour models that can deform themselves while preserving their topology. This enables us to jointly train the network and correct potential errors in the original annotations. The result is an approach that boosts performance of deep networks trained with potentially inaccurate annotations. Code has been released at https://github.com/doruk-oner/AdjustingAnnotationswithSnakes.
Multi-User Holographic MIMO Surfaces: Channel Modeling and Spectral Efficiency Analysis
Venue: arXiv cs.IT 2022
Year: 2022

The multi-user Holographic Multiple-Input and Multiple-Output Surface (MU-HMIMOS) paradigm, which is capable of realizing large continuous apertures with minimal power consumption, has been recently considered as an energyefficient solution for future wireless networks, offering increased flexibility in impacting electromagnetic (EM) wave propagation according to the desired communication, localization, and sensing objectives. The tractable channel modeling in MU-HMIMOS wireless systems is one of the most critical research challenges, mainly due to the coupling effect induced by the excessively large number of closely spaced patch antennas. In this paper, we focus on this challenge for the downlink of multi-user MIMO communications and extend an EM-compliant channel model to multiuser case, which is expressed in the wavenumber domain using the Fourier plane wave approximation. Based on the presented channel model, we investigate the spectral efficiency of maximumratio transmission and Zero-Forcing (ZF) precoding schemes. We also introduce a novel hardware efficient ZF precoder, leveraging Neumann series (NS) expansion to replace the required matrix inversion operation, which is very hard to be computed in the conventional way due to the extremely large number of patch antennas in the envisioned MU-HMIMOS communication systems. In comparison with the conventional independent and identical Rayleigh fading channels that ignore antenna coupling effects, the proposed EM-compliant channel model captures the mutual couplings induced by the very small antenna spacing. Our extensive performance evaluation results demonstrate that our theoretical performance expressions approximate sufficiently well ...
Near-optimal bounds for signal recovery from blind phaseless periodic short-time Fourier transform
Venue: arXiv cs.IT 2022
Year: 2022

We study the problem of recovering a signal $x\in\mathbb{C}^N$ from samples of its phaseless periodic short-time Fourier transform (STFT): the magnitude of the Fourier transform of the signal multiplied by a sliding window $w\in \mathbb{C}^W$. We show that if the window $w$ is known, then a generic signal can be recovered, up to a global phase, from less than 4N phaseless STFT measurements. In the blind case, when the window is unknown, we show that the signal and the window can be determined simultaneously, up to a group of unavoidable ambiguities, from less than 4N+2W measurements. In both cases, our bounds are optimal, up to a constant smaller than two.
Indian Kidney Exchange Program: A Game Theoretic Perspective
Venue: arXiv cs.GT 2022
Year: 2022

We propose a way in which Kidney exchange can be feasibly, economically and efficiently implemented in Indian medical space, named as Indian Kidney Exchange Program(IKEP) along with Indian specific influence on compatibility and final outcomes. Kidney exchange is a boon for those suffering from renal kidney failure and do have a donor with an incompatible kidney (compatible kidney also encouraged for better matches). In such situations the patient, donor pair is matched to another patient, donor pair having the same problem and are compatible to each other. Hospitals put up their patient-donor data. Using the biological data, compatibility scores(or weights) are generated and preferences are formed accordingly. Indian influence on weights, modify the compatibility scores generated and hence, the preferences. The pairs are then allocated using game theoretic matching algorithms for markets without money.
How News Evolves? Modeling News Text and Coverage using Graphs and Hawkes Process
Venue: arXiv cs.CL 2022
Year: 2022

Monitoring news content automatically is an important problem. The news content, unlike traditional text, has a temporal component. However, few works have explored the combination of natural language processing and dynamic system models. One reason is that it is challenging to mathematically model the nuances of natural language. In this paper, we discuss how we built a novel dataset of news articles collected over time. Then, we present a method of converting news text collected over time to a sequence of directed multi-graphs, which represent semantic triples (Subject -> Predicate} ->Object). We model the dynamics of specific topological changes in these graphs using a set of multivariate count series, which we fit the discrete-time Hawkes process. With our real-world data, we show that the multivariate time series contain both dynamic information of how many articles/words were published each day and semantic information of the content of the articles. This yields novel insights into how news events are covered. We show with the experiment that our approach can be used to infer from a sequence of news articles if the articles were published by major or entertainment news outlets.
3D Hierarchical Refinement and Augmentation for Unsupervised Learning of Depth and Pose from Monocular Video
Venue: arXiv cs.CV 2022
Year: 2022

Depth and ego-motion estimations are essential for the localization and navigation of autonomous robots and autonomous driving. Recent studies make it possible to learn the per-pixel depth and ego-motion from the unlabeled monocular video. A novel unsupervised training framework is proposed with 3D hierarchical refinement and augmentation using explicit 3D geometry. In this framework, the depth and pose estimations are hierarchically and mutually coupled to refine the estimated pose layer by layer. The intermediate view image is proposed and synthesized by warping the pixels in an image with the estimated depth and coarse pose. Then, the residual pose transformation can be estimated from the new view image and the image of the adjacent frame to refine the coarse pose. The iterative refinement is implemented in a differentiable manner in this paper, making the whole framework optimized uniformly. Meanwhile, a new image augmentation method is proposed for the pose estimation by synthesizing a new view image, which creatively augments the pose in 3D space but gets a new augmented 2D image. The experiments on KITTI demonstrate that our depth estimation achieves state-of-the-art performance and even surpasses recent approaches that utilize other auxiliary tasks. Our visual odometry outperforms all recent unsupervised monocular learning-based methods and achieves competitive performance to the geometry-based method, ORB-SLAM2 with back-end optimization.
Encouraging Disentangled and Convex Representation with Controllable Interpolation Regularization
Venue: arXiv cs.CV 2022
Year: 2022

We focus on controllable disentangled representation learning (C-Dis-RL), where users can control the partition of the disentangled latent space to factorize dataset attributes (concepts) for downstream tasks. Two general problems remain under-explored in current methods: (1) They lack comprehensive disentanglement constraints, especially missing the minimization of mutual information between different attributes across latent and observation domains. (2) They lack convexity constraints, which is important for meaningfully manipulating specific attributes for downstream tasks. To encourage both comprehensive C-Dis-RL and convexity simultaneously, we propose a simple yet efficient method: Controllable Interpolation Regularization (CIR), which creates a positive loop where disentanglement and convexity can help each other. Specifically, we conduct controlled interpolation in latent space during training, and we reuse the encoder to help form a 'perfect disentanglement' regularization. In that case, (a) disentanglement loss implicitly enlarges the potential understandable distribution to encourage convexity; (b) convexity can in turn improve robust and precise disentanglement. CIR is a general module and we merge CIR with three different algorithms: ELEGANT, I2I-Dis, and GZS-Net to show the compatibility and effectiveness. Qualitative and quantitative experiments show improvement in C-Dis-RL and latent convexity by CIR. This further improves downstream tasks: controllable image synthesis, cross-modality image translation, and zero-shot synthesis.
Simulation Intelligence: Towards a New Generation of Scientific Methods
Venue: arXiv cs.AI 2022
Year: 2022

The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simulation, and artificial intelligence. We call this merger simulation intelligence (SI), for short. We argue the motifs of simulation intelligence are interconnected and interdependent, much like the components within the layers of an operating system. Using this metaphor, we explore the nature of each layer of the simulation intelligence operating system stack (SI-stack) and the motifs therein: (1) Multi-physics and multi-scale modeling; (2) Surrogate modeling and emulation; (3) Simulation-based inference; (4) Causal modeling and inference; (5) Agent-based modeling; (6) Probabilistic programming; (7) Differentiable programming; (8) Open-ended optimization; (9) Machine programming. We believe coordinated efforts between motifs offers immense opportunity to accelerate scientific discovery, from solving inverse problems in synthetic biology and climate science, to directing nuclear energy experiments and predicting emergent behavior in socioeconomic settings. We elaborate on each layer of the SI-stack, detailing the state-of-art methods, presenting examples to highlight challenges and opportunities, and advocating for specific ways to advance the motifs and the synergies from their combinations. Advancing and integrating these technologies can enable a robust and efficient hypothesis-simulation-analysis type of scientific method, which we introduce with several use-cases for human-machine teaming and automated science.
A Family of Density-Scaled Filtered Complexes
Venue: arXiv cs.CG 2022
Year: 2022

We develop novel methods for using persistent homology to infer the homology of an unknown Riemannian manifold $(M, g)$ from a point cloud sampled from an arbitrary smooth probability density function. Standard distance-based filtered complexes, such as the Čech complex, often have trouble distinguishing noise from features that are simply small. We address this problem by defining a family of "density-scaled filtered complexes" that includes a density-scaled Čech complex and a density-scaled Vietoris--Rips complex. We show that the density-scaled Čech complex is homotopy-equivalent to $M$ for filtration values in an interval whose starting point converges to $0$ in probability as the number of points $N \to \infty$ and whose ending point approaches infinity as $N \to \infty$. By contrast, the standard Čech complex may only be homotopy-equivalent to $M$ for a very small range of filtration values. The density-scaled filtered complexes also have the property that they are invariant under conformal transformations, such as scaling. We implement a filtered complex $\widehat{DVR}$ that approximates the density-scaled Vietoris--Rips complex, and we empirically test the performance of our implementation. As examples, we use $\widehat{DVR}$ to identify clusters that have different densities, and we apply $\widehat{DVR}$ to a time-delay embedding of the Lorenz dynamical system. Our implementation is stable (under conditions that are almost surely satisfied) and designed to handle outliers in the point cloud that do not lie on $M$.
Associative Memories Using Complex-Valued Hopfield Networks Based on Spin-Torque Oscillator Arrays
Venue: arXiv cs.ET 2022
Year: 2022

Simulations of complex-valued Hopfield networks based on spin-torque oscillators can recover phase-encoded images. Sequences of memristor-augmented inverters provide tunable delay elements that implement complex weights by phase shifting the oscillatory output of the oscillators. Pseudo-inverse training suffices to store at least 12 images in a set of 192 oscillators, representing 16$\times$12 pixel images. The energy required to recover an image depends on the desired error level. For the oscillators and circuitry considered here, 5 % root mean square deviations from the ideal image require approximately 5 $μ$s and consume roughly 130 nJ. Simulations show that the network functions well when the resonant frequency of the oscillators can be tuned to have a fractional spread less than $10^{-3}$, depending on the strength of the feedback.