OpenRaiser/Intern-Atlas huggingface dataset:
Anti-aliasing Predictive Coding Network for Future Video Frame Prediction
Venue: arXiv cs.CV 2023
Year: 2023

We introduce here a predictive coding based model that aims to generate accurate and sharp future frames. Inspired by the predictive coding hypothesis and related works, the total model is updated through a combination of bottom-up and top-down information flows, which can enhance the interaction between different network levels. Most importantly, We propose and improve several artifacts to ensure that the neural networks generate clear and natural frames. Different inputs are no longer simply concatenated or added, they are calculated in a modulated manner to avoid being roughly fused. The downsampling and upsampling modules have been redesigned to ensure that the network can more easily construct images from Fourier features of low-frequency inputs. Additionally, the training strategies are also explored and improved to generate believable results and alleviate inconsistency between the input predicted frames and ground truth. Our proposals achieve results that better balance pixel accuracy and visualization effect.
Trends in Explainable AI (XAI) Literature
Venue: arXiv cs.AI 2023
Year: 2023

The XAI literature is decentralized, both in terminology and in publication venues, but recent years saw the community converge around keywords that make it possible to more reliably discover papers automatically. We use keyword search using the SemanticScholar API and manual curation to collect a well-formatted and reasonably comprehensive set of 5199 XAI papers, available at https://github.com/alonjacovi/XAI-Scholar . We use this collection to clarify and visualize trends about the size and scope of the literature, citation trends, cross-field trends, and collaboration trends. Overall, XAI is becoming increasingly multidisciplinary, with relative growth in papers belonging to increasingly diverse (non-CS) scientific fields, increasing cross-field collaborative authorship, increasing cross-field citation activity. The collection can additionally be used as a paper discovery engine, by retrieving XAI literature which is cited according to specific constraints (for example, papers that are influential outside of their field, or influential to non-XAI research).
LVRNet: Lightweight Image Restoration for Aerial Images under Low Visibility
Venue: arXiv cs.CV 2023
Year: 2023

Learning to recover clear images from images having a combination of degrading factors is a challenging task. That being said, autonomous surveillance in low visibility conditions caused by high pollution/smoke, poor air quality index, low light, atmospheric scattering, and haze during a blizzard becomes even more important to prevent accidents. It is thus crucial to form a solution that can result in a high-quality image and is efficient enough to be deployed for everyday use. However, the lack of proper datasets available to tackle this task limits the performance of the previous methods proposed. To this end, we generate the LowVis-AFO dataset, containing 3647 paired dark-hazy and clear images. We also introduce a lightweight deep learning model called Low-Visibility Restoration Network (LVRNet). It outperforms previous image restoration methods with low latency, achieving a PSNR value of 25.744 and an SSIM of 0.905, making our approach scalable and ready for practical use. The code and data can be found at https://github.com/Achleshwar/LVRNet.
Towards Single Camera Human 3D-Kinematics
Venue: arXiv cs.CV 2023
Year: 2023

Markerless estimation of 3D Kinematics has the great potential to clinically diagnose and monitor movement disorders without referrals to expensive motion capture labs; however, current approaches are limited by performing multiple de-coupled steps to estimate the kinematics of a person from videos. Most current techniques work in a multi-step approach by first detecting the pose of the body and then fitting a musculoskeletal model to the data for accurate kinematic estimation. Errors in training data of the pose detection algorithms, model scaling, as well the requirement of multiple cameras limit the use of these techniques in a clinical setting. Our goal is to pave the way toward fast, easily applicable and accurate 3D kinematic estimation \xdeleted{in a clinical setting}. To this end, we propose a novel approach for direct 3D human kinematic estimation D3KE from videos using deep neural networks. Our experiments demonstrate that the proposed end-to-end training is robust and outperforms 2D and 3D markerless motion capture based kinematic estimation pipelines in terms of joint angles error by a large margin (35\% from 5.44 to 3.54 degrees). We show that D3KE is superior to the multi-step approach and can run at video framerate speeds. This technology shows the potential for clinical analysis from mobile devices in the future.
Structuring ontologies in a context of collaborative system modelling
Venue: arXiv cs.AI 2023
Year: 2023

Prospective studies require discussing and collaborating with the stakeholders to create scenarios of the possible evolution of the studied value-chain. However, stakeholders don't always use the same words when referring to one idea. Constructing an ontology and homogenizing vocabularies is thus crucial to identify key variables which serve in the construction of the needed scenarios. Nevertheless, it is a very complex and timeconsuming task. In this paper we present the method we used to manually build ontologies adapted to the needs of two complementary system-analysis models (namely the "Godet" and the "MyChoice" models), starting from interviews of the agri-food system's stakeholders.
RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised Medical Image Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Medical image segmentation methods are generally designed as fully-supervised to guarantee model performance, which require a significant amount of expert annotated samples that are high-cost and laborious. Semi-supervised image segmentation can alleviate the problem by utilizing a large number of unlabeled images along with limited labeled images. However, learning a robust representation from numerous unlabeled images remains challenging due to potential noise in pseudo labels and insufficient class separability in feature space, which undermines the performance of current semi-supervised segmentation approaches. To address the issues above, we propose a novel semi-supervised segmentation method named as Rectified Contrastive Pseudo Supervision (RCPS), which combines a rectified pseudo supervision and voxel-level contrastive learning to improve the effectiveness of semi-supervised segmentation. Particularly, we design a novel rectification strategy for the pseudo supervision method based on uncertainty estimation and consistency regularization to reduce the noise influence in pseudo labels. Furthermore, we introduce a bidirectional voxel contrastive loss to the network to ensure intra-class consistency and inter-class contrast in feature space, which increases class separability in the segmentation. The proposed RCPS segmentation method has been validated on two public datasets and an in-house clinical dataset. Experimental results reveal that the proposed method yields better segmentation performance compared with the state-of-the-art methods in semi-supervised medical image segmentation. The source code is available at https://github.com/hsiangyuzhao/RCPS.
Self-Training Guided Disentangled Adaptation for Cross-Domain Remote Sensing Image Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Deep convolutional neural networks (DCNNs) based remote sensing (RS) image semantic segmentation technology has achieved great success used in many real-world applications such as geographic element analysis. However, strong dependency on annotated data of specific scene makes it hard for DCNNs to fit different RS scenes. To solve this problem, recent works gradually focus on cross-domain RS image semantic segmentation task. In this task, different ground sampling distance, remote sensing sensor variation and different geographical landscapes are three main factors causing dramatic domain shift between source and target images. To decrease the negative influence of domain shift, we propose a self-training guided disentangled adaptation network (ST-DASegNet). We first propose source student backbone and target student backbone to respectively extract the source-style and target-style feature for both source and target images. Towards the intermediate output feature maps of each backbone, we adopt adversarial learning for alignment. Then, we propose a domain disentangled module to extract the universal feature and purify the distinct feature of source-style and target-style features. Finally, these two features are fused and served as input of source student decoder and target student decoder to generate final predictions. Based on our proposed domain disentangled module, we further propose exponential moving average (EMA) based cross-domain separated self-training mechanism to ease the instability and disadvantageous effect during adversarial optimization. Extensive experiments and analysis on benchmark RS datasets show that ST-DASegNet outperforms previous methods on cross-domain RS image semantic segmentation task and achieves state-of-the-art (SOTA) results. Our code is available at https://github.com/cv516Buaa/ST-DASegNet.
An RTL Implementation of the Data Encryption Standard (DES)
Venue: arXiv cs.CR 2023
Year: 2023

Data Encryption Standard (DES) is based on the Feistel block cipher, developed in 1971 by IBM cryptography researcher Horst Feistel. DES uses 16 rounds of the Feistel structure. But with the changes in recent years, the internet is starting to be used more to connect devices to each other. These devices can range from powerful computing devices, such as desktop computers and tablets, to resource constrained devices, When it comes to these constrained devices, using a different key for each round cryptography algorithms fail to provide necessary security and performance.
Using the profile of publishers to predict barriers across news articles
Venue: arXiv cs.AI 2023
Year: 2023

Detection of news propagation barriers, being economical, cultural, political, time zonal, or geographical, is still an open research issue. We present an approach to barrier detection in news spreading by utilizing Wikipedia-concepts and metadata associated with each barrier. Solving this problem can not only convey the information about the coverage of an event but it can also show whether an event has been able to cross a specific barrier or not. Experimental results on IPoNews dataset (dataset for information spreading over the news) reveals that simple classification models are able to detect barriers with high accuracy. We believe that our approach can serve to provide useful insights which pave the way for the future development of a system for predicting information spreading barriers over the news.
From Ember to Blaze: Swift Interactive Video Adaptation via Meta-Reinforcement Learning
Venue: arXiv cs.MM 2023
Year: 2023

Maximizing quality of experience (QoE) for interactive video streaming has been a long-standing challenge, as its delay-sensitive nature makes it more vulnerable to bandwidth fluctuations. While reinforcement learning (RL) has demonstrated great potential, existing works are either limited by fixed models or require enormous data/time for online adaptation, which struggle to fit time-varying and diverse network states. Driven by these practical concerns, we perform large-scale measurements on WeChat for Business's interactive video service to study real-world network fluctuations. Surprisingly, our analysis shows that, compared to time-varying network metrics, network sequences exhibit noticeable short-term continuity, sufficient for few-shot learning requirements. We thus propose Fiammetta, the first meta-RL-based bitrate adaptation algorithm for interactive video streaming. Building on the short-term continuity, Fiammetta accumulates learning experiences through offline meta-training and enables fast online adaptation to changing network states through a few gradient updates. Moreover, Fiammetta innovatively incorporates a probing mechanism for real-time monitoring of network states, and proposes an adaptive meta-testing mechanism for seamless adaptation. We implement Fiammetta on a testbed whose end-to-end network follows the real-world WeChat for Business traces. The results show that Fiammetta outperforms prior algorithms significantly, improving video bitrate by 3.6%-16.2% without increasing stalling rate.
The 2022 n2c2/UW Shared Task on Extracting Social Determinants of Health
Venue: arXiv cs.CL 2023
Year: 2023

Objective: The n2c2/UW SDOH Challenge explores the extraction of social determinant of health (SDOH) information from clinical notes. The objectives include the advancement of natural language processing (NLP) information extraction techniques for SDOH and clinical information more broadly. This paper presents the shared task, data, participating teams, performance results, and considerations for future work. Materials and Methods: The task used the Social History Annotated Corpus (SHAC), which consists of clinical text with detailed event-based annotations for SDOH events such as alcohol, drug, tobacco, employment, and living situation. Each SDOH event is characterized through attributes related to status, extent, and temporality. The task includes three subtasks related to information extraction (Subtask A), generalizability (Subtask B), and learning transfer (Subtask C). In addressing this task, participants utilized a range of techniques, including rules, knowledge bases, n-grams, word embeddings, and pretrained language models (LM). Results: A total of 15 teams participated, and the top teams utilized pretrained deep learning LM. The top team across all subtasks used a sequence-to-sequence approach achieving 0.901 F1 for Subtask A, 0.774 F1 Subtask B, and 0.889 F1 for Subtask C. Conclusions: Similar to many NLP tasks and domains, pretrained LM yielded the best performance, including generalizability and learning transfer. An error analysis indicates extraction performance varies by SDOH, with lower performance achieved for conditions, like substance use and homelessness, that increase health risks (risk factors) and higher performance achieved for conditions, like substance abstinence and living with family, that reduce health risks (protective factors).
A Framework for the Evaluation of Network Reliability Under Periodic Demand
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we study network reliability in relation to a periodic time-dependent utility function that reflects the system's functional performance. When an anomaly occurs, the system incurs a loss of utility that depends on the anomaly's timing and duration. We analyze the long-term average utility loss by considering exponential anomalies' inter-arrival times and general distributions of maintenance duration. We show that the expected utility loss converges in probability to a simple form. We then extend our convergence results to more general distributions of anomalies' inter-arrival times and to particular families of non-periodic utility functions. To validate our results, we use data gathered from a cellular network consisting of 660 base stations and serving over 20k users. We demonstrate the quasi-periodic nature of users' traffic and the exponential distribution of the anomalies' inter-arrival times, allowing us to apply our results and provide reliability scores for the network. We also discuss the convergence speed of the long-term average utility loss, the interplay between the different network's parameters, and the impact of non-stationarity on our convergence results.
Differentially Private Continual Releases of Streaming Frequency Moment Estimations
Venue: arXiv cs.DS 2023
Year: 2023

The streaming model of computation is a popular approach for working with large-scale data. In this setting, there is a stream of items and the goal is to compute the desired quantities (usually data statistics) while making a single pass through the stream and using as little space as possible. Motivated by the importance of data privacy, we develop differentially private streaming algorithms under the continual release setting, where the union of outputs of the algorithm at every timestamp must be differentially private. Specifically, we study the fundamental $\ell_p$ $(p\in [0,+\infty))$ frequency moment estimation problem under this setting, and give an $\varepsilon$-DP algorithm that achieves $(1+η)$-relative approximation $(\forall η\in(0,1))$ with $\mathrm{poly}\log(Tn)$ additive error and uses $\mathrm{poly}\log(Tn)\cdot \max(1, n^{1-2/p})$ space, where $T$ is the length of the stream and $n$ is the size of the universe of elements. Our space is near optimal up to poly-logarithmic factors even in the non-private setting. To obtain our results, we first reduce several primitives under the differentially private continual release model, such as counting distinct elements, heavy hitters and counting low frequency elements, to the simpler, counting/summing problems in the same setting. Based on these primitives, we develop a differentially private continual release level set estimation approach to address the $\ell_p$ frequency moment estimation problem. We also provide a simple extension of our results to the harder sliding window model, where the statistics must be maintained over the past $W$ data items.
Co-manipulation of soft-materials estimating deformation from depth images
Venue: arXiv cs.RO 2023
Year: 2023

Human-robot co-manipulation of soft materials, such as fabrics, composites, and sheets of paper/cardboard, is a challenging operation that presents several relevant industrial applications. Estimating the deformation state of the co-manipulated material is one of the main challenges. Viable methods provide the indirect measure by calculating the human-robot relative distance. In this paper, we develop a data-driven model to estimate the deformation state of the material from a depth image through a Convolutional Neural Network (CNN). First, we define the deformation state of the material as the relative roto-translation from the current robot pose and a human grasping position. The model estimates the current deformation state through a Convolutional Neural Network, specifically a DenseNet-121 pretrained on ImageNet.The delta between the current and the desired deformation state is fed to the robot controller that outputs twist commands. The paper describes the developed approach to acquire, preprocess the dataset and train the model. The model is compared with the current state-of-the-art method based on a skeletal tracker from cameras. Results show that our approach achieves better performances and avoids the various drawbacks caused by using a skeletal tracker.Finally, we also studied the model performance according to different architectures and dataset dimensions to minimize the time required for dataset acquisition
Linear Computation Coding: Exponential Search and Reduced-State Algorithms
Venue: arXiv cs.IT 2023
Year: 2023

Linear computation coding is concerned with the compression of multidimensional linear functions, i.e. with reducing the computational effort of multiplying an arbitrary vector to an arbitrary, but known, constant matrix. This paper advances over the state-of-the art, that is based on a discrete matching pursuit (DMP) algorithm, by a step-wise optimal search. Offering significant performance gains over DMP, it is however computationally infeasible for large matrices and high accuracy. Therefore, a reduced-state algorithm is introduced that offers performance superior to DMP, while still being computationally feasible even for large matrices. Depending on the matrix size, the performance gain over DMP is on the order of at least 10%.
Reworking geometric morphometrics into a methodology of transformation grids
Venue: arXiv cs.CV 2023
Year: 2023

Today's typical application of geometric morphometrics to a quantitative comparison of organismal anatomies begins by standardizing samples of homologously labelled point configurations for location, orientation, and scale, and then renders the ensuing comparisons graphically by thin-plate spline as applied to group averages, principal components, regression predictions, or canonical variates. The scale-standardization step has recently come under criticism as inappropriate, at least for growth studies. This essay argues for a similar rethinking of the centering and rotation, and then the replacement of the thin-plate spline interpolant of the resulting configurations by a different strategy that leaves unexplained residuals at every landmark individually in order to simplify the interpretation of the displayed grid as a whole, the "transformation grid" that has been highlighted as the true underlying topic ever since D'Arcy Thompson's exposition of 1917. For analyses of comparisons involving gradients at large geometric scale, this paper argues for replacement of all the Procrustes conventions by a version of my two-point registration of 1986 (originally Francis Galton's of 1907). The choice of the two points interacts with another non-Procrustes concern, interpretability of the grid lines of a coordinate system deformed according to a fitted polynomial trend rather than an interpolating thin-plate spline. The paper works two examples using previously published cranial data; there result new findings pertinent to the interpretation of both of these classic data sets. A concluding discussion suggests that the current toolkit of geometric morphometrics, centered on Procrustes shape coordinates and thin-plate splines, is too restricted to suit many of the interpretive purposes of evolutionary and developmental biology.
Electromagnetic-Compliant Channel Modeling and Performance Evaluation for Holographic MIMO
Venue: arXiv cs.IT 2023
Year: 2023

Recently, the concept of holographic multiple-input multiple-output (MIMO) is emerging as one of the promising technologies beyond massive MIMO. Many challenges need to be addressed to bring this novel idea into practice, including electromagnetic (EM)-compliant channel modeling and accurate performance evaluation. In this paper, an EM-compliant channel model is proposed for the holographic MIMO systems, which is able to model both the characteristics of the propagation channel and the non-ideal factors caused by mutual coupling at the transceivers, including the antenna pattern distortion and the decrease of antenna efficiency. Based on the proposed channel model, a more realistic performance evaluation is conducted to show the performance of the holographic MIMO system in both the single-user and the multi-user scenarios. Key challenges and future research directions are further provided based on the theoretical analyses and numerical results.