OpenRaiser/Intern-Atlas huggingface dataset:
Towards the Understanding of Receptivity and Affect in EMAs using Physiological based Machine Learning Method: Analysis of Receptivity and Affect
Venue: arXiv cs.HC 2023
Year: 2023

As mobile health (mHealth) studies become increasingly productive due to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. The reliance on subjective responses for health constructs poses challenges, especially in populations with lower receptivity rates. Researchers have proposed machine-learning approaches to optimize survey timing and delivery to address this. However, there are concerns regarding potential biases or unintended influences on participant responses. Our study delves into factors impacting receptivity to ecological momentary assessments (EMA) in a 10-day mHealth study, exploring physiological relationships indicative of receptivity and affect. Utilizing data from 45 participants with wearable devices measuring various biometrics, we employ unsupervised (k-means clustering) and supervised (Random Forest and Neural Networks) machine learning methods to infer affect during non-responses. Findings reveal that triggering EMAs based on a receptivity model reduces reported negative affect by over 3 points (0.29 standard deviations). The predicted affect during non-responses exhibits a bimodal distribution, suggesting more frequent initiation during states of higher positive emotions. The study underscores a clear relationship between affect and receptivity, impacting mHealth study efficacy, especially those using machine learning for EMA triggering. Therefore, we propose a smart trigger that promotes EMA receptivity without influencing affect during sampled time points as future work.
Preoperative Prognosis Assessment of Lumbar Spinal Surgery for Low Back Pain and Sciatica Patients based on Multimodalities and Multimodal Learning
Venue: arXiv cs.LG 2023
Year: 2023

Low back pain (LBP) and sciatica may require surgical therapy when they are symptomatic of severe pain. However, there is no effective measures to evaluate the surgical outcomes in advance. This work combined elements of Eastern medicine and machine learning, and developed a preoperative assessment tool to predict the prognosis of lumbar spinal surgery in LBP and sciatica patients. Standard operative assessments, traditional Chinese medicine body constitution assessments, planned surgical approach, and vowel pronunciation recordings were collected and stored in different modalities. Our work provides insights into leveraging modality combinations, multimodals, and fusion strategies. The interpretability of models and correlations between modalities were also inspected. Based on the recruited 105 patients, we found that combining standard operative assessments, body constitution assessments, and planned surgical approach achieved the best performance in 0.81 accuracy. Our approach is effective and can be widely applied in general practice due to simplicity and effective.
IRIS: Interpretable Rubric-Informed Segmentation for Action Quality Assessment
Venue: arXiv cs.AI 2023
Year: 2023

AI-driven Action Quality Assessment (AQA) of sports videos can mimic Olympic judges to help score performances as a second opinion or for training. However, these AI methods are uninterpretable and do not justify their scores, which is important for algorithmic accountability. Indeed, to account for their decisions, instead of scoring subjectively, sports judges use a consistent set of criteria - rubric - on multiple actions in each performance sequence. Therefore, we propose IRIS to perform Interpretable Rubric-Informed Segmentation on action sequences for AQA. We investigated IRIS for scoring videos of figure skating performance. IRIS predicts (1) action segments, (2) technical element score differences of each segment relative to base scores, (3) multiple program component scores, and (4) the summed final score. In a modeling study, we found that IRIS performs better than non-interpretable, state-of-the-art models. In a formative user study, practicing figure skaters agreed with the rubric-informed explanations, found them useful, and trusted AI judgments more. This work highlights the importance of using judgment rubrics to account for AI decisions.
A bit-parallel tabu search algorithm for finding E($s^2$)-optimal and minimax-optimal supersaturated designs
Venue: arXiv cs.DM 2023
Year: 2023

We prove the equivalence of two-symbol supersaturated designs (SSDs) with $N$ (even) rows, $m$ columns, $s_{\rm max} = 4t +i$, where $i\in\{0,2\}$, $t \in \mathbb{Z}^{\geq 0}$ and resolvable incomplete block designs (RIBDs) whose any two blocks intersect in at most $(N+4t+i)/4$ points. Using this equivalence, we formulate the search for two-symbol E($s^2$)-optimal and minimax-optimal SSDs with $s_{\max} \in \{2,4,6\}$ as a search for RIBDs whose blocks intersect accordingly. This allows developing a bit-parallel tabu search (TS) algorithm. The TS algorithm found E($s^2$)-optimal and minimax-optimal SSDs achieving the sharpest known E($s^2$) lower bound with $s_{\max} \in \{2,4,6\}$ of sizes $(N,m)=(16,25), (16,26), (16,27), (18,23),(18,24),(18,25),(18,26),(18,27),(18, 28),$ $(18,29),(20,21),(22,22),(22,23),(24,24)$, and $(24,25)$. In each of these cases no such SSD could previously be found.
Formally Verified Animation for RoboChart using Interaction Trees
Venue: arXiv cs.LO 2023
Year: 2023

RoboChart is a core notation in the RoboStar framework. It is a timed and probabilistic domain-specific and state machine-based language for robotics. RoboChart supports shared variables and communication across entities in its component model. It has formal denotational semantics given in CSP. The semantic technique of Interaction Trees (ITrees) represents behaviours of reactive and concurrent programs interacting with their environments. Recent mechanisation of ITrees, along with ITree-based CSP semantics and a Z mathematical toolkit in Isabelle/HOL, bring new applications of verification and animation for state-rich process languages, such as RoboChart. In this paper, we use ITrees to give RoboChart novel operational semantics, implement it in Isabelle, and use Isabelle's code generator to generate verified and executable animations. We illustrate our approach using an autonomous chemical detector model and a patrol robot model additionally exhibiting nondeterminism and using shared variables. With animation, we show two concrete scenarios for the chemical detector when the robot encounters different environmental inputs and three concrete scenarios for the patrol robot when its calibrated position is in different sections of a corridor. We also verify that the animated scenarios are truly trace refinements of the CSP denotational semantics of the RoboChart models using FDR, a refinement model checker for CSP. This ensures that our approach to resolving nondeterminism using CSP operators with priority is sound and correct.
Minimum $L_\infty$ Hausdorff Distance of Point Sets Under Translation: Generalizing Klee's Measure Problem
Venue: arXiv cs.CG 2023
Year: 2023

We present a (combinatorial) algorithm with running time close to $O(n^d)$ for computing the minimum directed $L_\infty$ Hausdorff distance between two sets of $n$ points under translations in any constant dimension $d$. This substantially improves the best previous time bound near $O(n^{5d/4})$ by Chew, Dor, Efrat, and Kedem from more than twenty years ago. Our solution is obtained by a new generalization of Chan's algorithm [FOCS'13] for Klee's measure problem. To complement this algorithmic result, we also prove a nearly matching conditional lower bound close to $Ω(n^d)$ for combinatorial algorithms, under the Combinatorial $k$-Clique Hypothesis.
Emergence and fragility of a research-creation (2000-2007)
Venue: arXiv cs.GR 2023
Year: 2023

My research-creation process coincides with the encounter with the ''digital paradigm'' and the attempt to incorporate it into the foundation of my scenic writing. I propose in this paper to give an account from a director point of view of how I realized my shows between 2000 and 2007 and which researches influenced the process. I will formulate some remarks on the fragilities that can arise in a ''technological laboratory of staging''.
Regret, Delete, (Do Not) Repeat: An Analysis of Self-Cleaning Practices on Twitter After the Outbreak of the COVID-19 Pandemic
Venue: arXiv cs.SI 2023
Year: 2023

During the outbreak of the COVID-19 pandemic, many people shared their symptoms across Online Social Networks (OSNs) like Twitter, hoping for others' advice or moral support. Prior studies have shown that those who disclose health-related information across OSNs often tend to regret it and delete their publications afterwards. Hence, deleted posts containing sensitive data can be seen as manifestations of online regrets. In this work, we present an analysis of deleted content on Twitter during the outbreak of the COVID-19 pandemic. For this, we collected more than 3.67 million tweets describing COVID-19 symptoms (e.g., fever, cough, and fatigue) posted between January and April 2020. We observed that around 24% of the tweets containing personal pronouns were deleted either by their authors or by the platform after one year. As a practical application of the resulting dataset, we explored its suitability for the automatic classification of regrettable content on Twitter.
Facial Affective Behavior Analysis Method for 5th ABAW Competition
Venue: arXiv cs.CV 2023
Year: 2023

Facial affective behavior analysis is important for human-computer interaction. 5th ABAW competition includes three challenges from Aff-Wild2 database. Three common facial affective analysis tasks are involved, i.e. valence-arousal estimation, expression classification, action unit recognition. For the three challenges, we construct three different models to solve the corresponding problems to improve the results, such as data unbalance and data noise. For the experiments of three challenges, we train the models on the provided training data and validate the models on the validation data.
Reliable Image Dehazing by NeRF
Venue: arXiv cs.CV 2023
Year: 2023

We present an image dehazing algorithm with high quality, wide application, and no data training or prior needed. We analyze the defects of the original dehazing model, and propose a new and reliable dehazing reconstruction and dehazing model based on the combination of optical scattering model and computer graphics lighting rendering model. Based on the new haze model and the images obtained by the cameras, we can reconstruct the three-dimensional space, accurately calculate the objects and haze in the space, and use the transparency relationship of haze to perform accurate haze removal. To obtain a 3D simulation dataset we used the Unreal 5 computer graphics rendering engine. In order to obtain real shot data in different scenes, we used fog generators, array cameras, mobile phones, underwater cameras and drones to obtain haze data. We use formula derivation, simulation data set and real shot data set result experimental results to prove the feasibility of the new method. Compared with various other methods, we are far ahead in terms of calculation indicators (4 dB higher quality average scene), color remains more natural, and the algorithm is more robust in different scenarios and best in the subjective perception.
Emotional Reaction Intensity Estimation Based on Multimodal Data
Venue: arXiv cs.CV 2023
Year: 2023

This paper introduces our method for the Emotional Reaction Intensity (ERI) Estimation Challenge, in CVPR 2023: 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW). Based on the multimodal data provided by the originazers, we extract acoustic and visual features with different pretrained models. The multimodal features are mixed together by Transformer Encoders with cross-modal attention mechnism. In this paper, 1. better features are extracted with the SOTA pretrained models. 2. Compared with the baseline, we improve the Pearson's Correlations Coefficient a lot. 3. We process the data with some special skills to enhance performance ability of our model.
Network-based Control of Epidemic via Flattening the Infection Curve: High-Clustered vs. Low-Clustered Social Networks
Venue: arXiv cs.SI 2023
Year: 2023

Recent studies in network science and control have shown a meaningful relationship between the epidemic processes (e.g., COVID-19 spread) and some network properties. This paper studies how such network properties, namely clustering coefficient and centrality measures (or node influence metrics), affect the spread of viruses and the growth of epidemics over scale-free networks. The results can be used to target individuals (the nodes in the network) to \textit{flatten the infection curve}. This so-called flattening of the infection curve is to reduce the health service costs and burden to the authorities/governments. Our Monte-Carlo simulation results show that clustered networks are, in general, easier to flatten the infection curve, i.e., with the same connectivity and the same number of isolated individuals they result in more flattened curves. Moreover, distance-based centrality measures, which target the nodes based on their average network distance to other nodes (and not the node degrees), are better choices for targeting individuals for isolation/vaccination.
User Selection for Simple Passive Beamforming in Multi-RIS-Aided Multi-User Communications
Venue: arXiv cs.IT 2023
Year: 2023

This paper focuses on multi-user downlink signal transmission in a wireless system aided by multiple reconfigurable intelligent surfaces (RISs). In such a multi-RIS, multi-user, multi-antenna scenario, determining a set of RIS phase shifts to maximize the sum throughput becomes intractable. Hence, we propose a novel scheme that can substantially simplify the optimization of passive beamforming. By opportunistically selecting a user with the best channel condition as the only active transmitter in the system, it degrades to single-user passive beamforming, where two methods, i.e., joint optimization based on the semidefinite relaxation approach and alternating optimization, are applicable. The superiority of the proposed scheme is demonstrated through Monte-Carlo simulations.
Wiki-based Communities of Interest: Demographics and Outliers
Venue: arXiv cs.SI 2023
Year: 2023

In this paper, we release data about demographic information and outliers of communities of interest. Identified from Wiki-based sources, mainly Wikidata, the data covers 7.5k communities, such as members of the White House Coronavirus Task Force, and 345k subjects, e.g., Deborah Birx. We describe the statistical inference methodology adopted to mine such data. We release subject-centric and group-centric datasets in JSON format, as well as a browsing interface. Finally, we forsee three areas this research can have an impact on: in social sciences research, it provides a resource for demographic analyses; in web-scale collaborative encyclopedias, it serves as an edit recommender to fill knowledge gaps; and in web search, it offers lists of salient statements about queried subjects for higher user engagement.
Reduction of rain-induced errors for wind speed estimation on SAR observations using convolutional neural networks
Venue: arXiv cs.CV 2023
Year: 2023

Synthetic Aperture Radar is known to be able to provide high-resolution estimates of surface wind speed. These estimates usually rely on a Geophysical Model Function (GMF) that has difficulties accounting for non-wind processes such as rain events. Convolutional neural network, on the other hand, have the capacity to use contextual information and have demonstrated their ability to delimit rainfall areas. By carefully building a large dataset of SAR observations from the Copernicus Sentinel-1 mission, collocated with both GMF and atmospheric model wind speeds as well as rainfall estimates, we were able to train a wind speed estimator with reduced errors under rain. Collocations with in-situ wind speed measurements from buoys show a root mean square error that is reduced by 27% (resp. 45%) under rainfall estimated at more than 1 mm/h (resp. 3 mm/h). These results demonstrate the capacity of deep learning models to correct rain-related errors in SAR products.
GridCLIP: One-Stage Object Detection by Grid-Level CLIP Representation Learning
Venue: arXiv cs.CV 2023
Year: 2023

A vision-language foundation model pretrained on very large-scale image-text paired data has the potential to provide generalizable knowledge representation for downstream visual recognition and detection tasks, especially on supplementing the undersampled categories in downstream model training. Recent studies utilizing CLIP for object detection have shown that a two-stage detector design typically outperforms a one-stage detector, while requiring more expensive training resources and longer inference time. In this work, we propose a one-stage detector GridCLIP that narrows its performance gap to those of two-stage detectors, with approximately 43 and 5 times faster than its two-stage counterpart (ViLD) in the training and test process respectively. GridCLIP learns grid-level representations to adapt to the intrinsic principle of one-stage detection learning by expanding the conventional CLIP image-text holistic mapping to a more fine-grained, grid-text alignment. This differs from the region-text mapping in two-stage detectors that apply CLIP directly by treating regions as images. Specifically, GridCLIP performs Grid-level Alignment to adapt the CLIP image-level representations to grid-level representations by aligning to CLIP category representations to learn the annotated (especially frequent) categories. To learn generalizable visual representations of broader categories, especially undersampled ones, we perform Image-level Alignment during training to propagate broad pre-learned categories in the CLIP image encoder from the image-level to the grid-level representations. Experiments show that the learned CLIP-based grid-level representations boost the performance of undersampled (infrequent and novel) categories, reaching comparable detection performance on the LVIS benchmark.
Thermal Feedback for Transparency in Human-Robot Interaction
Venue: arXiv cs.HC 2023
Year: 2023

Robots can support humans in tedious tasks, as well as provide social support. However, the decision-making and behavior of robots is not always clear to the human interaction partner. In this work, we discuss the opportunity of using thermal feedback as an additional modality to create transparent interactions. We then present scenarios where thermal feedback is incorporated into the interaction e.g. to unobtrusively communicate the behavior of the robot. We highlight the limitations and challenges of temperature-based feedback, which can be explored in future research.
Depth Super-Resolution from Explicit and Implicit High-Frequency Features
Venue: arXiv cs.CV 2023
Year: 2023

We propose a novel multi-stage depth super-resolution network, which progressively reconstructs high-resolution depth maps from explicit and implicit high-frequency features. The former are extracted by an efficient transformer processing both local and global contexts, while the latter are obtained by projecting color images into the frequency domain. Both are combined together with depth features by means of a fusion strategy within a multi-stage and multi-scale framework. Experiments on the main benchmarks, such as NYUv2, Middlebury, DIML and RGBDD, show that our approach outperforms existing methods by a large margin (~20% on NYUv2 and DIML against the contemporary work DADA, with 16x upsampling), establishing a new state-of-the-art in the guided depth super-resolution task.
Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?
Venue: arXiv cs.AI 2023
Year: 2023

We evaluated the capability of generative pre-trained transformers (GPT), to pass assessments in introductory and intermediate Python programming courses at the postsecondary level. Discussions of potential uses (e.g., exercise generation, code explanation) and misuses (e.g., cheating) of this emerging technology in programming education have intensified, but to date there has not been a rigorous analysis of the models' capabilities in the realistic context of a full-fledged programming course with diverse set of assessment instruments. We evaluated GPT on three Python courses that employ assessments ranging from simple multiple-choice questions (no code involved) to complex programming projects with code bases distributed into multiple files (599 exercises overall). Further, we studied if and how successfully GPT models leverage feedback provided by an auto-grader. We found that the current models are not capable of passing the full spectrum of assessments typically involved in a Python programming course (<70% on even entry-level modules). Yet, it is clear that a straightforward application of these easily accessible models could enable a learner to obtain a non-trivial portion of the overall available score (>55%) in introductory and intermediate courses alike. While the models exhibit remarkable capabilities, including correcting solutions based on auto-grader's feedback, some limitations exist (e.g., poor handling of exercises requiring complex chains of reasoning steps). These findings can be leveraged by instructors wishing to adapt their assessments so that GPT becomes a valuable assistant for a learner as opposed to an end-to-end solution.
Real-time elastic partial shape matching using a neural network-based adjoint method
Venue: arXiv cs.AI 2023
Year: 2023

Surface matching usually provides significant deformations that can lead to structural failure due to the lack of physical policy. In this context, partial surface matching of non-linear deformable bodies is crucial in engineering to govern structure deformations. In this article, we propose to formulate the registration problem as an optimal control problem using an artificial neural network where the unknown is the surface force distribution that applies to the object and the resulting deformation computed using a hyper-elastic model. The optimization problem is solved using an adjoint method where the hyper-elastic problem is solved using the feed-forward neural network and the adjoint problem is obtained through the backpropagation of the network. Our process improves the computation speed by multiple orders of magnitude while providing acceptable registration errors.
The NCI Imaging Data Commons as a platform for reproducible research in computational pathology
Venue: arXiv cs.CV 2023
Year: 2023

Background and Objectives: Reproducibility is a major challenge in developing machine learning (ML)-based solutions in computational pathology (CompPath). The NCI Imaging Data Commons (IDC) provides >120 cancer image collections according to the FAIR principles and is designed to be used with cloud ML services. Here, we explore its potential to facilitate reproducibility in CompPath research. Methods: Using the IDC, we implemented two experiments in which a representative ML-based method for classifying lung tumor tissue was trained and/or evaluated on different datasets. To assess reproducibility, the experiments were run multiple times with separate but identically configured instances of common ML services. Results: The AUC values of different runs of the same experiment were generally consistent. However, we observed small variations in AUC values of up to 0.045, indicating a practical limit to reproducibility. Conclusions: We conclude that the IDC facilitates approaching the reproducibility limit of CompPath research (i) by enabling researchers to reuse exactly the same datasets and (ii) by integrating with cloud ML services so that experiments can be run in identically configured computing environments.