OpenRaiser/Intern-Atlas huggingface dataset:
Exploring Runtime Decision Support for Trauma Resuscitation
Venue: arXiv cs.AI 2022
Year: 2022

AI-based recommender systems have been successfully applied in many domains (e.g., e-commerce, feeds ranking). Medical experts believe that incorporating such methods into a clinical decision support system may help reduce medical team errors and improve patient outcomes during treatment processes (e.g., trauma resuscitation, surgical processes). Limited research, however, has been done to develop automatic data-driven treatment decision support. We explored the feasibility of building a treatment recommender system to provide runtime next-minute activity predictions. The system uses patient context (e.g., demographics and vital signs) and process context (e.g., activities) to continuously predict activities that will be performed in the next minute. We evaluated our system on a pre-recorded dataset of trauma resuscitation and conducted an ablation study on different model variants. The best model achieved an average F1-score of 0.67 for 61 activity types. We include medical team feedback and discuss the future work.
Learning Optimal Solutions via an LSTM-Optimization Framework
Venue: arXiv cs.LG 2022
Year: 2022

In this study, we present a deep learning-optimization framework to tackle dynamic mixed-integer programs. Specifically, we develop a bidirectional Long Short Term Memory (LSTM) framework that can process information forward and backward in time to learn optimal solutions to sequential decision-making problems. We demonstrate our approach in predicting the optimal decisions for the single-item capacitated lot-sizing problem (CLSP), where a binary variable denotes whether to produce in a period or not. Due to the dynamic nature of the problem, the CLSP can be treated as a sequence labeling task where a recurrent neural network can capture the problem's temporal dynamics. Computational results show that our LSTM-Optimization (LSTM-Opt) framework significantly reduces the solution time of benchmark CLSP problems without much loss in feasibility and optimality. For example, the predictions at the 85\% level reduce the CPLEX solution time by a factor of 9 on average for over 240,000 test instances with an optimality gap of less than 0.05\% and 0.4\% infeasibility in the test set. Also, models trained using shorter planning horizons can successfully predict the optimal solution of the instances with longer planning horizons. For the hardest data set, the LSTM predictions at the 25\% level reduce the solution time of 70 CPU hours to less than 2 CPU minutes with an optimality gap of 0.8\% and without any infeasibility. The LSTM-Opt framework outperforms classical ML algorithms, such as the logistic regression and random forest, in terms of the solution quality, and exact approaches, such as the ($\ell$, S) and dynamic programming-based inequalities, with respect to the solution time improvement. Our machine learning approach could be beneficial in tackling sequential decision-making problems similar to CLSP, which need to be solved repetitively, frequently, and in a fast manner.
Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithm
Venue: arXiv cs.CV 2022
Year: 2022

While artificial intelligence (AI) holds promise for supporting healthcare providers and improving the accuracy of medical diagnoses, a lack of transparency in the composition of datasets exposes AI models to the possibility of unintentional and avoidable mistakes. In particular, public and private image datasets of dermatological conditions rarely include information on skin color. As a start towards increasing transparency, AI researchers have appropriated the use of the Fitzpatrick skin type (FST) from a measure of patient photosensitivity to a measure for estimating skin tone in algorithmic audits of computer vision applications including facial recognition and dermatology diagnosis. In order to understand the variability of estimated FST annotations on images, we compare several FST annotation methods on a diverse set of 460 images of skin conditions from both textbooks and online dermatology atlases. We find the inter-rater reliability between three board-certified dermatologists is comparable to the inter-rater reliability between the board-certified dermatologists and two crowdsourcing methods. In contrast, we find that the Individual Typology Angle converted to FST (ITA-FST) method produces annotations that are significantly less correlated with the experts' annotations than the experts' annotations are correlated with each other. These results demonstrate that algorithms based on ITA-FST are not reliable for annotating large-scale image datasets, but human-centered, crowd-based protocols can reliably add skin type transparency to dermatology datasets. Furthermore, we introduce the concept of dynamic consensus protocols with tunable parameters including expert review that increase the visibility of crowdwork and provide guidance for future crowdsourced annotations of large image datasets.
NeuralGrasps: Learning Implicit Representations for Grasps of Multiple Robotic Hands
Venue: arXiv cs.RO 2022
Year: 2022

We introduce a neural implicit representation for grasps of objects from multiple robotic hands. Different grasps across multiple robotic hands are encoded into a shared latent space. Each latent vector is learned to decode to the 3D shape of an object and the 3D shape of a robotic hand in a grasping pose in terms of the signed distance functions of the two 3D shapes. In addition, the distance metric in the latent space is learned to preserve the similarity between grasps across different robotic hands, where the similarity of grasps is defined according to contact regions of the robotic hands. This property enables our method to transfer grasps between different grippers including a human hand, and grasp transfer has the potential to share grasping skills between robots and enable robots to learn grasping skills from humans. Furthermore, the encoded signed distance functions of objects and grasps in our implicit representation can be used for 6D object pose estimation with grasping contact optimization from partial point clouds, which enables robotic grasping in the real world.
The "Collections as ML Data" Checklist for Machine Learning & Cultural Heritage
Venue: arXiv cs.LG 2022
Year: 2022

Within the cultural heritage sector, there has been a growing and concerted effort to consider a critical sociotechnical lens when applying machine learning techniques to digital collections. Though the cultural heritage community has collectively developed an emerging body of work detailing responsible operations for machine learning in libraries and other cultural heritage institutions at the organizational level, there remains a paucity of guidelines created specifically for practitioners embarking on machine learning projects. The manifold stakes and sensitivities involved in applying machine learning to cultural heritage underscore the importance of developing such guidelines. This paper contributes to this need by formulating a detailed checklist with guiding questions and practices that can be employed while developing a machine learning project that utilizes cultural heritage data. I call the resulting checklist the "Collections as ML Data" checklist, which, when completed, can be published with the deliverables of the project. By surveying existing projects, including my own project, Newspaper Navigator, I justify the "Collections as ML Data" checklist and demonstrate how the formulated guiding questions can be employed and operationalized.
Learning Invariant World State Representations with Predictive Coding
Venue: arXiv cs.LG 2022
Year: 2022

Self-supervised learning methods overcome the key bottleneck for building more capable AI: limited availability of labeled data. However, one of the drawbacks of self-supervised architectures is that the representations that they learn are implicit and it is hard to extract meaningful information about the encoded world states, such as 3D structure of the visual scene encoded in a depth map. Moreover, in the visual domain such representations only rarely undergo evaluations that may be critical for downstream tasks, such as vision for autonomous cars. Herein, we propose a framework for evaluating visual representations for illumination invariance in the context of depth perception. We develop a new predictive coding-based architecture and a hybrid fully-supervised/self-supervised learning method. We propose a novel architecture that extends the predictive coding approach: PRedictive Lateral bottom-Up and top-Down Encoder-decoder Network (PreludeNet), which explicitly learns to infer and predict depth from video frames. In PreludeNet, the encoder's stack of predictive coding layers is trained in a self-supervised manner, while the predictive decoder is trained in a supervised manner to infer or predict the depth. We evaluate the robustness of our model on a new synthetic dataset, in which lighting conditions (such as overall illumination, and effect of shadows) can be be parametrically adjusted while keeping all other aspects of the world constant. PreludeNet achieves both competitive depth inference performance and next frame prediction accuracy. We also show how this new network architecture, coupled with the hybrid fully-supervised/self-supervised learning method, achieves balance between the said performance and invariance to changes in lighting. The proposed framework for evaluating visual representations can be extended to diverse task domains and invariance tests.
Extending Logical Neural Networks using First-Order Theories
Venue: arXiv cs.AI 2022
Year: 2022

Logical Neural Networks (LNNs) are a type of architecture which combine a neural network's abilities to learn and systems of formal logic's abilities to perform symbolic reasoning. LLNs provide programmers the ability to implicitly modify the underlying structure of the neural network via logical formulae. In this paper, we take advantage of this abstraction to extend LNNs to support equality and function symbols via first-order theories. This extension improves the power of LNNs by significantly increasing the types of problems they can tackle. As a proof of concept, we add support for the first-order theory of equality to IBM's LNN library and demonstrate how the introduction of this allows the LNN library to now reason about expressions without needing to make the unique-names assumption.
An Overview on Designs and Applications of Context-Aware Automation Systems
Venue: arXiv cs.SE 2022
Year: 2022

Automation systems are increasingly being used in dynamic and various operating conditions. With higher flexibility demands, they need to promptly respond to surrounding dynamic changes by adapting their operation. Context information collected during runtime can be useful to enhance the system's adaptability. Context-aware systems represent a design paradigm for modeling and applying context in various applications such as decision-making. In order to address context for automation systems, a state of the art assessment of existing approaches is necessary. Thus, the objective of this work is to provide an overview on the design and applications of context and context models for automation systems. A systematic literature review has been conducted, the results of which are represented as a knowledge graph.
Team Learning as a Lens for Designing Human-AI Co-Creative Systems
Venue: arXiv cs.HC 2022
Year: 2022

Generative, ML-driven interactive systems have the potential to change how people interact with computers in creative processes - turning tools into co-creators. However, it is still unclear how we might achieve effective human-AI collaboration in open-ended task domains. There are several known challenges around communication in the interaction with ML-driven systems. An overlooked aspect in the design of co-creative systems is how users can be better supported in learning to collaborate with such systems. Here we reframe human-AI collaboration as a learning problem: Inspired by research on team learning, we hypothesize that similar learning strategies that apply to human-human teams might also increase the collaboration effectiveness and quality of humans working with co-creative generative systems. In this position paper, we aim to promote team learning as a lens for designing more effective co-creative human-AI collaboration and emphasize collaboration process quality as a goal for co-creative systems. Furthermore, we outline a preliminary schematic framework for embedding team learning support in co-creative AI systems. We conclude by proposing a research agenda and posing open questions for further study on supporting people in learning to collaborate with generative AI systems.
Machine Learning based Interference Whitening in 5G NR MIMO Receiver
Venue: arXiv cs.IT 2022
Year: 2022

We address the problem of computing the interference-plus-noise covariance matrix from a sparsely located demodulation reference signal (DMRS) for spatial domain interference whitening (IW). The IW procedure is critical at the user equipment (UE) to mitigate the co-channel interference in 5G new radio (NR) systems. A supervised learning based algorithm is proposed to compute the covariance matrix with goals of minimizing both the block-error rate (BLER) and the whitening complexity. A single neural network is trained to select an IW option for covariance computation in various interference scenarios consisting of different interference occupancy, signal-to-interference ratio, signal-to-noise ratio, modulation order, coding rate, etc. In interference-dominant scenarios, the proposed algorithm computes the covariance matrix using DMRS in one resource block (RB) due to the frequency selectivity of the interference channel. On the other hand, in noise-dominant scenarios, the covariance matrix is computed from DMRS in entire signal bandwidth. Further, the proposed algorithm approximates the covariance matrix into a diagonal matrix when the spatial correlation of interference-plus-noise is low. This approximation reduces the complexity of whitening from $\mathcal{O}(N^3)$ to $\mathcal{O}(N)$ where $N$ is the number of receiver antennas. Results show that the selection algorithm can minimize the BLER under both trained as well as untrained interference scenarios.
Single-image Defocus Deblurring by Integration of Defocus Map Prediction Tracing the Inverse Problem Computation
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we consider the problem in defocus image deblurring. Previous classical methods follow two-steps approaches, i.e., first defocus map estimation and then the non-blind deblurring. In the era of deep learning, some researchers have tried to address these two problems by CNN. However, the simple concatenation of defocus map, which represents the blur level, leads to suboptimal performance. Considering the spatial variant property of the defocus blur and the blur level indicated in the defocus map, we employ the defocus map as conditional guidance to adjust the features from the input blurring images instead of simple concatenation. Then we propose a simple but effective network with spatial modulation based on the defocus map. To achieve this, we design a network consisting of three sub-networks, including the defocus map estimation network, a condition network that encodes the defocus map into condition features, and the defocus deblurring network that performs spatially dynamic modulation based on the condition features. Moreover, the spatially dynamic modulation is based on an affine transform function to adjust the features from the input blurry images. Experimental results show that our method can achieve better quantitative and qualitative evaluation performance than the existing state-of-the-art methods on the commonly used public test datasets.
Multi-Label Learning to Rank through Multi-Objective Optimization
Venue: arXiv cs.IR 2022
Year: 2022

Learning to Rank (LTR) technique is ubiquitous in the Information Retrieval system nowadays, especially in the Search Ranking application. The query-item relevance labels typically used to train the ranking model are often noisy measurements of human behavior, e.g., product rating for product search. The coarse measurements make the ground truth ranking non-unique with respect to a single relevance criterion. To resolve ambiguity, it is desirable to train a model using many relevance criteria, giving rise to Multi-Label LTR (MLLTR). Moreover, it formulates multiple goals that may be conflicting yet important to optimize for simultaneously, e.g., in product search, a ranking model can be trained based on product quality and purchase likelihood to increase revenue. In this research, we leverage the Multi-Objective Optimization (MOO) aspect of the MLLTR problem and employ recently developed MOO algorithms to solve it. Specifically, we propose a general framework where the information from labels can be combined in a variety of ways to meaningfully characterize the trade-off among the goals. Our framework allows for any gradient based MOO algorithm to be used for solving the MLLTR problem. We test the proposed framework on two publicly available LTR datasets and one e-commerce dataset to show its efficacy.
Centralised Connectivity-Preserving Transformations by Rotation: 3 Musketeers for all Orthogonal Convex Shapes
Venue: arXiv cs.DS 2022
Year: 2022

We study a model of programmable matter systems consisting of $n$ devices lying on a 2-dimensional square grid, which are able to perform the minimal mechanical operation of rotating around each other. The goal is to transform an initial shape A into a target shape B. We are interested in characterising the class of shapes which can be transformed into each other in such a scenario, under the additional constraint of maintaining global connectivity at all times. This was one of the main problems left open by $[$Michail et al., JCSS'19$]$. Note that the considered question is about structural feasibility of transformations, which we exclusively deal with via centralised constructive proofs. Distributed solutions are left for future work and form an interesting research direction. Past work made some progress for the special class of nice shapes. We here consider the class of orthogonal convex shapes, where for any two nodes $u, v$ in a horizontal or vertical line on the grid, there is no empty cell between $u$ and $v$. We develop a generic centralised transformation and prove that, for any pair $A$, $B$ of colour-consistent orthogonal convex shapes, it can transform $A$ into $B$. In light of the existence of blocked shapes in the considered class, we use a minimal 3-node seed to trigger the transformation. The running time of our transformation is an optimal $O(n^2)$ sequential moves, where $n=|A|=|B|$. We leave as an open problem the existence of a universal connectivity-preserving transformation with a small seed. Our belief is that the techniques developed in this paper might prove useful to answer this.
Pre-training helps Bayesian optimization too
Venue: arXiv cs.LG 2022
Year: 2022

Bayesian optimization (BO) has become a popular strategy for global optimization of many expensive real-world functions. Contrary to a common belief that BO is suited to optimizing black-box functions, it actually requires domain knowledge on characteristics of those functions to deploy BO successfully. Such domain knowledge often manifests in Gaussian process priors that specify initial beliefs on functions. However, even with expert knowledge, it is not an easy task to select a prior. This is especially true for hyperparameter tuning problems on complex machine learning models, where landscapes of tuning objectives are often difficult to comprehend. We seek an alternative practice for setting these functional priors. In particular, we consider the scenario where we have data from similar functions that allow us to pre-train a tighter distribution a priori. To verify our approach in realistic model training setups, we collected a large multi-task hyperparameter tuning dataset by training tens of thousands of configurations of near-state-of-the-art models on popular image and text datasets, as well as a protein sequence dataset. Our results show that on average, our method is able to locate good hyperparameters at least 3 times more efficiently than the best competing methods.
Batch Evaluation Metrics in Information Retrieval: Measures, Scales, and Meaning
Venue: arXiv cs.IR 2022
Year: 2022

A sequence of recent papers has considered the role of measurement scales in information retrieval (IR) experimentation, and presented the argument that (only) uniform-step interval scales should be used, and hence that well-known metrics such as reciprocal rank, expected reciprocal rank, normalized discounted cumulative gain, and average precision, should be either discarded as measurement tools, or adapted so that their metric values lie at uniformly-spaced points on the number line. These papers paint a rather bleak picture of past decades of IR evaluation, at odds with the community's overall emphasis on practical experimentation and measurable improvement. Our purpose in this work is to challenge that position. In particular, we argue that mappings from categorical and ordinal data to sets of points on the number line are valid provided there is an external reason for each target point to have been selected. We first consider the general role of measurement scales, and of categorical, ordinal, interval, ratio, and absolute data collections. In connection with the first two of those categories we also provide examples of the knowledge that is captured and represented by numeric mappings to the real number line. Focusing then on information retrieval, we argue that document rankings are categorical data, and that the role of an effectiveness metric is to provide a single value that represents the usefulness to a user or population of users of any given ranking, with usefulness able to be represented as a continuous variable on a ratio scale. That is, we argue that current IR metrics are well-founded, and, moreover, that those metrics are more meaningful in their current form than in the proposed "intervalized" versions.
Energy-Efficient Communication Networks via Multiple Aerial Reconfigurable Intelligent Surfaces: DRL and Optimization Approach
Venue: arXiv cs.NI 2022
Year: 2022

In the realm of wireless communications in 5G, 6G and beyond, deploying unmanned aerial vehicle (UAV) has been an innovative approach to extend the coverage area due to its easy deployment. Moreover, reconfigurable intelligent surface (RIS) has also emerged as a new paradigm with the goals of enhancing the average sum-rate as well as energy efficiency. By combining these attractive features, an energy-efficient RIS-mounted multiple UAVs (aerial RISs: ARISs) assisted downlink communication system is studied. Due to the obstruction, user equipments (UEs) can have a poor line of sight to communicate with the base station (BS). To solve this, multiple ARISs are implemented to assist the communication between the BS and UEs. Then, the joint optimization problem of deployment of ARIS, ARIS reflective elements on/off states, phase shift, and power control of the multiple ARISs-assisted communication system is formulated. The problem is challenging to solve since it is mixed-integer, non-convex, and NP-hard. To overcome this, it is decomposed into three sub-problems. Afterwards, successive convex approximation (SCA), actor-critic proximal policy optimization (AC-PPO), and whale optimization algorithm (WOA) are employed to solve these sub-problems alternatively. Finally, extensive simulation results have been generated to illustrate the efficacy of our proposed algorithms.