Miracle-dz/UniFAR-FTUs huggingface dataset:
The Limits of Computation in Solving Equity Trade-Offs in Machine Learning and Justice System Risk Assessment

This paper explores how different ideas of racial equity in machine learning, in justice settings in particular, can present trade-offs that are difficult to solve computationally. Machine learning is often used in justice settings to create risk assessments, which are used to determine interventions, resources, and punitive actions. Overall aspects and performance of these machine learningbased tools, such as distributions of scores, outcome rates by levels, and the frequency of false positives and true positives, can be problematic when examined by racial group. Models that produce different distributions of scores or produce a different relationship between level and outcome are problematic when those scores and levels are directly linked to the restriction of individual liberty and to the broader context of racial inequity. While computation can help highlight these aspects, data and computation are unlikely to solve them. This paper explores where values and mission might have to fill the spaces computation leaves.
Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration

Outcome prediction from clinical text can prevent doctors from overlooking possible risks and help hospitals to plan capacities. We simulate patients at admission time, when decision support can be especially valuable, and contribute a novel *admission to discharge* task with four common outcome prediction targets: Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction. The ideal system should infer outcomes based on symptoms, pre-conditions and risk factors of a patient. We evaluate the effectiveness of language models to handle this scenario and propose *clinical outcome pre-training* to integrate knowledge about patient outcomes from multiple public sources. We further present a simple method to incorporate ICD code hierarchy into the models. We show that our approach improves performance on the outcome tasks against several baselines. A detailed analysis reveals further strengths of the model, including transferability, but also weaknesses such as handling of vital values and inconsistencies in the underlying data.
Markovian approximation for Pauli Fierz operators

The purpose of this article is to derive a Markovian approximation of the reduced time dynamics of observables for the Pauli-Fierz Hamiltonian with a precise control of the error terms. In that aim, we define a Lindblad operator associated to the corresponding quantum master equation. In a particular case, this allows to study the transition probability matrix.
SkillBot: Identifying Risky Content for Children in Alexa Skills

Many households include children who use voice personal assistants (VPA) such as Amazon Alexa. Children benefit from the rich functionalities of VPAs and third-party apps but are also exposed to new risks in the VPA ecosystem. In this article, we first investigate “risky” child-directed voice apps that contain inappropriate content or ask for personal information through voice interactions. We build SkillBot—a natural language processing-based system to automatically interact with VPA apps and analyze the resulting conversations. We find 28 risky child-directed apps and maintain a growing dataset of 31,966 non-overlapping app behaviors collected from 3,434 Alexa apps. Our findings suggest that although child-directed VPA apps are subject to stricter policy requirements and more intensive vetting, children remain vulnerable to inappropriate content and privacy violations. We then conduct a user study showing that parents are concerned about the identified risky apps. Many parents do not believe that these apps are available and designed for families/kids, although these apps are actually published in Amazon’s “Kids” product category. We also find that parents often neglect basic precautions, such as enabling parental controls on Alexa devices. Finally, we identify a novel risk in the VPA ecosystem: confounding utterances or voice commands shared by multiple apps that may cause a user to interact with a different app than intended. We identify 4,487 confounding utterances, including 581 shared by child-directed and non-child-directed apps. We find that 27% of these confounding utterances prioritize invoking a non-child-directed app over a child-directed app. This indicates that children are at real risk of accidentally invoking non-child-directed apps due to confounding utterances.
Learning Curve Theory

Recently a number of empirical"universal"scaling law papers have been published, most notably by OpenAI. `Scaling laws' refers to power-law decreases of training or test error w.r. t. more data, larger neural networks, and/or more compute. In this work we focus on scaling w.r. t. data size $n$. Theoretical understanding of this phenomenon is largely lacking, except in finite-dimensional models for which error typically decreases with $n^{-1/2}$ or $n^{-1}$, where $n$ is the sample size. We develop and theoretically analyse the simplest possible (toy) model that can exhibit $n^{-\beta}$ learning curves for arbitrary power $\beta>0$, and determine whether power laws are universal or depend on the data distribution.
WiSleep: Scalable Sleep Monitoring and Analytics Using Passive WiFi Sensing

Sleep deprivation is a public health concern that significantly impacts one’s well-being and performance. Sleep is an intimate experience, and state-of-the-art sleep monitoring solutions are highly-personalized to individual users. With a motivation to expand sleep monitoring at a large-scale and contribute sleep data to public health understanding, we present WiSleep , a sleep monitoring and analytics platform using smartphone network connections that are passively sensed from WiFi infrastructure. We propose an unsupervised ensemble model of Bayesian change point detection to predict sleep and wake-up times. Then, we validate our approach using ground truth from a user study in campus dormitories and a private home. Our results find WiSleep outperforming established methods for users with irregular sleep patterns while yielding comparable accuracy for regular sleepers with an average 79.5% accuracy. This is comparable to client-side based methods, albeit utilizing only coarse-grained information. Finally, we show that WiSleep can process data from 20,000 users on a single commodity server, allowing it to scale to large campus populations with low server requirements.
Life cycle of streaks in the buffer layer of wall-bounded turbulence

Streaks in the buffer layer of wall-bounded turbulence are tracked in time to study their life-cycle. Spatially and temporally resolved direct numerical simulation data is used to analyze the strong wall-parallel movements conditioned to low-speed streamwise flow. The analysis of the streaks shows that there is a clear distinction between wall-attached and detached streaks, and that the former can be further categorized into streaks that are contained in the buffer layer and the ones that reach the outer region. The results reveal that streaks are born in the buffer layer, coalescing with each other to create larger streaks that are still attached to the wall. Once the streak becomes large enough, it starts to meander due to the large streamwise-to-wall-normal aspect ratio, and consequently the elongation in the streamwise direction, which makes it more difficult for the streak to be oriented strictly in the streamwise direction. While the continuous interaction of the streaks allows the super-structure to span extremely long temporal and length scales, individual streak components are relatively small and short-lived. Tall-attached streaks eventually split into wall-attached and wall-detached components. These wall-detached streaks have a strong wall-normal velocity away from the wall, similar to ejections or bursts observed in the literature. Conditionally averaging the flow fields to these split events show that the detached streak has not only a larger wall-normal velocity compared to the wall-attached counterpart, it also has a larger (less negative) streamwise velocity, similar to the velocity field at the tip of a vortex cluster.
Riiid! Answer Correctness Prediction Kaggle Challenge: 4th Place Solution Summary

This paper presents my solution to the challenge "Riiid! Answer Correctness Prediction" on Kaggle hosted by Riiid Labs (2020), which scores 0.817 (AUC) and ranks 4th on the final private leaderboard. It is a single transformer-based model heavily inspired from previous works such as SAKT, SAINT and SAINT+. Novel ingredients that I believed to have made a difference are the time-aware attention mechanism, the concatenation of the embeddings of the input sequences and the embedding of continuous features.
Bandits for Learning to Explain from Explanations

We introduce Explearn, an online algorithm that learns to jointly output predictions and explanations for those predictions. Explearn leverages Gaussian Processes (GP)-based contextual bandits. This brings two key benefits. First, GPs naturally capture different kinds of explanations and enable the system designer to control how explanations generalize across the space by virtue of choosing a suitable kernel. Second, Explearn builds on recent results in contextual bandits which guarantee convergence with high probability. Our initial experiments hint at the promise of the approach.
Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learning

Although reinforcement learning has been successfully applied in many domains in recent years, we still lack agents that can systematically generalize. While relational inductive biases that fit a task can improve generalization of RL agents, these biases are commonly hard-coded directly in the agent's neural architecture. In this work, we show that we can incorporate relational inductive biases, encoded in the form of relational graphs, into agents. Based on this insight, we propose Grid-to-Graph (GTG), a mapping from grid structures to relational graphs that carry useful spatial relational inductive biases when processed through a Relational Graph Convolution Network (R-GCN). We show that, with GTG, R-GCNs generalize better both in terms of in-distribution and out-of-distribution compared to baselines based on Convolutional Neural Networks and Neural Logic Machines on challenging procedurally generated environments and MinAtar. Furthermore, we show that GTG produces agents that can jointly reason over observations and environment dynamics encoded in knowledge bases.
A surgical dataset from the da Vinci Research Kit for task automation and recognition

The use of large datasets is essential in surgical robotics to advance in the field of recognition and automation of surgical tasks. Furthermore, public datasets allow the comparison of different algorithms and methods to evaluate their performance. The objective of this work is to provide a complete dataset of three common training surgical tasks performed with the da Vinci Research Kit (dVRK). The dataset contains a total of 206 trials performed by twelve different subjects. For each trial, the dataset includes 154 kinematic variables from the dVRK (both master and slave sides) together with the associated video recorded with a camera. All the data has been carefully timestamped and provided in a readable csv format. A MATLAB interface integrated with ROS for using and replicating the data is also provided.
Solid Texture Synthesis using Generative Adversarial Networks

Solid texture synthesis (STS), as an effective way to extend 2D exemplar to a 3D solid volume, exhibits advantages in numerous application domains. However, existing methods generally synthesize solid texture with speciﬁc features, which may result in the failure of capturing diversiﬁed textural information. In this paper, we propose a novel generative adversarial nets-based approach (STS-GAN) to hierarchically learn solid texture with a feature-free nature. Our multi-scale discriminators evaluate the similarity between patch from exemplar and slice from the generated volume, promoting the generator to synthesize realistic solid textures. Experimental results demonstrate that the proposed method can generate high-quality solid textures with similar visual characteristics to the exemplar.
Graph Neural Network to Dilute Outliers for Refactoring Monolith Application

Microservices are becoming the defacto design choice for software architecture. It involves partitioning the software components into finer modules such that the development can happen independently. It also provides natural benefits when deployed on the cloud since resources can be allocated dynamically to necessary components based on demand. Therefore, enterprises as part of their journey to cloud, are increasingly looking to refactor their monolith application into one or more candidate microservices; wherein each service contains a group of software entities (e.g., classes) that are responsible for a common functionality. Graphs are a natural choice to represent a software system. Each software entity can be represented as nodes and its dependencies with other entities as links. Therefore, this problem of refactoring can be viewed as a graph based clustering task. In this work, we propose a novel method to adapt the recent advancements in graph neural networks in the context of code to better understand the software and apply them in the clustering task. In that process, we also identify the outliers in the graph which can be directly mapped to top refactor candidates in the software. Our solution is able to improve state-of-the-art performance compared to works from both software engineering and existing graph representation based techniques.
A Deep Learning Approach for Ontology Enrichment from Unstructured Text

Information Security in the cyber world is a major cause for concern, with a significant increase in the number of attack surfaces. Existing information on vulnerabilities, attacks, controls, and advisories available on the web provides an opportunity to represent knowledge and perform security analytics to mitigate some of the concerns. Representing security knowledge in the form of ontology facilitates anomaly detection, threat intelligence, reasoning and relevance attribution of attacks, and many more. This necessitates dynamic and automated enrichment of information security ontologies. However, existing ontology enrichment algorithms based on natural language processing and ML models have issues with contextual extraction of concepts in words, phrases, and sentences. This motivates the need for sequential Deep Learning architectures that traverse through dependency paths in text and extract embedded vulnerabilities, threats, controls, products, and other security-related concepts and instances from learned path representations. In the proposed approach, Bidirectional LSTMs trained on a large DBpedia dataset and Wikipedia corpus of 2.8 GB along with Universal Sentence Encoder is deployed to enrich ISO 27001-based information security ontology. The model is trained and tested on a high-performance computing (HPC) environment to handle Wiki text dimensionality. The approach yielded a test accuracy of over 80% when tested with knocked-out concepts from ontology and web page instances to validate the robustness.
On the Broadcast Independence Number of Circulant Graphs

An independent broadcast on a graph $G$ is a function $f: V \longrightarrow \{0,\ldots,{\rm diam}(G)\}$ such that $(i)$ $f(v)\leq e(v)$ for every vertex $v\in V(G)$, where $\operatorname{diam}(G)$ denotes the diameter of $G$ and $e(v)$ the eccentricity of vertex $v$, and $(ii)$ $d(u,v)>\max \{f(u), f(v)\}$ for every two distinct vertices $u$ and $v$ with $f(u)f(v)>0$. The broadcast independence number $\beta_b(G)$ of $G$ is then the maximum value of $\sum_{v \in V} f(v)$, taken over all independent broadcasts on $G$. We prove that every circulant graph of the form $C(n;1,a)$, $3\le a\le \lfloor\frac{n}{2} \rfloor$, admits an optimal $2$-bounded independent broadcast, that is, an independent broadcast~$f$ satisfying $f(v)\le 2$ for every vertex $v$, except when $n=2a+1$, or $n=2a$ and $a$ is even. We then determine the broadcast independence number of various classes of such circulant graphs, and prove that, for most of these classes, the equality $\beta_b(C(n;1,a)) = \alpha(C(n;1,a))$ holds, where $\alpha(C(n;1,a))$ denotes the independence number of $C(n;1,a)$.
The Fault in the Stars: Understanding the Underground Market of Amazon Reviews

Reviews play a crucial role in a product’s visibility and eventually customers’ buying decisions on e-commerce sites. While there is suspicion and anecdotal reports about the prevalence of fake reviews on popular e-commerce sites, little is known about how these reputation manipulation services continue to operate despite attempts to thwart them. We investigate an emerging fraud ecosystem of incentivized reviews – fake reviews written by real customers. In this kind of review fraud, sellers give away products for free to complicit buyers in exchange for a positive review. Like other fraud services, these incentivized review services are operated through specialized agents that mediate the exchange between buyers and sellers. In this paper, we conduct an in-depth quantitative and qualitative study of incentivized review services for Amazon.com. We inﬁltrate multiple Facebook groups that are used for recruitment. We are able to identify more than 16 thousand products ranging seeking incentivized reviews. We ﬁrst evaluate off-the-shelf machine learning classiﬁers as well as Amazon’s own countermeasures against incentivized reviews and corresponding products. Our results demonstrate the ineffectiveness of these countermeasures – even when incentivized reviews are removed, new incentivized reviews are promptly added back to make up for the removed reviews. To understand the inner-workings of incentivized reviews services, we survey 38 agents and 36 buyers. Our qualitative analysis reveals that the incentivized reviews services spans multiple countries, with sellers typically in China, Bangladesh, and buyers the France, and We that groups of agents organize using sophisticated training
Dynamic Sasvi: Strong Safe Screening for Norm-Regularized Least Squares

A recently introduced technique for a sparse optimization problem called"safe screening"allows us to identify irrelevant variables in the early stage of optimization. In this paper, we first propose a flexible framework for safe screening based on the Fenchel-Rockafellar duality and then derive a strong safe screening rule for norm-regularized least squares by the framework. We call the proposed screening rule for norm-regularized least squares"dynamic Sasvi"because it can be interpreted as a generalization of Sasvi. Unlike the original Sasvi, it does not require the exact solution of a more strongly regularized problem; hence, it works safely in practice. We show that our screening rule can eliminate more features and increase the speed of the solver in comparison with other screening rules both theoretically and experimentally.
Regularization for convolutional kernel tensors to avoid unstable gradient problem in convolutional neural networks

Convolutional neural networks are very popular nowadays. Training neural networks is not an easy task. Each convolution corresponds to a structured transformation matrix. In order to help avoid the exploding/vanishing gradient problem, it is desirable that the singular values of each transformation matrix are not large/small in the training process. We propose three new regularization terms for a convolutional kernel tensor to constrain the singular values of each transformation matrix. We show how to carry out the gradient type methods, which provides new insight about the training of convolutional neural networks.
Concerning three classes of non-Diophantine arithmetics

We present three classes of abstract prearithmetics, $\{\mathbf{A}_M\}_{M \geq 1}$, $\{\mathbf{A}_{-M,M}\}_{M \geq 1}$, and $\{\mathbf{B}_M\}_{M>0}$. The first one is weakly projective with respect to the nonnegative real Diophantine arithmetic $\mathbf{R_+}=(\mathbb{R}_+,+,\times,\leq_{\mathbb{R}_+})$, the second one is weakly projective with respect to the real Diophantine arithmetic $\mathbf{R}=(\mathbb{R},+,\times,\leq_{\mathbb{R}})$, while the third one is projective with respect to the extended real Diophantine arithmetic $\overline{\mathbf{R}}=(\overline{\mathbb{R}},+,\times,\leq_{\overline{\mathbb{R}}})$. In addition, we have that every $\mathbf{A}_M$ and every $\mathbf{B}_M$ are a complete totally ordered semiring, while every $\mathbf{A}_{-M,M}$ is not. We show that the projection of any series of elements of $\mathbb{R}_+$ converges in $\mathbf{A}_M$, for any $M \geq 1$, and that the projection of any non-oscillating series series of elements of $\mathbb{R}$ converges in $\mathbf{A}_{-M,M}$, for any $M \geq 1$, and in $\mathbf{B}_M$, for all $M>0$. We also prove that working in $\mathbf{A}_M$ and in $\mathbf{A}_{-M,M}$, for any $M \geq 1$, and in $\mathbf{B}_M$, for all $M>0$, allows to overcome a version of the paradox of the heap.
Points2Vec: Unsupervised Object-level Feature Learning from Point Clouds

Unsupervised representation learning techniques, such as learning word embeddings, have had a significant impact on the field of natural language processing. Similar representation learning techniques have not yet become commonplace in the context of 3D vision. This, despite the fact that the physical 3D spaces have a similar semantic structure to bodies of text: words are surrounded by words that are semantically related, just like objects are surrounded by other objects that are similar in concept and usage. In this work, we exploit this structure in learning semantically meaningful low dimensional vector representations of objects. We learn these vector representations by mining a dataset of scanned 3D spaces using an unsupervised algorithm. We represent objects as point clouds, a flexible and general representation for 3D data, which we encode into a vector representation. We show that using our method to include context increases the ability of a clustering algorithm to distinguish different semantic classes from each other. Furthermore, we show that our algorithm produces continuous and meaningful object embeddings through interpolation experiments.