Miracle-dz/UniFAR-FTUs huggingface dataset:
Towards Probabilistic Verification of Machine Unlearning

Right to be forgotten, also known as the right to erasure, is the right of individuals to have their data erased from an entity storing it. The General Data Protection Regulation in the European Union legally solidified the status of this long held notion. As a consequence, there is a growing need for the development of mechanisms whereby users can verify if service providers comply with their deletion requests. In this work, we take the first step in proposing a formal framework to study the design of such verification mechanisms for data deletion requests -- also known as machine unlearning -- in the context of systems that provide machine learning as a service. We propose a backdoor-based verification mechanism and demonstrate its effectiveness in certifying data deletion with high confidence using the above framework. Our mechanism makes a novel use of backdoor attacks in ML as a basis for quantitatively inferring machine unlearning. In our mechanism, each user poisons part of its training data by injecting a user-specific backdoor trigger associated with a user-specific target label. The prediction of target labels on test samples with the backdoor trigger is then used as an indication of the user's data being used to train the ML model. We formalize the verification process as a hypothesis testing problem, and provide theoretical guarantees on the statistical power of the hypothesis test. We experimentally demonstrate that our approach has minimal effect on the machine learning service but provides high confidence verification of unlearning. We show that with a $30\%$ poison ratio and merely $20$ test queries, our verification mechanism has both false positive and false negative ratios below $10^{-5}$. Furthermore, we also show the effectiveness of our approach by testing it against an adaptive adversary that uses a state-of-the-art backdoor defense method.
Inverse Feature Learning: Feature Learning Based on Representation Learning of Error

This paper proposes inverse feature learning (IFL) as a novel supervised feature learning technique that learns a set of high-level features for classification based on an error representation approach. The key contribution of this method is to learn the representation of error as high-level features, while current representation learning methods interpret error by loss functions which are obtained as a function of differences between the true labels and the predicted ones. One advantage of this error representation is that the learned features for each class can be obtained independently of learned features for other classes; therefore, IFL can learn simultaneously meaning that it can learn new classes’ features without retraining. Error representation learning can also help with generalization and reduce the chance of over-fitting by adding a set of impactful features to the original data set which capture the relationships between each instance and different classes through an error generation and analysis process. This method can be particularly effective in data sets, where the instances of each class have diverse feature representations or the ones with imbalanced classes. The experimental results show that the proposed IFL results in better performance compared to the state-of-the-art classification techniques for several popular data sets. We hope this paper can open a new path to utilize the proposed perspective of error representation learning in different feature learning domains.
NoSQL Databases: Yearning for Disambiguation

The demanding requirements of the new Big Data intensive era raised the need for flexible storage systems capable of handling huge volumes of unstructured data and of tackling the challenges that traditional databases were facing. NoSQL Databases, in their heterogeneity, are a powerful and diverse set of databases tailored to specific industrial and business needs. However, the lack of theoretical background creates a lack of consensus even among experts about many NoSQL concepts, leading to ambiguity and confusion. In this paper, we present a survey of NoSQL databases and their classification by data model type. We also conduct a benchmark in order to compare different NoSQL databases and distinguish their characteristics. Additionally, we present the major areas of ambiguity and confusion around NoSQL databases and their related concepts, and attempt to disambiguate them.
Models for knot spaces and Atiyah duality

Let $\mathrm{Emb}(S^1,M)$ be the space of smooth embeddings from the circle to a closed manifold $M$ of dimension $\geq 4$. We study a cosimplicial model of $\mathrm{Emb}(S^1,M)$ in stable categories, using a spectral version of Poincare-Lefschetz duality called Atiyah duality. We actually deal with a notion of a comodule instead of the cosimplicial model, and prove a comodule version of the duality. As an application, we introduce a new spectral sequence converging to $H^*(\mathrm{Emb}(S^1,M))$ for simply connected $M$ and for major coefficient rings. Using this, we compute $H^*(\mathrm{Emb}(S^1, S^k\times S^l))$ in low degrees with some conditions on $k$, $l$. We also prove the inclusion $\mathrm{Emb}(S^1,M)\to \mathrm{Imm}(S^1,M)$ to the immersions induces an isomorphism on $\pi_1$ for some simply connected $4$-manifolds, related to a question posed by Arone and Szymik. We also prove an equivalence of singular cochain complex of $\mathrm{Emb}(S^1,M)$ and a homotopy colimit of chain complexes of a Thom spectrum of a bundle over a comprehensible space. Our key ingredient is a structured version of the duality due to R. Cohen.
MATNet: Motion-Attentive Transition Network for Zero-Shot Video Object Segmentation

In this paper, we present a novel end-to-end learning neural network, i.e., MATNet, for zero-shot video object segmentation (ZVOS). Motivated by the human visual attention behavior, MATNet leverages motion cues as a bottom-up signal to guide the perception of object appearance. To achieve this, an asymmetric attention block, named Motion-Attentive Transition (MAT), is proposed within a two-stream encoder network to firstly identify moving regions and then attend appearance learning to capture the full extent of objects. Putting MATs in different convolutional layers, our encoder becomes deeply interleaved, allowing for close hierarchical interactions between object apperance and motion. Such a biologically-inspired design is proven to be superb to conventional two-stream structures, which treat motion and appearance independently in separate streams and often suffer severe overfitting to object appearance. Moreover, we introduce a bridge network to modulate multi-scale spatiotemporal features into more compact, discriminative and scale-sensitive representations, which are subsequently fed into a boundary-aware decoder network to produce accurate segmentation with crisp boundaries. We perform extensive quantitative and qualitative experiments on four challenging public benchmarks, i.e., DAVIS16, DAVIS17, FBMS and YouTube-Objects. Results show that our method achieves compelling performance against current state-of-the-art ZVOS methods. To further demonstrate the generalization ability of our spatiotemporal learning framework, we extend MATNet to another relevant task: dynamic visual attention prediction (DVAP). The experiments on two popular datasets (i.e., Hollywood-2 and UCF-Sports) further verify the superiority of our model (our code is available at https://github.com/tfzhou/MATNet).
When can Wasserstein GANs minimize Wasserstein Distance?

Generative Adversarial Networks (GANs) are widely used models to learn complex real-world distributions. In GANs, the training of the generator usually stops when the discriminator can no longer distinguish the generator's output from the set of training examples. A central question of GANs is that when the training stops, whether the generated distribution is actually close to the target distribution. Previously, it was found that such closeness can only be achieved when there is a strict capacity trade-off between the generator and discriminator: Neither of the two models can be too powerful than the other. In this paper, we established one of the first theoretical results in explaining this trade-off. We show that when the generator is a class of two-layer neural networks, then it is necessary and sufficient for the discriminator to be a one-layer network with ReLU-type activation functions. With this trade-off, using polynomially many training examples, when the training stops, the generator will indeed output a distribution that is inverse-polynomially close to the target. Our result also sheds light on how GANs training can find such a generator efficiently.
Manifold Regularization for Adversarial Robustness

Manifold regularization is a technique that penalizes the complexity of learned functions over the intrinsic geometry of input data. We develop a connection to learning functions which are "locally stable", and propose new regularization terms for training deep neural networks that are stable against a class of local perturbations. These regularizers enable us to train a network to state-of-the-art robust accuracy of 70% on CIFAR-10 against a PGD adversary using $\ell_\infty$ perturbations of size $\epsilon = 8/255$. Furthermore, our techniques do not rely on the construction of any adversarial examples, thus running orders of magnitude faster than standard algorithms for adversarial training.
AirCROP: Airline Crew Pairing Optimizer for Complex Flight Networks Involving Multiple Crew Bases & Billion-Plus Variables

Airline scheduling poses some of the most challenging problems in the entire Operations Research (OR) domain. In that, crew scheduling (CS) constitutes one of the most important and challenging planning activities. Notably, the crew operating cost is the second-largest component of an airline's total operating cost (after the fuel cost). Hence, its optimization promises enormous benefits, and even marginal improvements may translate to annual savings worth millions of dollars for an airline. However, CS is a combination of complex combinatorial optimization problems (with NP-hard computational complexity), namely \textit{crew pairing} and \textit{crew assignment}, which are solved sequentially. Here, \textit{crew pairing optimization} aims at generating a set of flight sequences (each called a crew pairing) to cover a finite set of flight legs from an airline's timetable at minimum cost, while satisfying several legality constraints linked to the federations' safety rules, airline-specific regulations, labor laws, etc. Subsequently, \textit{crew assignment} aims at assigning crew members to these optimal crew pairings. This research focuses on the critically fundamental step of CS, in that, a crew pairing optimization problem (CPOP) is formulated, and an optimization framework to solve the posed CPOP has been proposed. The distinctive contribution of this research relates to \textit{its tackling of a large-scale complex flight network for an airline (leading to more than a billion legal pairings/variables), and validation of the results by the research consortium's Industrial partner, GE Aviation}.
NYTWIT: A Dataset of Novel Words in the New York Times

We present the New York Times Word Innovation Types dataset, or NYTWIT, a collection of over 2,500 novel English words published in the New York Times between November 2017 and March 2019, manually annotated for their class of novelty (such as lexical derivation, dialectal variation, blending, or compounding). We present baseline results for both uncontextual and contextual prediction of novelty class, showing that there is room for improvement even for state-of-the-art NLP systems. We hope this resource will prove useful for linguists and NLP practitioners by providing a real-world environment of novel word appearance.
Total Reflection of Two Guided Waves for Embedded Trapped Modes

To investigate the mechanism of wave trapping, acoustic embedded trapped modes associated with two-resonant-mode interference in two-dimensional duct-cavity structures are calculated by the feedback-loop closure principle, which allows us to analyse the travelling modes that construct the trapped modes. The exact two coexisting resonant modes that underpin an embedded trapped mode in a cavity open to two semi-infinite ducts are numerically demonstrated. At the interface between the cavity segment and a duct, total reflection can occur for a particular combination of two propagative guided waves in the cavity. With a particular cavity length and a particular frequency, total reflection also happens for the two reflected waves at the opposite end of the cavity. In this way, the two coexisting standing waves underpin a trapped mode. It is found that the two standing waves are not two closed-cavity modes. Thus, this work presents a new understanding of such embedded trapped modes as a product of total reflection of two guided waves at the interface between two waveguides, rather than interference between two eigenmodes of a closed cavity. For quasi-trapped modes, the Fano scattering phenomenon owing to the effects of two acoustic channels is also shown.
Local Hypergraph Clustering using Capacity Releasing Diffusion

Local graph clustering is an important machine learning task that aims to find a well-connected cluster near a set of seed nodes. Recent results have revealed that incorporating higher order information significantly enhances the results of graph clustering techniques. The majority of existing research in this area focuses on spectral graph theory-based techniques. However, an alternative perspective on local graph clustering arises from using max-flow and min-cut on the objectives, which offer distinctly different guarantees. For instance, a new method called capacity releasing diffusion (CRD) was recently proposed and shown to preserve local structure around the seeds better than spectral methods. The method was also the first local clustering technique that is not subject to the quadratic Cheeger inequality by assuming a good cluster near the seed nodes. In this paper, we propose a local hypergraph clustering technique called hypergraph CRD (HG-CRD) by extending the CRD process to cluster based on higher order patterns, encoded as hyperedges of a hypergraph. Moreover, we theoretically show that HG-CRD gives results about a quantity called motif conductance, rather than a biased version used in previous experiments. Experimental results on synthetic datasets and real world graphs show that HG-CRD enhances the clustering quality.
Capacity of Continuous Channels with Memory via Directed Information Neural Estimator

Calculating the capacity (with or without feedback) of channels with memory and continuous alphabets is a challenging task. It requires optimizing the directed information (DI) rate over all channel input distributions. The objective is a multi-letter expression, whose analytic solution is only known for a few specific cases. When no analytic solution is present or the channel model is unknown, there is no unified framework for calculating or even approximating capacity. This work proposes a novel capacity estimation algorithm that treats the channel as a ‘black-box’, both when feedback is or is not present. The algorithm has two main ingredients: (i) a neural distribution transformer (NDT) model that shapes a noise variable into the channel input distribution, which we are able to sample, and (ii) the DI neural estimator (DINE) that estimates the communication rate of the current NDT model. These models are trained by an alternating maximization procedure to both estimate the channel capacity and obtain an NDT for the optimal input distribution. The method is demonstrated on the moving average additive Gaussian noise channel, where it is shown that both the capacity and feedback capacity are estimated without knowledge of the channel transition kernel. The proposed estimation framework opens the door to a myriad of capacity approximation results for continuous alphabet channels that were inaccessible until now.
Principles of diagnosis and treatment of patients with head and neck squamous cancer

(RPGS) rozwijają się w strukturach anatomicznych obejmujących wargi, jamę ustną, krtań, gardło, jamę nosową wraz z zatokami przynosowymi oraz gruczoły ślinowe. Stanowią ok. 10% wszystkich nowotworów złośliwych, mężczyźni chorują 4-krotnie częściej niż kobiety. Podstawowe czynniki rakotwórcze dla RPGS to narażenie błon śluzowych na dym tytoniowy i wysokoprocentowy alkohol. Typowymi stanami przedrakowymi są rogowacenie białe (leukoplakia) i czerwone (erytroplakia). RPGS pomimo różnego przebiegu klinicznego posiadają wiele wspólnych cech. Z uwagi na umiejscowienie prowadzą do różnego stopnia dysfunkcji dotyczących podstawowych czynności życiowych, takich jak oddychanie, odżywianie i mowa. Spływem chłonki RPGS są węzły chłonne szyjno-nadobojczykowe, w których w chwili rozpoznania u ok. 40% chorych stwierdza się zmiany przerzutowe. Fakt zachorowania na RPGS jest czynnikiem wystąpienia drugiego nowotworu w obrębie dróg oddechowych. Umiejscowienie ogniska pierwotnego związane jest ze stopniem zróżnicowania nowotworu: wysoki stopień dotyczy raka wargi, jamy ustnej, języka i krtani (głośni). W pozostałych lokalizacjach przeważają nowotwory nisko oraz niezróżnicowane. Podstawą rozpoznania RPGS jest wycinek pobrany z ogniska pierwotnego, w przypadku klinicznie podejrzanych węzłów chłonnych zawsze należy wykonać biopsję aspiracyjną cienkoigłową. Podstawowymi metodami leczenia są chirurgia, radioterapia i chemioterapia, stosowane samodzielnie lub w skojarzeniu. Wybór optymalnego sposobu postępowania zależy od stanu sprawności chorego, stopnia zaawansowania nowotworu, jego umiejscowienia i zróżnicowania oraz doświadczenia ośrodka prowadzącego leczenie. Najważniejszym czynnikiem rokowniczym jest stan regionalnych węzłów chłonnych. Odsetek 5-letnich przeżyć chorych na RPGS bez przerzutów do węzłów chłonnych wynosi ok. 50%, w porównaniu do niespełna 30% w przypadku węzłów zmienionych przerzutowo.
Metodologia alternativa para o estudo dos efeitos da radiação infravermelha-A sobre a pele humana

Infrared radiation (IR-A) causes structural changes in the skin, similar to those caused by prolonged exposure to ultraviolet radiation. Evaluation of efficacy and safety of cosmetic products concentrates in in vitro tests and clinical trials. A promising alternative is the use of fragments of human skin from elective cosmetic surgery, to evaluate the actual clinical benefits of a product applied topically. Objective: The objective of this study was to correlate IR-A radiation effects in biopsies and in ex vivo skin fragments and in human fibroblasts culture by quantifying MMP-1, TIMP-1 and GADD45a mediators. Methods: Collection of biopsies from 15 volunteers after IR-A applications for 5 consecutive days. Exposure to IR-A radiation of human skin fragments from elective cosmetic surgery, and human fibroblasts culture. Measurement of MMP-1, TIMP-1 and GADD45a mediators for further comparison of results. Results: In the three models used, the IR-A radiation induced an increase in MMP-1, inhibited the synthesis of GADD45a, and did not changed TIMP-1 values. Conclusion: Due to the positive correlation of the models studied, it may be suggested the use of ex vivo skin as plausible and sustainable tool to overcome differences between knowledge generated from in vitro and clinical experiments
High‐dimensional, multiscale online changepoint detection

We introduce a new method for high‐dimensional, online changepoint detection in settings where a p‐variate Gaussian data stream may undergo a change in mean. The procedure works by performing likelihood ratio tests against simple alternatives of different scales in each coordinate, and then aggregating test statistics across scales and coordinates. The algorithm is online in the sense that both its storage requirements and worst‐case computational complexity per new observation are independent of the number of previous observations; in practice, it may even be significantly faster than this. We prove that the patience, or average run length under the null, of our procedure is at least at the desired nominal level, and provide guarantees on its response delay under the alternative that depend on the sparsity of the vector of mean change. Simulations confirm the practical effectiveness of our proposal, which is implemented in the R package ocd, and we also demonstrate its utility on a seismology data set.
Gait velocity as a tool for evaluating the effects of gait training in patients with chronic stroke

Background. Gait velocity is a simple but very essential parameter which may be applied as an indicator of functional efficiency. Gait of stroke patients is characterized by reduced speed. As a result, the patients have significantly limited functioning capabilities, including walking independently outside home. The study was designed to assess selected temporal gait parameters, including gait velocity, stride velocity and swing phase velocity in patients with chronic stroke following a rehabilitation program with the use
Heavy traffic scaling limits for shortest remaining processing time queues with heavy tailed processing time distributions

We study a single server queue operating under the shortest remaining processing time (SRPT) scheduling policy; that is, the server preemptively serves the job with the shortest remaining processing time first. In this work we are interested in studying the asymptotic behavior of suitably scaled measure-valued state descriptors that describe the evolution of a sequence of SRPT queuing systems. Gromoll, Kruk, and Puha (2011) have studied this problem under diffusive scaling. In the setting where the processing time distributions have unbounded support, under suitable conditions, they show that the diffusion scaled measures converge in distribution to the process that is identically zero. In Puha (2015) for the setting where the processing time distributions have unbounded support and light tails, a non-standard scaling of the queue length process is shown to give rise to a form of state space collapse that results in a nonzero limit. In the current work we consider the case where processing time distributions have finite second moments and regularly varying tails. We show that the measure valued process, under a non-standard scaling, converges in distribution in the space of paths of measures. In sharp contrast with previous results, there is no state space collapse. Nevertheless, the description of the limit is simple and given explicitly in terms of a certain $\mathbb{R}_+$ valued random field which is determined from a single Brownian motion. Along the way we establish convergence of suitably scaled workload and queue length processes. We also show that as the tail of the distribution of job processing times becomes lighter in an appropriate fashion, the difference between the limiting queue length process and the limiting workload process converges to zero, thereby approaching the behavior of state space collapse.
Overview of Tools Supporting Planning for Automated Driving

Planning is an essential topic in the realm of automated driving. Besides planning algorithms that are widely covered in the literature, planning requires different software tools for its development, validation, and operation. This paper presents a survey of such tools including map representations, communication, traffic rules, open-source planning stacks and middleware, simulation, and visualization tools as well as benchmarks. We start by defining the planning task and different supporting tools. Next, we provide a comprehensive review of state-of-the-art developments and analysis of relations among them. Afterwards, a systematic method to opt for superior tools with respect to specific planning tasks is proposed. Finally, we discuss the current gaps and suggest future research directions. The survey as well as methodology of selecting tools can speed up the pace of planning research for automated driving.