Miracle-dz/UniFAR-FTUs huggingface dataset:
Interaction of a priori Anatomic Knowledge with Self-Supervised Contrastive Learning in Cardiac Magnetic Resonance Imaging

Training deep learning models on cardiac magnetic resonance imaging (CMR) can be a challenge due to the small amount of expert generated labels and inherent complexity of data source. Self-supervised contrastive learning (SSCL) has recently been shown to boost performance in several medical imaging tasks. However, it is unclear how much the pre-trained representation reﬂects the primary organ of interest compared to spurious surrounding tissue. In this work, we evaluate the optimal method of incorporating prior knowledge of anatomy into a SSCL training paradigm. Speciﬁcally, we evaluate using a segmentation network to explicitly local the heart in CMR images, followed by SSCL pretraining in multiple diagnostic tasks. We ﬁnd that using a priori knowledge of anatomy can greatly improve the downstream diagnostic performance. Furthermore, SSCL pre-training with in-domain data generally improved downstream performance and more human-like saliency compared to end-to-end training and ImageNet pre-trained networks. However, introducing anatomic knowledge to pre-training generally does not have signiﬁcant impact.
Would You Ask it that Way?: Measuring and Improving Question Naturalness for Knowledge Graph Question Answering

Knowledge graph question answering (KGQA) facilitates information access by leveraging structured data without requiring formal query language expertise from the user. Instead, users can express their information needs by simply asking their questions in natural language (NL). Datasets used to train KGQA models that would provide such a service are expensive to construct, both in terms of expert and crowdsourced labor. Typically, crowdsourced labor is used to improve template-based pseudo-natural questions generated from formal queries. However, the resulting datasets often fall short of representing genuinely natural and fluent language. In the present work, we investigate ways to characterize and remedy these shortcomings. We create the IQN-KGQA test collection by sampling questions from existing KGQA datasets and evaluating them with regards to five different aspects of naturalness. Then, the questions are rewritten to improve their fluency. Finally, the performance of existing KGQA models is compared on the original and rewritten versions of the NL questions. We find that some KGQA systems fare worse when presented with more realistic formulations of NL questions. The IQN-KGQA test collection is a resource to help evaluate KGQA systems in a more realistic setting. The construction of this test collection also sheds light on the challenges of constructing large-scale KGQA datasets with genuinely NL questions.
Step Size is a Consequential Parameter in Continuous Cellular Automata

Step size in continuous cellular automata (CA) plays an important role in the stability and behavior of self-organizing patterns. Continous CA dynamics are defined by formula very similar to numerical estimation of physics-based ordinary differential equations, specifically Euler's method, for which a large step size is often inaccurate and unstable. Rather than asymptotically approaching more accurate estimates of CA dynamics with decreasing step size, continuous CA may support different self-organizing patterns at different ranges of step size. We discuss several examples of mobile patterns that become unstable at step sizes that are too small as well as too large. Additionally, an individual mobile pattern may exhibit qualitatively different behavior across a range of step sizes. We demonstrate examples of the effects of step size in pattern stability and qualitative behavior in continuous CA implemented in the Lenia framework and its variant, Glaberish.
Towards a Fair Comparison and Realistic Design and Evaluation Framework of Android Malware Detectors

—As in other cybersecurity areas, machine learning (ML) techniques have emerged as a promising solution to detect Android malware. In this sense, many proposals employing a variety of algorithms and feature sets have been presented to date, often reporting impresive detection performances. However, the lack of reproducibility and the absence of a standard evaluation framework make these proposals difﬁcult to compare. In this paper, we perform an analysis of 10 inﬂuential research works on Android malware detection using a common evaluation framework. We have identiﬁed ﬁve factors that, if not taken into account when creating datasets and designing detectors, signiﬁcantly affect the trained ML models and their performances. In particular, we analyze the effect of (1) the presence of duplicated samples, (2) label (goodware/greyware/malware) attribution, (3) class imbalance, (4) the presence of apps that use evasion techniques and, (5) the evolution of apps. Based on this extensive experimentation, we conclude that the studied ML-based detectors have been evaluated optimistically, which justiﬁes the good published results. Our ﬁndings also highlight that it is imperative to generate realistic datasets, taking into account the factors mentioned above, to enable the design and evaluation of better solutions for Android malware detection.
Structured Uncertainty in the Observation Space of Variational Autoencoders

Variational autoencoders (VAEs) are a popular class of deep generative models with many variants and a wide range of applications. Improvements upon the standard VAE mostly focus on the modelling of the posterior distribution over the latent space and the properties of the neural network decoder. In contrast, improving the model for the observational distribution is rarely considered and typically defaults to a pixel-wise independent categorical or normal distribution. In image synthesis, sampling from such distributions produces spatially-incoherent results with uncorrelated pixel noise, resulting in only the sample mean being somewhat useful as an output prediction. In this paper, we aim to stay true to VAE theory by improving the samples from the observational distribution. We propose SOS-VAE, an alternative model for the observation space, encoding spatial dependencies via a low-rank parameterisation. We demonstrate that this new observational distribution has the ability to capture relevant covariance between pixels, resulting in spatially-coherent samples. In contrast to pixel-wise independent distributions, our samples seem to contain semantically-meaningful variations from the mean allowing the prediction of multiple plausible outputs with a single forward pass.
Using what you know: Learning dynamics from partial observations with structured neural ODEs

Identifying dynamical systems from experimental data is a notably difﬁcult task. Prior knowledge generally helps, but the extent of this knowledge varies with the application, and customized models are often needed. We propose a ﬂexible framework to incorporate a broad spectrum of physical insight into neural ODE-based system identiﬁcation, giving physical interpretability to the resulting latent space. This insight is either enforced through hard constraints in the optimization problem or added in its cost function. In order to link the partial and possibly noisy observations to the latent state, we rely on tools from nonlinear observer theory to build a recognition model. We demonstrate the performance of the proposed approach on numerical simulations and on an experimental dataset from a robotic exoskeleton.
Resampling-Based Multisplit Inference for High-Dimensional Regression

We propose a novel resampling-based method to construct an asymptotically exact test for any subset of hypotheses on coefficients in high-dimensional linear regression. It can be embedded into any multiple testing procedure to make confidence statements on relevant predictor variables. The method constructs permutation test statistics for any individual hypothesis by means of repeated splits of the data and a variable selection technique; then it defines a test for any subset by suitably aggregating its variables' test statistics. The resulting procedure is extremely flexible, as it allows different selection techniques and several combining functions. We present it in two ways: an exact method and an approximate one, that requires less memory usage and shorter computation time, and can be scaled up to higher dimensions. We illustrate the performance of the method with simulations and the analysis of real gene expression data.
Multi-Head Online Learning for Delayed Feedback Modeling

In online advertising, it is highly important to predict the probability and the value of a conversion (e.g., a purchase). It not only impacts user experience by showing relevant ads, but also affects ROI of advertisers and revenue of marketplaces. Unlike clicks, which often occur within minutes after impressions, conversions are expected to happen over a long period of time (e.g., 30 days for online shopping). It creates a challenge, as the true labels are only available after the long delays. Either inaccurate labels (partial conversions) are used, or models are trained on stale data (e.g., from 30 days ago). The problem is more eminent in online learning, which focuses on the live performance on the latest data. In this paper, a novel solution is presented to address this challenge using multi-head modeling. Unlike traditional methods, it directly quantizes conversions into multiple windows, such as day 1, day 2, day 3-7, and day 8-30. A sub-model is trained specifically on conversions within each window. Label freshness is maximally preserved in early models (e.g., day 1 and day 2), while late conversions are accurately utilized in models with longer delays (e.g., day 8-30). It is shown to greatly exceed the performance of known methods in online learning experiments for both conversion rate (CVR) and value per click (VPC) predictions. Lastly, as a general method for delayed feedback modeling, it can be combined with any advanced ML techniques to further improve the performance.
Toxicity Detection with Generative Prompt-based Inference

Due to the subtleness, implicity, and different possible interpretations perceived by different people, detecting undesirable content from text is a nuanced difficulty. It is a long-known risk that language models (LMs), once trained on corpus containing undesirable content, have the power to manifest biases and toxicity. However, recent studies imply that, as a remedy, LMs are also capable of identifying toxic content without additional fine-tuning. Prompt-methods have been shown to effectively harvest this surprising self-diagnosing capability. However, existing prompt-based methods usually specify an instruction to a language model in a discriminative way. In this work, we explore the generative variant of zero-shot prompt-based toxicity detection with comprehensive trials on prompt engineering. We evaluate on three datasets with toxicity labels annotated on social media posts. Our analysis highlights the strengths of our generative classification approach both quantitatively and qualitatively. Interesting aspects of self-diagnosis and its ethical implications are discussed.
Constrained Sampling from Language Models via Langevin Dynamics in Embedding Spaces

Large pre-trained language models are well-established for their ability to generate text seemingly indistinguishable from humans. In this work, we study the problem of constrained sampling from such language models. That is, generating text that satisﬁes user-deﬁned constraints. Typical decoding strategies which generate samples left-to-right are not always conducive to imposing such constraints globally. Instead, we propose M U - C O L A —a sampling procedure that combines the log-likelihood of the language model with arbitrary differentiable constraints into a single energy function; and generates samples by initializing the entire output sequence with noise and following a Markov chain deﬁned by Langevin Dynamics using the gradients of this energy. We evaluate our approach on text generation with soft and hard constraints as well as their combinations with competitive results for toxicity avoidance, sentiment control, and keyword guided generation. 1
An Appeal to Discuss Ethical Issues in Context with Cooperative User Localization

Rapid technical developments in GNSS and other ubiquitous positioning methods led to new applications and technical possibilities. Technical researchers and developers mostly imply that it is mainly about further enhancing localization technologies and algorithms including the development of new advanced Apps for Location-based Services (LBS). They are aiming to deliver personal navigation with a higher availability, integrity and reliability in any environment. Hence, localization technologies have become very powerful tools when tracking an individual or a group of users. As meanwhile LBS influences every individual’s life there is a need that ethical and political issues have to be addressed within our research community. Although there is a lot of research going on in developing algorithms to keep ones data and LBS search request in private, researchers can no longer keep their credibility without cooperating with ethical experts or an ethical committee. From our point of view, we should include the issue of the user’s privacy and ethical implications in mind from the very beginning of our research process. At the Department of Geodesy and Geoinformation of the Vienna University of Technology, Austria, cooperation with social scientists in this respect started recently. A study for the acceptance and user needs in a new project proposal dealing with cooperative positioning was initiated. In this project a mobile LBS user is guided with an App and located with assistance of the whole user group. Then users can help each other to find their way through complex environments such as multimodal public transport interchanges, airports, shopping malls, etc. Users of such services should get the right to withdraw their consent for transferring location based and other personal data at any time. They also should get clear and comprehensive information when and for what they give away their personal data and location and its further use.
A Neural Tangent Kernel Formula for Ensembles of Soft Trees with Arbitrary Architectures

A soft tree is an actively studied variant of a decision tree that updates splitting rules using the gradient method. Although it can have various tree architectures, the theoretical properties of their impact are not well known. In this paper, we formulate and analyze the Neural Tangent Kernel (NTK) induced by soft tree ensembles for arbitrary tree architectures. This kernel leads to the remarkable ﬁnding that only the number of leaves at each depth is relevant for the tree architecture in ensemble learning with inﬁnitely many trees. In other words, if the number of leaves at each depth is ﬁxed, the training behavior in function space and the generalization performance are exactly the same across different tree architectures, even if they are not isomorphic. We also show that the NTK of asymmetric trees like decision lists does not degenerate when they get inﬁnitely deep. This is in contrast to the perfect binary trees, whose NTK is known to degenerate and leads to worse generalization performance for deeper trees.
Mathematical Modelling of TEAM and VTEAM Memristor Model Using VerilogA

Anyone who looks into the circuitry world will be familiar with the three fundamental circuit elements - capacitor, resistor, and inductor. These circuit elements are defined by the relation between two of the four fundamental circuit variables current, voltage, charge, and flux. However, in 1971, Prof. Leon Chua proposed on the grounds of symmetry that there should be a fourth fundamental circuit element that gives the relation between flux and charge. He named this the memristor, which is short for memory resistor. This theory was then practicallymodeled, in May 2008 when the researchers at HP Labs published a paper announcing a model for a physical realization of a memristor. This report mainly focuses on the model of memristor and its applications. The advantages of variable resistance, flexibility, no leakage current, and compatibility with CMOS. The element memristor exhibits different characteristics for different applications which results in the formation of different models of the memristor. This paper gives a review of different models of the memristor. Memristors devices can be used in many applications such as memory, logic, and neuromorphic systems. A computer model of the memristor would be a useful tool for analyzing circuit behavior to help in developing the application of this memristor is a passive circuit element via simulation. In this paper, various VerilogA model of memristor devices are simulated for sinusoidal inputs and output are verified Various window functions has been used. The circuit analysis of the various memristor models is done
Jupiter's Temperature Structure: A Reassessment of the Voyager Radio Occultation Measurements

The thermal structure of planetary atmospheres is an essential input for predicting and retrieving the distribution of gases and aerosols, as well as the bulk chemical abundances. In the case of Jupiter, the temperature at a reference level—generally taken at 1 bar—serves as the anchor in models used to derive the planet’s interior structure and composition. Most models assume the temperature measured by the Galileo probe. However, those data correspond to a single location, an unusually clear, dry region, affected by local atmospheric dynamics. On the other hand, the Voyager radio occultation observations cover a wider range of latitudes, longitudes, and times. The Voyager retrievals were based on atmospheric composition and radio refractivity data that require updating and were never properly tabulated; the few existing tabulations are incomplete and ambiguous. Here we present a systematic electronic digitization of all available temperature profiles from Voyager, followed by their reanalysis, employing currently accepted values of the abundances and radio refractivities of atmospheric species. We find the corrected temperature at the 1 bar level to be up to 4 K greater than the previously published values, i.e., 170.3 ± 3.8 K at 12°S (Voyager 1 ingress) and 167.3 ± 3.8 K at 0°N (Voyager 1 egress). This is to be compared with the Galileo probe value of 166.1 ± 0.8 K at the edge of an unusual feature at 6. °57N. Altogether, this suggests that Jupiter’s tropospheric temperatures may vary spatially by up to 7 K between 7°N and 12°S.
Apport des ontologies pour le calcul de la similarité sémantique au sein d'un système de recommandation

Measurement of the semantic relatedness or likeness between terms, words, or text data plays an important role in different applications dealing with textual data such as knowledge acquisition, recommender system, and natural language processing. Over the past few years, many ontologies have been developed and used as a form of structured representation of knowledge bases for information systems. The calculation of semantic similarity from ontology has developed and depending on the context is complemented by other similarity calculation methods. In this paper, we propose and carry on an approach for the calculation of ontology-based semantic similarity using in the context of a recommender system.
Learning Action Conditions from Instructional Manuals for Instruction Understanding

The ability to infer pre- and postconditions of an action is vital for comprehending complex instructions, and is essential for applications such as autonomous instruction-guided agents and assistive AI that supports humans to perform physical tasks.In this work, we propose a task dubbed action condition inference, which extracts mentions of preconditions and postconditions of actions in instructional manuals.We propose a weakly supervised approach utilizing automatically constructed large-scale training instances from online instructions, and curate a densely human-annotated and validated dataset to study how well the current NLP models do on the proposed task.We design two types of models differ by whether contextualized and global information is leveraged, as well as various combinations of heuristics to construct the weak supervisions.Our experiments show a > 20% F1-score improvement with considering the entire instruction contexts and a > 6% F1-score benefit with the proposed heuristics. However, the best performing model is still well-behind human performance.
Vanishing Noether Current in Weyl Invariant Gravities

We revisit the issue that the Noether current associated with a local scale symmtery, or equivalently the Weyl symmetry, identically vanishes. Based on only the second Noether theorem for a local symmetry, we prove that the Noether current associated with the Weyl symmetry is in general vanishing in any Weyl invariant gravitational theories in four dimensional Riemannian geometry. We also clarify the reason: The Weyl transformation is non-dynamical in the sense that it does not contain the derivative term of the transformation parameter as opposed to the conventional gauge transformation. Finally, we apply this result to a quantum theory of a general Weyl invariant gravity and derive currents associated with choral symmetry, which is the $IOSp(10|10)$ supersymmetry.
Refining Query Representations for Dense Retrieval at Test Time

Dense retrieval uses a contrastive learning framework to learn dense representations of queries and contexts. 1 Trained encoders are directly used for each test query, but they often fail to accurately represent out-of-domain queries. In this paper, we introduce a framework that reﬁnes instance-level query representations at test time, with only the signals coming from the intermediate retrieval results. We optimize the query representation based on the retrieval result similar to pseudo relevance feedback (PRF) in information retrieval. Speciﬁcally, we adopt a cross-encoder labeler to provide pseudo labels over the retrieval result and iteratively reﬁne the query representation with a gradient descent method, treating each test query as a single data point to train on. Our theoretical analysis reveals that our framework can be viewed as a generalization of the classical Rocchio’s algorithm for PRF, which leads us to propose interesting variants of our method. We show that our test-time query reﬁnement strategy improves the performance of phrase retrieval (+8.1% Acc@1) and passage retrieval (+3.7% Acc@20) for open-domain QA with large improvements on out-of-domain queries.
Robust Differential Dynamic Programming

Differential Dynamic Programming is an optimal control technique often used for trajectory generation. Many variations of this algorithm have been developed in the literature, including algorithms for stochastic dynamics or state and input constraints. In this contribution, we develop a robust version of Differential Dynamic Programming that uses generalized plants and multiplier relaxations for uncertainties. To this end, we study a version of the Bellman principle and use convex relaxations to account for uncertainties in the dynamic program. The resulting algorithm can be seen as a robust trajectory generation tool for nonlinear systems.