Miracle-dz/UniFAR-FTUs huggingface dataset:
Neurashed: A Phenomenological Model for Imitating Deep Learning Training

To advance deep learning methodologies in the next decade, a theoretical framework for reasoning about modern neural networks is needed. While efforts are increasing toward demystifying why deep learning is so effective, a comprehensive picture remains lacking, suggesting that a better theory is possible. We argue that a future deep learning theory should inherit three characteristics: a \textit{hierarchically} structured network architecture, parameters \textit{iteratively} optimized using stochastic gradient-based methods, and information from the data that evolves \textit{compressively}. As an instantiation, we integrate these characteristics into a graphical model called \textit{neurashed}. This model effectively explains some common empirical patterns in deep learning. In particular, neurashed enables insights into implicit regularization, information bottleneck, and local elasticity. Finally, we discuss how neurashed can guide the development of deep learning theories.
THE USE OF REAL EARTHQUAKE ACCELEROGRAMS AS INPUT TO DYNAMIC ANALYSIS

The increasing availability of strong-motion accelerograms, and the relative ease with which they can be obtained compared to synthetic or artificial records, makes the use of real records an ever more attractive option for defining the input to dynamic analyses in geotechnical and structural engineering. Guidelines on procedures for the selection of appropriate suites of acceleration time-series for this purpose are lacking, and seismic design codes are particularly poor in this respect. Criteria for selecting records in terms of earthquake scenarios and in terms of response spectral ordinates are presented, together with options and criteria for adjusting the selected accelerograms to match the elastic design spectrum. The application of both geophysical and response spectral search criteria is illustrated using compatible scenarios, and the selected records are analysed and adjusted to produce suites of acceleration time-series suitable for dynamic analyses. The paper concludes with suggestions for making use of real records in engineering analysis and design, and recommendations are given for improving the current guidelines provided in seismic design codes.
FlowPool: Pooling Graph Representations with Wasserstein Gradient Flows

In several machine learning tasks for graph structured data, the graphs under consideration may be composed of a varying number of nodes. Therefore, it is necessary to design pooling methods that aggregate the graph representations of varying size to representations of ﬁxed size which can be used in downstream tasks, such as graph classiﬁcation. Existing graph pooling methods oﬀer no guarantee with regards to the similarity of a graph representation and its pooled version. In this work, we address this limitation by proposing FlowPool, a pooling method that optimally preserves the statistics of a graph representation to its pooled counterpart by minimizing their Wasserstein distance. This is achieved by performing a Wasserstein gradient ﬂow with respect to the pooled graph representation. Our method relies on a versatile implementation which can take into account the geometry of the representation space through any ground cost and computes the gradient of the Wasserstein distance with automatic diﬀerentiation. We propose the diﬀerentiation of the Wasserstein ﬂow layer using an implicit diﬀerentiation scheme. Therefore, our pooling method is amenable to automatic diﬀerentiation and can be integrated in end-to-end deep learning architectures. Further, FlowPool is invariant to permutations and can therefore be combined with permutation equivariant feature extraction layers in GNNs in order to obtain predictions that are independent of the ordering of the nodes. Experimental results demonstrate that our method leads to an increase in performance compared to existing pooling methods when evaluated on graph classiﬁcation.
Raw High-Definition Radar for Multi-Task Learning

With their robustness to adverse weather conditions and ability to measure speeds, radar sensors have been part of the automotive landscape for more than two decades. Recent progress toward High Definition (HD) Imaging radar has driven the angular resolution below the degree, thus approaching laser scanning performance. However, the amount of data a HD radar delivers and the computational cost to estimate the angular positions remain a challenge. In this paper, we propose a novel HD radar sensing model, FFT-RadNet, that eliminates the overhead of computing the range-azimuth-Doppler 3D tensor, learning instead to recover angles from a range-Doppler spectrum. FFT-RadNet is trained both to detect vehicles and to segment free driving space. On both tasks, it competes with the most recent radar-based models while requiring less compute and memory. Also, we collected and annotated 2-hour worth of raw data from synchronized automotive-grade sensors (camera, laser, HD radar) in various environments (city street, highway, countryside road). This unique dataset, nick-named RADIal for “Radar, LiDAR et al.”, is available at https://github.com/valeoai/RADIal.
Improving Ranking Quality and Fairness in Swiss-System Chess Tournaments

The International Chess Federation (FIDE) imposes a voluminous and complex set of player pairing criteria in Swiss-system chess tournaments and endorses computer programs that are able to calculate the prescribed pairings. The purpose of these formalities is to ensure that players are paired fairly during the tournament and that the final ranking corresponds to the players' true strength order. We contest the official FIDE player pairing routine by presenting alternative pairing rules. These can be enforced by computing maximum weight matchings in a carefully designed graph. We demonstrate by extensive experiments that a tournament format using our mechanism (1) yields fairer pairings in the rounds of the tournament and (2) produces a final ranking that reflects the players' true strengths better than the state-of-the-art FIDE pairing system.
Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

Transformer based architectures have shown notable results on many down streaming tasks including question answering. The availability of data, on the other hand, impedes obtaining legitimate performance for low-resource languages. In this paper, we investigate the applicability of pre-trained multilingual models to improve the performance of question answering in low-resource languages. We tested four combinations of language and task adapters using multilingual transformer architectures on seven languages similar to MLQA dataset. Additionally, we have also proposed zero-shot transfer learning of low-resource question answering using language and task adapters. We observed that stacking the language and the task adapters improves the multilingual transformer models’ performance significantly for low-resource languages. Our code and trained models are available at: https://github.com/CALEDIPQALL/
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP

What are the units of text that we want to model? From bytes to multi-word expressions, text can be analyzed and generated at many granularities. Until recently, most natural language processing (NLP) models operated over words, treating those as discrete and atomic tokens, but starting with byte-pair encoding (BPE), subword-based approaches have become dominant in many areas, enabling small vocabularies while still allowing for fast inference. Is the end of the road character-level model or byte-level processing? In this survey, we connect several lines of work from the pre-neural and neural era, by showing how hybrid approaches of words and characters as well as subword-based approaches based on learned segmentation have been proposed and evaluated. We conclude that there is and likely will never be a silver bullet singular solution for all applications and that thinking seriously about tokenization remains important for many applications.
Explicit Numerical Methods for High Dimensional Stochastic Nonlinear Schrödinger Equation: Divergence, Regularity and Convergence

This paper focuses on the construction and analysis of explicit numerical methods of high dimensional stochastic nonlinear Schrodinger equations (SNLSEs). We first prove that the classical explicit numerical methods are unstable and suffer from the numerical divergence phenomenon. Then we propose a kind of explicit splitting numerical methods and prove that the structure-preserving splitting strategy is able to enhance the numerical stability. Furthermore, we establish the regularity analysis and strong convergence analysis of the proposed schemes for SNLSEs based on two key ingredients. One ingredient is proving new regularity estimates of SNLSEs by constructing a logarithmic auxiliary functional and exploiting the Bourgain space. Another one is providing a dedicated error decomposition formula and a novel truncated stochastic Gronwall's lemma, which relies on the tail estimates of underlying stochastic processes. In particular, our result answers the strong convergence problem of numerical methods for 2D SNLSEs emerged from [C. Chen, J. Hong and A. Prohl, Stoch. Partial Differ. Equ. Anal. Comput. 4 (2016), no. 2, 274-318] and [J. Cui and J. Hong, SIAM J. Numer. Anal. 56 (2018), no. 4, 2045-2069].
Study of neutron irradiation effects in Depleted CMOS detector structures

In this paper the results of Edge-TCT and I-V measurements with passive test structures made in LFoundry 150 nm HV-CMOS process on p-type substrates with different initial resistivities ranging from 0.5 to 3 kΩcm are presented. Samples were irradiated with reactor neutrons up to a fluence of 2·1015 neq/cm2. The depletion depth was measured with Edge-TCT. The effective space charge concentration Neff was estimated from the dependence of the depletion depth on bias voltage and studied as a function of neutron fluence. The dependence of Neff on fluence changes with initial acceptor concentration in agreement with other measurements with p-type silicon. A long term accelerated annealing study of Neff and detector current up to 1280 minutes at 60°C was made. It was found that Neff and current in reverse biased detector behave as expected for irradiated silicon.
Certified Federated Adversarial Training

In federated learning (FL), robust aggregation schemes have been developed to protect against malicious clients. Many robust aggregation schemes rely on certain numbers of benign clients being present in a quorum of workers. This can be hard to guarantee when clients can join at will, or join based on factors such as idle system status, and connected to power and WiFi. We tackle the scenario of securing FL systems conducting adversarial training when a quorum of workers could be completely malicious. We model an attacker who poisons the model to insert a weakness into the adversarial training such that the model displays apparent adversarial robustness, while the attacker can exploit the inserted weakness to bypass the adversarial training and force the model to misclassify adversarial examples. We use abstract interpretation techniques to detect such stealthy attacks and block the corrupted model updates. We show that this defence can preserve adversarial robustness even against an adaptive attacker.
SSDNet: State Space Decomposition Neural Network for Time Series Forecasting

In this paper, we present SSDNet, a novel deep learning approach for time series forecasting. SSDNet combines the Transformer architecture with state space models to provide probabilistic and interpretable forecasts, including trend and seasonality components and previous time steps important for the prediction. The Transformer architecture is used to learn the temporal patterns and estimate the parameters of the state space model directly and efficiently, without the need for Kalman filters. We comprehensively evaluate the performance of SSDNet on five data sets, showing that SSDNet is an effective method in terms of accuracy and speed, outperforming state-of-the-art deep learning and statistical methods, and able to provide meaningful trend and seasonality components.
Object Pursuit: Building a Space of Objects via Discriminative Weight Generation

We propose a framework to continuously learn object-centric representations for visual learning and understanding. Existing object-centric representations either rely on supervisions that individualize objects in the scene, or perform unsupervised disentanglement that can hardly deal with complex scenes in the real world. To mitigate the annotation burden and relax the constraints on the statistical complexity of the data, our method leverages interactions to effectively sample diverse variations of an object and the corresponding training signals while learning the object-centric representations. Throughout learning, objects are streamed one by one in random order with unknown identities, and are associated with latent codes that can synthesize discriminative weights for each object through a convolutional hypernetwork. Moreover, re-identification of learned objects and forgetting prevention are employed to make the learning process efficient and robust. We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations. Furthermore, we demonstrate the capability of the proposed framework in learning representations that can improve label efficiency in downstream tasks. Our code and trained models are made publicly available at: https://github.com/pptrick/Object-Pursuit.
Neural Field Models: A mathematical overview and unifying framework

Mathematical modelling of the macroscopic electrical activity of the brain is highly non-trivial and requires a detailed understanding of not only the associated mathematical techniques, but also the underlying physiology and anatomy. Neural field theory is a population-level approach to modelling the non-linear dynamics of large populations of neurons, while maintaining a degree of mathematical tractability. This class of models provides a solid theoretical perspective on fundamental processes of neural tissue such as state transitions between different brain activities as observed during epilepsy or sleep. Various anatomical, physiological, and mathematical assumptions are essential for deriving a minimal set of equations that strike a balance between biophysical realism and mathematical tractability. However, these assumptions are not always made explicit throughout the literature. Even though neural field models (NFMs) first appeared in the literature in the early 1970's, the relationships between them have not been systematically addressed. This may partially be explained by the fact that the inter-dependencies between these models are often implicit and non-trivial. Herein we provide a review of key stages of the history and development of neural field theory and contemporary uses of this branch of mathematical neuroscience. First, the principles of the theory are summarised throughout a discussion of the pioneering models by Wilson and Cowan, Amari and Nunez. Upon thorough review of these models, we then present a unified mathematical framework in which all neural field models can be derived by applying different assumptions. We then use this framework to i) derive contemporary models by Robinson, Jansen and Rit, Wendling, Liley, and Steyn-Ross, and ii) make explicit the many significant inherited assumptions that exist in the current literature.
LAMBO: Landmarks Augmentation With Manifold-Barycentric Oversampling

We propose the first data augmentation method based on optimal transport theory, with the generated data being guaranteed to belong to the original data manifold. The proposed algorithm randomly samples a clique in the nearest-neighbors graph representing the data knowledge and computes the Wasserstein barycenter between the neighbours with random uniform weights. Being extremely natural-looking, many such barycenters are then produced iteratively to overpopulate the original dataset. We apply this approach to the problem of landmarks detection in unsupervised and semi-supervised scenarios in the popular tasks of face keypoints extraction, pose detection, and the segmentation of anatomical contours in medical imaging. The barycentric oversampling approach is shown to outperform state-of-the-art data augmentation methods. The code is available at https://github.com/cviaai/LAMBO/.
Sketching approximability of all finite CSPs

A constraint satisfaction problem (CSP), $\textsf{Max-CSP}(\mathcal{F})$, is specified by a finite set of constraints $\mathcal{F} \subseteq \{[q]^k \to \{0,1\}\}$ for positive integers $q$ and $k$. An instance of the problem on $n$ variables is given by $m$ applications of constraints from $\mathcal{F}$ to subsequences of the $n$ variables, and the goal is to find an assignment to the variables that satisfies the maximum number of constraints. In the $(\gamma,\beta)$-approximation version of the problem for parameters $0 \leq \beta<\gamma \leq 1$, the goal is to distinguish instances where at least $\gamma$ fraction of the constraints can be satisfied from instances where at most $\beta$ fraction of the constraints can be satisfied. In this work we consider the approximability of this problem in the context of sketching algorithms and give a dichotomy result. Specifically, for every family $\mathcal{F}$ and every $\beta<\gamma$, we show that either a linear sketching algorithm solves the problem in polylogarithmic space, or the problem is not solvable by any sketching algorithm in $o(\sqrt{n})$ space. In particular, we give non-trivial approximation algorithms using polylogarithmic space for infinitely many constraint satisfaction problems. We also extend previously known lower bounds for general streaming algorithms to a wide variety of problems, and in particular the case of $q=k=2$, where we get a dichotomy, and the case when the satisfying assignments of the constraints of $\mathcal{F}$ support a distribution on $[q]^k$ with uniform marginals. Prior to this work, other than sporadic examples, the only systematic classes of CSPs that were analyzed considered the setting of Boolean variables $q=2$, binary constraints $k=2$, singleton families $|\mathcal{F}|=1$ and only considered the setting where constraints are placed on literals rather than variables.
Taxonomy of Croton flavens (Euphorbiaceae), a Caribbean Endemic, and a Preliminary Subsectional Classification of Croton section Adenophylli

Abstract Croton flavens shares a general morphology with multiple species throughout Croton section Adenophylli, which has complicated the application of names to these taxa. We generated nuclear (ITS) and chloroplast (trnL-F) phylogenies of C. sect. Adenophylli in order to identify the phylogenetic placement of C. flavens and morphologically similar species. Based on these phylogenies, we recognize six subsections within the section, two of which (C. subsects. Adenophylli and Velamea) required new combinations, three others (C. subsects. Caribaeus, Laceratoglandulosus, and Meridionalis) are newly described, and one (C. subsect. Pungentes) was previously described. Croton flavens is restricted to the islands of the West Indies, extending from the Bahamas to Trinidad, but absent from the Leeward Antilles. It appears to be rare at the extremes of its range in the Bahamas and Trinidad, but otherwise it is often a ubiquitous member of dry, shrubby vegetation of many Caribbean islands. Within C. flavens we recognize a widespread typical variety and C. flavens var. pallidus from Hispaniola and Cuba.
Incorporating Team-Based Learning into a Fluid Mechanics Module: First Insights

The aim of this study is to analyze the impact of the use of Team-Based Learning (TBL) methodology in a module of a Fluid Mechanics course, namely fluids´ viscosity. A questionnaire was answered by students, which comprised the degree of satisfaction with various types of teaching methodologies (traditional, TBL, among others). Students’ grades obtained in that module, using TBL methodology, in average, show a positive tendency although not statistically significant when compared with the previous year, when traditional teaching methodology was used. Concerning students’ perceptions, data collected through the questionnaire show that TBL was the least preferred teaching methodology.
Multimodal Learning Analytics in Students with Learning Difficulties: How the Environment Can Affect their Learning Positively or Negatively

The present paper summarizes the 3-year research plan that will be conducted with the support of the University of Salamanca within the Doctoral Programme Education in the Knowledge Society. During this period, and through the analysis of multi-modal data, specially from the physical environment, we will study how the environment affects the learning of students with special education needs, which factors are the ones that make a positive impact in them and how to enhance the learning of these pupils.
Optimization of Resource Allocation and Leveling Using Genetic Algorithms

Resource allocation and leveling are among the top challenges in project management. Due to the complexity of projects, resource allocation and leveling have been dealt with as two distinct subproblems solved mainly using heuristic procedures that cannot guarantee optimum solutions. In this paper, improvements are proposed to resource allocation and leveling heuristics, and the Genetic Algorithms (GAs) technique is used to search for near-optimum solution, considering both aspects simultaneously. In the improved heuristics, random priorities are introduced into selected tasks and their impact on the schedule is monitored. The GA procedure then searches for an optimum set of tasks' priorities that produces shorter project duration and better-leveled resource profiles. One major advantage of the procedure is its simple applicability within commercial project management software systems to improve their performance. With a widely used system as an example, a macro program is written to automate the GA proced.
Monitoring load levels in multi-wire strands by nonlinear ultrasonic waves

Monitoring load levels in multi-wire steel strands is relevant to ensuring the proper structural performance of post-tensioned concrete structures, suspension bridges, and cable-stayed bridges. This article investigates the use of ultrasonic nonlinearity as a means to determine the level of load applied to the strands. Since an axial load on a multi-wire strand generates proportional contact stresses between adjacent wires, ultrasonic nonlinearity from the inter-wire contact must be related to the level of axial load. This article shows that the higher harmonic generation of ultrasonic guided waves propagating in individual wires of the strand indeed changes monotonically with the applied load, with smaller higher harmonic amplitudes with increasing load levels. This trend is consistent with known studies on higher harmonic generation from ultrasonic plane waves incident on a contact interface under a changing contact pressure. The article presents experimental studies on free strands and embedded strands, and numerical studies (nonlinear Finite Element Analysis) on free strands.
Yield learning and the sources of profitability in semiconductor manufacturing and process development

A numerical model that identifies the high-leverage variables associated with profitability in semiconductor manufacturing is presented. Varying the parameters of the model demonstrates that a rapid yield-learning rate determines profitability more than any other factor does. Factors such as ramping up early, adding fab capacity, depressing the terminal fault density, and shrinking die size all yield diminishing returns. The model also suggests that preparations in the early stages of process development are the key to successful yield learning.