Miracle-dz/UniFAR-FTUs huggingface dataset:
Phenomenology of Double Descent in Finite-Width Neural Networks

`Double descent' delineates the generalization behaviour of models depending on the regime they belong to: under- or over-parameterized. The current theoretical understanding behind the occurrence of this phenomenon is primarily based on linear and kernel regression models -- with informal parallels to neural networks via the Neural Tangent Kernel. Therefore such analyses do not adequately capture the mechanisms behind double descent in finite-width neural networks, as well as, disregard crucial components -- such as the choice of the loss function. We address these shortcomings by leveraging influence functions in order to derive suitable expressions of the population loss and its lower bound, while imposing minimal assumptions on the form of the parametric model. Our derived bounds bear an intimate connection with the spectrum of the Hessian at the optimum, and importantly, exhibit a double descent behaviour at the interpolation threshold. Building on our analysis, we further investigate how the loss function affects double descent -- and thus uncover interesting properties of neural networks and their Hessian spectra near the interpolation threshold.
Les déterminants du succès des systèmes de gestion des connaissances dans les entreprises Tunisiennes ( The determinants of success of knowledge management systems in Tunisian companies )

With the development of new information technologies and communication, organizations are implementing new management styles into account the knowledge of intangible capital. Therefore, organizations are struggling to survive and competitor. One of the strategies employed in these organizations is knowledge management (KM) with the support of Knowledge Management System (KMS). In fact, the effectiveness of KMS is intended to help companies achieve sustainable competitive advantage by using the existing knowledge base (Alavi, 2000). The benefits of KMS were observed, in many companies. However, it is not easy to successfully adopt KMS (Yogesh Malhotra, 2005).
Discrete, recurrent, and scalable patterns in human judgement underlie affective picture ratings

Operant keypress tasks, where each action has a consequence, have been analogized to the construct of “wanting” and produce lawful relationships in humans that quantify preferences for approach and avoidance behavior. It is unknown if rating tasks without an operant framework, which can be analogized to “liking”, show similar lawful relationships. We studied three independent cohorts of participants (N = 501, 506, and 4,019 participants) collected by two distinct organizations, using the same 7-point Likert scale to rate negative to positive preferences for pictures from the International Affective Picture Set. Picture ratings without an operant framework produced similar value functions, limit functions, and tradeoff functions to those reported in the literature for operant keypress tasks, all with R goodness of fits above 0.75. These value, limit, and trade-off functions were discrete in their mathematical formulation, recurrent across all three independent cohorts, and demonstrated scaling between individual and group curves. In all three experiments, the computation of loss aversion showed 95% confidence intervals below the value of 2, arguing against a strong overweighting of losses relative to gains, as has previously been reported for keypress tasks or games of chance with calibrated uncertainty. Graphed features from the three cohorts were similar and argue that preference assessments meet three of four criteria for lawfulness, providing a simple, short, and low-cost method for the quantitative assessment of preference without forced choice decisions, games of chance, or operant keypressing. This approach can easily be implemented on any digital device with a screen (e.g., cellphones).
Dissection of nitrate signalling in Arabidopsis thaliana

12 Chapter 1 Introduction 13 1.1 Importance of nitrogen 13 1.2 Nitrogen availability in the soil 13 1.3 Nitrogen in agriculture 14 1.4 Nitrogen uptake 14 1.5 Nitrogen in the cell 15 1.6 Nitrogen metabolism 16 1.6. 1 Nitrate assimilation 16 1.6. 2 Diurnal regulation of nitrate assimilation 17 1.7 Plant growth responses to nitrogen 18 1.7. 1 Low nitrogen 18 1.7. 2 High nitrogen 18 1.7 3 Nitrogen 'Hotspots' 19 1.8 Plant gene responses to nitrogen 19 1.9 Signalling Pathways 20
Exploration of High-level Synthesis Techniques to Improve Computational Intensive VLSI Designs

KIM, TAEMIN. Exploration of High-level Synthesis Techniques to Improve Computational Intensive VLSI Designs. (Under the direction of Professor Xun Liu). Optimization techniques during high level synthesis procedure are often preferred since design decisions at early stages of a design flow are believed to have a large impact on design quality. In this dissertation, we present three high-level synthesis schemes to improve the power, speed and reliability of deep submicron VLSI systems. Specifically, we first describe a simultaneous register and functional unit (FU) binding algorithm. Our algorithm targets the reduction of multiplexer inputs, shortening the total length of global interconnects. In this algorithm, we introduce three graph parameters that guide our FU and register binding. They are flow dependencies, common primary inputs and common register inputs. We maximize the interconnect sharing among FUs and registers. We then present an interconnect binding algorithm during high-level synthesis for global interconnect reduction. Our scheme is based on the observation that not all FUs operate at all time. When idle, FUs can be reconfigured as pass-through logic for data transfer, reducing interconnect requirement. Our scheme not only reduces the overall length of global interconnects but also minimizes the power overhead without introducing any timing violations. Lastly, we present a register binding algorithm with the objective of register minimization. We have observed that not all pipelined FUs are operating at all time. Idle pipelined FUs can be used to store data temporarily, reducing stand-alone registers. Exploration of High-level Synthesis Techniques to Improve Computational Intensive VLSI Designs
BioSimulators: a central registry of simulation engines and services for recommending specific tools

Abstract Computational models have great potential to accelerate bioscience, bioengineering, and medicine. However, it remains challenging to reproduce and reuse simulations, in part, because the numerous formats and methods for simulating various subsystems and scales remain siloed by different software tools. For example, each tool must be executed through a distinct interface. To help investigators find and use simulation tools, we developed BioSimulators (https://biosimulators.org), a central registry of the capabilities of simulation tools and consistent Python, command-line and containerized interfaces to each version of each tool. The foundation of BioSimulators is standards, such as CellML, SBML, SED-ML and the COMBINE archive format, and validation tools for simulation projects and simulation tools that ensure these standards are used consistently. To help modelers find tools for particular projects, we have also used the registry to develop recommendation services. We anticipate that BioSimulators will help modelers exchange, reproduce, and combine simulations.
Robust Event Triggering Control for Lateral Dynamics of Intelligent Vehicles with Designable Inter-event Times

In this brief, an improved event-triggered update mechanism (ETM) for the linear quadratic regulator is proposed to solve the lateral motion control problem of intelligent vehicle under bounded disturbances. Based on a novel event function using a clock-like variable to determine the triggering time, we further introduce two new design parameters to improve control performance. Distinct from existing event-based control mechanisms, the inter-event times (IETs) derived from the above control framework are designable, meaning that the proposed ETM can be deployed on practical vehicle more easily and effectively. In addition, the improved IETs-designable ETM features a global robust event-separation property that is extremely required for practical lateral motion control of vehicle subject to diverse disturbances. Theoretical analysis proves the feasibility and stability of the proposed control strategy for trajectory tracking under bounded disturbances. Finally, simulation results verify the theoretical results and show the advantages of the proposed control strategy.
Infrastructure-free, Deep Learned Urban Noise Monitoring at ~100mW

The Sounds of New York City (SONYC) wireless sensor network (WSN) has been fielded in Manhattan and Brooklyn over the past five years, as part of a larger human-in-the-loop cyber-physical control system for monitoring, analyzing, and mitigating urban noise pollution. We describe the evolution of the 2-tier SONYC WSN from an acoustic data collection fabric into a 3-tier in situ noise complaint monitoring WSN, and its current evaluation. The added tier consists of long range (LoRa), multi-hop networks of a new low-power acoustic mote, MKII (“Mach 2”), that we have designed and fabricated. MKII motes are notable in three ways: First, they advance machine learning capability at mote-scale in this application domain by introducing a real-time Convolutional Neural Network (CNN) based embedding model that is competitive with alternatives while also requiring 10x lesser training data and ~2 orders of magnitude fewer runtime resources. Second, they are conveniently deployed relatively far from higher-tier base station nodes without assuming power or network infrastructure support at operationally relevant sites (such as construction zones), yielding a relatively low-cost solution. And third, their networking is frequency agile, unlike conventional LoRa networks: it tolerates in a distributed, self-stabilizing way the variable external interfer-ence and link fading in the cluttered 902-928MHz ISM band urban environment by dynamically choosing good frequencies using an efficient new method that combines passive and active measure-ments.
Muon Collider Physics Summary

The perspective of designing muon colliders with high energy and luminosity, which is being investigated by the International Muon Collider Collaboration, has triggered a growing interest in their physics reach. We present a concise summary of the muon colliders potential to explore new physics, leveraging on the unique possibility of combining high available energy with very precise measurements.
Differentiated Relevances Embedding for Group-based Referring Expression Comprehension

The key of referring expression comprehension lies in capturing the cross-modal visual-linguistic relevance. Existing works typically model the cross-modal relevance in each image, where the anchor object/expression and their positive expression/object have the same attribute as the negative expression/object, but with different attribute values. These objects/expressions are exclusively utilized to learn the implicit representation of the attribute by a pair of different values, which however impedes the accuracies of the attribute representations, expression/object representations, and their cross-modal relevances since each anchor object/expression usually has multiple attributes while each attribute usually has multiple potential values. To this end, we investigate a novel REC problem named Group-based REC, where each object/expression is simultaneously employed to construct the multiple triplets among the semantically similar images. To tackle the explosion of the negatives and the differentiation of the anchor-negative relevance scores, we propose the multi-group self-paced relevance learning schema to adaptively assign within-group object-expression pairs with different priorities based on their cross-modal relevances. Since the average cross-modal relevance varies a lot across different groups, we further design an across-group relevance constraint to balance the bias of the group priority. Experiments on three standard REC benchmarks demonstrate the effectiveness and superiority of our method.
Spectral Graph Clustering for Intentional Islanding Operations in Resilient Hybrid Energy Systems

Establishing cleaner energy generation, therefore, improving the sustainability of the power system is a crucial task in this century, and one of the key strategies being pursued is to shift the dependence on fossil fuel to renewable technologies, such as wind, solar, and nuclear. However, with the increasing number of heterogeneous components included, the complexity of the hybrid energy system becomes more significant, and the complex system imposes a more stringent requirement of the contingency plan to enhance the overall system resilience. Among different strategies to ensure a reliable system, intentional islanding is commonly applied in practical applications for power systems and attracts abundant interest in the literature. In this study, we propose a hierarchical spectral clustering-based intentional islanding strategy at the transmission level with renewable generations. To incorporate the renewable generation that relies on the inverter technology, the frequency measurements are considered to represent the transient response, and it has been further used as embedded information in the clustering algorithm along with other important electrical information from the system to enrich the modeling capability of the proposed framework. To demonstrate the effectiveness of the islanding strategy, the modified IEEE 9-bus system coupled with wind farms is considered as the test case.
Feature space reduction as data preprocessing for the anomaly detection

In this paper, we present two pipelines in order to reduce the feature space for anomaly detection using the One Class SVM. As a first stage of both pipelines, we compare the performance of three convolutional autoencoders. We use the PCA method together with t-SNE as the first pipeline and the reconstruction errors based method as the second. Both methods have potential for the anomaly detection, but the reconstruction error metrics prove to be more robust for this task. We show that the convolutional autoencoder architecture doesn't have a significant effect for this task and we prove the potential of our approach on the real world dataset.
What Makes Reading Comprehension Questions Difficult?

For a natural language understanding benchmark to be useful in research, it has to consist of examples that are diverse and difficult enough to discriminate among current and near-future state-of-the-art systems. However, we do not yet know how best to select text sources to collect a variety of challenging examples. In this study, we crowdsource multiple-choice reading comprehension questions for passages taken from seven qualitatively distinct sources, analyzing what attributes of passages contribute to the difficulty and question types of the collected examples. To our surprise, we find that passage source, length, and readability measures do not significantly affect question difficulty. Through our manual annotation of seven reasoning types, we observe several trends between passage sources and reasoning types, e.g., logical reasoning is more often required in questions written for technical passages. These results suggest that when creating a new benchmark dataset, selecting a diverse set of passages can help ensure a diverse range of question types, but that passage difficulty need not be a priority.
Engineering entanglement between resonators by hot environment

Autonomous quantum thermal machines do not require an external coherent drive or work input to perform the desired tasks, making them a promising candidate for thermal management in quantum systems. Here, we propose an autonomous quantum thermal machine in which two uncoupled macroscopic mechanical resonators or microwave resonators achieve considerable entanglement via a hot thermal bath. This becomes possible by coupling the resonators to a common two-level system or third harmonic oscillator and driving it by the hot incoherent thermal bath. The critical step to make the entanglement involves suitable engineering of the hot bath, realized by bath spectrum filtering. Our results suggest that the bath spectrum filtering can be an alternative to typical non-autonomous reservoir engineering schemes to create exotic quantum states.
Toward Ethical AIED

In 1960, Norbert Weiner discussed “Some Moral and Technical Consequences of Automation” (Weiner, 1960). A pioneer of cybernetics, often credited with an early formulation of intelligent behaviour (of living things and machines) as a feedback mechanism, Weiner made two related observations which are not only pertinent to this day, but whose implications have since acquired a somewhat prophetic quality.
Bringing Rolling Shutter Images Alive with Dual Reversed Distortion

Rolling shutter (RS) distortion can be interpreted as the result of picking a row of pixels from instant global shutter (GS) frames over time during the exposure of the RS camera. This means that the information of each instant GS frame is partially, yet sequentially, embedded into the row-dependent distortion. Inspired by this fact, we address the challenging task of reversing this process, i.e., extracting undistorted GS frames from images suffering from RS distortion. However, since RS distortion is coupled with other factors such as readout settings and the relative velocity of scene elements to the camera, models that only exploit the geometric correlation between temporally adjacent images suffer from poor generality in processing data with different readout settings and dynamic scenes with both camera motion and object motion. In this paper, instead of two consecutive frames, we propose to exploit a pair of images captured by dual RS cameras with reversed RS directions for this highly challenging task. Grounded on the symmetric and complementary nature of dual reversed distortion, we develop a novel end-to-end model, IFED, to generate dual optical flow sequence through iterative learning of the velocity field during the RS time. Extensive experimental results demonstrate that IFED is superior to naive cascade schemes, as well as the state-of-the-art which utilizes adjacent RS images. Most importantly, although it is trained on a synthetic dataset, IFED is shown to be effective at retrieving GS frame sequences from real-world RS distorted images of dynamic scenes. Code is available at https://github.com/zzh-tech/Dual-Reversed-RS.
Spiking Neural Network Integrated Circuits: A Review of Trends and Future Directions

The rapid growth of deep learning, spurred by its successes in various fields ranging from face recognition [1] to game playing [2], has also triggered a growing interest in the design of specialized hardware accelerators to support these algorithms. This specialized hardware targets one of two categories-either operating in datacenters or on mobile devices at the network edge. While energy efficiency is important in both cases, the need is extremely stringent in the latter class of applications due to limited battery life. Several techniques have been used in the past to improve the energy efficiency of these accelerators [3], including reducing off-chip DRAM access, managing data flow across processing elements as well as in-memory computing (IMC) by exploiting analog processing of data within digital memory arrays [4].
Neural Theorem Provers Delineating Search Area Using RNN

Although traditional symbolic reasoning methods are highly interpretable, their application in knowledge graphs link prediction has been limited due to their computational inefficiency. A new RNNNTP method is proposed in this paper, using a generalized EM-based approach to continuously improve the computational efficiency of Neural Theorem Provers(NTPs). The RNNNTP is divided into relation generator and predictor. The relation generator is trained effectively and interpretably, so that the whole model can be carried out according to the development of the training, and the computational efficiency is also greatly improved. In all four data-sets, this method shows competitive performance on the link prediction task relative to traditional methods as well as one of the current strong competitive methods.
Real-Robot Deep Reinforcement Learning: Improving Trajectory Tracking of Flexible-Joint Manipulator with Reference Correction

Flexible-joint manipulators are governed by complex nonlinear dynamics, defining a challenging control problem. In this work, we propose an approach to learn an outer-loop joint trajectory tracking controller with deep reinforcement learning. The controller represented by a stochastic policy is learned in under two hours directly on the real robot. This is achieved through bounded reference correction actions and use of a model-free off-policy learning method. In addition, an informed policy initialization is proposed, where the agent is pre-trained in a learned simulation. We test our approach on the 7 DOF manipulator of a Baxter robot. We demonstrate that the proposed method is capable of consistent learning across multiple runs when applied directly on the real robot. Our method yields a policy which significantly improves the trajectory tracking accuracy in comparison to the vendor-provided controller, generalizing to an unseen payload.
Label-only Model Inversion Attack: The Attack that Requires the Least Information

In a model inversion attack, an adversary attempts to reconstruct the data records, used to train a target model, using only the model's output. In launching a contemporary model inversion attack, the strategies discussed are generally based on either predicted confidence score vectors, i.e., black-box attacks, or the parameters of a target model, i.e., white-box attacks. However, in the real world, model owners usually only give out the predicted labels; the confidence score vectors and model parameters are hidden as a defense mechanism to prevent such attacks. Unfortunately, we have found a model inversion method that can reconstruct the input data records based only on the output labels. We believe this is the attack that requires the least information to succeed and, therefore, has the best applicability. The key idea is to exploit the error rate of the target model to compute the median distance from a set of data records to the decision boundary of the target model. The distance, then, is used to generate confidence score vectors which are adopted to train an attack model to reconstruct the data records. The experimental results show that highly recognizable data records can be reconstructed with far less information than existing methods.
Thermodynamic Limits of Photon-Multiplier Luminescent Solar Concentrators

Luminescent solar concentrators (LSCs) are theoretically able to concentrate both direct and diffuse solar radiation with extremely high efficiencies. Photon-multiplier luminescent solar concentrators (PM-LSCs) contain chromophores which exceed 100\% photoluminescence quantum efficiency. PM-LSCs have recently been experimentally demonstrated and hold promise to outcompete traditional LSCs. However, we find that the thermodynamic limits of PM-LSCs are different and are sometimes more extreme relative to traditional LSCs. As might be expected, to achieve very high concentration factors a PM-LSC design must also include a free energy change, analogous to the Stokes shift in traditional LSCs. Notably, unlike LSCs, the maximum concentration ratio of a PM-LSC is dependent on brightness of the incident photon field. For some brightnesses, but equivalent energy loss, the PM-LSC has a greater maximum concentration factor than that of the traditional LSC. We find that the thermodynamic requirements to achieve highly concentrating PM-LSCs differ from traditional LSCs. The new model gives insight into the limits of concentration of PM-LSCs and may be used to extract design rules for further PM-LSC design.