Miracle-dz/UniFAR-FTUs huggingface dataset:
Patent-KG: A Patent Knowledge Graph for Engineering Design

To facilitate the knowledge reuse in engineering design, several dataset approaches have been proposed and applied by designers. This paper builds a patent-based knowledge graph, patent-KG, to represent the knowledge facts in patents for engineering design. The arising patent-KG approach proposes a new unsupervised mechanism to extract the knowledge facts in patent, by searching the attention graph in language models. This method avoids using expensive labelled data in supervised learning or listing complex syntactic rules in rule-based extraction. The extracted entities are compared with other benchmarks and shows a higher coverage of engineering words. The extracted relationships are also compared with other benchmarks, and the result shows meaningful advantages.
From prethermalization to chaos in periodically driven coupled rotors

Periodically driven (Floquet) systems are said to prethermalize when their energy absorption is very slow for long time. This effect was first discovered in quantum spin models, where the heating rate is exponentially small in the ratio between the driving frequency and the spin bandwidth. Recently, it was shown that prethermalization occurs also in classical systems with an infinite bandwidth. Here, we address the open question of which small parameter controls the lifetime of the prethermal state in these systems. We, first, numerically study the dependence of the lifetime on the initial conditions and on the connectivity in a system of periodically driven coupled rotors. We find that the lifetime is controlled by the temperature of the prethermal state, which is quasi-conserved when the heating is slow. This finding allows us to develop a simple analytical model that describes the crossover from prethermalization to chaos in many-body classical systems.
Determining the origin of impulsive noise events using paired wireless sound sensors

This work investigates how to identify the source of impulsive noise events using a pair of wireless noise sensors. One sensor is placed at a known noise source, and another sensor is placed at the noise receiver. Machine learning models receive data from the two sensors and estimate whether a given noise event originates from the known noise source or another source. To avoid privacy issues, the approach uses on-edge preprocessing that converts the sound into privacy compatible spectrograms. The system was evaluated at a shooting range and explosives training facility, using data collected during noise emission testing. The combination of convolutional neural networks with cross-correlation achieved the best results. We created multiple alternative models using different spectrogram representations. The best model detected 70.8\% of the impulsive noise events and correctly predicted 90.3\% of the noise events in the optimal trade-off between recall and precision.
Machine Learning for Mediation in Armed Conflicts

Today’s conflicts are becoming increasingly complex, fluid and fragmented, often involving a host of national and international actors with multiple and often divergent interests. This development poses significant challenges for conflict mediation, as mediators struggle to make sense of conflict dynamics, such as the range of conflict parties and the evolution of their political positions, the distinction between relevant and less relevant actors in peace making, or the identification of key conflict issues and their interdependence. International peace efforts appear increasingly ill-equipped to successfully address these challenges. While technology is being increasingly used in a range of conflict related fields, such as conflict predicting or information gathering, less attention has been given to how technology can contribute to conflict mediation. This case study is the first to apply state-of-the-art machine learning technologies to data from an ongoing mediation process. Using dialogue transcripts from peace negotiations in Yemen, this study shows how machine-learning tools can effectively support international mediators by managing knowledge and offering additional conflict analysis tools to assess complex information. Apart from illustrating the potential of machine learning tools in conflict mediation, the paper also emphasises the importance of interdisciplinary and participatory research design for the development of context-sensitive and targeted tools and to ensure meaningful and responsible implementation. Policy Significance Statement This study offers insights into how machine learning tools can be used to assist conflict mediators in organising and analysing data from highly complex and dynamic conflict situations. Machine learning tools can bring significant efficiency improvements to mediation by organising complex data and making it more easily accessible, giving mediators more control over existing information. They can also support moves towards consensus by highlighting areas in which political actors are converging or diverging; point to potentially overlooked areas of conflict or dialogue bottlenecks; and challenge prejudices that may have built up during a mediation process. This study offers a concrete example of how innovative machine learning tools can be used to address mediation in complex, fluid and protracted conflicts.
Spatio-Temporal Dynamic Inference Network for Group Activity Recognition

Group activity recognition aims to understand the activity performed by a group of people. In order to solve it, modeling complex spatiotemporal interactions is the key. Previous methods are limited in reasoning on a predefined graph, which ignores the inherent person-specific interaction context. Moreover, they adopt inference schemes that are computationally expensive and easily result in the over-smoothing problem. In this paper, we manage to achieve spatio-temporal person-specific inferences by proposing Dynamic Inference Network (DIN), which composes of Dynamic Relation (DR) module and Dynamic Walk (DW) module. We firstly propose to initialize interaction fields on a primary spatio-temporal graph. Within each interaction field, we apply DR to predict the relation matrix and DW to predict the dynamic walk offsets in a joint-processing manner, thus forming a person-specific interaction graph. By updating features on the specific graph, a person can possess a global-level interaction field with a local initialization. Experiments indicate both modules’ effectiveness. Moreover, DIN1 achieves significant improvement compared to previous state-of-the-art methods on two popular datasets under the same setting, while costing much less computation overhead of the reasoning module.
Exact Decoding Probability of Sparse Random Linear Network Coding for Reliable Multicast

Sparse random linear network coding (SRLNC) used as a class of erasure codes to ensure the reliability of multicast communications has been widely investigated. However, an exact expression for the decoding success probability of SRLNC is still unknown, and existing expressions are either asymptotic or approximate. In this paper, we derive an exact expression for the decoding success probability of SRLNC. The key to achieving this is to propose a criterion that a vector is contained in a subspace. To obtain this criterion, we construct a basis of a subspace, with respect to this basis, the coordinates of a vector are known, based on a maximal linearly independent set of the columns of a matrix. The exactness and the computation of the derived expression are demonstrated by a simple example.
Design and Scaffolded Training of an Efficient DNN Operator for Computer Vision on the Edge

Massively parallel systolic arrays and resource-efficient depthwise separable convolutions are two promising hardware and software techniques to accelerate DNN inference on the edge. Interestingly, their combination is inefficient: Computational patterns of depthwise separable convolutions do not exhibit a rhythmic systolic flow and lack sufficient data reuse to saturate systolic arrays. In this article, we formally analyse this inefficiency and propose an efficient operator, an optimal hardware dataflow, and a superior training methodology towards alleviating this. The efficient operator, called Fully-Separable Convolutions (FuSeConv),1 is a drop-in replacement for depthwise-separable convolutions. FuSeConv generalizes factorization of convolution fully along their spatial and depth dimensions. The resultant computation is systolic and efficiently maps to systolic arrays. The optimal hardware dataflow, called Spatial-Tiled Output Stationary(ST-OS), maximizes the efficiency of FuSeConv on systolic arrays. It maps independent convolutions to rows of the systolic array to maximise resource-utilization with negligible VLSI overheads. Neural Operator Scaffolding (NOS) scaffolds the training of FuSeConv operators by distilling knowledge from the more expensive depthwise separable convolution operation. This bridges the accuracy gap between FuSeConv networks and networks with depthwise-separable convolutions. Additionally, NOS can be combined with Neural Architecture Search (NAS) to trade off latency and accuracy. The hardware-software co-design of FuSeConv with ST-OS achieves a significant speedup of 4.1-9.25× with state-of-the-art efficient networks for the ImageNet dataset. The parameter efficiency of FuSeConv and its significant superiority over depthwise-separable convolutions on systolic arrays illustrates their promise as a strong solution on the edge. Training FuSeConv networks with NOS achieves accuracy comparable to the depthwise-separable convolution baselines. Further, by combining NOS with NAS, we design networks that define state-of-the-art models improving on both accuracy and latency for computer vision on systolic arrays.
Enabling Longitudinal Exploratory Analysis of Clinical COVID Data

As the COVID-19 pandemic continues to impact the world, data is being gathered and analyzed to better understand the disease. Recognizing the potential for visual analytics technologies to support exploratory analysis and hypothesis generation from longitudinal clinical data, a team of collaborators worked to apply existing event sequence visual analytics technologies to a longitudinal clinical data from a cohort of 998 patients with high rates of COVID-19 infection. This paper describes the initial steps toward this goal, including: (1) the data transformation and processing work required to prepare the data for visual analysis, (2) initial findings and observations, and (3) qualitative feedback and lessons learned which highlight key features as well as limitations to address in future work.
Higher order fluctuations of extremal eigenvalues of sparse random matrices

We consider extremal eigenvalues of sparse random matrices, a class of random matrices including the adjacency matrices of Erd\H{o}s-R\'{e}nyi graphs $\mathcal{G}(N,p)$. Recently, it was shown that the leading order fluctuations of extremal eigenvalues are given by a single random variable associated with the total degree of the graph (Ann. Probab., 48(2):916-962, 2020; Probab. Theory Related Fields, 180:985-1056, 2021). We construct a sequence of random correction terms to capture higher (sub-leading) order fluctuations of extremal eigenvalues in the regime $N^{\epsilon}<pN<N^{1/3-\epsilon}$. Using these random correction terms, we prove a local law up to a shifted edge and recover the rigidity of extremal eigenvalues under some corrections for $pN>N^{\epsilon}$.
Stably projectionless Fra\"iss\'e limits

We realise the algebra W , the algebra Z 0 and the algebras Z 0 ⊗ A , where A is a unital separable UHF algebra as Fra¨ıss´e limits of suitable classes of structures. In doing so, we show that such algebras are generic objects without the use of any classiﬁcation result.
On Moments of Multiplicative Coalescents

We prove existence of all moments of the multiplicative coalescent at all times. We obtain as byproducts a number of related results which could be of general interest. In particular, we show the finiteness of the second moment of the $l^2$ norm for any extremal eternal version of multiplicative coalescent. Our techniques are in part inspired by percolation, and in part are based on tools from stochastic analysis, notably the semi-martingale and the excursion theory.
Unsupervised Reservoir Computing for Solving Ordinary Differential Equations

There is a wave of interest in using unsupervised neural networks for solving differential equations. The existing methods are based on feed-forward networks, {while} recurrent neural network differential equation solvers have not yet been reported. We introduce an unsupervised reservoir computing (RC), an echo-state recurrent neural network capable of discovering approximate solutions that satisfy ordinary differential equations (ODEs). We suggest an approach to calculate time derivatives of recurrent neural network outputs without using backpropagation. The internal weights of an RC are fixed, while only a linear output layer is trained, yielding efficient training. However, RC performance strongly depends on finding the optimal hyper-parameters, which is a computationally expensive process. We use Bayesian optimization to efficiently discover optimal sets in a high-dimensional hyper-parameter space and numerically show that one set is robust and can be used to solve an ODE for different initial conditions and time ranges. A closed-form formula for the optimal output weights is derived to solve first order linear equations in a backpropagation-free learning process. We extend the RC approach by solving nonlinear system of ODEs using a hybrid optimization method consisting of gradient descent and Bayesian optimization. Evaluation of linear and nonlinear systems of equations demonstrates the efficiency of the RC ODE solver.
SomnNET: An SpO2 Based Deep Learning Network for Sleep Apnea Detection in Smartwatches

The abnormal pause or rate reduction in breathing is known as the sleep-apnea hypopnea syndrome and affects the quality of sleep of an individual. A novel method for the detection of sleep apnea events (pause in breathing) from peripheral oxygen saturation (SpO2) signals obtained from wearable devices is discussed in this paper. The paper details an apnea detection algorithm of a very high resolution on a per-second basis for which a 1-dimensional convolutional neural network- which we termed SomnNET- is developed. This network exhibits an accuracy of 97.08% and outperforms several lower resolution state-of-the-art apnea detection methods. The feasibility of model pruning and binarization to reduce the computational complexity is explored. The pruned network with 80% sparsity exhibited an accuracy of 89.75%, and the binarized network exhibited an accuracy of 68.22%. The performance of the proposed networks is compared against several state-of-the-art algorithms.
Efficient Out-of-Distribution Detection Using Latent Space of β-VAE for Cyber-Physical Systems

Deep Neural Networks are actively being used in the design of autonomous Cyber-Physical Systems (CPSs). The advantage of these models is their ability to handle high-dimensional state-space and learn compact surrogate representations of the operational state spaces. However, the problem is that the sampled observations used for training the model may never cover the entire state space of the physical environment, and as a result, the system will likely operate in conditions that do not belong to the training distribution. These conditions that do not belong to training distribution are referred to as Out-of-Distribution (OOD). Detecting OOD conditions at runtime is critical for the safety of CPS. In addition, it is also desirable to identify the context or the feature(s) that are the source of OOD to select an appropriate control action to mitigate the consequences that may arise because of the OOD condition. In this article, we study this problem as a multi-labeled time series OOD detection problem over images, where the OOD is defined both sequentially across short time windows (change points) as well as across the training data distribution. A common approach to solving this problem is the use of multi-chained one-class classifiers. However, this approach is expensive for CPSs that have limited computational resources and require short inference times. Our contribution is an approach to design and train a single β-Variational Autoencoder detector with a partially disentangled latent space sensitive to variations in image features. We use the feature sensitive latent variables in the latent space to detect OOD images and identify the most likely feature(s) responsible for the OOD. We demonstrate our approach using an Autonomous Vehicle in the CARLA simulator and a real-world automotive dataset called nuImages.
Resource Efficiency and Circular Economy in European SMEs: Investigating the Role of Green Jobs and Skills

This study explored the size and potential of green employment for circular economy (CE) in small and medium enterprises (SMEs) in the European Union, and investigated the role of green jobs and skills for the implementation of CE practices. The data were collected in a Eurobarometer survey, and refer to resource efficiency, green markets, and CE procedures. Lack of environmental expertise is one of the factors that might be perceived as an obstacle when trying to implement resource-efficiency actions. Previous research has shown that, although resource-efficiency practices are adopted by firms in all European countries, there are differences both within and between countries. The analysis of the determinants of green behavior by European SMEs was completed by a study of heterogeneity across firms and within countries with a multilevel latent class model, a hierarchical clustering method. A general important observation is that having no workers dedicated to green jobs is strongly correlated to the probability of adopting resource-efficiency practices, while perceiving the need of extra environmental skills has a positive effect on the intention to implement actions in the future. Other characteristics of the firms play a significant impact on resource efficiency: in general, older and bigger firms, with larger yearly turnover, are more prone to implement actions.
CoMPM: Context Modeling with Speaker’s Pre-trained Memory Tracking for Emotion Recognition in Conversation

As the use of interactive machines grow, the task of Emotion Recognition in Conversation (ERC) became more important. If the machine-generated sentences reflect emotion, more human-like sympathetic conversations are possible. Since emotion recognition in conversation is inaccurate if the previous utterances are not taken into account, many studies reflect the dialogue context to improve the performances. Many recent approaches show performance improvement by combining knowledge into modules learned from external structured data. However, structured data is difficult to access in non-English languages, making it difficult to extend to other languages. Therefore, we extract the pre-trained memory using the pre-trained language model as an extractor of external knowledge. We introduce CoMPM, which combines the speaker’s pre-trained memory with the context model, and find that the pre-trained memory significantly improves the performance of the context model. CoMPM achieves the first or second performance on all data and is state-of-the-art among systems that do not leverage structured data. In addition, our method shows that it can be extended to other languages because structured knowledge is not required, unlike previous methods. Our code is available on github .
Recent Developments in Ruthenium-Catalyzed C–H Arylation: Array of Mechanistic Manifolds

Functionalized biaryl-containing motifs are among the most valuable synthetic scaffolds in organic chemistry with wide ranging commercial applications in pharmaceuticals, functional materials, and agrochemicals. Recently, there has been tremendous progress in the development of methods for functional-group-tolerant, cost-effective, and atom-economic ruthenium-catalyzed C–H arylation, providing attractive catalytic alternatives to traditional C–H functionalization and cross-coupling approaches. New ruthenium-catalyzed C–H arylation reactions have resulted in the development of attractive and highly efficient synthetic approaches to functionalized biaryls unavailable by other methods. New mechanistic manifolds have emerged, thus opening prospects for the development of a broad range of novel reactions. Strategies for the use of ruthenium catalysis in complex synthesis, including industrial applications, have been reported, validating the synthetic potential of this mode of reactivity. The development of new.
Neural Network iLQR: A Reinforcement Learning Architecture for Trajectory Optimization

—As a notable machine learning paradigm, the research efforts in the context of reinforcement learning have certainly progressed leaps and bounds. When compared with reinforcement learning methods with the given system model, the methodology of the reinforcement learning architecture based on the unknown model generally exhibits signiﬁcantly broader universality and applicability. In this work, a new reinforcement learning architecture based on iterative linear quadratic regulator (iLQR) is developed and presented without the requirement of any prior knowledge of the system model, which is termed as an approach of a “neural network iterative linear quadratic regulator (NNiLQR)”. Depending solely on measurement data, this method yields a completely new non-parametric routine for the establishment of the optimal policy (without the necessity of system modeling) through iterative reﬁnements of the neural network system. Rather importantly, this approach signiﬁcantly outperforms the classical iLQR method in terms of the given objective function because of the innovative utilization of further exploration in the methodology. As clearly indicated from the results attained in two illustrative examples, these signiﬁcant merits of the NNiLQR method are demonstrated rather evidently.
A simple article template

To date, several automated strategies for identification/segmentation of Multiple Sclerosis (MS) lesions by Magnetic Resonance Imaging (MRI) have been presented which are either outperformed by human experts or, at least, whose results are well distinguishable from humans. This is due to the ambiguity originated by MRI instabilities, peculiar MS Heterogeneity and MRI unspecific nature with respect to MS. Physicians partially treat the uncertainty generated by ambiguity relying on personal radiological/clinical/anatomical background and experience. We present an automated framework for MS lesions identification/segmentation based on three pivotal concepts to better emulate human reasoning: the modeling of uncertainty; the proposal of two, separately trained, CNN, one optimized with respect to lesions themselves and the other to the environment surrounding lesions, respectively repeated for axial, coronal and sagittal directions; the ensemble of the CNN output. The proposed framework is trained, validated and tested on the 2016 MSSEG benchmark public data set from a single imaging modality, FLuid-Attenuated Inversion Recovery (FLAIR). The comparison, performed on the segmented lesions by means of most of the metrics normally used with respect to the ground-truth and the 7 human raters in MSSEG, prove that there is no significant difference between the proposed framework and the other raters. Results are also shown for the uncertainty, though a comparison with the other raters is impossible.