Miracle-dz/UniFAR-FTUs huggingface dataset:
An Empirical Study of UMLS Concept Extraction from Clinical Notes using Boolean Combination Ensembles

Our objective in this study is to investigate the behavior of Boolean operators on combining annotation output from multiple Natural Language Processing (NLP) systems across multiple corpora and to assess how filtering by aggregation of Unified Medical Language System (UMLS) Metathesaurus concepts affects system performance for Named Entity Recognition (NER) of UMLS concepts. We used three corpora annotated for UMLS concepts: 2010 i2b2 VA challenge set (31,161 annotations), Multi-source Integrated Platform for Answering Clinical Questions (MiPACQ) corpus (17,457 annotations including UMLS concept unique identifiers), and Fairview Health Services corpus (44,530 annotations). Our results showed that for UMLS concept matching, Boolean ensembling of the MiPACQ corpus trended towards higher performance over individual systems. Use of an approximate grid-search can help optimize the precision-recall tradeoff and can provide a set of heuristics for choosing an optimal set of ensembles.
Hippodrome: Data Race Repair Using Static Analysis Summaries

Implementing bug-free concurrent programs is a challenging task in modern software development. State-of-the-art static analyses find hundreds of concurrency bugs in production code, scaling to large codebases. Yet, fixing these bugs in constantly changing codebases represents a daunting effort for programmers, particularly because a fix in the concurrent code can introduce other bugs in a subtle way. In this work, we show how to harness compositional static analysis for concurrency bug detection, to enable a new Automated Program Repair (APR) technique for data races in large concurrent Java codebases. The key innovation of our work is an algorithm that translates procedure summaries inferred by the analysis tool for the purpose of bug reporting into small local patches that fix concurrency bugs (without introducing new ones). This synergy makes it possible to extend the virtues of compositional static concurrency analysis to APR, making our approach effective (it can detect and fix many more bugs than existing tools for data race repair), scalable (it takes seconds to analyze and suggest fixes for sizeable codebases), and usable (generally, it does not require annotations from the users and can perform continuous automated repair). Our study, conducted on popular open-source projects, has confirmed that our tool automatically produces concurrency fixes similar to those proposed by the developers in the past.
Sparse Communication via Mixed Distributions

Neural networks and other machine learning models compute continuous representations, while humans communicate mostly through discrete symbols. Reconciling these two forms of communication is desirable for generating human-readable interpretations or learning discrete latent variable models, while maintaining end-to-end differentiability. Some existing approaches (such as the Gumbel-Softmax transformation) build continuous relaxations that are discrete approximations in the zero-temperature limit, while others (such as sparsemax transformations and the Hard Concrete distribution) produce discrete/continuous hybrids. In this paper, we build rigorous theoretical foundations for these hybrids, which we call"mixed random variables. "Our starting point is a new"direct sum"base measure defined on the face lattice of the probability simplex. From this measure, we introduce new entropy and Kullback-Leibler divergence functions that subsume the discrete and differential cases and have interpretations in terms of code optimality. Our framework suggests two strategies for representing and sampling mixed random variables, an extrinsic ("sample-and-project") and an intrinsic one (based on face stratification). We experiment with both approaches on an emergent communication benchmark and on modeling MNIST and Fashion-MNIST data with variational auto-encoders with mixed latent variables.
GIFAIR-FL: An Approach for Group and Individual Fairness in Federated Learning

In this paper we propose GIFAIR-FL : an approach that imposes group and individual fairness to federated learning settings. By adding a regularization term, our algorithm penalizes the spread in the loss of client groups to drive the optimizer to fair solutions. Theoretically, we show convergence in non-convex and strongly convex settings. Our convergence guarantees hold for both i.i. d. and non-i.i
Fairness Properties of Face Recognition and Obfuscation Systems

The proliferation of automated face recognition in the commercial and government sectors has caused significant privacy concerns for individuals. One approach to address these privacy concerns is to employ evasion attacks against the metric embedding networks powering face recognition systems: Face obfuscation systems generate imperceptibly perturbed images that cause face recognition systems to misidentify the user. Perturbed faces are generated on metric embedding networks, which are known to be unfair in the context of face recognition. A question of demographic fairness naturally follows: are there demographic disparities in face obfuscation system performance? We answer this question with an analytical and empirical exploration of recent face obfuscation systems. Metric embedding networks are found to be demographically aware: face embeddings are clustered by demographic. We show how this clustering behavior leads to reduced face obfuscation utility for faces in minority groups. An intuitive analytical model yields insight into these phenomena.
LSENet: Location and Seasonality Enhanced Network for Multi-Class Ocean Front Detection

Ocean fronts can cause the accumulation of nutrients and affect the propagation of underwater sound, so high-precision ocean front detection is of great significance to the marine fishery and national defense fields. However, the existing ocean front detection methods either have low detection accuracy, or regard it as a binary classification problem to only detect where the ocean front occurs, and cannot identify and distinguish various categories of ocean fronts with different behavior characteristics in different times and regions. In order to solve the above problems, we propose a semantic segmentation network called location and seasonality enhanced network (LSENet) for multi-class ocean fronts detection at pixel level. In this network, we first design a channel supervision unit structure, which integrates the seasonal characteristics of the ocean front itself and the contextual information to improve the detection accuracy. We also introduce a location attention mechanism to adaptively assign attention weights to the fronts according to their frequently occurred sea area, which can further improve the accuracy of multi-class ocean front detection. Compared with other semantic segmentation methods and current representative ocean front detection method at Offshore China and Gulf of Mexico, the experimental results demonstrate convincingly that our method is more effective. The source code and datasets are released and publicly available via https://github.com/lliusha1155/LSENET.
PSTN: Periodic Spatial-temporal Deep Neural Network for Traffic Condition Prediction

Accurate forecasting of traffic conditions is critical for improving safety, stability, and efficiency of a city transportation system. In reality, it is challenging to produce accurate traffic forecasts due to the complex and dynamic spatiotemporal correlations. Most existing works only consider partial characteristics and features of traffic data, and result in unsatisfactory performances on modeling and forecasting. In this paper, we propose a periodic spatial-temporal deep neural network (PSTN) with three pivotal modules to improve the forecasting performance of traffic conditions through a novel integration of three types of information. First, the historical traffic information is folded and fed into a module consisting of a graph convolutional network and a temporal convolutional network. Second, the recent traffic information together with the historical output passes through the second module consisting of a graph convolutional network and a gated recurrent unit framework. Finally, a multi-layer perceptron is applied to process the auxiliary road attributes and output the final predictions. Experimental results on two publicly accessible real-world urban traffic data sets show that the proposed PSTN outperforms the state-of-the-art benchmarks by significant margins for short-term traffic conditions forecasting
M2IOSR: Maximal Mutual Information Open Set Recognition

In this work, we aim to address the challenging task of open set recognition (OSR). Many recent OSR methods rely on auto-encoders to extract class-specific features by a reconstruction strategy, requiring the network to restore the input image on pixel-level. This strategy is commonly over-demanding for OSR since class-specific features are generally contained in target objects, not in all pixels. To address this shortcoming, here we discard the pixel-level reconstruction strategy and pay more attention to improving the effectiveness of class-specific feature extraction. We propose a mutual information-based method with a streamlined architecture, Maximal Mutual Information Open Set Recognition (M2IOSR). The proposed M2IOSR only uses an encoder to extract class-specific features by maximizing the mutual information between the given input and its latent features across multiple scales. Meanwhile, to further reduce the open space risk, latent features are constrained to class conditional Gaussian distributions by a KL-divergence loss function. In this way, a strong function is learned to prevent the network from mapping different observations to similar latent features and help the network extract class-specific features with desired statistical characteristics. The proposed method significantly improves the performance of baselines and achieves new state-of-the-art results on several benchmarks consistently.
Active Reinforcement Learning over MDPs

The past decade has seen the rapid development of Reinforcement Learning, which acquires impressive performance with numerous training resources. However, one of the greatest challenges in RL is generalization efficiency (i.e., generalization performance in a unit time). This paper proposes a framework of Active Reinforcement Learning (ARL) over MDPs to improve generalization efficiency in a limited resource by instance selection. Given a number of instances, the algorithm chooses out valuable instances as training sets while training the policy, thereby costing fewer resources. Unlike existing approaches, we attempt to actively select and use training data rather than train on all the given data, thereby costing fewer resources. Furthermore, we introduce a general instance evaluation metrics and selection mechanism into the framework. Experiments results reveal that the proposed framework with Proximal Policy Optimization as policy optimizer can effectively improve generalization efficiency than unselect-ed and unbiased selected methods.
Automated Discovery of Autonomous Quantum Error Correction Schemes

We can encode a qubit in the energy levels of a quantum system. Relaxation and other dissipation processes lead to decay of the fidelity of this stored information. Is it possible to preserve the quantum information for a longer time by introducing additional drives and dissipation? The existence of autonomous quantum error correcting codes answers this question in the positive. Nonetheless, discovering these codes for a real physical system, i.e., finding the encoding and the associated driving fields and bath couplings, remains a challenge that has required intuition and inspiration to overcome. In this work, we develop and demonstrate a computational approach based on adjoint optimization for discovering autonomous quantum error correcting codes given a description of a physical system. We implement an optimizer that searches for a logical subspace and control parameters to better preserve quantum information. We demonstrate our method on a system of a harmonic oscillator coupled to a lossy qubit, and find that varying the Hamiltonian distance in Fock space -- a proxy for the control hardware complexity -- leads to discovery of different and new error correcting schemes. We discover what we call the $\sqrt{3}$ code, realizable with a Hamiltonian distance $d=2$, and propose a hardware-efficient implementation based on superconducting circuits.
Automatically Labeling Low Quality Content on Wikipedia By Leveraging Patterns in Editing Behaviors

Wikipedia articles aim to be definitive sources of encyclopedic content. Yet, only 0.6% of Wikipedia articles have high quality according to its quality scale due to insufficient number of Wikipedia editors and enormous number of articles. Supervised Machine Learning (ML) quality improvement approaches that can automatically identify and fix content issues rely on manual labels of individual Wikipedia sentence quality. However, current labeling approaches are tedious and produce noisy labels. Here, we propose an automated labeling approach that identifies the semantic category (e.g., adding citations, clarifications) of historic Wikipedia edits and uses the modified sentences prior to the edit as examples that require that semantic improvement. Highest-rated article sentences are examples that no longer need semantic improvements. We show that training existing sentence quality classification algorithms on our labels improves their performance compared to training them on existing labels. Our work shows that editing behaviors of Wikipedia editors provide better labels than labels generated by crowdworkers who lack the context to make judgments that the editors would agree with.
A Distributed Application Placement and Migration Management Techniques for Edge and Fog Computing Environments

Fog/Edge computing model allows harnessing of resources in the proximity of the Internet of Things (IoT) devices to support various types of latency-sensitive IoT applications. However, due to the mobility of users and a wide range of IoT applications with different resource requirements, it is a challenging issue to satisfy these applications’ requirements. The execution of IoT applications exclusively on one fog/edge server may not be always feasible due to limited resources, while the execution of IoT applications on different servers requires further collaboration and management among servers. Moreover, considering user mobility, some modules of each IoT application may require migration to other servers for execution, leading to service interruption and extra execution costs. In this article, we propose a new weighted cost model for hierarchical fog computing environments, in terms of the response time of IoT applications and energy consumption of IoT devices, to minimize the cost of running IoT applications and potential migrations. Besides, a distributed clustering technique is proposed to enable the collaborative execution of tasks, emitted from application modules, among servers. Also, we propose an application placement technique to minimize the overall cost of executing IoT applications on multiple servers in a distributed manner. Furthermore, a distributed migration management technique is proposed for the potential migration of applications’ modules to other remote servers as the users move along their path. Besides, failure recovery methods are embedded in the clustering, application placement, and migration management techniques to recover from unpredicted failures. The performance results demonstrate that our technique significantly improves its counterparts in terms of placement deployment time, average execution cost of tasks, the total number of migrations, the total number of interrupted tasks, and cumulative migration cost.
A variational Bayesian spatial interaction model for estimating revenue and demand at business facilities

We study the problem of estimating potential revenue or demand at business facilities and understanding its generating mechanism. This problem arises in different fields such as operation research or urban science, and more generally, it is crucial for businesses’ planning and decision making. We develop a Bayesian spatial interaction model, henceforth BSIM, which provides probabilistic predictions about revenues generated by a particular business location provided their features and the potential customers’ characteristics in a given region. BSIM explicitly accounts for the competition among the competitive facilities through a probability value determined by evaluating a store-specific Gaussian distribution at a given customer location. We propose a scalable variational inference framework that, while being significantly faster than competing Markov Chain Monte Carlo inference schemes, exhibits comparable performances in terms of parameters identification and uncertainty quantification. We demonstrate the benefits of BSIM in various synthetic settings characterised by an increasing number of stores and customers. Finally, we construct a real-world, large spatial dataset for pub activities in London, UK, which includes over 1,500 pubs and 150,000 customer regions. We demonstrate how BSIM outperforms competing approaches on this large dataset in terms of prediction performances while providing results that are both interpretable and consistent with related indicators observed for the London region.
BOSS: Bidirectional One-Shot Synthesis of Adversarial Examples

The design of additive perturbations to the inputs of classifiers has become a central focus of adversarial machine learning. An alternative approach is to synthesize adversarial examples using structures akin to generative adversarial networks, albeit with the use of large amounts of training data. By contrast, this paper considers the one-shot synthesis of adversarial examples that requires only a single reference datum. In particular, we explore solutions where the generated data must simultaneously satisfy user-defined constraints on its structural similarity to the reference input datum and the output of the classifier of interest it induces. This gives rise to what we call the Bidirectional One-Shot Synthesis (BOSS) problem. We prove that the BOSS problem is NP-complete. The experimental results verify that the targeted and confidence reduction attack methods developed either outperform or on par with state-of-the-art methods.
Remote Teaching and Learning in Applied Engineering: A Post-Pandemic Perspective

The COVID-19 pandemic significantly disrupted the educational sector. Faced with this life-threatening pandemic, educators had to swiftly pivot to an alternate form of course delivery without severely impacting the quality of the educational experience. Following the transition to online learning, educators had to grapple with a host of challenges. With interrupted face-to-face delivery, limited access to state-of-the-art labs, barriers with educational technologies, challenges of academic integrity, and obstacles with remote teamwork and student participation, creative solutions were urgently needed. In this chapter, we provide a rationale for a variety of course delivery models at different stages of the pandemic and highlight the approaches we took to overcome some of the pressing challenges of remote education. We also discuss how we ensured that hands-on learning remains an integral part of engineering curricula, and we argue that some of the applied changes during the pandemic will likely serve as a catalyst for modernizing education.
Spartus: A 9.4 TOp/s FPGA-Based LSTM Accelerator Exploiting Spatio-Temporal Sparsity

Long short-term memory (LSTM) recurrent networks are frequently used for tasks involving time-sequential data, such as speech recognition. Unlike previous LSTM accelerators that either exploit spatial weight sparsity or temporal activation sparsity, this article proposes a new accelerator called “Spartus” that exploits spatio-temporal sparsity to achieve ultralow latency inference. Spatial sparsity is induced using a new column-balanced targeted dropout (CBTD) structured pruning method, producing structured sparse weight matrices for a balanced workload. The pruned networks running on Spartus hardware achieve weight sparsity levels of up to 96% and 94% with negligible accuracy loss on the TIMIT and the Librispeech datasets. To induce temporal sparsity in LSTM, we extend the previous DeltaGRU method to the DeltaLSTM method. Combining spatio-temporal sparsity with CBTD and DeltaLSTM saves on weight memory access and associated arithmetic operations. The Spartus architecture is scalable and supports real-time online speech recognition when implemented on small and large FPGAs. Spartus per-sample latency for a single DeltaLSTM layer of 1024 neurons averages 1 $\mu \text{s}$ . Exploiting spatio-temporal sparsity on our test LSTM network using the TIMIT dataset leads to $46\times $ speedup of Spartus over its theoretical hardware performance to achieve 9.4-TOp/s effective batch-1 throughput and 1.1-TOp/s/W power efficiency.
Improved Speech Emotion Recognition using Transfer Learning and Spectrogram Augmentation

Automatic speech emotion recognition (SER) is a challenging task that plays a crucial role in natural human-computer interaction. One of the main challenges in SER is data scarcity, i.e., insufficient amounts of carefully labeled data to build and fully explore complex deep learning models for emotion classification. This paper aims to address this challenge using a transfer learning strategy combined with spectrogram augmentation. Specifically, we propose a transfer learning approach that leverages a pre-trained residual network (ResNet) model including a statistics pooling layer from speaker recognition trained using large amounts of speaker-labeled data. The statistics pooling layer enables the model to efficiently process variable-length input, thereby eliminating the need for sequence truncation which is commonly used in SER systems. In addition, we adopt a spectrogram augmentation technique to generate additional training data samples by applying random time-frequency masks to log-mel spectrograms to mitigate overfitting and improve the generalization of emotion recognition models. We evaluate the effectiveness of our proposed approach on the interactive emotional dyadic motion capture (IEMOCAP) dataset. Experimental results indicate that the transfer learning and spectrogram augmentation approaches improve the SER performance, and when combined achieve state-of-the-art results.
Generation of High-Order Coarse Quad Meshes on CAD Models via Integer Linear Programming

We propose an end-to-end pipeline to robustly generate high-quality, high-order and coarse quadrilateral meshes on CAD models. This kind of mesh enables the use of high-order analysis techniques such as high-order finite element methods or isogeometric analysis. An initial unstructured mesh is generated; this mesh contains a low number of irregular vertices but these are not necessarily aligned, causing a very dense quad layout. A T-mesh is built on the mesh which allows to modify its topology by assigning new integer lengths to the T-mesh arcs. The task of simplifying the quad layout can be formulated as an Integer Linear Program which is solved efficiently using an adequate solver. Finally, a high-order quad mesh is extracted from the optimized topology. Validation on several CAD models shows that our approach is fast, robust, strictly respects the CAD features, and achieves interesting results in terms of coarseness and quality.