Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning Conceptual-Contexual Embeddings for Medical Text

External knowledge is often useful for natu-ral language understanding tasks. We introduce a contextual text representation model called Conceptual-Contextual (CC) embed-dings, which incorporates structured knowledge into text representations. Unlike entity embedding methods, our approach encodes a knowledge graph into a context model. CC embeddings can be easily reused for a wide range of tasks just like pre-trained language models. Our model effectively encodes the huge UMLS database by leveraging semantic generalizability. Experiments on electronic health records (EHRs) and medical text processing benchmarks showed our model gives a major boost to the performance of supervised medical NLP tasks.
Modeling the primary and secondary productions of the southern Benguela upwelling system: A comparative study through two biogeochemical models

A three‐dimensional primitive equation model, the Regional Ocean Modeling Systems (ROMS), coupled to two biogeochemical configurations (NPZD and N2P2Z2D2) was used to study the dynamics of the first trophic levels of the pelagic food web in the southern Benguela upwelling system. The domain extends from the Agulhas Bank bordered by the Agulhas Current to 27°S on the west coast of South Africa. The circulation is driven by monthly climatologies of atmospheric forcing fields. The NPZD ecosystem model consists of four state variables: nutrient (nitrate), phytoplankton, zooplankton and detritus. In the N2P2Z2D2 model, ammonium has been added and the three other variables have been divided into small and large organisms or detritus. Both models are able to reproduce the spatio‐temporal phytoplankton distribution. Along the west coast, chlorophyll concentrations maxima are associated to surface waters. Westward dominating winds generate the lowest chlorophyll concentrations encountered in winter. The small phytoplankton organisms simulated by the N2P2Z2D2 model are responsible for a weaker chlorophyll inshore/offshore gradient, in closer agreement with observations. Transitions from a regime dominated by new production (high f ratio) to one dominated by regenerated production (low f ratio) happen to be abrupt, underlying the constant competition between small and large organisms with regard to upwelling induced nutrient inputs. On the Agulhas Bank, the summer enrichment is associated with subsurface maxima, while in winter, mixing by storms results in a homogeneous phytoplankton distribution in the water column. Regenerated production plays an important role in maintaining the total phytoplankton growth. Zooplankton biomass reflects the overall patterns of chlorophyll a concentrations with differences between the west coast and the Agulhas Bank, consistent with data, and its distribution exhibits a clear seasonal contrast. The seasonality of small and large zooplankton in the N2P2Z2D2 model is quite distinct, which allows, from the Agulhas Bank to St. Helena Bay, a food continuum for fish larvae. This was not achieved with the simpler NPZD model, emphasizing the importance of representing the appropriate level of complexity to characterize food availability for higher trophic levels.
A Fast Path Loss Model for Wireless Channels Considering Environmental Factors

—We present a path loss model that accurately predicts the path loss with low computational complexity considering environmental factors. In the proposed model, the entire area under consideration is recognized and divided into regions from a raster map, and each type of region is assigned with a path loss exponent (PLE) value. We then extract the model parameters via measurement in a suburban area to verify the proposed model. The results show that the root mean square error (RMSE) of the proposed model is 1.4 dB smaller than the widely used log-distance model.
Logics for Word Transductions with Synthesis

We introduce a logic, called ℒT, to express properties of transductions, i.e. binary relations from input to output (finite) words. In ℒT, the input/output dependencies are modelled via an origin function which associates to any position of the output word, the input position from which it originates. ℒT is well-suited to express relations (which are not necessarily functional), and can express all regular functional transductions, i.e. transductions definable for instance by deterministic two-way transducers. Despite its high expressive power, ℒT has decidable satisfiability and equivalence problems, with tight non-elementary and elementary complexities, depending on specific representation of ℒT-formulas. Our main contribution is a synthesis result: from any transduction R defined in ℒT, it is possible to synthesise a regular functional transduction f such that for all input words u in the domain of R, f is defined and (u, f(u)) ∈ R. As a consequence, we obtain that any functional transduction is regular iff it is ℒT-definable. We also investigate the algorithmic and expressiveness properties of several extensions of ℒT, and explicit a correspondence between transductions and data words. As a side-result, we obtain a new decidable logic for data words.
Judging Amy: Automated Legal Assessment using OWL 2

One of the most salient tasks in law is legal assessment, and concerns the problem of determining whether some case is allowed or disallowed given an appropriate body of legal norms. In this paper we describe a system and Protege 4 plugin, called OWL Judge, that uses standard OWL 2 DL reasoning for legal assessment. Norms are represented in terms of the LKIF Core ontology, as generic situation descriptions in which something (state, action) is deemed obliged, prohibited or permitted. We demonstrate the design patterns for defining the norms and actual cases. Furthermore we show how a DL classifier can be used to assess individual cases and automatically generate a lex specialis exception structure using OWL Judge. We illustrate our approach with a worked-out example of university library regulations.
The nature and origin of taxonomic diversity gradients in marine bivalves

Abstract Bivalves have been fundamental in developing the understanding of large-scale biodiversity patterns in the marine realm. A new study based on 29 regional bivalve faunas indicates that both latitudinal and longitudinal gradients are not so regular in form as was once imagined. The northern hemisphere latitudinal gradient has a marked step, or inflection, at 20–30°N, and in the southern hemisphere Australia forms a distinct diversity hotspot. A longitudinal gradient runs between a large tropical high diversity focus in the southern China-Indonesia-Australia region and a somewhat smaller one in the Panamic-Caribbean region. It is unlikely that the marked asymmetry in both latitudinal and longitudinal gradients can be explained entirely by either the geometric patterns of species ranges in relation to geographical boundaries (the mid-domain effect) or the operation of contemporary, or equilibrium, factors. The role of history must be considered too, and in some cases this could involve a timescale of at least 60 Ma. For example, Australia has moved progressively northwards throughout the Cenozoic era and this seems to have led to the development of a unique assemblage of both tropical and temperate taxa around the continent. When Australia eventually collided with southeast Asia, some 15 Ma ago, three previously distinct tropical marine faunas were brought into close juxtaposition. However, something else seems to have been involved in order to generate a marked pantropical Neogene diversification event in bivalves and other marine invertebrate taxa. It is striking how this event coincided with a prolonged interval of global cooling and the suggestion has been made that glacioeustatic cycles may have promoted the formation of a tropical species diversity pump. In concert with regular and profound sea-level changes, new taxa were created in marginal tropical regions and then concentrated in certain core regions. These same glacioeustatic cycles may also have served to accentuate the latitudinal limits of many tropical taxa. As the knowledge of large-scale taxonomic diversity gradients in the marine realm has increased, so too has the realization that they are at least the partial products of major historical events.
WikiOnto: A System for Semi-automatic Extraction and Modeling of Ontologies Using Wikipedia XML Corpus

This paper introduces WikiOnto: a system that assists in the extraction and modeling of topic ontologies in a semi-automatic manner using a preprocessed document corpus of one of the largest knowledge bases in the world - the Wikipedia. Based on the Wikipedia XML Corpus, we present a three-tiered framework for extracting topic ontologies in quick time and a modeling environment to refine these ontologies. Using Natural Language Processing (NLP) and other Machine Learning (ML) techniques along with a very rich document corpus, this system proposes a solution to a task that is generally considered extremely cumbersome. The initial results of the prototype suggest strong potential of the system to become highly successful in ontology extraction and modeling and also inspire further research on extracting ontologies from other semi-structured document corpora as well.
An efficient mosaic algorithm for UAV aerial images in multiple-strips flying

This paper describes an effective mosaic algorithm for UAV aerial images under the circumstances that the vehicle is executing a multiple-strips flying task. The proposed algorithm relies on two parts. One part is the basic image mosaic method which has three steps: firstly, an PCA-GSIFT is constructed to get the key points of the two images to be stitched, secondly, to get the matched pairs, an innovated matched pairs filter is presented, thirdly, to complete the registration and fusion, a weighted average method is implemented. And the other part of the new algorithm is the multiple-strips images mosaic strategy based on a global modulation strategy. Experimental results show that the approach has strong stabilization as well as high efficiency and performs excellent in multiple-stips flying UAV’s panorama generating .
Model Sparsification Can Simplify Machine Unlearning

—Recent data regulations necessitate machine unlearn- ing (MU): The removal of the effect of speciﬁc examples from the model. While exact unlearning is possible by conducting a model retraining with the remaining data from scratch, its computational cost has led to the development of approximate but efﬁcient unlearning schemes. Beyond data-centric MU solutions, we advance MU through a novel model-based viewpoint: sparsiﬁ- cation via weight pruning. Our results in both theory and practice indicate that model sparsity can boost the multi-criteria unlearn- ing performance of an approximate unlearner, closing the approximation gap, while continuing to be efﬁcient. With this insight, we develop two new sparsity-aware unlearning meta-schemes, termed ‘prune ﬁrst, then unlearn’ and ‘sparsity-aware unlearn- ing’. Extensive experiments show that our ﬁndings and proposals consistently beneﬁt MU in various scenarios including class-wise data scrubbing, random data scrubbing, and backdoor data forgetting. One highlight is the 77% unlearning efﬁcacy gain of ﬁne-tuning (one of the simplest approximate unlearning methods) in the proposed sparsity-aware unlearning paradigm. Codes are available at https://github.com/OPTML-Group/Unlearn-Sparse.
A Forgotten Danger in DNN Supervision Testing: Generating and Detecting True Ambiguity

—Deep Neural Networks (DNNs) are becoming a crucial component of modern software systems, but they are prone to fail under conditions that are different from the ones observed during training (out-of-distribution inputs) or on inputs that are truly ambiguous, i.e., inputs that admit multiple classes with nonzero probability in their ground truth labels. Recent work proposed DNN supervisors to detect high-uncertainty inputs before their possible misclassiﬁcation leads to any harm. To test and compare the capabilities of DNN supervisors, researchers proposed test generation techniques, to focus the testing effort on high-uncertainty inputs that should be recognized as anomalous by supervisors. However, existing test generators can only produce out-of-distribution inputs. No existing model-and supervisor independent technique supports the generation of truly ambiguous test inputs. In this paper, we propose a novel way to generate ambiguous inputs to test DNN supervisors and used it to empirically compare several existing supervisor techniques. In particular, we propose A MBI G UESS to generate ambiguous samples for image classiﬁcation problems. A MBI G UESS is based on gradient-guided sampling in the latent space of a regularized adversarial autoencoder. Moreover, we conducted what is – to the best of our knowledge – the most extensive comparative study of DNN supervisors, considering their capabilities to detect 4 distinct types of high-uncertainty inputs, including truly ambiguous ones.
Bidirectional Self-Normalizing Neural Networks

The problem of exploding and vanishing gradients has been a long-standing obstacle that hinders the effective training of neural networks. Despite various tricks and techniques that have been employed to alleviate the problem in practice, there still lacks satisfactory theories or provable solutions. In this paper, we address the problem from the perspective of high-dimensional probability theory. We provide a rigorous result that shows, under mild conditions, how the exploding/vanishing gradient problem disappears with high probability if the neural networks have sufficient width. Our main idea is to constrain both forward and backward signal propagation in a nonlinear neural network through a new class of activation functions, namely Gaussian-Poincare normalized functions, and orthogonal weight matrices. Experiments on both synthetic and real-world data validate our theory and confirm its effectiveness on very deep neural networks when applied in practice.
Differential coexpression analysis using microarray data and its application to human cancer

MOTIVATION Microarrays have been used to identify differential expression of individual genes or cluster genes that are coexpressed over various conditions. However, alteration in coexpression relationships has not been studied. Here we introduce a model for finding differential coexpression from microarrays and test its biological validity with respect to cancer. RESULTS We collected 10 published gene expression datasets from cancers of 13 different tissues and constructed 2 distinct coexpression networks: a tumor network and normal network. Comparison of the two networks showed that cancer affected many coexpression relationships. Functional changes such as alteration in energy metabolism, promotion of cell growth and enhanced immune activity were accompanied with coexpression changes. Coregulation of collagen genes that may control invasion and metastatic spread of tumor cells was also found. Cluster analysis in the tumor network identified groups of highly interconnected genes related to ribosomal protein synthesis, the cell cycle and antigen presentation. Metallothionein expression was also found to be clustered, which may play a role in apoptosis control in tumor cells. Our results show that this model would serve as a novel method for analyzing microarrays beyond the specific implications for cancer.
Identifying Co-Adaptation of Algorithmic and Implementational Innovations in Deep Reinforcement Learning: A Taxonomy and Case Study of Inference-based Algorithms

Recently many algorithms were devised for reinforcement learning (RL) with function approximation. While they have clear algorithmic distinctions, they also have many implementation differences that are algorithm-agnostic and some-times subtle. Such mixing of algorithmic novelty and implementation craftsmanship makes rigorous analyses of the sources of performance improvements difﬁcult. In this work, we focus on a series of inference-based actor-critic algorithms – MPO, AWR, and SAC – to decouple their algorithmic innovations and implementation decisions. We present uniﬁed derivations through a single control-as-inference objective, where we can categorize each algorithm as based on either Expectation-Maximization (EM) or direct Kull-back–Leibler (KL) divergence minimization and treat the rest of speciﬁcations as implementation details. We performed extensive ablation studies, and identiﬁed substantial performance drops whenever implementation details are mismatched for algorithmic choices. These results show which implementation details are co-adapted and co-evolved with algorithms, and which are transferable across algorithms: as examples, we identi-ﬁed that tanh policy and network sizes are highly adapted to algorithmic types, while layer normalization and ELU are critical for MPO’s performances but also transfer to noticeable gains in SAC. We hope our work can inspire future work to further demystify sources of performance improvements across multiple algorithms and allow researchers to build on one another’s both algorithmic and implementational innovations 1 .
Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique

The papers in this special section focus on the technology and applications supported by deep learning. Deep learning is a growing trend in general data analysis and has been termed one of the 10 breakthrough technologies of 2013. Deep learning is an improvement of artificial neural networks, consisting of more layers that permit higher levels of abstraction and improved predictions from data. To date, it is emerging as the leading machine-learning tool in the general imaging and computer vision domains. In particular, convolutional neural networks (CNNs) have proven to be powerful tools for a broad range of computer vision tasks. Deep CNNs automatically learn mid-level and high-level abstractions obtained from raw data (e.g., images). Recent results indicate that the generic descriptors extracted from CNNs are extremely effective in object recognition and localization in natural images. Medical image analysis groups across the world are quickly entering the field and applying CNNs and other deep learning methodologies to a wide variety of applications.
Columbia University TRECVID 2007 High-Level Feature Extraction

One difficulty in the HLF task this year was changing the applied domain from news video to foreign documentary videos. Classifiers trained in prior years performed poorly if naively applied, and classifiers trained on the 2007 data alone may suffer from too few positive training samples. This year we address this new fundamental problem how to efficiently and effectively adapt models learned from an old domain to a significantly different one. Investigation of this topic complements very well the scalability issue discussed in TRECVID 2006 how to leverage the resource of a large concept detector pool (e.g., Columbia 374) to improve accuracy of individual detectors. We developed and tested a new cross-domain SVM (CDSVM) algorithm for adapting previously learned support vectors from one domain to help classification in another domain. Performance gain is obtained with almost no additional computational cost. Also, we conduct a comprehensive comparative study of the state-of-the-art SVM-based cross-domain learning methods. To further understand the underlying contributing factors, we propose an intuitive selection criterion to determine which cross-domain learning method to use for each concept. Such a prediction mechanism is important since there are a multitude of promising methods for adapting old models to new domains, and thus judicious selection is a key to applying the right method under the right context (e.g., size of training data in new/old domains, variation of content between two domains, etc). Although there is no single method that universally outperforms other options, with adequate prediction mechanisms, we will be able to apply the right adaptation approach in different conditions, and demonstrate 22% performance improvement for mid-frequency or rare concepts.
Production and Application of HPMS Recycled Bonded Permanent Magnets for a Traction Motor Application

Due to the volatility of the cost and sustainability concerns associated with the rare-earth permanent magnets, alternative product designs using less or no rare-earth contents have, recently, gained popularity. Another method to address this need is to apply a magnet recycling process, such as the novel hydrogen processing of magnetic scrap (HPMS) which can be applied to the end-of-life products such as hard drive disks. Despite the growing research on the background science of different recycling techniques, a practical make, use and evaluation of recycled magnets in a real-life application, is rarely attended. To address this gap, in this paper and for the first time, the viability of the HPMS recycled magnets for use in a permanent magnet traction motor is investigated. On this basis, a detailed description and testing of the recycling process and the magnet production for a customized traction motor design is provided. Furthermore, the behavior of the motor using the final magnet product is analyzed using simulations and prototype testing. Based on the results, the proposed recycled magnets satisfy the overall requirements, while demonstrating similar or better electromagnetic performance compared to the alternative low-cost ferrite magnets.
Reinforcement Learning Using Neural Networks, with Applications to Motor Control

This thesis is a study of practical methods to estimate value functions with feedforward neural networks in model-based reinforcement learning. Focus is placed on problems in continuous time and space, such as motor-control tasks. In this work, the continuous TD(lambda) algorithm is refined to handle situations with discontinuous states and controls, and the vario-eta algorithm is proposed as a simple but efficient method to perform gradient descent. The main contributions of this thesis are experimental successes that clearly indicate the potential of feedforward neural networks to estimate high-dimensional value functions. Linear function approximators have been often preferred in reinforcement learning, but successful value function estimations in previous works are restricted to mechanical systems with very few degrees of freedom. The method presented in this thesis was tested successfully on an original task of learning to swim by a simulated articulated robot, with 4 control variables and 12 independent state variables, which is significantly more complex than problems that have been solved with linear function approximators so far.
Investigating Intention To Use An Interactive Television Game

Besides the excellent quality of picture and sound, and the plethora of available channels, digital interactive television provides access to applications and services with the touch of a button. Interactive television games are becoming popular since television viewers are able to virtually participate in the game. The aim of the work presented in this paper is to investigate factors affecting users’ intention to use an interactive television game that is built upon a popular television game. Towards this aim, we formed a model based on the Technology Acceptance Model (TAM) as well as additional factors derived from previous research in the domains of online games and fun information systems.