Miracle-dz/UniFAR-FTUs huggingface dataset:
Interrelating Prediction and Control Objectives in Episodic Actor-Critic Reinforcement Learning

The reinforcement learning framework provides a simple way to study computational intelligence as the interaction between an agent and an environment. The goal of an agent is to accrue as much reward as possible by intelligently choosing actions given states. This problem of finding a policy that maximizes the expected total reward is called the control problem. Many algorithms that solve the control problem also solve a related problem called the prediction problem. The goal in prediction is to accurately estimate the expected total reward from different states while following a given policy. Both the prediction and control problems can be formulated as optimization problems with different objectives. In this thesis, we present the first attempt at interrelating the prediction and control objectives. Prediction has often been studied independently in the past, but it is a subproblem subservient to the primary control problem that we generally care about. There is a need for interrelating the objectives particularly when the number of states is large and agents cannot learn about all states equally well. Agents are forced to use function approximation, and we have to specify how to allocate function approximation resources. The Emphatic Temporal Difference algorithm is the first prediction algorithm that allows a specification of a degree of caring about value function accuracy at different states. This specification is done through an interest function that changes the prediction objective. We now have a way to communicate to the agent how much we care about different states beyond how ii often they occur. However, interest is strictly a problem-related concept, defining the objective, in prediction. There were no clear insights about how to set the interest. We take this opportunity to use interest as a solution-related concept in control and ground the choice of prediction objective in improving control performance. As a running example, we use the episodic discounted control problem where distal rewards are worth exponentially less, and interaction terminates upon reaching a terminal state. In this problem setting, we study the actor-critic class of solution methods that neatly separate the two problems with the actor solving the control problem and the critic solving the prediction subproblem. First, we discuss a recent controversy in discounting and the role it should play in the learning updates of the actor. After concluding how we should update the actor, we move on to the critic. We find the first suggestion of an interest over states for the prediction problem by analyzing the updates made by the actor and hence present a choice of prediction objective motivated by a control objective. Experimental results confirm that control performance is indeed improved when the critic uses the new prediction objective.
EPR studies on telechelic polymers : Characterization of ion multiplets

Endfunctionalized polyisoprenes (PI) and polystyrenes (PS) with either one sulfonate or ammonium end group and α,ω-macrozwitterionic PI and PS have been synthesized and investigated by CW-EPR using the spin probe technique. From the extrema separation of the temperature dependent spectra a characteristic temperature, T‘50G, is introduced which can be employed as a measure for the temperature dependence of the chain end association in these telechelic ionomers. In this way it is shown that the mobility of an ionic 2,2,6,6-tetramethylpiperidine-N-oxyl derivative is dramatically restricted when compared to the corresponding nonionic probe. For PI the reduction in probe mobility corresponds to temperature shifts as high as 140 K relative to the dynamics of the polymer, indicating the presence of highly immobilized ion multiplets. As shown by the temperature dependent data, the method is sensitive to additional immobilization of the ionic domains induced by bulky substituents adjacent to the ionic chain ends.
Detection of Cardiac Arrhythmias From Varied Length Multichannel Electrocardiogram Recordings Using Deep Convolutional Neural Networks

Automatic identification of different arrhythmias helps cardiologists better diagnose patients with cardiovascular diseases. Deep learning algorithms are used for the classification of multichannel ECG signals into different heart rhythms. The study dataset includes a cohort of 43101 12- lead ECG recordings with various lengths. Two options are tested to standardize the recordings length: zero padding and signal repetition. Downsampling the recordings to 100 Hz allow handling the problem of different sampling frequencies of data coming from different sources. We design a deep one-dimensional convolutional neural network (CNN) called VGG-ECG, a 13-layer fully CNN for multilabel classification. Our team is called MIndS and our approach achieved a challenge validation score of 0.368, and full test score of -0.128, placing us 38 out of 41 in the official ranking.
Identifying Ectopic Pregnancy in a Large Integrated Health Care Delivery System: Algorithm Validation

Background Surveillance of ectopic pregnancy (EP) using electronic databases is important. To our knowledge, no published study has assessed the validity of EP case ascertainment using electronic health records. Objective We aimed to assess the validity of an enhanced version of a previously validated algorithm, which used a combination of encounters with EP-related diagnostic/procedure codes and methotrexate injections. Methods Medical records of 500 women aged 15-44 years with membership at Kaiser Permanente Southern and Northern California between 2009 and 2018 and a potential EP were randomly selected for chart review, and true cases were identified. The enhanced algorithm included diagnostic/procedure codes from the International Classification of Diseases, Tenth Revision, used telephone appointment visits, and excluded cases with only abdominal EP diagnosis codes. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and overall performance (Youden index and F-score) of the algorithm were evaluated and compared to the validated algorithm. Results There were 334 true positive and 166 true negative EP cases with available records. True positive and true negative EP cases did not differ significantly according to maternal age, race/ethnicity, and smoking status. EP cases with only one encounter and non-tubal EPs were more likely to be misclassified. The sensitivity, specificity, PPV, and NPV of the enhanced algorithm for EP were 97.6%, 84.9%, 92.9%, and 94.6%, respectively. The Youden index and F-score were 82.5% and 95.2%, respectively. The sensitivity and NPV were lower for the previously published algorithm at 94.3% and 88.1%, respectively. The sensitivity of surgical procedure codes from electronic chart abstraction to correctly identify surgical management was 91.9%. The overall accuracy, defined as the percentage of EP cases with correct management (surgical, medical, and unclassified) identified by electronic chart abstraction, was 92.3%. Conclusions The performance of the enhanced algorithm for EP case ascertainment in integrated health care databases is adequate to allow for use in future epidemiological studies. Use of this algorithm will likely result in better capture of true EP cases than the previously validated algorithm.
Changes in the Amino Acid Composition of Decomposing Plant Materials in Soil: Species and Depth Effects

Abstract This project studied the amino acid composition of samples taken from an experiment of in situ incubation of plant debris, mixed with mineral earth, to identify changes during decomposition. Medicago sativa debris, and Eucalyptus globulus and Pinus halepensis ground litter, were mixed with a OM‐poor mineral earth, and buried in the soil at 5, 20, and 40 cm depth, for 2 years. The total amino acid composition of the final samples was compared with that of the initial ones. The amino acid composition of the several initial mixtures was quite similar. Also, the differences between initial and incubated samples were small, but still large enough to detect consistent changes. The pattern of change was consistent for only nine amino acids: Gly consistently increased, whereas Lys, His, Leu, and (to a lesser extent) Arg, Tyr, Phe, Ser, and Tau consistently decreased. The ratio Gly/(Lys+His+Leu+Arg+Tyr+Phe+Ser) is taken as the amino acid signature (AaS). This ratio increased from about 0.30–0.35 in the undecomposed mixtures, to about 0.55–0.60 after 2 years of field decomposition, and the comparison with data from other authors suggests that it could be a useful indicator of the degree of biochemical evolution of nitrogen compounds in soil samples. In contrast, changes in the traditional groups of amino acids, i.e., acid, basic, neutral, and sulfur (S)‐containing, were hardly detectable.
Zur Verfestigung von Schüttgütern – Mikroprozesse und Kinetikmodelle

Storage tanks are important elements of a self-operating closed processing system. The reliability and availability of storage equipment essentially depends on the flow behaviour of particulate solids in storage containers. A wide residence time distribution (i.e. too long a storage time at rest) in silos, bunkers or transportation containers, respectively, can lead to the hazardous problem of so-called time consolidation of particulate solids. During this hardening process, solid bridges are forming with resulting solidification and solid properties of bulk material. In principle, there are four main physico-chemical effects in bulk materials storing and handling which can produce solid briding between the particle contacts due to crystallisation, chemical reactions, solidification of high-viscous bonding agents and sintering. Generally, new adhesion force based models are presented to describe the consolidation kinetics of particulate solids. Preliminary solutions of kinetic model equations are discussed and compared with new test results and practical conclusions are drawn concerning the reliable processing, storage and transportation of bulk materials.
Friction of Gels. 3. Friction on Solid Surfaces

Frictions of several kinds of hydrogels, poly(vinyl alcohol) gel, gellan gel, poly(2-acrylamido-2-methylpropanesulfonic acid) gel, and its sodium salt gel, sliding on glass plates as well as on Teflon plates have been investigated both in air and in water. The frictional force and its dependencies on the load are quite different depending on the chemical structures of the gels, surface properties of the opposing substrates, and the measurement condition. The gel friction is explained in terms of interface interaction, either attractive or repulsive, between the polymer chain and the solid surface. Surface adhesion between glass particles and gels measured by atomic force microscopy showed a good correlation with the friction, which supports the polymer repulsion−adsorption model proposed by authors.
A Theory of Cooperative Phenomena. III. Detailed Discussions of the Cluster Variation Method

A series of approximations for the statistical mechanics of order‐disorder, proposed by Bethe, Takagi, Yang‐Li‐Hill, Kikuchi, and others, are investigated in detail in two ways. (1) A new interpretation of the method for constructing the combinatory factor is presented in order to give a better understanding of the nature of approximations. (2) The partition functions with approximate combinatory factors are expanded to compare with the rigorous expansion and the discrepancies between them are investigated in detail. One of the conclusions is that in order to obtain a higher approximation, it is necessary to use the basic figure ``closed'' with respect to the cluster of the preceding approximation. In appendices, an improved treatment of the body‐centered cubic lattice (Ising model) is given, and Bethe's fundamental assumptions are derived from our scheme.
The De-Biased Whittle Likelihood

The Whittle likelihood is a widely used and computationally efficient pseudo-likelihood. However, it is known to produce biased parameter estimates for large classes of models. We propose a method for de-biasing Whittle estimates for second-order stationary stochastic processes. The de-biased Whittle likelihood can be computed in the same $\mathcal{O}(n\log n)$ operations as the standard approach. We demonstrate the superior performance of the method in simulation studies and in application to a large-scale oceanographic dataset, where in both cases the de-biased approach reduces bias by up to two orders of magnitude, achieving estimates that are close to exact maximum likelihood, at a fraction of the computational cost. We prove that the method yields estimates that are consistent at an optimal convergence rate of $n^{-1/2}$, under weaker assumptions than standard theory, where we do not require that the power spectral density is continuous in frequency. We describe how the method can be easily combined with standard methods of bias reduction, such as tapering and differencing, to further reduce bias in parameter estimates.
Direct copolycondensation of biobased elastomers based on lactic acid with tunable and versatile properties

Novel poly(lactate/butanediol/sebacate/itaconate) (PLBSI) copolyesters with sustainability and biocompatibility were synthesized by using direct melting polycondensation. The molecular weights, chemical structures, and compositions of the copolyesters were confirmed by GPC, NMR, and FTIR. By controlling the content of lactic acid, the copolyesters were transformed from rigid plastics to flexible elastomers. The PLBSI elastomers were highly reinforced with nanosilica which can meet the tensile strength requirements for the majority of rubber products. More importantly, the PLBSI elastomers can be used as PLA tougheners owing to the similar lactate structures of PLA and PLBSI. The toughened PLA can be applied as excellent three-dimensional printing ink. Besides, the crosslinked PLBSI plastics were demonstrated to be promising shape memory polymers with excellent shape fixity and recovery rate. The renewable monomers, relatively simple polymerization, and versatile performance make our PLBSI copolyesters promising for various medical and engineering applications.
Cone beam computed tomography evaluation of variations in the sella turcica in a Turkish population.

Background : The aim of the study was to analyse the variations of the sella turcica and their relationship to age and gender. Materials and methods: The cone beam computed tomography (CBCT) images of the patients who referred to Gaziantep University, Faculty of Dentistry, Department of Dentomaxillofacial Radiology were analysed by two examiners and 200 CBCT images were evaluated retrospectively. Sella turcica shapes of individuals were described according to Axelsson’s classification as either normal or with aberrations including: oblique anterior wall, double contour of the floor, sella turcica bridging, irregularity in the posterior part of the dorsum sella, and pyramidal shape of the dorsum sella. Results: Normal sella turcica was the most common type seen in 100 (49.8%) individuals. The secondary common type was double contour of the floor of sella turcica seen in 46 (22.9%) patients. Thirty-two (15.9%) patients had oblique anterior wall. Seventeen (8.5%) individuals had pyramidal shape of the dorsum sella, 3% irregularity in the posterior part of the sella turcica. None of the patients had sella turcica bridging. Irregularity in the posterior part of the sella turcica and pyramidal shape of the dorsum sella were reported significantly more frequent in female than male. Conclusions: Sella turcica varies in shape and morphology regardless of age. Normal sella turcica was the most frequent type in the study group. Irregularity in the posterior part of the sella turcica and pyramidal shape of the dorsum sella were reported more frequent in female than male.
Detection and Classification of Cardiac Arrhythmias by Machine Learning: a Systematic Review

Machine learning (ML) techniques can perform as better as humans at key healthcare tasks. Recent advances make it possible to perform, using ML, automatic high-level feature extraction and classification of cardiac arrhythmia. In this work, we aimed through a systematic literature review to identify the principal methods, databases, and contributions of ML on cardiac arrhythmias classification. Electronic database including PubMed, Science Direct, IEEE, Scielo, Scopus, and Web of Science were searched, from 2014 to 2019, by combining the following keywords “ECG”, “heart signals”, “ar-rhythmia”, “classification” and “machine learning”. 28 studies were selected as eligible. Classifications classes ranged from 2 to 17, with prevalence of 2 classes (71.4% of the studies). The most frequent applied methods were Artificial Neural Network (13 articles), followed by Support Vector Machines and Mixed techniques (5 articles respectively). MIT-BIH Arrhythmia Database was used in 15 studies (54%), whereas 8 (28.5%) utilized their own data. The approach basis for evaluating the results is the confusion matrix, where up to 82% of the studies used accuracy, 67.8% precision, and 46% sensitivity/specificity. Classification of cardiac arrhythmias through ECG is of increasing interest from the research groups, and ML classification is showing rising levels of performance. It would benefit both patients and clinicians.
Convolutional Recurrent Neural Network and LightGBM Ensemble Model for 12-lead ECG Classification

Automatic abnormality detection of ECG signals is a challenging topic of great research and commercial interest. It can provide a cost-effective and accessible tool for early and accurate diagnosis, which increases the chances of successful treatment. In this study, an ensemble classifier that identifies 24 types of cardiac abnormalities is proposed, as part of the PhysioNet/Computing in Cardiology Challenge 2020. The ensemble model consists of a convolutional recurrent neural network that is able to automatically learn deep features, and LightGBM, a gradient boosting machine that relies on hand-engineered expert features. The individual models are combined using class-specific weights and thresholds, which are tuned by a genetic algorithm. Results from 5-fold cross validation on the full training set, report the Challenge metric of 0.593 that outperforms both individual models. On the full hidden test set, the proposed architecture by “AUTh Team” achieves a score of 0.281 with an official ranking of 13/41.
Sentiment Analysis and Deep Reinforcement Learning for Algorithmic Trading

In algorithmic trading, we buy/sell stocks using computers automatically. While high frequency algorithmic trading is pretty common in ﬁnancial market, we focus on long-term algorithmic trading based on historical stock price and news/tweets. To make the problem tractable, we model the transaction as a sequential decision making problem. The transaction period is one day. The return is calculated using the close price plus a ﬁxed rate of commission. In the midterm report, we implement three baselines models: RL using price-only observation, RL using news-only observation and pretrained sentiment classiﬁer plus manually designed policy. We implement the sentiment classiﬁer as a 2-layer convolutional nets using pretrained Bert embedding. We test our idea on Dow Jones Industrial Average (DJIA) and experiments show that all the methods overﬁt on training dataset and the performance on testing data amounts to random guessing. Our code is open source at https://github.com/ vermouth1992/CSCI699ml4know/ tree/master/project .
STRUCTURE OF UNIFORM HARD-SPHERE FLUID : A DENSITY FUNCTIONAL APPROACH

A simple weighted-density approximation (SWDA) based on both local average and bulk densities is used to investigate the equilibrium structure of a uniform hard-sphere fluid. The main advantage of SWDA is computationally much simpler than the WDA of Tarazona based on the local density. The scheme which was pointed out by Percus has been used to interconnect between the structure of a uniform fluid and that of its nonuniform counterpart. The weighting function of the uniform hard-sphere fluid which is the requirement input is taken from the free energy functional approximation based on the truncated density expansion. The calculated results for structure-related functions such as the radial distribution function, cavity function, and cavity function are found to be in good agreement with the computer simulations. Comparisons with other approximations show that the SWDA results are a significant improvement upon those of the Percus–Yevick approximation and of the WDA of Denton–Ashcroft [A. R. Denton and N.
Camera Image Based Method of Real Time Gaze Detection and Interaction

- There are many possible applications for tracking the gaze of a person. For example, it can be used to analyze which areas of a website are attracting the most attention in order to optimize its layout. Another possible application is creating a barrier-free computer interaction for people with a limited ability to move. Some companies are specialized in gaze tracking technologies and offer different solutions. The downside of those solutions is, that they require special hardware and therefore are not affordable for everyone. This paper presents a method using only one camera for gaze tracking. It is based completely on free accessible software and only needs an ordinary camera, which is integrated into almost every laptop or can be connected as external hardware. The target is to proof that it is possible to obtain a usable gaze interaction while using simple hardware and algorithms. By this method it is possible to achieve resolutions of up to 17 columns and 15 rows, which is enough for special applications.
Surface characterization and depth profiling of biological molecules by electrospray droplet impact/SIMS

Electrospray droplet impact (EDI) was applied to the analysis of peptides. The etching rate of bradykinin was estimated to be ~2 nm/min. This value is about one order of magnitude greater than the etching rate for SiO2 (0.2 nm/min). Considering that the etching rate of argon cluster ions Ar700+ for organic compounds is more than two orders of magnitude larger than that for inorganic materials, the rather small difference in etching rates of EDI for organic and inorganic materials is unique. When water/ethanol (1/1, vol%) solution of gramicidin S and arginine was dried in air, [gramicidin S + H]+ was observed as a predominant signal with little [Arg + H]+ right after the EDI irradiation, indicating that EDI is capable of detecting the analytes enriched on the sample surface. Copyright © 2011 John Wiley & Sons, Ltd.
Monte Carlo simulation of liquid–liquid benzene–water interface

A Monte Carlo simulation of liquid–liquid benzene–water interface is reported. Molecular pair potentials obtained from quantum mechanical calculations have been used. The two phases remain stable over 70 000 configurations/molecule and the interior of each phase is shown to have bulk properties. The intrinsic interface is molecularly sharp, but the interfacial region is broadened by capillary waves. The preferential water–benzene orientation at the interface is similar to that in a dilute aqueous solution of benzene. There is no longitudinal water ordering induced by the interface. The presence of the nonpolar phase gives an alignment of water dipoles parallel to the surface and a reduced probability of parallel orientation of adjacent dipoles. The hydrogen bonds between water neighbors are reinforced at the interface.