Miracle-dz/UniFAR-FTUs huggingface dataset:
REINFORCEMENT LEARNING: AN INTRODUCTION by Richard S. Sutton and Andrew G. Barto, Adaptive Computation and Machine Learning series, MIT Press (Bradford Book), Cambridge, Mass., 1998, xviii + 322 pp, ISBN 0-262-19398-1, (hardback, £31.95).

Reinforcement learning, as understood by Sutton and Barto, is a fusion of the trial-and-error “law-of-effect” tradition in psychology, optimal control theory in engineering, the secondary reinforcement tradition in learning, and the use of decaying stimulus traces in, for example, Hull’s (1952) concept of a goal gradient and, more recently, Wagner’s (1981) model of conditioning. This fusion has given researchers in artiacial intelligence a number of ideas for computer algorithms that learn a policy that maximizes the agent’s long term return (amount of reward) from the performance of a task. Although many of the ideas behind reinforcement learning originated in psychological theorizing, in recent years these ideas have been most extensively developed within the artiacial learning community, particularly by the authors of this important summary, their students, and colleagues. The book is intended for use in a one-semester course for students interested in machine learning and artiacial intelligence. It would probably not be suitable for a course intended for psychology and neuroscience students, because it does not present models of experimentally established behavioral or neuroscientiac phenomena, and the problems given to illustrate how reinforcement learning algorithms may be applied are not necessarily problems that animals (even human animals) are notably good at solving (e.g., efacient scheduling problems). However, reinforcement learning, incentive, and utility remain central concepts in contemporary work on the neurobiological basis of learned, goaldirected behavior (Schultz, Dayan, & Montague, 1997; Shizgal, 1997), and this book is the place to look for the latest ideas on how these concepts may be developed into effective models for the direction of action. The preface says that the only mathematical background assumed is familiarity with elementary concepts of probability, such as expectations of random variables. Most students will feel that rather more than that is in fact assumed. Nonetheless, the material is presented in an intuitively understandable form, emphasizing the basic ideas and giving helpful illustrations of their application, rather than elaborating proofs. It can be read with proat by motivated neuroscience and psychology students interested in a more rigorous development of these psychologically important ideas. The biographical and historical sections at the end of each chapter are useful for the perspective they give. The many suggested exercises are challenging, open ended, and thought provoking.
T Cell Tolerance Induced by Cross-Reactive TCR Ligands Can Be Broken by Superagonist Resulting in Anti-Inflammatory T Cell Cytokine Production1

Cross-reactive activation of potentially autoreactive T cells by high-affinity nonself ligands may be important in breaking self-tolerance in autoimmunity. In a mouse transgenic for a cross-reactive TCR, we have previously shown that a hyperstimulating altered peptide ligand, L144, induced unresponsiveness to the self peptide, proteolipid protein 139–151. In this study, we demonstrate that a superagonist ligand can break T cell tolerance induced by the lower affinity cognate Ag. T cells tolerant to the cognate ligand, Q144, responded to superagonist, L144, by proliferation and the production of mainly IL-4 and IL-10 in vitro. In contrast, T cells that were tolerized to the superagonist were unable to respond to any peptide that cross-reacted with the transgenic TCR. Low-dose immunization with the superagonist L144 was able to break tolerance to the cognate ligand in vivo and resulted in a blunted proliferative response with production of Th2 cytokines.
Did the progenitor of SN 2011dh have a binary companion

We present late-time Hubble Space Telescope ultraviolet (UV) and optical observations of the site of SN 2011dh in the galaxy M51, ∼1164 days post-explosion. At the supernova (SN) location, we observe a point source that is visible at all wavelengths, which is significantly fainter than the spectral energy distribution (SED) of the yellow supergiant progenitor observed prior to explosion. The previously reported photometry of the progenitor is, therefore, completely unaffected by any sources that may persist at the SN location after explosion. In comparison with the previously reported late-time photometric evolution of SN 2011dh, we find that the light curve has plateaued at all wavelengths. The SED of the late-time source is clearly inconsistent with an SED of stellar origin. Although the SED is bright at UV wavelengths, there is no strong evidence that the late-time luminosity originates solely from a stellar source corresponding to the binary companion, although a partial contribution to the observed UV flux from a companion star cannot be ruled out.
A Description of the Trials, Reviews, and Practice Guidelines Indexed in the PEDro Database

This perspective provides an overview of the randomized controlled trials, systematic reviews, and evidence-based clinical practice guidelines in physical therapy. Data from the Physiotherapy Evidence Database (PEDro) are used to describe key events in the history of physical therapy research and the growth of evidence of effects of interventions used in the various subdisciplines of physical therapy. The 11,494 records that were identified reveal a rich history of physical therapy research dating back to the first trial in 1929. Most of the randomized controlled trials, systematic reviews, and evidence-based clinical practice guidelines in physical therapy have been published since the year 2000. This rapid growth presents a challenge for physical therapists who want to keep up to date in clinical practice.
Measuring the off-axis angle and the rotational movements of phonating sperm whales using a single hydrophone.

The common use of the bent-horn model of the sperm whale sound generator describes sperm whale clicks as the pulse series {p0, p1, p2, p3,. }. Clicks, however, deviate from this standard when recorded using off-axis hydrophones. The existence of additional pulses within the {p0, p1, p2, p3, . } series can be explained still using the bent-horn model. Multiple reflections on the whale's frontal and distal sacs of the p0 pulse lead to additional sets of pulses detectable using a farfield, off-axis hydrophone. The travel times of some of these additional pulses depend on the whale's orientation. The authors propose a method to estimate the off-axis angle of sperm whale clicks. They also propose a method to determine the nature of the movement (if it is pitch, yaw, or roll) of phonating sperm whales. The application of both methods requires the measurement of the travel time differences between pulses composing a sperm whale click. They lead, using a simple apparatus consisting of a single hydrophone at an unknown depth, to new measurements of the underwater movements of sperm whales. Using these methods shows that sperm whales would methodically scan seawater while searching for prey, by making periodic pitch and yaw movements in sync with their acoustic activity.
Spinal afferents to the lateral cervical nucleus in the cat. An experimental study

I n a previous paper (Rexed and Brodal, '51) we drew attention to and described a hitherto neglected nucleus, forming a longitudinal cell column ventrolateral to the dorsal horn in the lateral funiculus of the first and second cervical segments of the eat's spinal cord. This nucleus which must be kept apart from the reticular formation of the spinal cord as f a r as can be judged from its cellular structure was named by us the lateral cervical nucleus. The reasons for choosing this name were given. Evidence was brought forward that the nucleus sends its fibers to the cerebellum and that it receives afferent fibers ascending in the spinal cord, but the material then available did not permit any closer analysis of its connections. The present paper deals with the results of experimental studies undertaken to elucidate more fully the origin and course of these spinal afferents, in an attempt to obtain some information of the functional' significance of the lateral cervical nucleus. The results of concomitant electrophysiological studies by Rexed and Str6m are published separately.
Barrier thickness and hydrostatic pressure effects on hydrogenic impurity states in wurtzite GaN/AAl x Ga 1− x N strained quantum dots

Within the framework of the effective mass approximation, barrier thickness and hydrostatic pressure effects on the ground-state binding energy of hydrogenic impurity are investigated in wurtzite (WZ) GaN/AlxGa1-xN strained quantum dots (QDs) by means of a variational approach. The hydrostatic pressure dependence of physical parameters such as electron effective mass, energy band gaps, lattice constants, and dielectric constants is considered in the calculations. Numerical results show that the donor binding energy for any impurity position increases when the hydrostatic pressure increases. The donor binding energy for the impurity located at the central of the QD increases firstly and then begins to drop quickly with the decrease of QD radius (height) in strong built-in electric fields. Moreover, the influence of barrier thickness along the QD growth direction and Al concentration on donor binding energy is also investigated. In addition, we also found that impurity positions have great influence on the donor binding energy.
5/6 Nephrectomy as a Validated Rat Model Mimicking Human Warfarin-Related Nephropathy

Background/Aims: We previously reported that patients with chronic kidney disease (CKD) receiving warfarin therapy and whose international normalized ratio increases to >3.0 may develop acute kidney injury (AKI) as a result of glomerular hemorrhage and formation of obstructive red blood cell (RBC) casts. We named this condition warfarin-related nephropathy (WRN). We also previously reported that acute excessive anticoagulation with brodifacoum (superwarfarin) induces AKI in 5/6 nephrectomy (5/6NE) rats. Limitations of the brodifacoum model precluded a careful assessment of dose-response relationships. Methods: Warfarin treatment was used in 5/6NE. Results: Herein we report that warfarin treatment of 5/6NE rats resulted in a dose-dependent increase in serum creatinine (SC). The increase in SC following warfarin treatment was greater at 3 and 19 weeks after the ablative surgery, than that observed 8 weeks after the ablative surgery. The SC increase was correlated with the prothrombin time increase. Morphologically, 5/6NE, but not control rats, had acute tubular injury with RBC and RBC casts in the tubules. Treatment with vitamin K prevented SC increase and morphologic changes in the kidney associated with warfarin treatment. A single episode of WRN did not affect the progression of CKD in 5/6NE. Conclusion: (1) The 5/6NE model of CKD is an appropriate animal model to study the pathogenesis of WRN. (2) The pharmacokinetics of warfarin is better suited to the study of WRN than that of brodifacoum. (3) The more advanced stages of 5/6NE are more susceptible to WRN than the earlier stages. (4) Vitamin K treatment prevents WRN.
Renal Replacement Therapy for Acute Kidney Injury: Let's follow the Evidence

Selection of a dialysis modality for critically ill patients with acute kidney injury (AKI) continues to be a controversial topic. Available studies can be divided into: 1) randomized controlled trials assessing the effect of treatment dose on patient outcome; 2) randomized controlled outcome comparisons of CRRT and IRRT; and 3) observational outcome comparisons of CRRT and IRRT Clinical outcome comparisons of CRRT and HD continue to be very problematic. With the availability of the recent CRRT dose/outcome data, the interpretation of CRRT vs HD trial data now becomes nearly impossible because of the low-dose “handicap” that has existed in all of the comparative trials in the CRRT arms. Randomized and observational outcome studies suffer from several methodological problems and a final statement cannot be made. In the end, CRRT and HD are vastly different therapies that are not mutually exclusive but rather are complementary - they provide different options for patients at variable stages of their AKI course. Very preliminary data from SLED studies suggest this therapy also can be part of the continuum of AKI dialytic modalities. However, the lack of any meaningful outcome data for this therapy invalidates any claims of clinical “equivalence” to the other established modalities.
Bounding Interference in Wireless Ad Hoc Networks With Nodes in Random Position

Given a set of positions for wireless nodes, the interference minimization problem is to assign a transmission radius (i.e., a power level) to each node such that the resulting communication graph is connected while minimizing the maximum (respectively, average) interference. We consider the model introduced by von Rickenbach (2005), in which each wireless node is represented by a point in Euclidean space on which is centered a transmission range represented by a ball, and edges in the corresponding graph are symmetric. The problem is NP-complete in two or more dimensions (Buchin 2008), and no polynomial-time approximation algorithm is known. We show how to solve the problem efficiently in settings typical for wireless ad hoc networks. If nodes are represented by a set P of n points selected uniformly and independently at random over a d-dimensional rectangular region, then the topology given by the closure of the Euclidean minimum spanning tree of P has O(log n) maximum interference with high probability and O(1) expected interference. We extend the first bound to a general class of communication graphs over a broad set of probability distributions. We present a local algorithm that constructs a graph from this class; this is the first local algorithm to provide an upper bound on expected maximum interference. Finally, we disprove a conjecture of Devroye and Morin (2012) relating the maximum interference of the Euclidean minimum spanning tree to the optimal maximum interference attainable.
Continuous Subcutaneous Insulin Infusion Therapy with Rapid-ActingInsulin Analogs in Insulin Pumps: Does it Work, How Does it Work, andwhat Therapies Work Better than Others?

Many patients are not optimally controlled on conventional insulin regimens. This review evaluates the practicalities and clinical success of continuous subcutaneous insulin infusion therapy with rapid-acting insulin analogs in insulin pumps in patients with type 1 and type 2 diabetes. In contrast to a multiple daily injection regimen, CSII provides patients with greater flexibility in the timing of meals and insulin dosing, resulting in improved quality of life and greater treatment adherence. CSII therapy with rapid-acting insulin analogs offers reliable glycemic control and proactive response to glucose variability, with a reduced risk of weight gain and hypoglycemia compared with regular human insulin in CSII. While the clinical benefits of CSII versus multiple daily injection therapy in patients with type 1 diabetes are quite evident in studies in appropriately-selected patients, the benefits are more equal in patients with type 2 diabetes. Appropriate patient selection for CSII therapy may help ensure successful outcomes in diabetes treatment. Patients who are very poorly controlled on multiple daily injections may benefit from CSII therapy, with the greatest glycemic improvement associated with high baseline HbA1c levels. CSII therapy may be of particular benefit to patients with type 1 diabetes who are prone to hypoglycemia and patients with type 2 diabetes who are obese and uncontrolled on high doses of insulin; however, most important as a predictor of success is that patients desire CSII therapy.
BugRedux: Reproducing field failures for in-house debugging

A recent survey conducted among developers of the Apache, Eclipse, and Mozilla projects showed that the ability to recreate field failures is considered of fundamental importance when investigating bug reports. Unfortunately, the information typically contained in a bug report, such as memory dumps or call stacks, is usually insufficient for recreating the problem. Even more advanced approaches for gathering field data and help in-house debugging tend to collect either too little information, and be ineffective, or too much information, and be inefficient. To address these issues, we present BugRedux, a novel general approach for in-house debugging of field failures. BugRedux aims to synthesize, using execution data collected in the field, executions that mimic the observed field failures. We define several instances of BugRedux that collect different types of execution data and perform, through an empirical study, a cost-benefit analysis of the approach and its variations. In the study, we apply BugRedux to 16 failures of 14 real-world programs. Our results are promising in that they show that it is possible to synthesize in-house executions that reproduce failures observed in the field using a suitable set of execution data.
Channel quantizers that maximize random coding exponents for binary-input memoryless channels

The problem of finding the optimum output quantizer for a given discrete memoryless channel is investigated, where the quantizer output has fewer values than the channel output. While mutual information has received attention as an objective function for optimization, the focus of this paper is use of the random coding exponent, which was originally derived by Gallager, as criteria. Two problems are addressed, where one problem is a partial problem of the other. The main result is a quantizer design algorithm, and a proof that it finds the optimum quantizer in the partial problem. The quantizer design algorithm is based on a dynamic programming approach, and is an extension of a mutual-information maximization method. For the binary-input case, it is shown that the optimum quantizer can be found with complexity that is polynomial in the number of channel outputs.
Effect of Respiratory Rehabilitation Before Open Cardiac Surgery on Respiratory Function: A Randomized Clinical Trial

Introduction: Prevention of pulmonary complications after coronary artery bypass graft is attended as a very important issue. The aim of this study was to evaluate the role of pulmonary rehabilitation before surgery for reducing the risk of pulmonary complications after surgery. Methods: In a randomized clinical trial, 60 patients undergoing heart surgery were randomly divided into two groups A and B. Chest physiotherapy was performed before and after surgery on group A patients however it was done on group B’s, only after surgery. Effects of preoperative pulmonary rehabilitation were compared between two groups, using spirometry and arterial blood gas (ABG). Results: Thirty nine males (65%) and 21 females (35%) with mean age of 8.10 ± 9.56 were analyzed. The mean differences were statistically significant for predicted forced vital capacity (FVC) (CI 95%:1.3 to 8.7) and Predicted Peak Flow indices (PEF) (CI 95%: 1.9 to 9.4) of spirometry indicator, PCO2 index (of ABG parameter) (CI 95%: 1.4 to 8.9) and mean oxygen saturation (mean Spo2) (CI 95%: 0.6 to 1.7) of ABG index in two groups. Conclusion: The performance of pulmonary rehabilitation program before surgery is recommended, as it may result in the reduction of complications of heart surgery
Temperature effects on the forces, moments and moment to force ratio of nickel-titanium and TMA symmetrical T-loops.

OBJECTIVE To test the null hypothesis that temperature has no effect on the force, moment, and moment to force ratio (M:F) of nickel-titanium (NiTi) and titanium molybdenum alloy (TMA) T-loops. MATERIALS AND METHODS Twelve symmetrical T-loops were constructed from NiTi and TMA rectangular wires (N = 24). For each material, four specimens had 30 degrees preactivation bent into the legs, four had 15 degrees , and the remainder no preactivation. The specimens were mounted on a custom stand incorporating a force and a moment transducer housed in an insulated oven in which the temperature was varied from 10 degrees C to 50 degrees C in 10 degrees increments (+/-0.5 degrees C). The loops were opened in 1-mm increments to a maximum of 8 mm and allowed to return to the rest position. Forces and moments were recorded at each interval; from this, the M:F was calculated. Mixed-model statistical analysis was used to detect differences between mean results of material type, preactivation, and temperature. RESULTS Temperature significantly influenced the forces and moments (P < .000) produced by NiTi closing loops, with values increasing as the temperature increased. The M:F ratios of NiTi loops were less affected, with no significant changes with temperature for the 15 degrees and 30 degrees preactivation loops, although some change was noted for the non-preactivated loops. TMA wires showed significance for some force measurements, but were generally not influenced by temperature. CONCLUSION The hypothesis is rejected. Temperature significantly affected the forces and moments produced by NiTi T-loops, though these changes generally do not affect the overall M:F ratio.
Soluble Guanylate Cyclase a1–Deficient Mice: a novel murine model for Primary Open Angle Glaucoma

Glaucoma is an optic neuropathy characterized by retinal ganglion cell degeneration leading to vision loss. Primary open angle glaucoma is a subtype of glaucoma that results in visual field loss due to optic nerve damage caused by increase in intraocular pressure. Events in primary open angle glaucoma (POAG) pathogenesis has been associated with nitric oxide (NO) which activates the soluble guanylate cyclase (sGC), a heterodimeric enzyme consisting of α and β subunits by c GMP signaling. 1-2 In POAG patients NO metabolites and level of c GMP has been found to be decreased in aqueous humor. Impaired signaling can be a contributing risk factor to the etiology of the POAG.3 However whether signal impairment can result in POAG and mechanism underlying is yet to be elucidated. doi : 10.5214/ans.0972. 7531.200207
Direct Feature Visualization Using Morse-Smale Complexes

In this paper, we characterize the range of features that can be extracted from an Morse-Smale complex and describe a unified query language to extract them. We provide a visual dictionary to guide users when defining features in terms of these queries. We demonstrate our topology-rich visualization pipeline in a tool that interactively queries the MS complex to extract features at multiple resolutions, assigns rendering attributes, and combines traditional volume visualization with the extracted features. The flexibility and power of this approach is illustrated with examples showing novel features.