LLukas22/scidocs huggingface dataset:
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition

This paper presents a new approach to hierarchical reinforcement learning based on decomposing the target Markov decision process (MDP) into a hierarchy of smaller MDPs and decomposing the value function of the target MDP into an additive combination of the value functions of the smaller MDPs. The decomposition, known as the MAXQ decomposition, has both a procedural semantics|as a subroutine hierarchy|and a declarative semantics|as a representation of the value function of a hierarchical policy. MAXQ uni es and extends previous work on hierarchical reinforcement learning by Singh, Kaelbling, and Dayan and Hinton. It is based on the assumption that the programmer can identify useful subgoals and de ne subtasks that achieve these subgoals. By de ning such subgoals, the programmer constrains the set of policies that need to be considered during reinforcement learning. The MAXQ value function decomposition can represent the value function of any policy that is consistent with the given hierarchy. The decomposition also creates opportunities to exploit state abstractions, so that individual MDPs within the hierarchy can ignore large parts of the state space. This is important for the practical application of the method. This paper de nes the MAXQ hierarchy, proves formal results on its representational power, and establishes ve conditions for the safe use of state abstractions. The paper presents an online model-free learning algorithm, MAXQ-Q, and proves that it converges with probability 1 to a kind of locally-optimal policy known as a recursively optimal policy, even in the presence of the ve kinds of state abstraction. The paper evaluates the MAXQ representation and MAXQ-Q through a series of experiments in three domains and shows experimentally that MAXQ-Q (with state abstractions) converges to a recursively optimal policy much faster than at Q learning. The fact that MAXQ learns a representation of the value function has an important bene t: it makes it possible to compute and execute an improved, non-hierarchical policy via a procedure similar to the policy improvement step of policy iteration. The paper demonstrates the e ectiveness of this non-hierarchical execution experimentally. Finally, the paper concludes with a comparison to related work and a discussion of the design tradeo s in hierarchical reinforcement learning. c 2000 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.
Modelling Radiocesium in Lakes and Coastal Areas: New Approaches for Ecosystem Modellers

Prologue. 1. Modelling radiocesium in lakes. 2. Modelling radiocesium in coastal areas. 3. Epilogue. 4. Literature references. 5. Appendix. Subject index.
Primipaternity and duration of exposure to sperm antigens as risk factors for pre-eclampsia.

OBJECTIVE
To establish whether primipaternity and duration of unprotected sexual cohabitation is associated with an increased risk of pre-eclampsia.


METHOD
At a tertiary referral center, the study had a case and control group of 60 multigravid women each, as well as a case and control group of 50 primigravid women each. Information was compiled by means of a confidential questionnaire.


RESULT
After multiple logistic regression analysis using age, smoking, hypertension in previous pregnancies, change of paternity and duration of unprotected sexual cohabitation as predictors, the regression coefficients for change of paternity and sexual cohabitation of longer than 6 months in multigravid women were -0.4 (P = 0.15) and -1.4 (P = 0.03), respectively.


CONCLUSION
Multigravid women with a period of unprotected sexual cohabitation of longer than 6 months had a decreased risk of pre-eclampsia. Primipaternity was not a significant risk factor for pre-eclampsia.
Unramified extensions and geometric $\mathbb{Z}_p$-extensions of global function fields

We study on finite unramified extensions of global function fields (function fields of one valuable over a finite field). We show two results. One is an extension of Perret's result about the ideal class group problem. Another is a construction of a geometric $\mathbb{Z}_p$-extension which has a certain property.
A Beginner ’ s Course in Legal Translation : the Case of Culture-bound Terms

The translation course consists of an introductory lecture followed by twelve hours of seminars held over one semester. It is complemented by a nine-hour course that presents an overview of the English legal system and some aspects of comparative law. Authentic English-language documents (contracts, guarantees, judgments, summonses etc.) are studied and subsequently used as models for translation into English.
Leg orientation as a clinical sign for pusher syndrome

BACKGROUND
Effective control of (upright) body posture requires a proper representation of body orientation. Stroke patients with pusher syndrome were shown to suffer from severely disturbed perception of own body orientation. They experience their body as oriented 'upright' when actually tilted by nearly 20 degrees to the ipsilesional side. Thus, it can be expected that postural control mechanisms are impaired accordingly in these patients. Our aim was to investigate pusher patients' spontaneous postural responses of the non-paretic leg and of the head during passive body tilt.


METHODS
A sideways tilting motion was applied to the trunk of the subject in the roll plane. Stroke patients with pusher syndrome were compared to stroke patients not showing pushing behaviour, patients with acute unilateral vestibular loss, and non brain damaged subjects.


RESULTS
Compared to all groups without pushing behaviour, the non-paretic leg of the pusher patients showed a constant ipsiversive tilt across the whole tilt range for an amount which was observed in the non-pusher subjects when they were tilted for about 15 degrees into the ipsiversive direction.


CONCLUSION
The observation that patients with acute unilateral vestibular loss showed no alterations of leg posture indicates that disturbed vestibular afferences alone are not responsible for the disordered leg responses seen in pusher patients. Our results may suggest that in pusher patients a representation of body orientation is disturbed that drives both conscious perception of body orientation and spontaneous postural adjustment of the non-paretic leg in the roll plane. The investigation of the pusher patients' leg-to-trunk orientation thus could serve as an additional bedside tool to detect pusher syndrome in acute stroke patients.
Patent document categorization based on semantic structural information

The number of patent documents is currently rising rapidly worldwide, creating the need for an automatic categorization system to replace time-consuming and labor-intensive manual categorization. Because accurate patent classification is crucial to search for relevant existing patents in a certain field, patent categorization is a very important and useful field. As patent documents are structural documents with their own characteristics distinguished from general documents, these unique traits should be considered in the patent categorization process. In this paper, we categorize Japanese patent documents automatically, focusing on their characteristics: patents are structured by claims, purposes, effects, embodiments of the invention, and so on. We propose a patent document categorization method that uses the k-NN (k-Nearest Neighbour) approach. In order to retrieve similar documents from a training document set, some specific components to denote the socalled semantic elements, such as claim, purpose, and application field, are compared instead of the whole texts. Because those specific components are identified by various user-defined tags, first all of the components are clustered into several semantic elements. Such semantically clustered structural components are the basic features of patent categorization. We can achieve a 74% improvement of categorization performance over a baseline system that does not use the structural information of the patent. 2007 Published by Elsevier Ltd.
Relationship between oxidative stress, semen characteristics, and clinical diagnosis in men undergoing infertility investigation.

OBJECTIVE
To determine whether particular semen characteristics in various clinical diagnoses of infertility are associated with high oxidative stress and whether any group of infertile men is more likely to have high seminal oxidative stress. Reactive oxygen species (ROS) play an important role in sperm physiological functions, but elevated levels of ROS or oxidative stress are related to male infertility.


DESIGN
Measurement of sperm concentration, motility, morphology, seminal ROS, and total antioxidant capacity (TAC) in patients seeking infertility treatment and controls.


SETTING
Male infertility clinic of a tertiary care center.


PATIENT(S)
One hundred sixty-seven infertile patients and 19 controls.


INTERVENTION(S)
None.


MAIN OUTCOME MEASURE(S)
Semen characteristics, seminal ROS, and TAC in samples from patients with various clinical diagnoses and controls.


RESULT(S)
Fifteen patients (9.0%) were Endtz positive and 152 (91.0%) Endtz negative. Sperm concentration, motility, and morphology were significantly reduced in all groups compared with the controls (P =.02), except in varicocele associated with infection group. Mean (+/-SD) ROS levels in patient groups ranged from 2.2 +/- 0.13 to 3.2 +/- 0.35, significantly higher than controls (1.3 +/- 0.3; P<.005). Patient groups had a significantly lower mean (+/-SD) TAC from 1014.75 +/- 79.22 to 1173.05 +/- 58.07 than controls (1653 +/- 115.28, P<.001), except in the vasectomy reversal group (1532.02 +/- 74.24). Sperm concentration was negatively correlated with ROS both overall and within all groups (P</=.007), with the exception of idiopathic infertility.


CONCLUSION(S)
Irrespective of the clinical diagnosis and semen characteristics, the presence of seminal oxidative stress in infertile men suggests its role in the pathophysiology of infertility. Medical or surgical treatments for infertility in these men should include strategies to reduce oxidative stress.
3D analysis of brace treatment in idiopathic scoliosis

We have evaluated the effect of bracing in scoliosis on coronal alignment in a cohort of patients. Current literature has not described the specific effect of bracing on the 3D shape of the scoliotic curves. The purpose of this study was to analyze the variability of the 3D effect of bracing on idiopathic scoliosis. The spines of 30 patients with adolescent idiopathic scoliosis were reconstructed using biplanar stereoradiography with and without the brace. The Cobb angle, sagittal and pelvic parameters and transverse plane parameters were calculated. The variability and the mean values of each parameter, with and without a brace, were analyzed and compared using a student t test. The Cobb angle improved in 50 % of patients but remained unchanged in 50 % cases. In 90 % of the cases lordosis was decreased. The thoracic kyphosis was decreased in 26 % cases, unchanged in 57 % of cases and increased in 17 % cases. The AVR was improved (>5°) in 26 % cases, worsened in 23 % and unchanged in 50 %. Only the differences of Cobb angle and the lordosis were statistically significant. Global statistics of this study concur with the literature. The Cobb angle was significantly improved. It also showed a significant hypolordotic effect. However, the results showed a high variability of the brace treatment effect in almost every parameter. Analysis of this variability by means of 3D reconstructions instead of global statistics should help characterize the mechanisms of correction of brace treatment.
Raltegravir in HIV-1-Infected Pregnant Women: Pharmacokinetics, Safety, and Efficacy.

BACKGROUND
The use of raltegravir in human immunodeficiency virus (HIV)-infected pregnant women is important in the prevention of mother-to-child HIV transmission, especially in circumstances when a rapid decline of HIV RNA load is warranted or when preferred antiretroviral agents cannot be used. Physiological changes during pregnancy can reduce antiretroviral drug exposure. We studied the effect of pregnancy on the pharmacokinetics of raltegravir and its safety and efficacy in HIV-infected pregnant women.


METHODS
An open-label, multicenter, phase 4 study in HIV-infected pregnant women receiving raltegravir 400 mg twice daily was performed (Pharmacokinetics of Newly Developed Antiretroviral Agents in HIV-Infected Pregnant Women Network). Steady-state pharmacokinetic profiles were obtained in the third trimester and postpartum along with cord and maternal delivery concentrations. Safety and virologic efficacy were evaluated.


RESULTS
Twenty-two patients were included, of which 68% started raltegravir during pregnancy. Approaching delivery, 86% of the patients had an undetectable viral load (<50 copies/mL). None of the children were HIV-infected. Exposure to raltegravir was highly variable. Overall area under the plasma concentration-time curve (AUC) and plasma concentration at 12 hours after intake (C12h) plasma concentrations in the third trimester were on average 29% and 36% lower, respectively, compared with postpartum: Geometric mean ratios (90% confidence interval) were 0.71 (.53-.96) for AUC0-12h and 0.64 (.34-1.22) for C12h. The median ratio of raltegravir cord to maternal blood was 1.21 (interquartile range, 1.02-2.17; n = 9).


CONCLUSIONS
Raltegravir was well tolerated during pregnancy. The pharmacokinetics of raltegravir showed extensive variability. The observed mean decrease in exposure to raltegravir during third trimester compared to postpartum is not considered to be of clinical importance. Raltegravir can be used in standard dosages in HIV-infected pregnant women.


CLINICAL TRIALS REGISTRATION
NCT00825929.
The Stable Marriage Problem

The original work of Gale and Shapley on an assignment method using the stable marriage criterion has been extended to find all the stable marriage assignments. The algorithm derived for finding all the stable marriage assignments is proved to satisfy all the conditions of the problem. Algorithm 411 applies to this paper.
Deep Learning for Detecting Robotic Grasps

We consider the problem of detecting robotic grasps in an RGB-D view of a scene containing objects. In this work, we apply a deep learning approach to solve this problem, which avoids time-consuming hand-design of features. This presents two main challenges. First, we need to evaluate a huge number of candidate grasps. In order to make detection fast, as well as robust, we present a two-step cascaded structure with two deep networks, where the top detections from the first are re-evaluated by the second. The first network has fewer features, is faster to run, and can effectively prune out unlikely candidate grasps. The second, with more features, is slower but has to run only on the top few detections. Second, we need to handle multimodal inputs well, for which we present a method to apply structured regularization on the weights based on multimodal group regularization. We demonstrate that our method outperforms the previous state-of-the-art methods in robotic grasp detection, and can be used to successfully execute grasps on a Baxter robot. 1
Intensive cholesterol reduction lowers blood pressure and large artery stiffness in isolated systolic hypertension.

OBJECTIVES
We sought to investigate the effects of intensive cholesterol reduction on large artery stiffness and blood pressure in normolipidemic patients with isolated systolic hypertension (ISH).


BACKGROUND
Isolated systolic hypertension is associated with elevated cardiovascular morbidity and mortality and is primarily due to large artery stiffening, which has been independently related to cardiovascular mortality. Cholesterol-lowering therapy has been efficacious in reducing arterial stiffness in patients with hypercholesterolemia, and thus may be beneficial in ISH.


METHODS
In a randomized, double-blinded, cross-over study design, 22 patients with stage I ISH received three months of atorvastatin therapy (80 mg/day) and three months of placebo treatment. Systemic arterial compliance was measured noninvasively using carotid applanation tonometry and Doppler velocimetry of the ascending aorta.


RESULTS
Atorvastatin treatment reduced total and low-density lipoprotein cholesterol and triglyceride levels by 36 +/- 2% (p < 0.001), 48 +/- 3% (p < 0.001) and 23 +/- 5% (p = 0.003), respectively, and increased high density lipoprotein cholesterol by 7 +/- 3% (p = 0.03). Systemic arterial compliance was higher after treatment (placebo vs. atorvastatin: 0.36 +/- 0.03 vs. 0.43 +/- 0.05 ml/mm Hg, p = 0.03). Brachial systolic blood pressure was lower after atorvastatin treatment (154 +/- 3 vs. 148 +/- 2 mm Hg, p = 0.03), as were mean (111 +/- 2 vs. 107 +/- 2 mm Hg, p = 0.04) and diastolic blood pressures (83 +/- 1 vs. 81 +/- 2 mm Hg, p = 0.04). There was a trend toward a reduction in pulse pressure (71 +/- 3 vs. 67 +/- 2 mm Hg, p = 0.08).


CONCLUSIONS
Intensive cholesterol reduction may be beneficial in the treatment of patients with ISH and normal lipid levels, through a reduction in large artery stiffness.
Thinking with Mahatma Gandhi: Beyond Liberal Democracy

Gandhi's case against the West looks... infinitely stronger than it looked, to us Westerners, thirty years ago.G. D. H. cole
Simplified Adaptive Spatial Modulation for Limited-Feedback MIMO Systems

Adaptive spatial modulation (ASM), where the modulation orders assigned to transmit antennas are dynamically adapted to the changing channel conditions, is a new limited-feedback multiple-input-multiple-output (MIMO) transmission technique. However, the exhaustive search for the optimal modulation order vector in ASM requires very high computational complexity and feedback load. In this paper, we propose two simplified ASM (SASM) schemes that reduce the high computational complexity and feedback load with negligible performance loss. More specifically, a candidate-reduction-based ASM (CR-ASM) that exploits the candidate selection probability is first developed to shrink the search space. The CR-ASM scheme only considers the candidates with high probability. To further reduce the complexity, another SASM scheme that uses a one-bit reallocation (OBRA) algorithm for modulation order assignment is proposed. Regardless of the number of transmit antennas, OBRA only needs to consider the transmit antenna pair with the highest correlation and selects their optimum orders among four possible candidates, which significantly reduce the search complexity. Simulation results show that the proposed schemes substantially reduce computational cost and feedback load while maintaining the benefits of the conventional ASM, particularly for a high number of transmit antennas.
Does Governance Travel Around the World? Evidence from Institutional Investors

We examine whether institutional investors affect corporate governance by analyzing portfolio holdings of institutions in companies from 23 countries during the period 2003– 2008. We find that firm-level governance is positively associated with international institutional investment. Changes in institutional ownership over time positively affect subsequent changes in firm-level governance, but the opposite is not true. Foreign institutions and institutions from countries with strong shareholder protection play a role in promoting governance improvements outside of the U.S. Institutional investors affect not only which corporate governance mechanisms are in place, but also outcomes. Firms with higher institutional ownership are more likely to terminate poorly performing Chief Executive Officers (CEOs) and exhibit improvements in valuation over time. Our results suggest that international portfolio investment by institutional investors promotes good corporate governance practices around the world. & 2010 Elsevier B.V. All rights reserved.
Exploring aesthetical gameplay design patterns: camaraderie in four games

This paper explores how a vocabulary supporting design-related discussions of gameplay preferences can be developed. Using the preference of experiencing camaraderie as an example, we have analyzed four games: the board games Space Alert and Battlestar Galactica, the massively multiplayer online game World of Warcraft, and the cooperative FPS series Left for Dead. Through a combination of the MDA model on how game mechanics give rise to game aesthetics via game dynamics, and the concept of aesthetic ideals in gameplay, we present gameplay design patterns related to achieving camaraderie. We argue that some of these patterns can be seen as aesthetic gameplay design patterns in that they are closely related to aesthetic ideals. Further, as a consequence, gameplay design pattern collections which include patterns related to all levels of the MDA model can be used as design tools when aiming for certain gameplay aesthetics.
Macro to Micro: Country Exposures, Firm Fundamentals and Stock Returns

We outline a systematic approach to incorporate macroeconomic information into firm level forecasting from the perspective of an equity investor. Using a global sample of 198,315 firm-years over the 1998-2010 time period, we find that combining firm level exposures to countries (via geographic segment data) with forecasts of country level performance, is able to generate superior forecasts for firm fundamentals. This result is particularly evident for purely domestic firms. We further find that this forecasting benefit is associated with future excess stock returns. These relations are stronger after periods of higher dispersion in expected country level performance. JEL classification: G12; G14; M41
Estimating customer service in a two-location continuous review inventory model with emergency transshipments

In this paper, an approximate analytical two-location inventory transshipment model is developed that combines the popular order-quantity, reorder-point ðQ;RÞ continuous review ordering policy with a third parameter, the hold-back amount, which limits the level of outgoing transshipments. The degree to which transshipments improve both Type I (no-stockout probability) and Type II (fill rate) customer service levels can be calculated using the model. Simulation studies conducted to test the validity of the approximations in the analytical model indicate that it performs very well over a wide range of inputs. 2002 Elsevier Science B.V. All rights reserved.
Automatic Generation of Game Level Solutions as Storyboards

Game programmers rely on artificial intelligence techniques to encode characters' behaviors initially specified by game designers. Although significant efforts have been made to assist their collaboration, the formalization of behaviors remains a time-consuming process during the early stages of game development. We propose an authoring tool allowing game designers to formalize, visualize, modify, and validate game level solutions in the form of automatically generated storyboards. This system uses planning techniques to produce a level solution consistent with gameplay constraints. The main planning agent corresponds to the player character, and the system uses the game actions as planning operators and level objectives as goals to plan the level solutions. Generated solutions are presented as 2-D storyboards similar to comic strips. We present in this paper the first version of a fully implemented prototype as well as examples of generated storyboards, adapted from the original design documents of the blockbuster game Hitman.