Miracle-dz/UniFAR-FTUs huggingface dataset:
Reinforcement Learning and Dynamic Programming Using Function Approximators

From household appliances to applications in robotics, engineered systems involving complex dynamics can only be as effective as the algorithms that control them. While Dynamic Programming (DP) has provided researchers with a way to optimally solve decision and control problems involving complex dynamic systems, its practical value was limited by algorithms that lacked the capacity to scale up to realistic problems. However, in recent years, dramatic developments in Reinforcement Learning (RL), the model-free counterpart of DP, changed our understanding of what is possible. Those developments led to the creation of reliable methods that can be applied even when a mathematical model of the system is unavailable, allowing researchers to solve challenging control problems in engineering, as well as in a variety of other disciplines, including economics, medicine, and artificial intelligence. Reinforcement Learning and Dynamic Programming Using Function Approximators provides a comprehensive and unparalleled exploration of the field of RL and DP. With a focus on continuous-variable problems, this seminal text details essential developments that have substantially altered the field over the past decade. In its pages, pioneering experts provide a concise introduction to classical RL and DP, followed by an extensive presentation of the state-of-the-art and novel methods in RL and DP with approximation. Combining algorithm development with theoretical guarantees, they elaborate on their work with illustrative examples and insightful comparisons. Three individual chapters are dedicated to representative algorithms from each of the major classes of techniques: value iteration, policy iteration, and policy search. The features and performance of these algorithms are highlighted in extensive experimental studies on a range of control applications. The recent development of applications involving complex systems has led to a surge of interest in RL and DP methods and the subsequent need for a quality resource on the subject. For graduate students and others new to the field, this book offers a thorough introduction to both the basics and emerging methods. And for those researchers and practitioners working in the fields of optimal and adaptive control, machine learning, artificial intelligence, and operations research, this resource offers a combination of practical algorithms, theoretical analysis, and comprehensive examples that they will be able to adapt and apply to their own work. Access the authors' website at www.dcsc. tudelft.nl/rlbook/ for additional material, including computer code used in the studies and information concerning new developments.
Qualitative Data Analysis with NVivo

This straightforward, jargon-free book provides an invaluable introduction to planning and conducting qualitative data analysis with NVivo. Written by leading authorities, with over 40 years combined experience in computer-assisted analysis of qualitative and mixed-mode data, the new edition of this best selling textbook is an ideal mix of practical instruction, methodology and real world examples. Practical, clear and focused the book effectively shows how NVivo software can accommodate and assist analysis across a wide range of research questions, data types, perspectives and methodologies. It sets out: - The power and flexibility of the NVivo software - How best to use NVivo at each stage in your research project - Examples from the authors' own research and the sample data that accompanies the software, supplemented with vignettes drawn from across the social sciences - Annotated screen shots - A website with links to data, sample projects, supplementary/updated instructions, and SAGE journal content This second edition contains new chapters on handling a literature review, visualizing data, working in mixed methods and social media datasets, and approaching NVivo as a team. An insightful step-by-step guide to the messy reality of doing computer-assisted analysis, this successful book is essential reading for anyone considering using NVivo software.
Robust control of mobile robots using the computed torque plus H/sub /spl infin// compensation method

This paper introduces robust control architecture for nonholonomic mobile robots. The proposed technique linearizes and decouples the nonlinear and coupled dynamic equations of the robot by computed torque approach. Since the computed torque technique requires the exact knowledge of the corresponding nonlinear dynamics, therefore, any mismatch in this regard may leave some nonlinear and also uncertain parts in the resulting system. The effect of mentioned parts may be compensated via a robust controller. As a result, in the present work, the remaining uncertainty is compensated by the H/sub /spl infin// technique, which requires the boundedness of the uncertain part. It is shown that the elements of the said uncertainty are L/sub 2/ -bounded. Numerical simulations demonstrate the effectiveness of the proposed H/sub /spl infin// based robust control scheme.
Cervical cancer studies on prevention and treatment

Cervical cancer is the fourth most common reason for cancer death amongst women worldwide, with 500 000 new cases every year. Most of these women live in developing countries. Cervical screening programs can reduce the incidence by 50%, but are not common in the developing world. In Sweden there are approximately 400 new cases of cervical cancer yearly, despite the existence of a screening program. Cervical cancer and dysplastic lesions in the cervix are caused by infection by the Human Papilloma Virus, mainly HPV 16 and 18. An alternative to cytological screening is screening for HPV DNA. The aim of this thesis was to analyze the screening history of women with cervical cancer, to find new techniques to screen women for cervical cancer, and to find out why some women do not attend screening. Furthermore, standard treatment for early cervical cancer is surgery including lymph node mapping, and patients have adverse effects from lymphadenectomies, so if a sentinel node procedure is accurate, it would be of value for the women. Of the population of cervical cancer patients diagnosed in the years 2009-2010, one third of the women were over 60 years of age, and one fourth were over 65. The women that followed the screening program had a better prognosis compared to women above the screening age (Hazard Ratio (HR) =5.3 95% CI 2.4-12.0, p<0.001). All the women whose cancer was identified via the screening program were alive at the median follow-up (36 months). This contrasted with the women that had symptoms, where 68 of 98 patients were alive at the median follow-up (p<0.0001). The vaginal self-sampling device that we validated showed good agreement with HPV analysis of liquid-based cytology (LBC) (kappa value 0.67 95% CI 0.53-0.81). When 1000 women who had not taken a cytology sample for over nine years were offered self-sampling at home, 15% responded. In contrast, when 500 women were offered flexible and free open clinic appointments, 4.2% came. The most common reason for non-attending was “uncomfortable with vaginal examination”. The sentinel node concept is safe when a sentinel node is identified on both sides, and the tumor is 2 cm or less. The negative predictive value was then 100%. The conclusions are that since so many women develop their cervical cancer after screening age, it would be advisable with an exit test. This exit test should ideally be an HPV test. In the case of hr-HPV positivity, the women could be offered a reflex cytology test. To reduce the non-attendance rate for cervical cancer screening, vaginal self-sampling for HPV-DNA is a possible method to use. Since so many women seem to be non-attendant due to being uncomfortable with vaginal examination, a self-test could be an attractive alternative. The final conclusion is that the sentinel node concept should be further investigated to make it possible to offer a sharp sentinel node for women with cervical cancer.
A statistical model for line-of-sight (LOS) factory radio channels

A statistical model, based on experimental data, is presented to characterize the multipath propagation medium in typical manufacturing environments. This model includes the statistics of multipath propagation characteristics for cases of light and heavy clutter in a factory where there is a direct path from transmitter to receiver. The dynamic behavior of the medium is described by a time-varying impulse response which includes the distribution of the parameters of the channel such as the gain, the interarrival time, and the number of paths. The peaks and the inflections of a number of experimentally measured pulse responses of channels were used to estimate the channel statistics. <<ETX>>
La simulation de concours d’orchestre : analyse qualitative et située de l’activité des musiciens

Cet essai consiste en une analyse, fondee sur la theorie et l’approche du cours d’action, de l’activite de sept musiciens-etudiants en situation de simulation de concours d’orchestre. Les strategies d’adaptation des musiciens sont explorees selon les principes de l’enaction. Des entretiens replongeant le musicien dans cette situation ont permis de faire ressortir, a travers le codage des verbatim en « unites significatives elementaires » (USE), le sens que chacun des musiciens donne aux differents moments de la simulation de concours, ainsi qu’une multitude de strategies d’adaptation developpees pour y faire face. Les differentes macro-sequences identifiees (la decouverte du programme, la mise en condition pour le concours, les adaptations, et la construction de l’experience apres concours) ont une influence sur les musiciens et sont percues de maniere changeante selon l’avancee du concours. A la suite de cette analyse, des strategies sont proposees afin de faire face aux differentes dynamiques generees par le concours. Cette etude, empruntant la methode pluridisciplinaire du cours d’action, cherche a apporter de nouvelles connaissances dans la gestion de la performance musicale.
Potential cases, database types, and selection methodologies for searching distributed text databases

The rapid proliferation of online textual databases on the Internet has made it difficult to effectively and efficiently search desired information for the users. Often, the task of locating the most relevant databases with respect to a given user query is hindered by the heterogeneities among the underlying local textual databases. In this chapter, we first identify various potential selection cases in distributed textual databases (DTDs) and classify the types of DTDs. Based on these results, the relationships between selection cases and types of DTDs are recognized and necessary constraints of database selection methods in different cases are given which can be used to develop a more effective and suitable selection algorithm. This chapter appears in the book, Intelligent Agents for Data Mining and Information Retrieval, edited by Masoud Mohammadian. Copyright © 2004, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. 701 E. Chocolate Avenue, Suite 200, Hershey PA 17033-1240, USA Tel: 717/533-8845; Fax 717/533-8661; URL-http://www.idea-group.com IDEA GROUP PUBLISHING
How different is translated Chinese from native Chinese

Corpus-based translation studies focus on translation as a product by comparing comparable corpora of translational and non-translational texts. A number of distinctive features of translational English in relation to native English have been uncovered. Nevertheless, research of this area has so far been confined largely to translational English translated from closely related European languages. If the features of translational language that have been reported on the basis of translated English are to be generalized as ‘translation universals’, it is of vital importance to find supporting evidence from non-European languages. Clearly, evidence from “genetically” distinct language pairs such as English and Chinese is arguably more convincing, if not indispensable. This article explores potential features of translational Chinese on the basis of two balanced monolingual comparable corpora of translated and native Mandarin Chinese. The implications of the study for translation universal hypotheses are also discussed.
Algorithmes et processeurs temps réel de traitement de signaux neuronaux pour une plateforme optogénétique sans fil

The electrical signals acquisition from the brain’s neurons allows neuroscientists to better understand its functioning. In this work, new neural signals compression algorithms are designed and presented. These new algorithms are incorporated into three new miniature optogenetic wireless devices. These devices are capable to optically stimulate neural activity and to wirelessly transmit the biopotentials captured by several microelectrodes. Two of these systems are able to compress the signals from two microelectrodes and to stimulate optically via two high-power LED. Both systems feature a new spike detection algorithm to reduce the bandwidth used by the wireless transceiver. This new spike detection algorithm differs from existing algorithms by achieving better detection rate while using less material resources and processing time. A third device incorporating the detection and compression algorithms was designed. This device is the only optogenetic wireless system including 32 optical stimulation channels and 32 electrophysiological recording channels in parallel. This new system has the ability to compress the neural signals using a new wavelet compression technique that significantly increase the number of channels under observation without increasing the consumption of the wireless transceiver. In particular, this new compression technique differs from the existing wavelet based compression methods by achieving better compression ratio while allowing to reconstruct the compressed signals with better quality. At the time of writing this thesis, these are the first three devices that offer simultaneous multichannel optical stimulation, multichannel electrophysiological signals recording and on-the-fly spike detection. The resulting systems are more compact and lightweight than previous systems, making these devices essentials to conduct long term experiments on the brains of small freely moving animals. The three systems were validated within in vivo experiments using transgenic mice at the Centre de Recherche de l’Institut Universitaire en Santé Mentale de Québec (CRIUSMQ).
Teaching programming strategies explicitly to novice programmers

[Abstract]: The traditional approach to training novice programmers has been to provide explicit programming knowledge instruction but to rely on implicit instruction of programming strategies. Studies, reported in literature, have discovered universally poor results on standardised tests for novices studying under this traditional approach. This dissertation describes the explicit integration of programming strategies into instruction and assessment of novice programmers, and the impact of this change on their learning outcomes. An initial experiment was used to measure the performance of students studying under a traditional curriculum with implicitly taught programming strategies. This experiment uncovered common flaws in the strategy skills of novices and revealed weaknesses in the curriculum. Incorporation of explicit strategy instruction was proposed. To validate a model of strategies as being authentic and appropriate for novice instruction, an experiment with experts was conducted. Experts were asked to solve three problems that a novice would typically be expected to solve at the end of an introductory programming course. Experts‟ solutions were analysed using Goal/Plan Analysis and it was discovered that experts consistently applied plans, the subalgorithmic strategies suggested by Soloway (1986). It was proposed that plans could be adapted for explicit inclusion in an introductory programming curriculum. Initially a curriculum incorporating explicit strategy instruction was tested in an artificial setting with a small number of volunteers, divided into control and experimental groups. The control group was taught using a simplified traditional curriculum and the experimental group were exposed to a curriculum which explicitly included programming strategies. Testing revealed that experimental group participants applied plans more than control group participants, who had been expected to learn these strategies implicitly. In interviews, experimental participants used strategy-related terminology and were more confident in the solutions they had created. These results justified a trial of the curriculum in an actual introductory programming course. When explicit instruction of programming strategies was incorporated into an actual introductory programming curriculum, novices achieved superior results when compared to results from the initial experiment. Novices used strategies significantly more when these strategies were incorporated explicitly into instructional materials and assessment items. This series of experiments focussed on explicitly teaching specific programming strategies rather than teaching problem-solving more generally. These experimental results demonstrate that explicit incorporation of programming strategies may improve outcomes for novices and potentially improve the potential of expert programmers in future.
Academic Effects of Writer's Workshop

Marla Smithson. ACADEMIC EFFECTS OF WRITER’S WORKSHOP. (Under the direction of Dr. Randy Dunn) School of Education, September, 2008. The purpose of this dissertation was to study the effects of a process writing curriculum, Writer’s Workshop, on the writing achievement of fifth grade students as measured by the Georgia Fifth Grade Writing Assessment. The study used three years of data and a total of 321 subjects in determining improved writing scores. The investigation equated and compared the performance levels using a Chi-square contingency table, finding that the group prior to implementation surpassed the performance of the cohorts in the following two years. An independent samples t-test evaluated scale scores from the latter two years, resulting in no significant difference between the two groupings. Data also includes interviews from the principal and academic coach. The discussion cites several reasons for the findings and suggests continued use of the curriculum with alterations.
Multi‐Level Modeling

Whenever research is concerned with the analysis of relationships between lowerlevel units (e.g., individuals) and the higherlevel unit they belong to (e.g., group), a hierarchical – multilevel or nested – data structure occurs. In such cases, multi-level modeling (MLM) approaches, also known as hierarchical linear modeling, are suited to empirically assess respective data structures in a methodologically correct way. The article gives an introduction to the background of multilevel modeling, outlines the methodological procedure for the basic case of a two-level model, and provides an outlook to more advanced applications of MLM. Keywords: multilevel modeling; levels of analysis; nested data; hierarchical linear modeling; within-group regression; between group regression
Two-dimensional PHMM for handwriting digits recognition

We used two-dimensional pseudo hidden Markov models (2D PHMM) for handwriting digits recognition. We used a 2D pseudo Viterbi algorithm for training the model. This algorithm compared to the traditional forward-backward algorithm had some advantages as follow. (1) It can get rig of the down overflow that may happen in the forward-backward algorithm. (2) As the algorithm used log calculation the accuracy of the calculation was much higher than that of the forward-backward algorithm. (3) The speed of calculation was much faster than the forward-backward algorithm. The database of handwriting digits used was collected at the Beijing Postal Center, the digits were scanned from letters zip code; of the 4000 handwriting digits collected, 2000 were used for the training set, and 2000 were used for the testing set. We used the pixel value as the recognition feature. The test result showed, the recognition rate of training set was 95%. The recognition rate of testing set was 92.6%. We used the horizontal and vertical directions as a super plane to build two types models at same time, and at the recognition phase if the result of the two models are different as a rejection condition, the recognition rate of training set is 98%, the recognition rate of testing set is 96%. If we used some feature which is rotate and move invariant the recognition rate may get higher.
Scheduling jobs with machine-dependent release dates on unrelated machines

This paper considers the job scheduling problem with varying release dates on unrelated parallel machines. Our model involves the machine-dependent release dates and the makespan criterion. The two developed and presented constructive deterministic algorithms use different decomposition strategies that result logically from exploiting the problem structure. The approximation factor of the greedy polynomial algorithm depends on the machines. An efficient adaptation of the well-known brute-force technique to solve $Rm\vert r_{i,j}\vert C_{\max}$ is also considered in this paper. A series of numerical experiments are conducted to compare the quality of schedules.
Using a personal response system as an in-class assessment tool in the teaching of basic college chemistry

Since the introduction of personal response systems (PRS) (also referred to as "clickers") nearly a decade ago, their use has been extensively adopted on college campuses, and they are particularly popular with lecturers of large classes. Available evidence supports that PRS offers a promising avenue for future developments in pedagogy, although findings on the advantages of its effective use related to improving or enhancing student learning remain inconclusive. This study examines the degree to which students perceive that using PRS in class as an assessment tool effects their understanding of course content, engagement in classroom learning, and test preparation. Multiple, student-performance evaluation data was used to explore correlations between student perceptions of PRS and their actual learning outcomes. This paper presents the learning experiences of 151 undergraduate students taking basic chemistry classes and incorporating PRS as an in-class assessment tool at the National Chung Hsing University in Taiwan. While the research revealed positive student perceived benefits and effectiveness of PRS use, it also indicated the need for further studies to discover what specific contribution PRS can make to certain learning outcomes of a large chemistry class in higher education.
Downlink relative co-channel interference powers in cellular radio systems

One method to achieve high capacity in cellular radio systems involves interference cancellation techniques at the receivers. To design and evaluate the performance of such interference cancellation receivers, it is essential to use realistic models of the co-channel interference. In the paper we study the ordered statistic of the relative co-channel interference, i.e. the ratio between the power level of i:th strongest interferer and the total co-channel interference power. Results from Monte-Carlo simulations show that for base stations on a symmetric grid, hexagonal cells or half-square Manhattan like street cells, the probability density functions (pdf) of the interference ratios are almost independent of the cluster size and of the base station activities. The results demonstrate the dominance of the strongest interference and show that this dominance is even more pronounced in those situations when the signal to interference ratio is low. The presented results provide a base for more realistic models for performance evaluation in cellular systems than the commonly used Gaussian interference model.