Miracle-dz/UniFAR-FTUs huggingface dataset:
An Empirical Study of Exploration Strategies for Model-Free Reinforcement Learning

Reinforcement Learning is a formalism for learning by trial and error. Unfortunately, trial and error can take a long time to find a solution if the agent does not efficiently explore the behaviours available to it. Moreover, how an agent ought to explore depends on the task that the agent is trying to learn. In this thesis we study how an agent’s exploration strategy affects how quickly it learns to solve different problems. In particular, we focus on model-free algorithms that learn value functions. We first examine the space of problems, or environments, that reinforcement learning agents are expected to solve. We identify six properties of environments that can make exploration difficult, and design a prototypical environment expressing each property. We also survey the exploration literature and categorize existing exploration methods by the heuristic that guides their behaviour. Lastly, we conduct an empirical study evaluating the performance of several explorationmethods on our prototypical exploration environments. We found that only one method, Linear Upper Confidence Least Squares, was able to consistently perform well in every environment. We also found that methods which add a bonus to their value function tended to explore much more effectively than methods which add a bonus to their rewards. Our investigation of value-based exploration provides a novel, systematic approach to understanding the strengths and weaknesses of exploration algorithms in reinforcement learning.
Spring-Summer Drought Induces Extremely Low Radial Growth Reactions in North-Tyrrhenian Pinus pinea L.

Climate projections predict shifts in environmental conditions, with cascade effects on forest growth dynamics. As such, Pinus pinea L. , an ecologically important low-elevation Mediterranean tree, can be threatened by drought events. The occurrence of negative stem growth anomalies (“negative pointer years”, or NPY) and its relation to climatic conditions were analyzed, as well as the influence of extreme dry spells upon the species growth. NPY were temporally independent among the analyzed forest stands, likely due to local factors. We observed that NPY depended on dry and hot conditions during the spring-summer period at both sites, while differences in the NPYclimate reflected the species medium term dendroclimatological signal. Extremely dry years directly reduced stem growth rates. Water stress differentially affected growth at each site, likely reflecting local adaptation to droughts. Because of the increasing drought trend expected for the Mediterranean basin, our findings must be considered regarding the conservation and management of these forests.
GC-MS Characterization of Antioxidative Compounds from the Stem Bark and Flower Extracts of Schefflera Species, from Western Ghats

Objective: Investigation and identification of antioxidative compounds of methanolic flower extract of S. venulosa and ethyl acetate stem bark extract of S. wallichiana (Araliaceae) from Western Ghats. Methods: Powdered stem bark and flower materials were subjected to Soxhlet extraction using various solvents according to their polarity. The methanolic flower extract of S. venulosa and the ethyl acetate stem bark extract of S. wallichiana were reconstituted in the respective solvents. One μl of the extracts was injected for GC-MS analysis. The fragmentation patterns of mass spectra were compared with those stored in the spectrometer database using the National Institute of Standards and Technology Mass Spectral (NIST-MS) database. Results: The methanolic flower extract of S. venulosa revealed the presence of 120 compounds whereas ethyl acetate stem bark extract of S. wallichiana showed the presence of 115 compounds. Among those, 23 compounds from methanol flower extract of S. venulosa and 25 compounds from the ethyl acetate stem bark extract of S. wallichiana have been identified as the probable antioxidant compounds based on the functional groups they possess. Conclusion: Many bioactive compounds were identified from two Schefflera species. Thus, the identification of different biologically active compounds in these extracts warrants further biological and pharmacological studies.
Visual Sensor Fusion With Error Compensation Strategy Toward a Rapid and Low-Cost 3D Scanning System for the Lower Residual Limb

Rapid and low-cost 3D scanning system becomes a common solution for prosthetic designers and engineers to acquire accurate 3D geometrical model of the involved patient. Multiple Kinect sensor fusion solutions have been proposed but still, there is a lack of fusion scheme within a single workstation and a systematic reconstruction error analysis. The objective of the present study was to develop a new Kinect sensor fusion scheme running on a single workstation and an error compensation strategy. Architecture of a 3D scanning system for the lower residual limb was proposed. Four Kinect sensors were used to develop the fusion scheme. An error compensation approach based on linear regression method was performed. Then, an iterative rigid multi-set registration method was proposed. Different primitive geometries (cylindrical objects, box) with well-known physical dimensions were designed, fabricated and used for the evaluation process. Obtained results showed that the error compensation strategy leads to a reduction of around 40% of 3D geometrical reconstruction error for the residual stump-like structure. The total scanning and processing time is 25.4 ± 2.5 seconds. Moreover, our novel iterative rigid multi-set registration method leads to an improvement of accuracy of 60% compared to conventional registration method. Our proposed 3D scanning system using multiple Kinect cameras with error compensation strategy is expected to reduce the design cost and improve the design accuracy. As perspective, this 3D scanning system with fusion scheme will be integrated into a Computer-Aided Design (CAD) system with soft tissue deformation feedbacks for lower limb prostheses.
Effects of Cypermethrin on the Hematological Parameters, Biochemical Components of Blood and Histopathological Changes in Different Organs of Chirruh Snow Trout (Schizothorax esocinus)

Naveed Akhtar1*, Muhammad Fiaz Khan1*, Sadia Tabassum1, Munawar Saleem Ahmad2 and Khan Dil Badshah3 1Department of Zoology, Hazara University, Mansehra, Pakistan 2Department of Zoology, University of Swabi, Pakistan 3Department of Chemistry, Government Postgraduate College, Haripur, Pakistan Article Information Received 10 April 2019 Revised 04 June 2019 Accepted 20 June 2019 Available online 03 August 2020
Estimating the relative fitness of escaped farmed salmon offspring in the wild and modelling the consequences of invasion for wild populations

Throughout their native range, wild Atlantic salmon populations are threatened by hybridization and introgression with escapees from net‐pen salmon aquaculture. Although domestic–wild hybrid offspring have shown reduced fitness in laboratory and field experiments, consequential impacts on population abundance and genetic integrity remain difficult to predict in the field, in part because the strength of selection against domestic offspring is often unknown and context‐dependent. Here, we follow a single large escape event of farmed Atlantic salmon in southern Newfoundland and monitor changes in the in‐river proportions of hybrids and feral individuals over time using genetically based hybrid identification. Over a three‐year period following the escape, the overall proportion of wild parr increased consistently (total wild proportion of 71.6%, 75.1% and 87.5% each year, respectively), with subsequent declines in feral (genetically pure farmed individuals originating from escaped, farmed adults) and hybrid parr. We quantify the strength of selection against parr of aquaculture ancestry and explore the genetic and demographic consequences for populations in the region. Within‐cohort changes in the relative proportions of feral and F1 parr suggest reduced relative survival compared to wild individuals over the first (0.15 and 0.81 for feral and F1, respectively) and second years of life (0.26, 0.83). These relative survivorship estimates were used to inform an individual‐based salmon eco‐genetic model to project changes in adult abundance and overall allele frequency across three invasion scenarios ranging from short‐term to long‐term invasion and three relative survival scenarios. Modelling results indicate that total population abundance and time to recovery were greatly affected by relative survivorship and predict significant declines in wild population abundance under continued large escape events and calculated survivorship. Overall, this work demonstrates the importance of estimating the strength of selection against domestic offspring in the wild to predict the long‐term impact of farmed salmon escape events on wild populations.
Constant-Amplitude Sequences for Spectrally Compact OFDM Training Waveforms

Constant-amplitude (CA) sequences are desirable to construct training waveforms in orthogonal frequency division multiplexing (OFDM) systems in order to achieve accurate channel estimation and robust initial time and frequency synchronization. However, widely adopted CA pseudorandom-noise and Zadoff-Chu sequences suffer from large power spectral sidelobes and thus render low spectral compactness in OFDM training waveforms. This arouses the design of new CA sequences to provide compact spectrum. In this paper, new CA sequences of length $N$ are developed in explicit sequence expressions for all composite sequence lengths and all prime sequence lengths larger than 11 under all parametric conditions, and shown to provide higher spectral compactness than existing CA sequences.
Direct and indirect interactions with vegetation shape crocodylian ecology at multiple scales

1CSIRO Land and Water, Floreat, Western Australia, Australia 2School of Biological Sciences, University of Western Australia, Crawley, Western Australia, Australia 3Queensland Parks and Wildlife, Bentley Park, Queensland, Australia 4Wildlife Conservation Society, Bronx, New York, USA 5Wildlife Management International Pty. Limited, Karama, Northern Territory, Australia 6Western Australian Biodiversity Science Institute, Perth, Western Australia , Australia
Resolving the individual contribution of key microbial populations to enhanced biological phosphorus removal with Ramanâ€“FISH

Enhanced biological phosphorus removal (EBPR) is a globally important biotechnological process and relies on the massive accumulation of phosphate within special microorganisms. Candidatus Accumulibacter conform to the classical physiology model for polyphosphate accumulating organisms and are widely believed to be the most important player for the process in full-scale EBPR systems. However, it was impossible till now to quantify the contribution of specific microbial clades to EBPR. In this study, we have developed a new tool to directly link the identity of microbial cells to the absolute quantification of intracellular poly-P and other polymers under in situ conditions, and applied it to eight full-scale EBPR plants. Besides Ca. Accumulibacter, members of the genus Tetrasphaera were found to be important microbes for P accumulation, and in six plants they were the most important. As these Tetrasphaera cells did not exhibit the classical phenotype of poly-P accumulating microbes, our entire understanding of the microbiology of the EBPR process has to be revised. Furthermore, our new single-cell approach can now also be applied to quantify storage polymer dynamics in individual populations in situ in other ecosystems and might become a valuable tool for many environmental microbiologists.
Time dependent change of ethanol consumption biomarkers, ethyl glucuronide and ethyl sulphate, after single dose ethanol intake

Abstract Background and objective Ethyl glucuronide (EtG) and ethyl sulfate (EtS), minor metabolites of ethanol, aid to forensic scientist and clinicians to determine recent ethanol uptake when ethanol elimination is occurred. Present study aimed to show (a) kinetics of EtG and EtS in blood and urine after 0.5 g/kg ethanol intake (b) peak concentrations and time of disappearance in Turkish people. Material and methods Seventeen (10 male, 7 female) healthy volunteers participated in the study. Blood and urine samples were obtained during 48 h after consumption and analyzed in LC-MS/MS. Results and conclusion Blood peak concentrations of EtG and EtS were 0.13–0.389 mg/L and 0.211–0.5 mg/L, urine peak concentrations were 6.89–30.42 mg/L and 10.5–58.17 mg/L, respectively. There was no EtG and EtS in all samples 24 h later in blood and 48 h later in urine. Correlation was found between EtG and EtS concentrations in blood. Most of obtained data are similar to literature, except EtS dominancy to EtG in urine and blood.
A Systematic Survey of Industrial Internet of Things Security: Requirements and Fog Computing Opportunities

A key application of the Internet of Things (IoT) paradigm lies within industrial contexts. Indeed, the emerging Industrial Internet of Things (IIoT), commonly referred to as Industry 4.0, promises to revolutionize production and manufacturing through the use of large numbers of networked embedded sensing devices, and the combination of emerging computing technologies, such as Fog/Cloud Computing and Artificial Intelligence. The IIoT is characterized by an increased degree of inter-connectivity, which not only creates opportunities for the industries that adopt it, but also for cyber-criminals. Indeed, IoT security currently represents one of the major obstacles that prevent the widespread adoption of IIoT technology. Unsurprisingly, such concerns led to an exponential growth of published research over the last few years. To get an overview of the field, we deem it important to systematically survey the academic literature so far, and distill from it various security requirements as well as their popularity. This paper consists of two contributions: our primary contribution is a systematic review of the literature over the period 2011–2019 on IIoT Security, focusing in particular on the security requirements of the IIoT. Our secondary contribution is a reflection on how the relatively new paradigm of Fog computing can be leveraged to address these requirements, and thus improve the security of the IIoT.
The Effect of Flipped Professional Development on Novice English Teachers’ Achievement

Education experts are always looking for new teaching methods in teaching and learning environments (Biggs & Tang, 2011). Researchers are working to enhance student learning through these new teaching methods. “Flipped Learning “is one of the new approaches that has been proposed recently. In this educational approach, the direct teaching of a teacher is transferred to the outside of the classroom. In contrast, the classroom time is spent on doing homework and activities under the supervision of the teacher as a facilitator (Ojalvo & Doyne, 2011). In any system of education, the teacher training program is a necessity for developing teachers’ knowledge, efficiency, and effectiveness. For this reason, pre-service and in-service classes are held each year for teachers. Research showed that these classes are held in a traditional way in which instructors, who are ABSTRACT
Fluorodeoxyglucose positron-emission tomography-magnetic resonance hybrid imaging: An emerging tool for staging of cancer of the uterine cervix

Positron-emission tomography-magnetic resonance imaging (PET-MRI) is an emerging hybrid imaging modality that utilizes the superior soft tissue resolution of MR with the metabolic data from PET. In this study, we sought to assess the clinical value of fluorodeoxyglucose (FDG) PET-MRI with dedicated pelvic PET-MR in the initial staging of cervical cancer. In this institutional-approved study, we identified 23 adult females who underwent FDG PET-MRI on hybrid camera for staging of primary uterine cervical cancer that included a dedicated PET-MR of the pelvis. A nuclear medicine physician and a radiologist reviewed the PET, MRI, and fusion-body and pelvis images alone and then with consensus read characterizing PET and MR abnormal findings. There were 23 patients who underwent FDG PET-MRI for initial staging of cervical cancer with an average age of 52.2 ± 14.0 years. A total of 23 suspected lymph nodes in eight different patients were detected within the pelvis with increased metabolic activity on PET. Both the dedicated pelvis and whole-body PET imaging detected the same corresponding pelvic lymph nodes, although the pelvic PET imaging had better lymph node uptake delineation due to longer acquisition time. Using a 10-mm short-axis criterion, MRI identified only 43.5% of the FDG avid lymph nodes. The average SUVmax on the pelvis PET sequences was higher with SUV 8.9 ± 5.2 compared to the whole-body PET with SUV 7.8 ± 5.4 but was not statistically significant (P > 0.05). Primary cervical cancer was identified in 18 patients on both PET imaging and MRI with dedicated MR pelvis providing better characterization. Based on our results of the patients with cervical cancer evaluated for initial staging, combining dedicated pelvic PET-MRI with whole-body PET/MR provides the most complete status of malignant disease in reference to delineation of primary tumor, involvement of surrounding tissues, and regional lymph nodes.
Adaptive Identifier-Critic-Based Optimal Tracking Control for Nonlinear Systems With Experimental Validation

This article presents and practically validates an identifier-critic-based approximate dynamic programming (ADP) method to online address the optimal tracking control problem for nonlinear continuous-time unknown systems. The imposed assumption on precisely known system dynamics is obviated via a neural network (NN) identifier. A static control is first adopted to retain the steady-state tracking response, while an optimal control derived via the ADP method is proposed to regulate the tracking error by minimizing a cost function. A critic NN is then trained online to obtain the solution of the associated Hamilton–Jacobi–Bellman (HJB) equation. The learning of the identifier NN and critic NN is performed online simultaneously by tailoring a novel adaptation method, which can guarantee the convergence of the estimated NN weights. Consequently, the critic NN can be used to construct the optimal control policy directly, such that the actor NN used in the previous ADP schemes is avoided. Simulations are performed to verify the suggested control, and experiments on a helicopter plant are carried out to show its feasibility and improved control response.
Chapter 10. Personalized Nutrition to Treat and Prevent Obesity and Diabetes

Obesity and Type 2 diabetes are chronic diseases that are increasing rapidly in many countries going through a nutrition transition. Namely, traditional societies that have shifted towards a Western-culture diet and lifestyle are mainly at risk for obesity-related chronic disease. On the other hand, diet-related adaptive genes encoding alternative alleles may be related to modern-day chronic diseases. Notably, the risk of dyslipidemias and insulin resistance will depend upon the predominance of the “non-risk” or “risk” allele and the surrounding nutrients (food habits). Under this perspective, obesity and Type 2 diabetes are complex diseases that arise from the interaction between a specific risk allele and obesogenic environmental factors. In this chapter, genes encoding receptors for sugar and fat taste perception, lipid transporters, starch, and milk digestive enzymes, energy homeostasis, and food reward systems, as well as epigenetics and host–gut microbiota interactions, will be reviewed. Populations worldwide differ in both the distribution of metabolic risk alleles and dietary patterns; therefore, one standard diet will not fit all. Currently, personalized treatment strategies should aim to customize nutritional needs based on the individual’s or population’s genetic background, food resources, and culture to treat or prevent obesity and Type 2 diabetes.
Preservation technology investment for an inventory system with variable deterioration rate under expiration dates and price sensitive demand

The proposed study addresses a supplier-retailer inventory problem by considering the detrimental impacts of deterioration. The inventory produced by the supplier undergo a machine-shift and hence, produces non-conforming items. The retailer uses the preservation technology to deal with deteriorating items ingeniously where the demand is price-sensitive at the buyers end. Two models are developed through different means viz. Integrated and Bi-level approach and compared so to impart some constructive organizational insights. Sensitivity analysis was done over a numerical example to validate strength of the developed models.
Impact of Insect Infestation on Plant Damage and Yield of Roselle [Hibiscus sabdariffa L.] in Benue State, Nigeria

Impact of insect infestation on growth and yield of Roselle (Hibiscus sabdariffa L.) was evaluated at the Teaching and Research Farm of University of Agriculture, Makurdi, in the 2016 cropping season. An early and late crop (as main plot), of the red (H. sabdariffa sabdariffa) and green (H. sabdariffa altissima) types (as subplot) were planted in Randomized Complete Block Design (RCBD) with split-split-plot arrangement. Four weekly application of 100 g a.i/ha of cypermethrin + 400 g a. i. /ha of dimethoate constituted sub-subplot treatments. All treatments were replicated three times. Insects were visually counted in 1 m area in two rows of each plot. The dominant insect pests included Monolepta thomsoni, Nisotra sjostedti, Dysdercus volkeri and Oxycarenus hyalinipennis. The early crop differed significantly (having 9 % wider stem girth, 2x more branches/stem, and 1.5x more leaf damage) from the late crop. The green Roselle had more pod Original Research Article Simon et al.; IJPSS, 26(1): 1-10, 2018; Article no.IJPSS. 45446 2 (2.5x) and seed (1.1x) damage and gave from 1.3 – 1.5x lower calyx, pod and seed yield. Plant growth and productivity were significantly higher in sprayed than in the unsprayed plots. Plants sprayed at both vegetative and reproductive stages were the most productive having significantly more fresh leaf biomass (2.5-103.6x), calyx yield (2.6-2.8x), pod yield (2.2-7.4x), seed yield (3.111.0x) sequel to more vigorous growth and less pod damage (2.0-44.6x) and seed damage (1.88.6x). Cost-benefit analysis indicated that the red Roselle was more profitable than the green, the late crop was more profitable than the early, and protection at both vegetative and reproductive stages was more profitable than other spray regimes returning N440,291.25/ha, N755,291.5/ha, and N397,236.0/ha for leaf, calyx and seed valuation, respectively. Insecticidal protection of the crop has been shown to mitigate drop damage and return profit.
Vietnamese Consumers Search For Information Online: Some Research Hypotheses

This study provided the proposed hypotheses to encourage researcher to do more study to gain knowledge of online information search behavior of Vietnamese customer. In online customer context, some psychology theories have proved its power in explaining customer action and there were lot of empirical research in the world supported this viewpoint. In addition, due to information available on the Internet, customers are willing to search product or service information online and this action is likely to be habitual behavior. Habit is special psychology construct and thus the empirically study to exam the role of this factor is pragmatic and urgent.