Miracle-dz/UniFAR-FTUs huggingface dataset:
Strategic Exploration in Reinforcement Learning - New Algorithms and Learning Guarantees

Reinforcement learning (RL) focuses on an essential aspect of intelligent behavior – how an agent can learn to make good decisions given experience and rewards in a stochasticworld. Yet popular RL algorithms that have enabled exciting successes in domains with good simulators (Go, Atari, etc) still often fail to learn in other domains because they rely onsimple heuristics for exploration. This provides additional empirical justification for essential questions around RL, specifically around algorithms that learn in a provably efficient manner through strategic exploration in any considered domain. This thesis provides new algorithmsand theory that enable good performance with respect to existing theoretical frameworks for evaluating RL algorithms (specifically, probably approximately correct) and introduces new stronger evaluation criteria, that may be particularly of interest as RL is applied to more real world problems.For the first line of work on probably approximately correct (PAC) RL algorithms, we introduce a series of algorithms for episodic tabular domains with substantially better PACsample complexity bounds that culminate in a new algorithm with close to minimax optimal PAC and regret bounds. Look up tables are required by most sample efficient and computationally tractable algorithms, but cannot represent many practical domains. We therefore also present a new RL algorithm that can learn a good policy in environments with high dimensional observations and hidden deterministic states; unlike predecessors, this algorithm provablyexplores not only in a statistically but also computationally efficient manner assuming access to function classes with efficient optimization oracles. To make progress it is critical to have the right measures of success. While empiricaldemonstrations are quite clear, we find that for theoretical properties, two of the most commonly used learning frameworks, PAC guarantees and regret guarantees, each allow undesirable algorithm behavior (e.g. ignoring new observations that could improve the policy). We presenta new stronger learning framework called Uniform-PAC that unifies the existing frameworks and prevents undesirable algorithm properties. One caveat of all existing learning frameworks is that for any particular episode, we do notknow how well the algorithm will perform. To address this, we introduce the IPOC framework that requires algorithms to provide a certificate before each episode bounding how suboptimal the current policy can be. Such certifications may be of substantial interest in high stakes scenarios when an organization may wish to track or even pause an online RL system should the potential expected performance bound drop below a required expected outcome.
ThunderX2 Performance and Energy-Efficiency for HPC Workloads

In the last years, the energy efficiency of HPC systems is increasingly becoming of paramount importance for environmental, technical, and economical reasons. Several projects have investigated the use of different processors and accelerators in the quest of building systems able to achieve high energy efficiency levels for data centers and HPC installations. In this context, Arm CPU architecture has received a lot of attention given its wide use in low-power and energy-limited applications, but server grade processors have appeared on the market just recently. In this study, we targeted the Marvell ThunderX2, one of the latest Arm-based processors developed to fit the requirements of high performance computing applications. Our interest is mainly focused on the assessment in the context of large HPC installations, and thus we evaluated both computing performance and energy efficiency, using the ERT benchmark and two HPC production ready applications. We finally compared the results with other processors commonly used in large parallel systems and highlight the characteristics of applications which could benefit from the ThunderX2 architecture, in terms of both computing performance and energy efficiency. Pursuing this aim, we also describe how ERT has been modified and optimized for ThunderX2, and how to monitor power drain while running applications on this processor.
Review on E-Waste Management Strategies for Implementing Green Computing

Green computing is considered to the procedure of designing, manufacturing, using the products and managing them in a correct way to minimize the hazards from the environment. E-waste has become an emerging concern in the whole environment. The utilization of e-waste has become a non-avoidable problem that the environment and ecology face. This paper focuses on the description of e-waste management for implementing green computing. This paper helps out the reader to get knowledge about e-waste management and green computing and how they can influence each other in the activation process. Therefore, E-waste management is working as an approach to green computing.
Dinornis - An insular oddity, a taxonomic conundrum reviewed

The taxonomic history of the extinct moa genus Dinornis (Aves: Dinornithiformes) is reviewed. Until recently limb bone dimensions and island of origin (North or South) were the pre-eminent factors in species determination within the genus Dinornis due to the expectation that flightless birds on distinct landmasses could not be the same species. Recent morphological analyses applying modern concepts of biological variation reduced the number of acceptable taxa, but size remained of paramount importance in defining species boundaries. Recent analyses of mitochondrial and nuclear DNA have resulted in a radical new explanation of the size variation in Dinornis. Here we assess the new genetics-derived hypothesis of one species per island where the size variation seen in the morphometric data is due to reversed sexual dimorphism. Length data from main limb bones is analysed by region or site and demonstrates clear bimodality where averages for the male and female forms vary between regions/sites but move up or down in parallel. The regional datasets demonstrate that in the mid-late Holocene, birds were smallest in subalpine zones and montane forests and largest in low altitude and low rainfall regions such as Canterbury (in eastern South Island) and the Horowhenua coast north of Wellington in southern North Island.
INCREASING DIETARY HIGH-LINOLEATE SAFFLOWER OIL AFFECTS DUODENAL FLOW OF ESTERIFIED LINOLEATE IN WETHERS

We hypothesize that lambs fed a highconcentrate diet with high-linoleate safflower oil will have increased duodenal flow of 18:2c9,c12 (18:2, linoleic acid) due primarily to flow of esterified 18:2. Four crossbred wethers (BW = 44.3 ± 15.7 kg) fitted with ruminal and duodenal cannulae were used in a 4 × 4 Latin square experiment to determine duodenal fatty acid flow when fed an 80% concentrate diet with increasing levels of highlinoleate (77%) safflower oil (oil; 0, 3, 6, or 9% of DM). Wethers were fed diets, at 2% of BW, that were formulated to be isonitrogenous and included bromegrass hay, cracked corn, corn gluten meal, urea, limestone, plus oil. Duodenal digesta, collected to represent every 2 h in a theoretical 24h period, was extracted with chloroform, methanol, and water (1:2:0.8) in tubes containing tritridecanoate as internal standard. Half of each lipid extract was used for methyl ester preparation using 1.09 N HCl in methanol to catalyze methyl esterification of total (free plus esterified) fatty acids. The other half of each lipid extract was transesterified with 0.2 N KOH in methanol to catalyze methyl esterification of only esterified fatty acids. Fatty acid intake increased (linear, P < 0.004) with increased dietary oil. Duodenal flow of both total (linear, P = 0.007) and esterified (linear, P = 0.002) fatty acids increased from 0 to 9% dietary oil. Total to esterified fatty acid ratio of duodenal digesta increased (linear, P = 0.08) with increased dietary oil, indicating greater ruminal lipolysis with increased dietary oil. Ratio of total 18:2 to esterified 18:2 in duodenal digesta increased (linear, P = 0.06) from 0 to 9% dietary oil. However, within each treatment, duodenal flow of total 18:2 was not different (P ≥ 0.32) than that of esterified 18:2. Duodenal flow of 18:1t-11, a major biohydrogenation intermediate, increased (linear, P = 0.04) as dietary oil increased suggesting that biohydrogenation was less complete as dietary oil increased. We conclude that the increase in duodenal flow of 18:2 that occurred from increased dietary oil could be accounted for by increased flow of esterified 18:2.
Exploring quantum computation through the lens of classical simulation

It is widely believed that quantum computation has the potential to offer an ex- ponential speedup over classical devices. However, there is currently no definitive proof of this separation in computational power. Such a separation would in turn imply that quantum circuits cannot be efficiently simulated classically. However, it is well known that certain classes of quantum computations nonetheless admit an efficient classical description. Recent work has also argued that efficient classical simulation of quantum circuits would imply the collapse of the Polynomial Hierarchy, something which is commonly invoked in clas- sical complexity theory as a no-go theorem. This suggests a route for studying this ‘quantum advantage’ through classical simulations. This project looks at the problem of classically simulating quantum circuits through decompositions into stabilizer circuits. These are a restricted class of quantum computation which can be efficiently simulated classically. In this picture, the rank of the decomposition determines the temporal and spatial complexity of the simulation. We approach the problem by considering classical simulations of stabilizer circuits, introducing two new representations with novel features compared to previous meth- ods. We then examine techniques for building these so-called ‘stabilizer rank’ decom- positions, both exact and approximate. Finally, we combine these two ingredients to introduce an improved method for classically simulating broad classes of circuits using the stabilizer rank method.
Controller Placement for Resilient Network State Synchronization in Multi-Controller SDN

In a multi-controller SDN, the network state is replicated or distributed between the controllers requiring repeated synchronization. This synchronization cost depends on controller locations, as well as needs to be low and resilient to network failures. Leveraging a Steiner tree-based inter-controller latency model, a multi-objective integer linear program is presented to deduce the controller placement optimizing (a) synchronization cost in failure-free scenarios, and, (b) resilience against single-link failures. The proposed approach incurs low computational complexity and post-failure network reconfiguration cost, while achieving near-optimal post-failure synchronization cost. We extensively evaluate the proposed model over real networks, and demonstrate its effectiveness.
Dual‐Gated MoS2 Transistors for Synaptic and Programmable Logic Functions

Neuromorphic computers, which can store information and compute at the same time, have been considered to be a potential candidate for greatly improving computing efficiency. The development of high‐performance artificial synapses, which are the basic unit of brain‐like chips, is very important for realizing efficient neuromorphic computing. Here, a dual‐gated MoS2 transistor is designed to realize synaptic functions and programmable logic operations. The channel conductance is modulated via top electrolyte gating to mimic important synaptic functions, such as excitatory postsynaptic current, paired‐pulse facilitation, and spike‐timing dependent plasticity. The synaptic transistor exhibits ultra‐low energy consumption with 12.7 fJ. Furthermore, the MoS2 transistor can dynamically reconfigure the logic operations of “AND,” “OR,” and “NOT” by combining top electrolyte gating with back gating. Classical Pavlov's dog experiment can be simulated by the dual gated device. These results indicate that the proposed synaptic transistor has potential applications in realizing neuromorphic and programmable logic devices.
A 12-Gb/s 10-ns Turn-On Time Rapid ON/OFF Baud-Rate DFE Receiver in 65-nm CMOS

Rapid ON/OFF (ROO) operation helps scale power in accordance with link utilization. In this article, we present a baud-rate ROO receiver that can turn on in just 10 ns (~120 UI). Baud-rate clock and data recovery (CDR) is implemented using a new timing function that is amenable to operation with a loop un-rolled decision feedback equalizer (DFE). The receiver is turned on rapidly by sweeping the recovered clock phase across the received data bit by offsetting the digitally controlled oscillator (DCO) frequency at each power-ON event. This first ROO DFE receiver also includes a continuous-time linear equalizer (CTLE) and three-tap DFE to compensate up to 20-dB channel loss at Nyquist. Fabricated in a 65-nm CMOS process, the prototype receiver recovers 12 Gb/s with BER < 10−12 and achieves a >30-MHz JTOL corner, 377 $fs_{\text {rms}}$ recovered clock jitter, and 3.8-pJ/bit energy efficiency.
Opportunistic machine learning methods for effective insider threat detection

The topic of insider threat detection is getting an increased concern from academia, industry, and governments due to the growing number of malicious insider incidents. A malicious insider threat is devised of a set of anomalous behaviours attributed to an insider who exploit their privileges with the intention to compromise the confidentiality, integrity, or availability of the system or data. The existing approaches for detecting insider threats still have a common shortcoming, which is the high number of false alarms (false positives), which deceives the system administrator(s) about suspicious behaviour of many users. To address the shortcoming of false alarms, in this thesis, we formulate an opportunistic approach to detect insider threats with the aim of any-behaviour-all-threat detection. As a preliminary step, we apply feature engineering on the data logs of users’ behaviour. This work is conducted on synthetic CMU-CERT data sets which implement a variety of malicious insider threat scenarios. The maturity of data in an organisation is defined into three cases based on the availability of labelled data. We address the different cases of data maturity by proposing, developing, and evaluating machine learning approaches that incorporate techniques to reduce false alarms. The first presents a class imbalance approach, namely CD-AMOTRE, which combines the concept of Class Decomposition (CD) and a novel Artificial Minority Oversampling and Trapper REmoval (AMOTRE) technique. The second builds an adaptive one-class ensemble-based anomaly detection framework which introduces a progressive update method with an outlier aware artificial oversampling procedure. The third proposes a real-time anomaly detection approach, namely Ensemble of Random subspace Anomaly detectors In Data Streams (E-RAIDS). The proposed approaches detect most/all of the malicious insider threats, and achieve the minimum FP over the data sets compared to the existing machine learning approaches.
Review on Speed Breaker Detection System

Roads are designed such that vehicles can speed up and people can travel with comfort, from one place to another within the required time period. But at certain locations, like zebra crossings, schools, hospitals, sharp turns, residential streets and other crowded places there is a need for certain speed controlling devices for the safety of human beings, animals. This is achievable by installation of various different safety measures such road signs, speed cameras, road markings, illumination devices and speed breakers. In few situations, speed breakers are essential, but their use on roads is not a well thought-out engineering practice. There are many unmarked speed breakers on the roads which when are un-noticed cause severe accidents, suspension damage and tyre bursts. In this report the method of detecting humps and potholes and provide timely alerts to drivers to shun accidents or vehicle damages are put forward. An image processing system is used to become aware of the speed breaker. This output will be utilized for alerting the driver. Further, we plan to devise an automatic speed reducing mechanism, which will reduce the speed of the vehicle in case it is travelling at a speed higher than a threshold value. IndexTerms speed breakers, unmarked, detection, alerting.
Measures of Machine Translation Quality

We explore both manual and automatic methods of machine translation evaluation. We propose a manual evaluation method in which annotators rank only translations of short segments instead of whole sentences. This results in easier and more efficient annotation. We have conducted an annotation experiment and evaluated a set of MT systems using this method. The obtained results are very close to the official WMT14 evaluation results. We also use the collected database of annotations to automatically evaluate new, unseen systems and to tune parameters of a statistical machine translation system. The evaluation of unseen systems, however, does not work and we analyze the reasons. To explore the automatic methods, we organized Metrics Shared Task held during the Workshop of Statistical Machine Translation in years 2013 and 2014. We report the results of the last shared task, discuss various metaevaluation methods and analyze some of the participating metrics.
Prioritization of Failure Modes in Manufacturing Processes

In the contemporary automotive manufacturing industry, service providers are continuously working to improve system optimization in order to remain competitive in the market and deliver quality products to satisfy their customers. With this comes the possibility of failure, rejection and reworking of the components or services in the system, which can incur high costs and impact the reputation of an organization. This book uses Failure Mode and Effect Analysis (FMEA) to assess, investigate and predict the Risk Priority Number (RPN) of potential failures for three companies within the manufacturing industry: A metal component supplier in the automotive sector Part manufacturer for the automobile and engineering industries Manufacturer of suspension components for commercial vehicles
Inferring Passengers’ Interactive Choices on Public Transits via MA-AL: Multi-Agent Apprenticeship Learning

Public transports, such as subway lines and buses, offer affordable ride-sharing services and reduce the road network traffic. Extracting passengers’ preferences from their public transit choices is important to city planners but technically non-trivial. When traveling by taking public transits, passengers make sequences of transit choices, and their rewards are usually influenced by other passengers’ choices. This process can be modeled as a Markov Game (MG). In this paper, we make the first effort to model travelers’ preferences of making transit choices using MGs. Based on the discovery that passengers usually do not change their policies, we propose novel algorithms to extract reward functions from the observed deterministic equilibrium joint policy of all agents in a general-sum MG to infer travelers’ preferences. First, we assume we have the access to the entire joint policy. We characterize the set of all reward functions for which the given joint policy is a Nash equilibrium policy. In order to remove the degeneracy of the solution, we then attempt to pick reward functions so as to maximize the sum of the deviation between the the observed policy and the sub-optimal policy of each agent. This results in a skillfully solvable linear programming algorithm for the multi-agent inverse reinforcement learning (MA-IRL) problem. Then, we deal with the case where we have access to the equilibrium joint policy through a set of actual trajectories. We propose an iterative algorithm inspired by single-agent apprenticeship learning algorithms and the cyclic coordinate descent approach. We evaluate the proposed algorithms on both a simple Grid Game and a unique real-world dataset (from Shenzhen, China). Results show that when we have access to the full policy, our algorithm can efficiently recover most of the reward structure, especially the interaction of agents. In the case where we only have access to a set of sampled expert trajectories, our algorithm can provide an explanation of the expert trajectories. Measured with respect to the experts’ unknown reward function, the performance of the policy output by our algorithm is close to that of the expert policy.
Verification of Scheduling of Conditional Behaviors in High-Level Synthesis

High-level synthesis (HLS) technique translates the behaviors written in high-level languages like C/C++ into register transfer level (RTL) design. Due to its complexity, proving the correctness of an HLS tool is prohibitively expensive. Translation validation is the process of proving that the target code is a correct translation of the source program being compiled. The path-based equivalence checking (PBEC) method is a widely used translation validation method for verification of the scheduling phase of HLS. The existing PBEC methods cannot handle significant control structure modification that occurs in the efficient scheduling of conditional behaviors. Hence, they produce a false-negative result. In this article, we identify some scenarios involving path merge/split where the state-of-the-art PBEC approaches fail to show the equivalence even though behaviors are equivalent. We propose a value propagation-based PBEC method along with a new cutpoint selection scheme to overcome this limitation. Our method can also handle the scenario where adjacent conditional blocks (CBs) having an equivalent conditional expression are combined into one CB. Experimental results demonstrate the usefulness of our method over the existing methods.
A Survey of RANSAC enhancements for Plane Detection in 3D Point Clouds

Planar surfaces are distinguished features of man-made environment, which are used in many computer vision applications such as object detection, motion segmentation, 3D scene reconstruction, and 3D mapping. One of the most used technique for robust plane detection is the RANdom SAmple Consensus (RANSAC), which is a global iterative method for estimating the parameters of a certain model from input data points contaminated by a set of outliers (noisy data). Unfortunately, the standard RANSAC suffers from some problems regarding the processing time, accuracy of fitting data, and finding an optimal solution. This paper gives a review study of the most recent RANSAC enhancements techniques. In addition, it covers the solving techniques for the speed, accuracy and optimality problems.