Miracle-dz/UniFAR-FTUs huggingface dataset:
IBP Regularization for Verified Adversarial Robustness via Branch-and-Bound

Recent works have tried to increase the verifiability of adversarially trained networks by running the attacks over domains larger than the original perturbations and adding various regularization terms to the objective. However, these algorithms either underperform or require complex and expensive stage-wise training procedures, hindering their practical applicability. We present IBP-R, a novel verified training algorithm that is both simple and effective. IBP-R induces network verifiability by coupling adversarial attacks on enlarged domains with a regularization term, based on inexpensive interval bound propagation, that minimizes the gap between the non-convex verification problem and its approximations. By leveraging recent branch-and-bound frameworks, we show that IBP-R obtains state-of-the-art verified robustness-accuracy trade-offs for small perturbations on CIFAR-10 while training significantly faster than relevant previous work. Additionally, we present UPB, a novel branching strategy that, relying on a simple heuristic based on $\beta$-CROWN, reduces the cost of state-of-the-art branching algorithms while yielding splits of comparable quality.
Meta-learning over time for destination prediction tasks

A need to understand and predict vehicles' behavior underlies both public and private goals in the transportation domain, including urban planning and management, ride-sharing services, and intelligent transportation systems. Individuals' preferences and intended destinations vary throughout the day, week, and year: for example, bars are most popular in the evenings, and beaches are most popular in the summer. Despite this principle, we note that recent studies on a popular benchmark dataset from Porto, Portugal have found, at best, only marginal improvements in predictive performance from incorporating temporal information. We propose an approach based on hypernetworks, a variant of meta-learning ("learning to learn") in which a neural network learns to change its own weights in response to an input. In our case, the weights responsible for destination prediction vary with the metadata, in particular the time, of the input trajectory. The time-conditioned weights notably improve the model's error relative to ablation studies and comparable prior work, and we confirm our hypothesis that knowledge of time should improve prediction of a vehicle's intended destination.
NERDA-Con: Extending NER models for Continual Learning - Integrating Distinct Tasks and Updating Distribution Shifts

With increasing applications in areas such as biomedical information extraction pipelines and social media analytics, Named Entity Recognition (NER) has become an indispensable tool for knowledge extraction. However, with the gradual shift in language structure and vocabulary, NERs are plagued with distribution shifts, making them redundant or not as profitable without re-training. Re-training NERs based on Large Language Models (LLMs) from scratch over newly acquired data poses economic disadvantages. In contrast, re-training only with newly acquired data will result in Catastrophic Forgetting of previously acquired knowledge. Therefore, we propose NERDA-Con, a pipeline for training NERs with LLM bases by incorporating the concept of Elastic Weight Consolidation (EWC) into the NER fine-tuning NERDA pipeline. As we believe our work has implications to be utilized in the pipeline of continual learning and NER, we open-source our code as well as provide the fine-tuning library of the same name NERDA-Con at https://github.com/SupritiVijay/NERDA-Con and https://pypi.org/project/NERDA-Con/.
Method Description : ” Agile Requirements Prioritization : What Happens in Practice and What Is Described in Literature ” Garm Lucassen , 3484572 , Group 3 By :

Agile software development consists of a multitude of iterations in between which the decision what requirements to implement is re-evaluated by a process of requirements re-prioritization. In 2010, Racheva, Daneva, Herrmann, and Wieringa (2010a) examined 22 different Requirements Prioritization (RP) methods and extracted an abstracted, conceptual model of of the RP process from a generic perspective. Four months later, Racheva, Daneva, and Herrmann (2010b) present an improved version of this model based on a case study among 11 practitioners of companies that follow agile methodologies. The paper discussed in this method description (Bakalova, Daneva, Herrmann, & Wieringa, 2011) presents the same improved version of this model. In fact, a majority of this paper is identical to Racheva et al. (2010b), with the addition of mapping the concepts of the aforementioned 22 RP methods to the aspects of the improved model. The end-goal of the method is to inform decision-makers (clients) of the important concepts that are considered during a RP iteration by practitioners. In the end, this information helps the client in three ways: ”(i) to navigate through the agile process of delivering business value; (ii) to make explicit the tacit assumptions in different RP methods; (iii) to identify those possible pieces/sources of information important to the outcome of the prioritization and, consequently, to the project. ” The method consists of eight aspects that clients consider when making decisions on requirements priorities: (1) Project Context consisting of project settings such as ’project size’ or ’size of the client’s organization’, which also has a significant impact on (2) Prioritization Criteria together with estimated Business Value, (3) Estimated Size broken down into functional size of a requirement and a size/effort estimation, (4) Input from the Developer providing advice and alternative solutions, (5) External Change represents events during the project that impact the company, business environment or product, (6) Learning Experiences comprises of new insights acquired by the client and/or the developers, (7) Project Constraints such as release date or budget and (8) Dependencies between requirements both architectural or chronological. Figure 1 connects all aspects to the relevant steps in a RP process.
Cyclical Kernel Adaptive Metropolis

We propose cKAM, cyclical Kernel Adaptive Metropolis, which incorporates a cyclical stepsize scheme to allow control for exploration and sampling. We show that on a crafted bimodal distribution, existing Adaptive Metropolis type algorithms would fail to converge to the true posterior distribution. We point out that this is because adaptive samplers estimates the local/global covariance structure using past history of the chain, which will lead to adaptive algorithms be trapped in a local mode. We demonstrate that cKAM encourages exploration of the posterior distribution and allows the sampler to escape from a local mode, while maintaining the high performance of adaptive methods.
Spherical Channels for Modeling Atomic Interactions

Modeling the energy and forces of atomic systems is a fundamental problem in computational chemistry with the potential to help address many of the world's most pressing problems, including those related to energy scarcity and climate change. These calculations are traditionally performed using Density Functional Theory, which is computationally very expensive. Machine learning has the potential to dramatically improve the efficiency of these calculations from days or hours to seconds. We propose the Spherical Channel Network (SCN) to model atomic energies and forces. The SCN is a graph neural network where nodes represent atoms and edges their neighboring atoms. The atom embeddings are a set of spherical functions, called spherical channels, represented using spherical harmonics. We demonstrate, that by rotating the embeddings based on the 3D edge orientation, more information may be utilized while maintaining the rotational equivariance of the messages. While equivariance is a desirable property, we find that by relaxing this constraint in both message passing and aggregation, improved accuracy may be achieved. We demonstrate state-of-the-art results on the large-scale Open Catalyst dataset in both energy and force prediction for numerous tasks and metrics.
Technical Report for CVPR 2022 LOVEU AQTC Challenge

This technical report presents the 2nd winning model for AQTC, a task newly introduced in CVPR 2022 LOng-form VidEo Understanding (LOVEU) challenges. This challenge faces difficulties with multi-step answers, multi-modal, and diverse and changing button representations in video. We address this problem by proposing a new context ground module attention mechanism for more effective feature mapping. In addition, we also perform the analysis over the number of buttons and ablation study of different step networks and video features. As a result, we achieved the overall 2nd place in LOVEU competition track 3, specifically the 1st place in two out of four evaluation metrics. Our code is available at https://github.com/jaykim9870/ CVPR-22_LOVEU_unipyler.
Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing?

This work investigates the compatibility between label smoothing (LS) and knowledge distillation (KD). Contemporary findings addressing this thesis statement take dichotomous standpoints: Muller et al. (2019) and Shen et al. (2021b). Critically, there is no effort to understand and resolve these contradictory findings, leaving the primal question -- to smooth or not to smooth a teacher network? -- unanswered. The main contributions of our work are the discovery, analysis and validation of systematic diffusion as the missing concept which is instrumental in understanding and resolving these contradictory findings. This systematic diffusion essentially curtails the benefits of distilling from an LS-trained teacher, thereby rendering KD at increased temperatures ineffective. Our discovery is comprehensively supported by large-scale experiments, analyses and case studies including image classification, neural machine translation and compact student distillation tasks spanning across multiple datasets and teacher-student architectures. Based on our analysis, we suggest practitioners to use an LS-trained teacher with a low-temperature transfer to achieve high performance students. Code and models are available at https://keshik6.github. io/revisiting-ls-kd-compatibility/
Online vs. Offline Adaptive Domain Randomization Benchmark

Physics simulators have shown great promise for conveniently learning reinforcement learning policies in safe, unconstrained environments. However, transferring the acquired knowledge to the real world can be challenging due to the reality gap. To this end, several methods have been recently proposed to automatically tune simulator parameters with posterior distributions given real data, for use with domain randomization at training time. These approaches have been shown to work for various robotic tasks under different settings and assumptions. Nevertheless, existing literature lacks a thorough comparison of existing adaptive domain randomization methods with respect to transfer performance and real-data efficiency. In this work, we present an open benchmark for both offline and online methods (SimOpt, BayRn, DROID, DROPO), to shed light on which are most suitable for each setting and task at hand. We found that online methods are limited by the quality of the currently learned policy for the next iteration, while offline methods may sometimes fail when replaying trajectories in simulation with open-loop commands. The code used will be released at https://github.com/gabrieletiboni/adr-benchmark.
A Deep Learning Approach to Create DNS Amplification Attacks

In recent years, deep learning has shown itself to be an incredibly valuable tool in cybersecurity as it helps network intrusion detection systems to classify attacks and detect new ones. Adversarial learning is the process of utilizing machine learning to generate a perturbed set of inputs to then feed to the neural network to misclassify it. Much of the current work in the field of adversarial learning has been conducted in image processing and natural language processing with a wide variety of algorithms. Two algorithms of interest are the Elastic-Net Attack on Deep Neural Networks and TextAttack. In our experiment the EAD and TextAttack algorithms are applied to a Domain Name System amplification classifier. The algorithms are used to generate malicious Distributed Denial of Service adversarial examples to then feed as inputs to the network intrusion detection systems neural network to classify as valid traffic. We show in this work that both image processing and natural language processing adversarial learning algorithms can be applied against a network intrusion detection neural network.
Biesenbach Hemodialysis in patients older than 65 years with end-stage renal failure Comparison of outcome in patients with and without diabetes

Background During recent years, the number of patients with end-stage renal disease (ESRD) has been rising worldwide. Especially older patients and those with diabetes contribute to this rise. The aim of the present study was to evaluate whether patients older than 65 years with type 2 diabetes, who started first dialysis, have a higher prevalence of vascular diseases and co-morbidities, show a higher incidence of vascular complications and/or have a higher mortality rate than elderly non-diabetic patients with ESRD. Patients and methods In this study, 82 consecutive patients with ESRD, who had either type 2 diabetes or did not have diabetes and who had started chronic hemodialysis (HD) in our dialysis center during the years 1994 to 2002, were included. Patients were included when they were older than 65 years. Patients were divided into two groups: those with diabetes (DM) (n = 47) and those without diabetes (nDM) (n = 35). For both groups the number of co-morbidities as well as the prevalence of vascular diseases and vascular risk-factors at the start of HD was evaluated. In addition, the incidence of vascular complications was registered over a 3-year observation period. In both groups serum(S)-creatinine, blood urea nitrogen (BUN), creatinine clearance, hemoglobin, fasting blood glucose, HbA1c (in diabetic patients), cholesterol, triglycerides and phosphorus were evaluated. Results At the start of HD the creatinine clearance was significantly higher in diabetic subjects (nDM 7.1 ± 2.1 vs DM 9.5 ± 4.4 ml/min/1.73 m; p < 0.005). For S-creatinine the difference was not statistically significant (nDM 8.7 ± 3.3 mg/dl vs DM 7.4 ± 2.4; p = 0.07). Fasting blood glucose was significantly higher in diabetic patients (p < 0.001). BUN, hemoglobin, phosphorus and lipids were not significantly different in both groups. There was no statistically significant difference in systolic or diastolic blood pressure, but a higher amount of antihypertensive drugs were necessary in the DM group for blood pressure control (p < 0.01). In both groups there was a high prevalence of vascular diseases at the start of HD. In the diabetic patients the prevalence of peripheral vascular disease was significantly higher. Furthermore, in the first and second year significantly more vascular complications were observed in the DM group (p < 0.01). Survival was low in both groups. The 3-year survival was 20.0% in non-diabetic and 17.0% in diabetic patients (NS). Conclusion Patients older than 65 years with ESRD have a low survival with and without type 2 diabetes. The mortality rate was only slightly higher in the diabetic group and was not statistically significant. The prevalence of vascular diseases and co-morbidities is high in both groups; however, the incidence of cardiovascular complications was significantly higher in our diabetic patients.
Applications of Reinforcement Learning in Finance - Trading with a Double Deep Q-Network

This paper presents a Double Deep Q-Network algorithm for trading single assets, namely the E-mini S&P 500 continuous futures contract. We use a proven setup as the foundation for our environment with multiple extensions. The features of our trading agent are constantly being expanded to include additional assets such as commodities, resulting in four models. We also respond to environmental conditions, including costs and crises. Our trading agent is first trained for a specific time period and tested on new data and compared with the long-and-hold strategy as a benchmark (market). We analyze the differences between the various models and the in-sample/out-of-sample performance with respect to the environment. The experimental results show that the trading agent follows an appropriate behavior. It can adjust its policy to different circumstances, such as more extensive use of the neutral position when trading costs are present. Furthermore, the net asset value exceeded that of the benchmark, and the agent outperformed the market in the test set. We provide initial insights into the behavior of an agent in a financial domain using a DDQN algorithm. The results of this study can be used for further development.
Phase transitions for support recovery under local differential privacy

We address the problem of variable selection in a high-dimensional but sparse mean model, under the additional constraint that only privatised data are available for inference. The original data are vectors with independent entries having a symmetric, strongly log-concave distribution on $\mathbb{R}$. For this purpose, we adopt a recent generalisation of classical minimax theory to the framework of local $\alpha-$differential privacy. We provide lower and upper bounds on the rate of convergence for the expected Hamming loss over classes of at most $s$-sparse vectors whose non-zero coordinates are separated from $0$ by a constant $a>0$. As corollaries, we derive necessary and sufficient conditions (up to log factors) for exact recovery and for almost full recovery. When we restrict our attention to non-interactive mechanisms that act independently on each coordinate our lower bound shows that, contrary to the non-private setting, both exact and almost full recovery are impossible whatever the value of $a$ in the high-dimensional regime such that $n \alpha^2/ d^2\lesssim 1$. However, in the regime $n\alpha^2/d^2\gg \log(d)$ we can exhibit a critical value $a^*$ (up to a logarithmic factor) such that exact and almost full recovery are possible for all $a\gg a^*$ and impossible for $a\leq a^*$. We show that these results can be improved when allowing for all non-interactive (that act globally on all coordinates) locally $\alpha-$differentially private mechanisms in the sense that phase transitions occur at lower levels.
Squirrels Can Little Remember: A Random Walk with Jump Reversals Induced by a Discrete-Time Renewal Process

We consider a class of discrete-time random walks with directed unit steps on the integer line. The direction of the steps is reversed at the time instants of events in a discrete-time renewal process and is maintained at uneventful time instants. This model represents a discrete-time semi-Markovian generalization of the telegraph process. We derive exact formulae for the propagator using generating functions. We prove that for geometrically distributed waiting times in the diffusive limit, this walk converges to the classical telegraph process. We consider the large-time asymptotics of the expected position: For waiting time densities with ﬁnite mean the walker remains in the average localized close to the departure site whereas escapes for fat-tailed waiting-time densities (i.e. densities with inﬁnite mean) by a sublinear power-law. We explore anomalous diffusion features by accounting for the ‘aging effect’ as a hallmark of non-Markovianity where the discrete-time version of the ‘aging renewal process’ comes into play. By deriving pertinent distributions of this process we obtain explicit formulae for the variance when the waiting-times are Sibuya-distributed. In this case and generally for fat-tailed waiting time PDFs a t 2 -ballistic superdiffusive scaling emerges in the large time limit. In contrast if the waiting time PDF between the step reversals is light-tailed (‘narrow’ with ﬁnite mean and variance) the walk exhibits normal diffusion and for ‘broad’ waiting time PDFs (with ﬁnite mean and inﬁnite variance) superdiffusive large time scaling. We also consider time-changed versions where the walk is subordinated to a continuous-time point process such as the time-fractional Poisson process. This deﬁnes a new class of biased continuous-time random walks exhibiting several regimes of anomalous diffusion.
Traffic Management of Autonomous Vehicles using Policy Based Deep Reinforcement Learning and Intelligent Routing

Deep Reinforcement Learning (DRL) uses diverse, unstructured data and makes RL capable of learning complex policies in high dimensional environments. Intelligent Transportation System (ITS) based on Autonomous Vehicles (AVs) offers an excellent playground for policy-based DRL. Deep learning architectures solve computational challenges of traditional algorithms while helping in real-world adoption and deployment of AVs. One of the main challenges in AVs implementation is that it can worsen traffic congestion on roads if not reliably and efficiently managed. Considering each vehicle's holistic effect and using efficient and reliable techniques could genuinely help optimise traffic flow management and congestion reduction. For this purpose, we proposed a intelligent traffic control system that deals with complex traffic congestion scenarios at intersections and behind the intersections. We proposed a DRL-based signal control system that dynamically adjusts traffic signals according to the current congestion situation on intersections. To deal with the congestion on roads behind the intersection, we used re-routing technique to load balance the vehicles on road networks. To achieve the actual benefits of the proposed approach, we break down the data silos and use all the data coming from sensors, detectors, vehicles and roads in combination to achieve sustainable results. We used SUMO micro-simulator for our simulations. The significance of our proposed approach is manifested from the results.
CoAP-DoS: An IoT Network Intrusion Data Set

The need for secure Internet of Things (IoT) devices is growing as IoT devices are becoming more integrated into vital networks. Many systems rely on these devices to remain available and provide reliable service. Denial of service attacks against IoT devices are a real threat due to the fact these low power devices are very susceptible to denial-of-service attacks. Machine learning enabled network intrusion detection systems are effective at identifying new threats, but they require a large amount of data to work well. There are many network traffic data sets but very few that focus on IoT network traffic. Within the IoT network data sets there is a lack of CoAP denial of service data. We propose a novel data set covering this gap. We develop a new data set by collecting network traffic from real CoAP denial of service attacks and compare the data on multiple different machine learning classifiers. We show that the data set is effective on many classifiers.