OpenRaiser/Intern-Atlas huggingface dataset:
Predicting the Probability of Collision of a Satellite with Space Debris: A Bayesian Machine Learning Approach
Venue: arXiv cs.LG 2023
Year: 2023

Space is becoming more crowded in Low Earth Orbit due to increased space activity. Such a dense space environment increases the risk of collisions between space objects endangering the whole space population. Therefore, the need to consider collision avoidance as part of routine operations is evident to satellite operators. Current procedures rely on the analysis of multiple collision warnings by human analysts. However, with the continuous growth of the space population, this manual approach may become unfeasible, highlighting the importance of automation in risk assessment. In 2019, ESA launched a competition to study the feasibility of applying machine learning in collision risk estimation and released a dataset that contained sequences of Conjunction Data Messages (CDMs) in support of real close encounters. The competition results showed that the naive forecast and its variants are strong predictors for this problem, which suggests that the CDMs may follow the Markov property. The proposed work investigates this theory by benchmarking Hidden Markov Models (HMM) in predicting the risk of collision between two resident space objects by using one feature of the entire dataset: the sequence of the probability in the CDMs. In addition, Bayesian statistics are used to infer a joint distribution for the parameters of the models, which allows the development of robust and reliable probabilistic predictive models that can incorporate physical or prior knowledge about the problem within a rigorous theoretical framework and provides prediction uncertainties that nicely reflect the accuracy of the predicted risk. This work shows that the implemented HMM outperforms the naive solution in some metrics, which further adds to the idea that the collision warnings may be Markovian and suggests that this is a powerful method to be further explored.
Online Calibration of Deep Learning Sub-Models for Hybrid Numerical Modeling Systems
Venue: arXiv cs.LG 2023
Year: 2023

Artificial intelligence and deep learning are currently reshaping numerical simulation frameworks by introducing new modeling capabilities. These frameworks are extensively investigated in the context of model correction and parameterization where they demonstrate great potential and often outperform traditional physical models. Most of these efforts in defining hybrid dynamical systems follow {offline} learning strategies in which the neural parameterization (called here sub-model) is trained to output an ideal correction. Yet, these hybrid models can face hard limitations when defining what should be a relevant sub-model response that would translate into a good forecasting performance. End-to-end learning schemes, also referred to as online learning, could address such a shortcoming by allowing the deep learning sub-models to train on historical data. However, defining end-to-end training schemes for the calibration of neural sub-models in hybrid systems requires working with an optimization problem that involves the solver of the physical equations. Online learning methodologies thus require the numerical model to be differentiable, which is not the case for most modeling systems. To overcome this difficulty and bypass the differentiability challenge of physical models, we present an efficient and practical online learning approach for hybrid systems. The method, called EGA for Euler Gradient Approximation, assumes an additive neural correction to the physical model, and an explicit Euler approximation of the gradients. We demonstrate that the EGA converges to the exact gradients in the limit of infinitely small time steps. Numerical experiments are performed on various case studies, including prototypical ocean-atmosphere dynamics. Results show significant improvements over offline learning, highlighting the potential of end-to-end online learning for hybrid modeling.
Deep Neuromorphic Networks with Superconducting Single Flux Quanta
Venue: arXiv cs.ET 2023
Year: 2023

Conventional semiconductor-based integrated circuits are gradually approaching fundamental scaling limits. Many prospective solutions have recently emerged to supplement or replace both the technology on which basic devices are built and the architecture of data processing. Neuromorphic circuits are a promising approach to computing where techniques used by the brain to achieve high efficiency are exploited. Many existing neuromorphic circuits rely on unconventional and useful properties of novel technologies to better mimic the operation of the brain. One such technology is single flux quantum (SFQ) logic -- a cryogenic superconductive technology in which the data are represented by quanta of magnetic flux (fluxons) produced and processed by Josephson junctions embedded within inductive loops. The movement of a fluxon within a circuit produces a quantized voltage pulse (SFQ pulse), resembling a neuronal spiking event. These circuits routinely operate at clock frequencies of tens to hundreds of gigahertz, making SFQ a natural technology for processing high frequency pulse trains. Prior proposals for SFQ neural networks often require energy-expensive fluxon conversions, involve heterogeneous technologies, or exclusively focus on device level behavior. In this paper, a design methodology for deep single flux quantum neuromorphic networks is presented. Synaptic and neuronal circuits based on SFQ technology are presented and characterized. Based on these primitives, a deep neuromorphic XOR network is evaluated as a case study, both at the architectural and circuit levels, achieving wide classification margins. The proposed methodology does not employ unconventional superconductive devices or semiconductor transistors. The resulting networks are tunable by an external current, making this proposed system an effective approach for scalable cryogenic neuromorphic computing.
Unveiling the Potential of Big Data Analytics for Transforming Higher Education in Bangladesh; Needs, Prospects, and Challenges
Venue: arXiv cs.CY 2023
Year: 2023

Big Data Analytics has gained tremendous momentum in many sectors worldwide. Big Data has substantial influence in the field of Learning Analytics that may allow academic institutions to better understand the learners needs and proactively address them. Hence, it is essential to understand Big Data and its application. With the capability of Big Data to find a broad understanding of the scientific decision making process, Big Data Analytics (BDA) can be a piece of the answer to accomplishing Bangladesh Higher Education (BHE) objectives. This paper reviews the capacity of BDA, considers possible applications in BHE, gives an insight into how to improve the quality of education or uncover additional values from the data generated by educational institutions, and lastly, identifies needs and difficulties, opportunities, and some frameworks to probable implications about the BDA in BHE sector. Keywords; Big Data Analytics, Learning Analytics, Quality of Education, Challenges, Higher Education, Bangladesh
AI-enhanced Auto-correction of Programming Exercises: How Effective is GPT-3.5?
Venue: arXiv cs.CY 2023
Year: 2023

Timely formative feedback is considered as one of the most important drivers for effective learning. Delivering timely and individualized feedback is particularly challenging in large classes in higher education. Recently Large Language Models such as GPT-3 became available to the public that showed promising results on various tasks such as code generation and code explanation. This paper investigates the potential of AI in providing personalized code correction and generating feedback. Based on existing student submissions of two different real-world assignments, the correctness of the AI-aided e-assessment as well as the characteristics such as fault localization, correctness of hints, and code style suggestions of the generated feedback are investigated. The results show that 73 % of the submissions were correctly identified as either correct or incorrect. In 59 % of these cases, GPT-3.5 also successfully generated effective and high-quality feedback. Additionally, GPT-3.5 exhibited weaknesses in its evaluation, including localization of errors that were not the actual errors, or even hallucinated errors. Implications and potential new usage scenarios are discussed.
Evaluation of Crowdsourced Data on Unplowed Roads
Venue: arXiv cs.CY 2023
Year: 2023

Transportation agencies routinely collect weather data to support maintenance activities. With the proliferation of smartphones, many agencies have begun using crowdsourced data in operations. This study evaluates a novel unplowed roads dataset from the largest crowdsourced transportation data provider Waze. User-reported unplowed roads in Virginia were compared to national and state weather data for accuracy, and found 81% of reports were near known snow events, with false positives occurring at a regular rate of approximately 10 per day statewide. Reports were largely located on primary roads, limiting the usefulness for transportation agencies who may be most concerned with poorly monitored secondary roads. An effort to encourage unplowed road reporting in Waze through targeted messages on social media did not increase participation. Low reporting may be due to the feature's novelty, recent mild winters, or COVID-19 school and business closures.
Comparing Generalization in Learning with Limited Numbers of Exemplars: Transformer vs. RNN in Attractor Dynamics
Venue: arXiv cs.CL 2023
Year: 2023

ChatGPT, a widely-recognized large language model (LLM), has recently gained substantial attention for its performance scaling, attributed to the billions of web-sourced natural language sentences used for training. Its underlying architecture, Transformer, has found applications across diverse fields, including video, audio signals, and robotic movement. %The crucial question this raises concerns the Transformer's generalization-in-learning (GIL) capacity. However, this raises a crucial question about Transformer's generalization in learning (GIL) capacity. Is ChatGPT's success chiefly due to the vast dataset used for training, or is there more to the story? To investigate this, we compared Transformer's GIL capabilities with those of a traditional Recurrent Neural Network (RNN) in tasks involving attractor dynamics learning. For performance evaluation, the Dynamic Time Warping (DTW) method has been employed. Our simulation results suggest that under conditions of limited data availability, Transformer's GIL abilities are markedly inferior to those of RNN.
Memory Augmented Language Models through Mixture of Word Experts
Venue: arXiv cs.CL 2023
Year: 2023

Scaling up the number of parameters of language models has proven to be an effective approach to improve performance. For dense models, increasing model size proportionally increases the model's computation footprint. In this work, we seek to aggressively decouple learning capacity and FLOPs through Mixture-of-Experts (MoE) style models with large knowledge-rich vocabulary based routing functions and experts. Our proposed approach, dubbed Mixture of Word Experts (MoWE), can be seen as a memory augmented model, where a large set of word-specific experts play the role of a sparse memory. We demonstrate that MoWE performs significantly better than the T5 family of models with similar number of FLOPs in a variety of NLP tasks. Additionally, MoWE outperforms regular MoE models on knowledge intensive tasks and has similar performance to more complex memory augmented approaches that often require to invoke custom mechanisms to search the sparse memory.
Exponentially Faster Language Modelling
Venue: arXiv cs.CL 2023
Year: 2023

Language models only really need to use an exponential fraction of their neurons for individual inferences. As proof, we present UltraFastBERT, a BERT variant that uses 0.3% of its neurons during inference while performing on par with similar BERT models. UltraFastBERT selectively engages just 12 out of 4095 neurons for each layer inference. This is achieved by replacing feedforward networks with fast feedforward networks (FFFs). While no truly efficient implementation currently exists to unlock the full acceleration potential of conditional neural execution, we provide high-level CPU code achieving 78x speedup over the optimized baseline feedforward implementation, and a PyTorch implementation delivering 40x speedup over the equivalent batched feedforward inference. We publish our training code, benchmarking setup, and model weights.
ToolTalk: Evaluating Tool-Usage in a Conversational Setting
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) have displayed massive improvements in reasoning and decision-making skills and can hold natural conversations with users. Many recent works seek to augment LLM-based assistants with external tools so they can access private or up-to-date information and carry out actions on behalf of users. To better measure the performance of these assistants, this paper introduces ToolTalk, a benchmark consisting of complex user intents requiring multi-step tool usage specified through dialogue. ToolTalk contains 28 tools grouped into 7 plugins, and includes a complete simulated implementation of each tool, allowing for fully automated evaluation of assistants that rely on execution feedback. ToolTalk also emphasizes tools that externally affect the world rather than only tools for referencing or searching information. We evaluate GPT-3.5 and GPT-4 on ToolTalk resulting in success rates of 26% and 50% respectively. Our analysis of the errors reveals three major categories and suggests some future directions for improvement. We release ToolTalk at https://github.com/microsoft/ToolTalk.
Extending Neural Network Verification to a Larger Family of Piece-wise Linear Activation Functions
Venue: arXiv cs.LG 2023
Year: 2023

In this paper, we extend an available neural network verification technique to support a wider class of piece-wise linear activation functions. Furthermore, we extend the algorithms, which provide in their original form exact respectively over-approximative results for bounded input sets represented as start sets, to allow also unbounded input set. We implemented our algorithms and demonstrated their effectiveness in some case studies.
Towards a Standardized Reinforcement Learning Framework for AAM Contingency Management
Venue: arXiv cs.LG 2023
Year: 2023

Advanced Air Mobility (AAM) is the next generation of air transportation that includes new entrants such as electric vertical takeoff and landing (eVTOL) aircraft, increasingly autonomous flight operations, and small UAS package delivery. With these new vehicles and operational concepts comes a desire to increase densities far beyond what occurs today in and around urban areas, to utilize new battery technology, and to move toward more autonomously-piloted aircraft. To achieve these goals, it becomes essential to introduce new safety management system capabilities that can rapidly assess risk as it evolves across a span of complex hazards and, if necessary, mitigate risk by executing appropriate contingencies via supervised or automated decision-making during flights. Recently, reinforcement learning has shown promise for real-time decision making across a wide variety of applications including contingency management. In this work, we formulate the contingency management problem as a Markov Decision Process (MDP) and integrate the contingency management MDP into the AAM-Gym simulation framework. This enables rapid prototyping of reinforcement learning algorithms and evaluation of existing systems, thus providing a community benchmark for future algorithm development. We report baseline statistical information for the environment and provide example performance metrics.
A novel post-hoc explanation comparison metric and applications
Venue: arXiv cs.LG 2023
Year: 2023

Explanatory systems make the behavior of machine learning models more transparent, but are often inconsistent. To quantify the differences between explanatory systems, this paper presents the Shreyan Distance, a novel metric based on the weighted difference between ranked feature importance lists produced by such systems. This paper uses the Shreyan Distance to compare two explanatory systems, SHAP and LIME, for both regression and classification learning tasks. Because we find that the average Shreyan Distance varies significantly between these two tasks, we conclude that consistency between explainers not only depends on inherent properties of the explainers themselves, but also the type of learning task. This paper further contributes the XAISuite library, which integrates the Shreyan distance algorithm into machine learning pipelines.
The Otbot project: Dynamic modelling, parameter identification, and motion control of an omnidirectional tire-wheeled robot
Venue: arXiv cs.RO 2023
Year: 2023

In recent years, autonomous mobile platforms are finding an increasing range of applications in inspection or surveillance tasks, or to the transport of objects, in places such as smart warehouses, factories or hospitals. In these environments it is useful for the robot to have omnidirectional capability in the plane, so it can navigate through narrow or cluttered areas, or make position and orientation changes without having to maneuver. While this capability is usually achieved with directional sliding wheels, this work studies a particular robot that achieves omnidirectionality using conventional wheels, which are easier to manufacture and maintain, and support larger loads in general. This robot, which we call ``Otbot'' (for omnidirectional tire-wheeled robot), was already conceived in the late 1990s, but all the controllers that have been proposed for it are based on purely kinematic models so far. These controllers may be sufficient if the robot is light, or if its motors are powerful, but on platforms that have to carry large loads, or that have more limited motors, it is necessary to resort to control laws based on dynamic models if the full acceleration capacities are to be exploited. This work develops a dynamic model of Otbot, proposes a plausible methodology to identify its parameters, and designs a control law that, using this model, is able to track prescribed trajectories in an accurate and robust manner.
Formal concept analysis for evaluating intrinsic dimension of a natural language
Venue: arXiv cs.CL 2023
Year: 2023

Some results of a computational experiment for determining the intrinsic dimension of linguistic varieties for the Bengali and Russian languages are presented. At the same time, both sets of words and sets of bigrams in these languages were considered separately. The method used to solve this problem was based on formal concept analysis algorithms. It was found that the intrinsic dimensions of these languages are significantly less than the dimensions used in popular neural network models in natural language processing.
PACOL: Poisoning Attacks Against Continual Learners
Venue: arXiv cs.LG 2023
Year: 2023

Continual learning algorithms are typically exposed to untrusted sources that contain training data inserted by adversaries and bad actors. An adversary can insert a small number of poisoned samples, such as mislabeled samples from previously learned tasks, or intentional adversarial perturbed samples, into the training datasets, which can drastically reduce the model's performance. In this work, we demonstrate that continual learning systems can be manipulated by malicious misinformation and present a new category of data poisoning attacks specific for continual learners, which we refer to as {\em Poisoning Attacks Against Continual Learners} (PACOL). The effectiveness of labeling flipping attacks inspires PACOL; however, PACOL produces attack samples that do not change the sample's label and produce an attack that causes catastrophic forgetting. A comprehensive set of experiments shows the vulnerability of commonly used generative replay and regularization-based continual learning approaches against attack methods. We evaluate the ability of label-flipping and a new adversarial poison attack, namely PACOL proposed in this work, to force the continual learning system to forget the knowledge of a learned task(s). More specifically, we compared the performance degradation of continual learning systems trained on benchmark data streams with and without poisoning attacks. Moreover, we discuss the stealthiness of the attacks in which we test the success rate of data sanitization defense and other outlier detection-based defenses for filtering out adversarial samples.
Journey of Hallucination-minimized Generative AI Solutions for Financial Decision Makers
Venue: arXiv cs.CL 2023
Year: 2023

Generative AI has significantly reduced the entry barrier to the domain of AI owing to the ease of use and core capabilities of automation, translation, and intelligent actions in our day to day lives. Currently, Large language models (LLMs) that power such chatbots are being utilized primarily for their automation capabilities for software monitoring, report generation etc. and for specific personalized question answering capabilities, on a limited scope and scale. One major limitation of the currently evolving family of LLMs is 'hallucinations', wherein inaccurate responses are reported as factual. Hallucinations are primarily caused by biased training data, ambiguous prompts and inaccurate LLM parameters, and they majorly occur while combining mathematical facts with language-based context. Thus, monitoring and controlling for hallucinations becomes necessary when designing solutions that are meant for decision makers. In this work we present the three major stages in the journey of designing hallucination-minimized LLM-based solutions that are specialized for the decision makers of the financial domain, namely: prototyping, scaling and LLM evolution using human feedback. These three stages and the novel data to answer generation modules presented in this work are necessary to ensure that the Generative AI chatbots, autonomous reports and alerts are reliable and high-quality to aid key decision-making processes.
(Nearest) Neighbors You Can Rely On: Formally Verified k-d Tree Construction and Search in Coq
Venue: arXiv cs.LO 2023
Year: 2023

The k-d tree is a classic binary space-partitioning tree used to organize points in k-dimensional space. While used in computational geometry and graphics, the data structure has a long history of application in nearest neighbor search. The objective of the nearest neighbor search problem is to efficiently find the closest point(s) to a given query point, and is the basis, in turn, of common machine learning techniques. We present in this paper a case study in the certified implementation, using the Coq proof assistant, of k-d tree construction from a set of data and the accompanying K-nearest neighbors search algorithm. Our experience demonstrates an intuitive method for specifying properties of these algorithms using the notion of list permutations.
MATILDA: Inclusive Data Science Pipelines Design through Computational Creativity
Venue: arXiv cs.DB 2023
Year: 2023

We argue for the need for a new generation of data science solutions that can democratize recent advances in data engineering and artificial intelligence for non-technical users from various disciplines, enabling them to unlock the full potential of these solutions. To do so, we adopt an approach whereby computational creativity and conversational computing are combined to guide non-specialists intuitively to explore and extract knowledge from data collections. The paper introduces MATILDA, a creativity-based data science design platform, showing how it can support the design process of data science pipelines guided by human and computational creativity.