OpenRaiser/Intern-Atlas huggingface dataset:
Diversity in Software Engineering: A Survey about Scientists from Underrepresented Groups
Venue: arXiv cs.SE 2023
Year: 2023

Technology plays a crucial role in people's lives. However, software engineering discriminates against individuals from underrepresented groups in several ways, either through algorithms that produce biased outcomes or for the lack of diversity and inclusion in software development environments and academic courses focused on technology. This reality contradicts the history of software engineering, which is filled with outstanding scientists from underrepresented groups who changed the world with their contributions to the field. Ada Lovelace, Alan Turing, and Clarence Ellis are only some individuals who made significant breakthroughs in the area and belonged to the population that is so underrepresented in undergraduate courses and the software industry. Previous research discusses that women, LGBTQIA+ people, and non-white individuals are examples of students who often feel unwelcome and ostracized in software engineering. However, do they know about the remarkable scientists that came before them and that share background similarities with them? Can we use these scientists as role models to motivate these students to continue pursuing a career in software engineering? In this study, we present the preliminary results of a survey with 128 undergraduate students about this topic. Our findings demonstrate that students' knowledge of computer scientists from underrepresented groups is limited. This creates opportunities for investigations on fostering diversity in software engineering courses using strategies exploring computer science's history.
Exploring Gender Bias in Remote Pair Programming among Software Engineering Students: The twincode Original Study and First External Replication
Venue: arXiv cs.SE 2023
Year: 2023

Context. Software Engineering (SE) has low female representation due to gender bias that men are better at programming. Pair programming (PP) is common in industry and can increase student interest in SE, especially women; but if gender bias affects PP, it may discourage women from joining the field. Objective. We explore gender bias in PP. In a remote setting where students cannot see their peers' gender, we study how perceived productivity, technical competency and collaboration/interaction behaviors of SE students vary by perceived gender of their remote partner. Method. We developed an online PP platform (twincode) with a collaborative editing window and a chat pane. Control group had no gender information about their partner, while treatment group saw a gendered avatar as a man or woman. Avatar gender was swapped between tasks to analyze 45 variables on collaborative coding behavior, chat utterances and questionnaire responses of 46 pairs in original study at the University of Seville and 23 pairs in the replication at the University of California, Berkeley. Results. No significant effect of gender bias treatment or interaction between perceived partner's gender and subject's gender in any variable in original study. In replication, significant effects with moderate to large sizes in four variables within experimental group comparing subjects' actions when partner was male vs female.
Local-Global Methods for Generalised Solar Irradiance Forecasting
Venue: arXiv cs.LG 2023
Year: 2023

As the use of solar power increases, having accurate and timely forecasts will be essential for smooth grid operators. There are many proposed methods for forecasting solar irradiance / solar power production. However, many of these methods formulate the problem as a time-series, relying on near real-time access to observations at the location of interest to generate forecasts. This requires both access to a real-time stream of data and enough historical observations for these methods to be deployed. In this paper, we propose the use of Global methods to train our models in a generalised way, enabling them to generate forecasts for unseen locations. We apply this approach to both classical ML and state of the art methods. Using data from 20 locations distributed throughout the UK and widely available weather data, we show that it is possible to build systems that do not require access to this data. We utilise and compare both satellite and ground observations (e.g. temperature, pressure) of weather data. Leveraging weather observations and measurements from other locations we show it is possible to create models capable of accurately forecasting solar irradiance at new locations. This could facilitate use planning and optimisation for both newly deployed solar farms and domestic installations from the moment they come online. Additionally, we show that training a single global model for multiple locations can produce a more robust model with more consistent and accurate results across locations.
TSMixer: An All-MLP Architecture for Time Series Forecasting
Venue: arXiv cs.LG 2023
Year: 2023

Real-world time-series datasets are often multivariate with complex dynamics. To capture this complexity, high capacity architectures like recurrent- or attention-based sequential deep learning models have become popular. However, recent work demonstrates that simple univariate linear models can outperform such deep learning models on several commonly used academic benchmarks. Extending them, in this paper, we investigate the capabilities of linear models for time-series forecasting and present Time-Series Mixer (TSMixer), a novel architecture designed by stacking multi-layer perceptrons (MLPs). TSMixer is based on mixing operations along both the time and feature dimensions to extract information efficiently. On popular academic benchmarks, the simple-to-implement TSMixer is comparable to specialized state-of-the-art models that leverage the inductive biases of specific benchmarks. On the challenging and large scale M5 benchmark, a real-world retail dataset, TSMixer demonstrates superior performance compared to the state-of-the-art alternatives. Our results underline the importance of efficiently utilizing cross-variate and auxiliary information for improving the performance of time series forecasting. We present various analyses to shed light into the capabilities of TSMixer. The design paradigms utilized in TSMixer are expected to open new horizons for deep learning-based time series forecasting. The implementation is available at https://github.com/google-research/google-research/tree/master/tsmixer
Walking Down the Road to Independent Mobility: An Adaptive Route Training System for the Cognitively Impaired
Venue: arXiv cs.HC 2023
Year: 2023

In this paper we describe the design and development of a route training system for individuals with cognitive impairments (CIs) living in residential care facilities. Learning to move autonomously in public spaces is a fundamental skill for people with CI, who face several challenges to independently and safely move around. Yet, exploring opportunities for route training support, especially in residential settings, has received very little attention. To explore these opportunities, we followed a design and development process based on inclusive design practices that considered the organisational context and aimed at involving people with CI in the software design. To ensure our solution addressed the identified needs and abilities of this heterogeneous population, we further framed the route training definition as a design process that is enacted by the system, making the trainer and user co-creators of a personalised training. In this paper we report on the needs and challenges for mobility training in residential settings, introduce the design and formative evaluation of the route training system, to conclude with reflections and considerations on our methodological approach.
Jordan algebra in R
Venue: arXiv cs.SC 2023
Year: 2023

In this short article I introduce the "jordan" package which provides functionality for working with different types of Jordan algebra. I give some numerical verification of the Jordan identity for the five types of Jordan algebras. The package is available on CRAN at https://CRAN.R-project.org/package=stokes.
On the Fusion Strategies for Federated Decision Making
Venue: arXiv cs.LG 2023
Year: 2023

We consider the problem of information aggregation in federated decision making, where a group of agents collaborate to infer the underlying state of nature without sharing their private data with the central processor or each other. We analyze the non-Bayesian social learning strategy in which agents incorporate their individual observations into their opinions (i.e., soft-decisions) with Bayes rule, and the central processor aggregates these opinions by arithmetic or geometric averaging. Building on our previous work, we establish that both pooling strategies result in asymptotic normality characterization of the system, which, for instance, can be utilized to derive approximate expressions for the error probability. We verify the theoretical findings with simulations and compare both strategies.
Optimal Design of Validation Experiments for the Prediction of Quantities of Interest
Venue: arXiv cs.CE 2023
Year: 2023

Numerical predictions of quantities of interest measured within physical systems rely on the use of mathematical models that should be validated, or at best, not invalidated. Model validation usually involves the comparison of experimental data (outputs from the system of interest) and model predictions, both obtained at a specific validation scenario. The design of this validation experiment should be directly relevant to the objective of the model, that of predicting a quantity of interest at a prediction scenario. In this paper, we address two specific issues arising when designing validation experiments. The first issue consists in determining an appropriate validation scenario in cases where the prediction scenario cannot be carried out in a controlled environment. The second issue concerns the selection of observations when the quantity of interest cannot be readily observed. The proposed methodology involves the computation of influence matrices that characterize the response surface of given model functionals. Minimization of the distance between influence matrices allow one for selecting a validation experiment most representative of the prediction scenario. We illustrate our approach on two numerical examples. The first example considers the validation of a simple model based on an ordinary differential equation governing an object in free fall to put in evidence the importance of the choice of the validation experiment. The second numerical experiment focuses on the transport of a pollutant and demonstrates the impact that the choice of the quantity of interest has on the validation experiment to be performed.
NoiseCAM: Explainable AI for the Boundary Between Noise and Adversarial Attacks
Venue: arXiv cs.LG 2023
Year: 2023

Deep Learning (DL) and Deep Neural Networks (DNNs) are widely used in various domains. However, adversarial attacks can easily mislead a neural network and lead to wrong decisions. Defense mechanisms are highly preferred in safety-critical applications. In this paper, firstly, we use the gradient class activation map (GradCAM) to analyze the behavior deviation of the VGG-16 network when its inputs are mixed with adversarial perturbation or Gaussian noise. In particular, our method can locate vulnerable layers that are sensitive to adversarial perturbation and Gaussian noise. We also show that the behavior deviation of vulnerable layers can be used to detect adversarial examples. Secondly, we propose a novel NoiseCAM algorithm that integrates information from globally and pixel-level weighted class activation maps. Our algorithm is susceptible to adversarial perturbations and will not respond to Gaussian random noise mixed in the inputs. Third, we compare detecting adversarial examples using both behavior deviation and NoiseCAM, and we show that NoiseCAM outperforms behavior deviation modeling in its overall performance. Our work could provide a useful tool to defend against certain adversarial attacks on deep neural networks.
Why is That a Good or Not a Good Frying Pan? -- Knowledge Representation for Functions of Objects and Tools for Design Understanding, Improvement, and Generation
Venue: arXiv cs.AI 2023
Year: 2023

The understanding of the functional aspects of objects and tools is of paramount importance in supporting an intelligent system in navigating around in the environment and interacting with various objects, structures, and systems, to help fulfil its goals. A detailed understanding of functionalities can also lead to design improvements and novel designs that would enhance the operations of AI and robotic systems on the one hand, and human lives on the other. This paper demonstrates how a particular object - in this case, a frying pan - and its participation in the processes it is designed to support - in this case, the frying process - can be represented in a general function representational language and framework, that can be used to flesh out the processes and functionalities involved, leading to a deep conceptual understanding with explainability of functionalities that allows the system to answer "why" questions - why is something a good frying pan, say, or why a certain part on the frying pan is designed in a certain way? Or, why is something not a good frying pan? This supports the re-design and improvement on design of objects, artifacts, and tools, as well as the potential for generating novel designs that are functionally accurate, usable, and satisfactory.
Software Vulnerability Prediction Knowledge Transferring Between Programming Languages
Venue: arXiv cs.SE 2023
Year: 2023

Developing automated and smart software vulnerability detection models has been receiving great attention from both research and development communities. One of the biggest challenges in this area is the lack of code samples for all different programming languages. In this study, we address this issue by proposing a transfer learning technique to leverage available datasets and generate a model to detect common vulnerabilities in different programming languages. We use C source code samples to train a Convolutional Neural Network (CNN) model, then, we use Java source code samples to adopt and evaluate the learned model. We use code samples from two benchmark datasets: NIST Software Assurance Reference Dataset (SARD) and Draper VDISC dataset. The results show that proposed model detects vulnerabilities in both C and Java codes with average recall of 72\%. Additionally, we employ explainable AI to investigate how much each feature contributes to the knowledge transfer mechanisms between C and Java in the proposed model.
Interpretable Joint Event-Particle Reconstruction for Neutrino Physics at NOvA with Sparse CNNs and Transformers
Venue: arXiv cs.LG 2023
Year: 2023

The complex events observed at the NOvA long-baseline neutrino oscillation experiment contain vital information for understanding the most elusive particles in the standard model. The NOvA detectors observe interactions of neutrinos from the NuMI beam at Fermilab. Associating the particles produced in these interaction events to their source particles, a process known as reconstruction, is critical for accurately measuring key parameters of the standard model. Events may contain several particles, each producing sparse high-dimensional spatial observations, and current methods are limited to evaluating individual particles. To accurately label these numerous, high-dimensional observations, we present a novel neural network architecture that combines the spatial learning enabled by convolutions with the contextual learning enabled by attention. This joint approach, TransformerCVN, simultaneously classifies each event and reconstructs every individual particle's identity. TransformerCVN classifies events with 90\% accuracy and improves the reconstruction of individual particles by 6\% over baseline methods which lack the integrated architecture of TransformerCVN. In addition, this architecture enables us to perform several interpretability studies which provide insights into the network's predictions and show that TransformerCVN discovers several fundamental principles that stem from the standard model.
A Theoretical Analysis Of Nearest Neighbor Search On Approximate Near Neighbor Graph
Venue: arXiv cs.LG 2023
Year: 2023

Graph-based algorithms have demonstrated state-of-the-art performance in the nearest neighbor search (NN-Search) problem. These empirical successes urge the need for theoretical results that guarantee the search quality and efficiency of these algorithms. However, there exists a practice-to-theory gap in the graph-based NN-Search algorithms. Current theoretical literature focuses on greedy search on exact near neighbor graph while practitioners use approximate near neighbor graph (ANN-Graph) to reduce the preprocessing time. This work bridges this gap by presenting the theoretical guarantees of solving NN-Search via greedy search on ANN-Graph for low dimensional and dense vectors. To build this bridge, we leverage several novel tools from computational geometry. Our results provide quantification of the trade-offs associated with the approximation while building a near neighbor graph. We hope our results will open the door for more provable efficient graph-based NN-Search algorithms.
AUTODIAL: Efficient Asynchronous Task-Oriented Dialogue Model
Venue: arXiv cs.CL 2023
Year: 2023

As large dialogue models become commonplace in practice, the problems surrounding high compute requirements for training, inference and larger memory footprint still persists. In this work, we present AUTODIAL, a multi-task dialogue model that addresses the challenges of deploying dialogue model. AUTODIAL utilizes parallel decoders to perform tasks such as dialogue act prediction, domain prediction, intent prediction, and dialogue state tracking. Using classification decoders over generative decoders allows AUTODIAL to significantly reduce memory footprint and achieve faster inference times compared to existing generative approach namely SimpleTOD. We demonstrate that AUTODIAL provides 3-6x speedups during inference while having 11x fewer parameters on three dialogue tasks compared to SimpleTOD. Our results show that extending current dialogue models to have parallel decoders can be a viable alternative for deploying them in resource-constrained environments.
A Problem Space for Designing Visualizations
Venue: arXiv cs.HC 2023
Year: 2023

Visualization researchers and visualization professionals seek appropriate abstractions of visualization requirements that permit considering visualization solutions independently from specific problems. Abstractions can help us design, analyze, organize, and evaluate the things we create. The literature has many task structures (taxonomies, typologies, etc.), design spaces, and related ``frameworks'' that provide abstractions of the problems a visualization is meant to address. In this viewpoint, we introduce a different one, a problem space that complements existing frameworks by focusing on the needs that a visualization is meant to solve. We believe it provides a valuable conceptual tool for designing and discussing visualizations.
Non-Coherent Active Device Identification for Massive Random Access
Venue: arXiv cs.IT 2023
Year: 2023

Massive Machine-Type Communications (mMTC) is a key service category in the current generation of wireless networks featuring an extremely high density of energy and resource-limited devices with sparse and sporadic activity patterns. In order to enable random access in such mMTC networks, base station needs to identify the active devices while operating within stringent access delay constraints. In this paper, an energy efficient active device identification protocol is proposed in which active devices transmit On-Off Keying (OOK) modulated preambles jointly and base station employs non-coherent energy detection avoiding channel estimation overheads. The minimum number of channel-uses required by the active user identification protocol is characterized in the asymptotic regime of total number of devices $\ell$ when the number of active devices $k$ scales as $k=Θ(1)$ along with an achievability scheme relying on the equivalence of activity detection to a group testing problem. Several practical schemes based on Belief Propagation (BP) and Combinatorial Orthogonal Matching Pursuit (COMP) are also proposed. Simulation results show that BP strategies outperform COMP significantly and can operate close to the theoretical achievability bounds. In a partial-recovery setting where few misdetections are allowed, BP continues to perform well.
Singleton-Optimal LRCs and Perfect LRCs via Cyclic and Constacyclic Codes
Venue: arXiv cs.IT 2023
Year: 2023

Locally repairable codes (LRCs) have emerged as an important coding scheme in distributed storage systems (DSSs) with relatively low repair cost by accessing fewer non-failure nodes. Theoretical bounds and optimal constructions of LRCs have been widely investigated. Optimal LRCs via cyclic and constacyclic codes provide significant benefit of elegant algebraic structure and efficient encoding procedure. In this paper, we continue to consider the constructions of optimal LRCs via cyclic and constacyclic codes with long code length. Specifically, we first obtain two classes of $q$-ary cyclic Singleton-optimal $(n, k, d=6;r=2)$-LRCs with length $n=3(q+1)$ when $3 \mid (q-1)$ and $q$ is even, and length $n=\frac{3}{2}(q+1)$ when $3 \mid (q-1)$ and $q \equiv 1(\bmod~4)$, respectively. To the best of our knowledge, this is the first construction of $q$-ary cyclic Singleton-optimal LRCs with length $n>q+1$ and minimum distance $d \geq 5$. On the other hand, an LRC acheiving the Hamming-type bound is called a perfect LRC. By using cyclic and constacyclic codes, we construct two new families of $q$-ary perfect LRCs with length $n=\frac{q^m-1}{q-1}$, minimum distance $d=5$ and locality $r=2$.
Space-Invariant Projection in Streaming Network Embedding
Venue: arXiv cs.SI 2023
Year: 2023

Newly arriving nodes in dynamics networks would gradually make the node embedding space drifted and the retraining of node embedding and downstream models indispensable. An exact threshold size of these new nodes, below which the node embedding space will be predicatively maintained, however, is rarely considered in either theory or experiment. From the view of matrix perturbation theory, a threshold of the maximum number of new nodes that keep the node embedding space approximately equivalent is analytically provided and empirically validated. It is therefore theoretically guaranteed that as the size of newly arriving nodes is below this threshold, embeddings of these new nodes can be quickly derived from embeddings of original nodes. A generation framework, Space-Invariant Projection (SIP), is accordingly proposed to enables arbitrary static MF-based embedding schemes to embed new nodes in dynamics networks fast. The time complexity of SIP is linear with the network size. By combining SIP with four state-of-the-art MF-based schemes, we show that SIP exhibits not only wide adaptability but also strong empirical performance in terms of efficiency and efficacy on the node classification task in three real datasets.