Miracle-dz/UniFAR-FTUs huggingface dataset:
Open Research Empirical analyses of the factors aﬀecting conﬁrmation bias and the eﬀects of conﬁrmation bias on software developer/tester performance

Background: During all levels of software testing, the goal should be to fail the code. However, software developers and testers are more likely to choose positive tests rather than negative ones due to the phenomenon called confirmation bias. Confirmation bias is defined as the tendency of people to verify their hypotheses rather than refuting them. In the literature, there are theories about the possible effects of confirmation bias on software development and testing. Due to the tendency towards positive tests, most of the software defects remain undetected, which in turn leads to an increase in software defect density. Aims: In this study, we analyze factors affecting confirmation bias in order to discover methods to circumvent confirmation bias. The factors, we investigate are experience in software development/testing and reasoning skills that can be gained through education. In addition, we analyze the effect of confirmation bias on software developer and tester performance. Method: In order to measure and quantify confirmation bias levels of software developers/testers, we prepared pen-and-paper and interactive tests based on two tasks from cognitive psychology literature. These tests were conducted on the 36 employees of a large scale telecommunication company in Europe as well as 28 graduate computer engineering students of Bogazici University, resulting in a total of 64 subjects. We evaluated the outcomes of these tests using the metrics we proposed in addition to some basic methods which we inherited from the cognitive psychology literature. Results: Results showed that regardless of experience in software development/testing, abilities such as logical reasoning and strategic hypotheses testing are differentiating factors in low confirmation bias levels. Moreover, the results of the analysis to investigate the relationship between code defect density and confirmation bias levels of software developers and testers showed that there is a direct correlation between confirmation bias and defect proneness of the code. Conclusions: Our findings show that having strong logical reasoning and hypothesis testing skills are differentiating factors in the software developer/tester performance in terms of defect rates. We recommend that companies should focus on improving logical reasoning and hypothesis testing skills of their employees by designing training programs. As future work, we plan to replicate this study in other software development companies. Moreover, we will use confirmation bias metrics in addition to product and process metrics in for software defect prediction. We believe that confirmation bias metrics would improve the prediction performance of learning based defect prediction models which we have been building over a decade.
NL2Viz: natural language to visualization via constrained syntax-guided synthesis

Recent development in NL2CODE (Natural Language to Code) research allows end-users, especially novice programmers to create a concrete implementation of their ideas such as data visualization by providing natural language (NL) instructions. An NL2CODE system often fails to achieve its goal due to three major challenges: the user's words have contextual semantics, the user may not include all details needed for code generation, and the system results are imperfect and require further refinement. To address the aforementioned three challenges for NL to Visualization, we propose a new approach and its supporting tool named NL2VIZ with three salient features: (1) leveraging not only the user's NL input but also the data and program context that the NL query is upon, (2) using hard/soft constraints to reflect different confidence levels in the constraints retrieved from the user input and data/program context, and (3) providing support for result refinement and reuse. We implement NL2VIZ in the Jupyter Notebook environment and evaluate NL2VIZ on a real-world visualization benchmark and a public dataset to show the effectiveness of NL2VIZ. We also conduct a user study involving 6 data scientist professionals to demonstrate the usability of NL2VIZ, the readability of the generated code, and NL2VIZ's effectiveness in helping users generate desired visualizations effectively and efficiently.
Beyond human-in-the-loop: scaling occupation taxonomy at Indeed

A hierarchical occupation taxonomy helps Indeed better match job seekers and jobs. Historically, scaling this hierarchical taxonomy in international markets was a time intensive and highly manual process. Leveraging the strengths of both machine learning models and subject matter experts led to the creation of an improved human-in-the-loop system that met the needs of a growing business without sacrificing quality. This paper describes this system and discusses the challenges and insights from its implementation. Specifically, it highlights the value of involving subject matter expertise beyond just labeling and therefore offers the application of an expert-in-the-loop framework for scaling taxonomy.
Is there any long run Granger-causality between economic growth and energy consumption ? evidence from Singapore

The energy consumption and economic growth nexus has been an issue in energy economics in recent decades. Rapid technological advancement and the corresponding increase in energy demand compels countries worldwide to assess the defining crux between energy consumption and economic development prior to implementing socio-economic policies that may deter economic growth in the long-run. In an era of global warming, this is especially vital to countries implementing energy policies aimed at reducing the lingering effects of climate change. One such country is Singapore. A forerunning Asian Tiger, Singapore’s concerted efforts to monitor its commitment to reducing carbon emissions has the urban city-state in a position where it must understand the energy consumption and economic growth nexus before it can devise energy policies focused on preserving the global climate, while simultaneously ensuring its concerted efforts toward long-run stable economic growth remains unhindered. The purpose of this study is to evaluate the energy consumption and economic growth nexus in Singapore. The methodology of this study employs the Autoregressive Distributed Lag (ARDL) approach to cointegration as proposed by Pesaran et al. (2001), and extends existing literature via the application of this cointegrating approach as it is previously unused in similar case studies. Findings indicate that no long-run causality exist between economic growth and energy consumption in Singapore, therefore denoting the presence of a neutrality hypothesis in the country’s economy. This implies that Singapore’s government would not experience negative economic growth as a result of its targeted policies towards reducing energy consumption in the country, further enhancing its position as a member of the Doha Amendment to the Kyoto Protocol.
Children's use of social information from multiple models: Cognitive capacities underlying population size effects on cumulative culture

Population size has been proposed to promote cumulative culture in humans. Experimental evidence from adult humans suggests that one explanatory mechanism might involve combining beneficial information from multiple models. However, it is possible that such combinatory social learning requires cognitive capacities restricted to adult humans. In our task, children aged 5–10 were exposed to two models who consecutively searched a 3×3 array for rewards. Models revealed different correct and incorrect reward locations. This information could be used by the child to maximise their own score on the same task. We were interested in children's ability to select rewarded locations, and avoid unrewarded ones, revealed by both models. We also manipulated the spatial and temporal displacement of the information available. Results showed that the youngest children were unable to fully benefit from the additional information provided by the two models under spatial and/or temporal displacement. Such displacement likely applies in most real-world cases of cumulative culture therefore our result may offer insight into the constraints on cumulative culture in nonhumans.
A SPATIALLY ADAPTIVE ITERATIVE METHOD FOR A CLASS OF NONLINEAR OPERATOR EIGENPROBLEMS

We present a new algorithm for the iterative solution of nonlinear operator eigenvalue problems arising from partial differential equations. This algorithm combines automatic spatial resolution of linear operators with the infinite Arnoldi method for nonlinear matrix eigenproblems proposed in [E. JARLEBRING, W. MICHIELS, AND K. MEERBERGEN, A linear eigenvalue algorithm for the nonlinear eigenvalue problem, Numer. Math., 122 (2012)]. The iterates in this infinite Arnoldi method are functions, and each iteration requires the solution of an inhomogeneous differential equation. This formulation is independent of the spatial representation of the functions, which allows us to employ a dynamic representation with an accuracy of about the level of machine precision at each iteration, similar to what is done in the Chebfun system [Z. BATTLES AND L. N. TREFETHEN, An extension of MATLAB to continuous functions and operators, SIAM J. Sci. Comput. , 25 (2004)] with its chebop functionality [T. A. DRISCOLL, F. BORNEMANN, AND L. N. TREFETHEN, The chebop system for automatic solution of differential equations, BIT, 48 (2008)], although our function representation is entirely based on coefficients instead of function values. Our approach also allows for nonlinearities in the boundary conditions of the PDE. The algorithm is illustrated with several examples, e.g., the study of eigenvalues of a vibrating string with delayed boundary feedback control.
Semantic Web Analysis with Flavor of Micro-Services

The last decades witnessed a significant evolution in terms of data generation, management, and maintenance, especially in the RDF format. Moreover, in the energy domain, semantic data is finding its way and can be used for various data analytics tasks. However, since data set sizes are increasing and can now be enormous, technologies are evolving to scale with the increasing data set sizes. In this regard, tools and frameworks such as SANSA have been emerged to facilitate the analytic over semantic data. SANSA is using big data technologies such as Apache Spark (as an analytics engine for large-scale data processing) and Apache Hadoop (as a distributed file system) in its backbone to be able to perform analytics in a distributed manner over a cluster of nodes. However, to be able to use SANSA, one should set up a cluster of nodes with enabled Spark and Hadoop. This requires extensive knowledge and expertise in computer systems, networking, distributed computing and etc. Moreover, in case of having sufficient technical knowledge, setting up such a cluster consumes huge manpower and is labor-intensive. To tackle the aforementioned issues, in this paper we introduce a micro-service architecture that easily brings the power of SANSA and distributed semantic data analysis in the end-user ecosystem, without having technical knowledge in the mentioned areas. The introduced architecture is based on Docker technologies and can be installed on-premise or in the cloud systems.
Can large language models build causal graphs?

Building causal graphs can be a laborious process. To ensure all relevant causal pathways have been captured, researchers often have to discuss with clinicians and experts while also reviewing extensive relevant medical literature. By encoding common and medical knowledge, large language models (LLMs) represent an opportunity to ease this process by automatically scoring edges (i.e., connections between two variables) in potential graphs. LLMs however have been shown to be brittle to the choice of probing words, context, and prompts that the user employs. In this work, we evaluate if LLMs can be a useful tool in complementing causal graph development.
PSST: Enabling Blind or Visually Impaired Developers to Author Sonifications of Streaming Sensor Data

We present the first toolkit that equips blind and visually impaired (BVI) developers with the tools to create accessible data displays. Called PSST (Physical computing Streaming Sensor data Toolkit), it enables BVI developers to understand the data generated by sensors from a mouse to a micro:bit physical computing platform. By assuming visual abilities, earlier efforts to make physical computing accessible fail to address the need for BVI developers to access sensor data. PSST enables BVI developers to understand real-time, real-world sensor data by providing control over what should be displayed, as well as when to display and how to display sensor data. PSST supports filtering based on raw or calculated values, highlighting, and transformation of data. Output formats include tonal sonification, nonspeech audio files, speech, and SVGs for laser cutting. We validate PSST through a series of demonstrations and a user study with BVI developers.
Title-based News Summarization via MRC Framework

Existing studies on extractive summarization 001 have adopted an approach that scores and se-002 lects summary sentences independently. How-003 ever, these models are limited to sentence-level 004 extraction and tend to select highly general-005 ized sentences while overlooking the overall 006 content of a document. In this study, we pro-007 pose a machine reading comprehension (MRC) 008 framework for extractive news summarization 009 (MRCS UM ) by setting a query as the title. This 010 enables the model to consider the semantics of 011 a compact summary when selecting summary 012 sentences. In particular, when a title is not 013 available, a TITLE - LIKE QUERY is generated, 014 which is expected to achieve the same effect as 015 a title. The experimental results demonstrate 016 that MRCS UM is effective for the extractive 017 summarization of a dataset with titles as well 018 as without titles. 019
Bridge the Gap Between CV and NLP! Optimize Adversarial Texts in Continuous Space

Despite recent success on various tasks, deep 001 learning techniques still perform poorly on ad-002 versarial examples with small perturbations. 003 While optimization methods for adversarial at-004 tacks are well-explored in the ﬁeld of com-005 puter vision, it is impractical to directly ap-006 ply them in natural language processing due 007 to the discrete nature of the text. To ad-008 dress the problem, we propose a uniﬁed frame-009 work to extend the existing optimization-based 010 method in the vision domain to craft textual 011 adversarial samples. In this framework, con-012 tinuously optimized perturbations are added 013 to the embedding layer and ampliﬁed in the 014 forward propagation process. Then the ﬁnal 015 perturbed latent representations are decoded 016 with a masked language model head to obtain 017 potential adversarial samples. In this paper, 018 we instantiate our framework with an attack 019 algorithm named T extual P rojected G radient 020 D escent ( T-PGD ). We ﬁnd our algorithm ef-021 fective even using proxy gradient information. 022 Therefore, we perform more challenging trans-023 fer black-box attacks and conduct comprehen-024 sive experiments to evaluate our attack algo-025 rithm with BERT, RoBERTa, and ALBERT 026 on three benchmark datasets. Experimental 027 results demonstrate that our method achieves 028 an overall better performance and produces 029 more ﬂuent and grammatical adversarial sam-030 ples compared to strong baseline methods. All 031 the code and data will be made public. 032
Discursive Socratic Questioning: (Unsupervised) Interpreting Neural Language Models for Discourse Understanding

Do neural language models (NLMs) understand 001 the discourse they are processing? Traditional 002 interpretation methods that address this ques-003 tion require pre-annotated explanations, which 004 defeats the purpose of unsupervised explana-005 tion. We propose unsupervised Discursive So-006 cratic Questioning ( D I SQ ), a two-step interpre-007 tative measure. 008 D I SQ first generates Socratic-style questions 009 about the discourse and then queries NLMs 010 about these questions. A model’s understand-011 ing is measured by its responses to these ques-012 tions. We apply D I SQ to examine two fun-013 damental discourse phenomena, namely dis-014 course relation and discourse coherence. We 015 find NLMs demonstrate non-trivial capacities 016 without being trained on any discourse data: 017 Q&A pairs in D I SQ are shown to be evidence 018 for discourse relation and cohesive devices for 019 discourse coherence. D I SQ brings initial evi-020 dence that NLMs understand discourse through 021 reasoning. We find larger models perform bet-022 ter, but contradictions and hallucinations are 023 still problems. We recommend D I SQ as a uni-024 versal diagnostic for discursive NLMs and us-025 ing its output for self-supervision. 026
Predicting the price of second-hand vehicles using data mining techniques

The electronic commerce, known as “E - commerce”, has been boosted rapidly in recent years, and makes it possible to record all information such as price, location, customer’s review, search history, discount options, competitor’s price, and so on. Accessing to such rich source of data, companies can analyze their users’ behavior to improve the customer satisfaction as well as the revenue. This study aims to estimate the price of used light vehicles in a commercial website, Divar, which is a popular website in Iran for trading second-handed goods. At first, highlighted features were extracted from the description column using the three methods of Bag of Words (BOW), Latent Dirichlet Allocation (LDA), and Hierarchical Dirichlet Process (HDP). Second, a multiple linear regression model was fit to predict the product price based on its attributes and the highlighted features. The accuracy index of Actuals-Predictions Correlation, the min-max index, and MAPE methods were used to validate the proposed methods. Results showed that the BOW model is the best model with an Adjusted R-square of 0.7841.
L EARNING N EURAL C AUSAL M ODELS FROM U NKNOWN I NTERVENTIONS

Meta-learning over a set of distributions can be interpreted as learning different types of parameters corresponding to short-term vs long-term aspects of the mechanisms underlying the generation of data. These are respectively captured by quickly-changing parameters and slowly-changing meta-parameters . We present a new framework for meta-learning causal models where the relationship between each variable and its parents is modeled by a neural network, modulated by structural meta-parameters which capture the overall topology of a directed graphical model. Our approach avoids a discrete search over models in favour of a continuous optimization procedure. We study a setting where interventional distributions are induced as a result of a random intervention on a single unknown variable of an unknown ground truth causal model
Mass transfer from giant donors

The stability of mass transfer in binaries with convective giant donors remains an open question in modern astrophysics. There is a significant discrepancy between what the existing methods predict for a response to mass loss of the giant itself, as well as for the mass transfer rate during the Roche lobe overflow. Here we show that the recombination energy in the superadiabatic layer plays an important and hitherto unaccounted-for role in he donor's response to mass loss, in particular on its luminosity and effective temperature. Our improved optically thick nozzle method to calculate the mass transfer rate via $L_1$ allows us to evolve binary systems for a substantial Roche lobe overflow. We propose a new, strengthened criterion for the mass transfer instability, basing it on whether the donor experiences overflow through its outer Lagrangian point. We find that with the new criterion, if the donor has a well-developed outer convective envelope, the critical initial mass ratio for which a binary would evolve stably through the conservative mass transfer varies from $1.5$ to $2.2$, which is about twice as large as previously believed. In underdeveloped giants with shallow convective envelopes this critical ratio may be even larger. When the convective envelope is still growing, and in particular for most cases of massive donors, the critical mass ratio gradually decreases to this value, from that of radiative donors.
Open Research Online Deﬁning the boundaries between Artiﬁcial Intelligence in Education, Computer-Supported Collaborative Learning, Educational Data Mining, and Learning Analytics: A Need for Coherence

This review aims to provide a concise overview of four distinct research ﬁelds: Artiﬁcial Intelligence and EDucation (AIED), Computer-Supported Collaborative Learning (CSCL), Educational Data Mining (EDM), and Learning Analytics (LA). While all four ﬁelds are focused on understanding learning and teaching using technology, each ﬁeld has a relatively unique or common perspective on which theoretical frameworks, methods, and ontologies might be appropriate. In this review we argue that researchers should be encouraged to cross the boundaries of their respective ﬁeld and work together to address the complex challenges in education.
This paper is included in the Proceedings of the Eighteenth Symposium on Usable Privacy and Security (SOUPS 2022).

CAPTCHAs are necessary to protect websites from bots and malicious crawlers, yet are increasingly solvable by automated systems. This has led to more challenging tests that require greater human effort and cultural knowledge; they may prevent bots effectively but sacriﬁce usability and discourage the human users they are meant to admit. We propose a new class of challenge: a Cryptographic Attestation of Per-sonhood (CAP) as the foundation of a usable, pro-privacy alternative. Our challenge is constructed using the open Web Authentication API (WebAuthn) that is supported in most browsers. We evaluated the CAP challenge through a public demo, with an accompanying user survey. Our evaluation indicates that CAP has a strong likelihood of adoption by users who possess the necessary hardware, showing good results for effectiveness and efﬁciency as well as a strong expressed preference for using CAP over traditional CAPTCHA solutions. In addition to demonstrating a mechanism for more usable challenge tests, we identify some areas for improvement for the WebAuthn user experience, and reﬂect on the difﬁcult usable privacy problems in this domain and how they might be mitigated.