Reviewerly/exHarmony huggingface dataset:
Prophet: Conflict-Free Sharding Blockchain via Byzantine-Tolerant Deterministic Ordering

Sharding scales throughput by splitting blockchain nodes into parallel groups. However, different shards’ independent and random scheduling for cross-shard transactions results in numerous conflicts and aborts, since cross-shard transactions from different shards may access the same account. A deterministic ordering can eliminate conflicts by determining a global order for transactions before processing, as proved in the database field. Unfortunately, due to the intertwining of the Byzantine environment and information isolation among shards, there is no trusted party able to predetermine such an order for cross-shard transactions. To tackle this challenge, this paper proposes Prophet, a conflict-free sharding blockchain based on Byzantine-tolerant deterministic ordering. It first depends on untrusted self-organizing coalitions of nodes from different shards to pre-execute cross-shard transactions for prerequisite information about ordering. It then determines a trusted global order based on stateless ordering and post-verification for pre-executed results, through shard cooperation. Following the order, the shards thus orderly execute and commit transactions without conflicts. Prophet orchestrates the pre-execution, ordering, and execution processes in the sharding consensus for minimal overhead. We rigorously prove the determinism and serializability of transactions under the Byzantine and sharded environment. An evaluation of our prototype shows that Prophet improves the throughput by 3.11× and achieves nearly no aborts on 1 million Ethereum transactions compared with state-of-the-art sharding.
How to Navigate Shifting Tides: Mapping Technical Writing Students’ Use of Artificial Intelligence

This brief paper describes a pilot usability test that aims to better understand how students engage artificial intelligence (AI) chatbots like ChatGPT as a tool at various stages of the writing process. Using interviews, think-aloud protocols, and research observations, we evaluate students’ practical knowledge of using AI and collaboratively explore with students the potential of such technology as a writing tool. Through our pilot study’s findings and reflection, we offer methods for applying usability testing to better understand students’ engagement with AI chatbots and provide brief insights into how AI technologies may be used in localized pedagogical aims.
Secure and Robust Two Factor Authentication via Acoustic Fingerprinting

The two-factor authentication (2FA) has become pervasive as the mobile devices become prevalent. Existing 2FA solutions usually require some form of user involvement, which could severely affect user experience and bring extra burdens to users. In this work, we propose a secure 2FA that utilizes the individual acoustic fingerprint of the speaker/microphone on enrolled device as the second proof. The main idea behind our system is to use both magnitude and phase fingerprints derived from the frequency response of the enrolled device by emitting acoustic beep signals alternately from both enrolled and login devices and receiving their direct arrivals for 2FA. Given the input microphone samplings, our system designs an arrival time detection scheme to accurately identify the beginning point of the beep signal from the received signal. To achieve a robust authentication, we develop a new distance mitigation scheme to eliminate the impact of transmission distances from the sound propagation model for extracting stable fingerprint in both magnitude and phase domain. Our device authentication component then calculates a weighted correlation value between the device profile and fingerprints extracted from run-time measurements to conduct the device authentication for 2FA. Our experimental results show that our proposed system is accurate and robust to both random impersonation and Man-in-the-middle (MiM) attack across different scenarios and device models.
Workshop: Using AI-Generated Content to Support the Writing Process

Artificial intelligence (AI) writing tools have exploded, both in sophistication and quantity. So, too, perspectives on AI have exploded. Professional communicators of all types should consider how AI can play a role in our work. In this workshop, three university professors discuss AI writing tools, focusing primarily on generative AI, as resources and demonstrate ways these tools can be productively integrated into writing workflows. They also invite discussion about ways AI writing tools can be used to support writers and show how practitioners and educators can responsibly use AI tools as part of the communication process.
Structuring Genre Performance for Future Data Scientists

In our panel’s presentations, we will discuss how our approaches to curriculum design in data science can help researchers and instructors name the types of writing skills they are asking students to display—and to perform—in varying instantiations throughout their academic careers, as well as later workplace contexts. This is especially relevant for data-driven writing in technical and professional settings, which we address in the teaching of our respective courses at two universities. The panelists will present two complementary studies that use Write & Audit, a text visualization tool that displays disciplinary genre choices for students. The presenters stress that Write & Audit is a non-evaluative revision tool designed for students to make more rhetorically informed choices in their technical writing. The course and workshops we’ve designed represent an “ inter actionist” model, where writing and content knowledge are intertwined. Additionally, panelists will share survey results from their respective studies which capture students’ sense of communicative self-efficacy and motivation. Overall, both studies show that our interventions positively affected students’ learning in several areas. Therefore, we believe communication advances data analysis that is core to problem-solving efforts in the data science field.
UX Testing of Developer Documentation - A pilot study of OceanBase Database Documentation

This pilot UX study aims to establish a user experience (UX) approach for assessing the quality of developer documentation, specifically for the OceanBase database company. A usability test was designed to examine task completion rates, times, errors, and satisfaction. Additionally, biometric equipment, such as eye-tracking, EEG, facial expression recognition, and measures of visual and mental fatigue, were utilized to analyze participants’ experiences. The PANAS scale was employed to collect self-reported emotions. The hybrid evaluation method revealed that the OceanBase database documentation suffers from poor usability, understanding, and findability. The root causes of these problems include weak functionality, subpar interaction design, lack of conciseness and explanation, and inadequate structure and legibility. Participants experienced negative emotions and increased mental fatigue after using the documentation, indicating a substantial cognitive load. These findings will inform future improvements to the documentation and provide a foundational UX model for developer documentation. The UX study process may also be a reference for practitioners or researchers conducting similar research on technical documents.
Leveraging Cognitive Science for Testing Large Language Models

Cognitive science related research offers valuable knowledge for the design, development, and evaluation of artificial intelligence (AI) systems. In this short position paper, leveraging first principles from cognitive science, we propose novel methods for testing pre-trained large language models (LLMs), specifically to assess their common sense reasoning abilities. The test cases are meant to aid in assessing the model’s ability 1) to analyze various dimensions of a prototype, and 2) to discover subtle and implied meanings (e.g. proverbs) across languages. We hope the ideas presented in the paper will spark interdisciplinary discussions concerning robust audit and evaluation of LLMs.
Visualization Tool for Extraction of Various Attributes and Corresponding Data for Dataset Quality Assessment

Generally, the quality of artificial intelligence (AI) models depends on the training dataset. Therefore, evaluating dataset quality is crucial. Datasets contain considerable attribute information, and the presence of noise attributes in the dataset can result in inferior performance in tasks such as classification and object detection. Thus, the numerous attributes contained in the dataset have to be extracted, and attributes that affect the performance of an AI model, as well as the presence and quantity of data with those attributes, have to be identified to assess dataset quality. In this study, we proposed a visualization tool for the extraction of various attributes and corresponding data for dataset quality assessment (VisTEx) by using an existing unsupervised learning method, multi-facet clustering variational autoencoders (MFCVAE). The proposed visualization tool displays the features extracted by MFCVAE on a two-dimensional (2D) scatter plot. When the user selects a region on this scatterplot, the corresponding data are displayed. This enables users to efficiently extract data attributes. A set of data corresponding to the attributes can then be extracted. Because handwritten character datasets were used in the study, a group of data with common attributes, such as the line width of the characters and presence of ruled lines as well as the character type, were extracted. Data containing ruled line noise could be extracted with 99.40% precision and 99.96% recall.
A Decentralized Truth Discovery Approach to the Blockchain Oracle Problem

When a blockchain application runs on data from the real world, it relies on an oracle mechanism that transports data from external sources to the blockchain. The blockchain oracle problem arises around the need to procure trustworthy data from external sources. Previous works have addressed data authenticity/integrity by building a secure channel between blockchain and external sources while employing a decentralized oracle network to avoid a single point of failure. However, the truthful data challenge, which emerges when legitimate external sources submit fraudulent or deceitful data, remains unsolved. In this paper, we introduce a new decentralized truth-discovering oracle architecture called DecenTruth to address the truthful data challenge using a data-centric approach. DecenTruth aims to elevate the "truthfulness" of external data input by enabling decentralized oracle nodes to discover and reach consensus on truthful values of common data objects from multi-sourced inputs in an off-chain manner. It harmonizes techniques in both the data plane and consensus plane—truth discovery (TD) and asynchronous BFT consensus—and enables nodes to finalize the same estimated truths on data objects with high accuracy, amid the harsh asynchronous network condition and presence of Byzantine sources and nodes. We implemented DecenTruth and evaluated its performance in a simulated oracle service scenario. The results demonstrate significantly higher Byzantine resilience and long-term data feed accuracy of DecenTruth, compared to existing median-based aggregation methods.
Iterative Partial Fulfillment of Counterfactual Explanations: Benefits and Risks

Counterfactual (CF) explanations, also known as contrastive explanations and algorithmic recourses, are popular for explaining machine learning models in high-stakes domains. For a subject that receives a negative model prediction (e.g., mortgage application denial), the CF explanations are similar instances but with positive predictions, which informs the subject of ways to improve. While their various properties have been studied, such as validity and stability, we contribute a novel one: their behaviors under iterative partial fulfillment (IPF). Specifically, upon receiving a CF explanation, the subject may only partially fulfill it before requesting a new prediction with a new explanation, and repeat until the prediction is positive. Such partial fulfillment could be due to the subject’s limited capability (e.g., can only pay down two out of four credit card accounts at this moment) or an attempt to take the chance (e.g., betting that a monthly salary increase of $800 is enough even though $1,000 is recommended). Does such iterative partial fulfillment increase or decrease the total cost of improvement incurred by the subject? We mathematically formalize IPF and demonstrate, both theoretically and empirically, that different CF algorithms exhibit vastly different behaviors under IPF. We discuss implications of our observations, advocate for this factor to be carefully considered in the development and study of CF algorithms, and give several directions for future work.
Sealed Knowledges

This research examines the implications of the usage of large language models (LLMs) as search engines on knowledge. Drawing on feminist theories of knowledge, I argue that LLMs used to generate direct answers to search engine inquiries both rely on and reinforce a disembodied and non-situated view of knowledge. This, it is argued, can lead to a "sealing" of non-dominant knowledges. Through this sealing of knowledges, marginalized voices may be heard even less than before. Lastly, drawing on the works of feminist theorists such as Donna Haraway and Sara Ahmed, the research proposes that doubting the outputs of LLMs can function as a feminist intervention that resists the marginalization and sealing of certain knowledges and perspectives through the usage of LLMS as chatbots. This research as part of a wider discourse on the usage of LLMs as search engines is crucial considering the current trend of major search engine providers to integrate LLMs for the production of direct answers into their search engines.
GATE: A Challenge Set for Gender-Ambiguous Translation Examples

Although recent years have brought significant progress in improving translation of unambiguously gendered sentences, translation of ambiguously gendered input remains relatively unexplored. When source gender is ambiguous, machine translation models typically default to stereotypical gender roles, perpetuating harmful bias. Recent work has led to the development of "gender rewriters" that generate alternative gender translations on such ambiguous inputs, but such systems are plagued by poor linguistic coverage. To encourage better performance on this task we present and release GATE, a linguistically diverse corpus of gender-ambiguous source sentences along with multiple alternative target language translations. We also provide tools for evaluation and system analysis when using GATE and use them to evaluate our translation rewriter.
Protecting Children from Online Exploitation: Can a Trained Model Detect Harmful Communication Strategies?

The growing popularity of social media raises concerns about children’s online safety. Of particular concern are interactions between minors and adults with predatory intentions. Unfortunately, previous research on online sexual grooming has relied on time-intensive manual annotation by domain experts, limiting both the scale and scope of possible interventions. This work explores the possibility of detecting predatory behaviours with accuracy comparable to expert annotators using machine learning (ML). Using a dataset of 6771 chat messages sent by child sex offenders, labelled by two of the authors who are forensic psychology experts, we study how well can deep learning algorithms identify eleven known predatory behaviours. We find that the best-performing ML models are consistent but not on par with expert annotation. We therefore consider a system where an expert annotator validates the ML algorithms outputs. The combination of human decision-making and computer efficiency yields precision—but not recall—comparable to manual annotation, while taking only a fraction of the time needed by a human annotator. Our findings underscore the promise of ML as a tool for assisting researchers in this area, but also highlight the current limitations in reliably detecting online sexual exploitation using ML.
A Deep Dive into Dataset Imbalance and Bias in Face Identification

As the deployment of automated face recognition (FR) systems proliferates, bias in these systems is not just an academic question, but a matter of public concern. Media portrayals often center imbalance as the main source of bias, i.e., that FR models perform worse on images of non-white people or women because these demographic groups are underrepresented in training data. Recent academic research paints a more nuanced picture of this relationship. However, previous studies of data imbalance in FR have focused exclusively on the face verification setting, while the face identification setting has been largely ignored, despite being deployed in sensitive applications such as law enforcement. This is an unfortunate omission, as 'imbalance' is a more complex matter in identification; imbalance may arise in not only the training data, but also the testing data, and furthermore may affect the proportion of identities belonging to each demographic group or the number of images belonging to each identity. In this work, we address this gap in the research by thoroughly exploring the effects of each kind of imbalance possible in face identification, and discuss other factors which may impact bias in this setting.
Infants are sensitive to the Edge Orientation Entropy of Natural Scenes

Regularities in edge orientations aid efficient recognition and categorization of scenes and objects (Geisler, 2001; Perrinet & Bednar, 2015; Sigman et al., 2001). The spatial distribution and relationships of oriented edges (Edge Orientation Entropy, EOE; Redies et al., 2017) also predicts aesthetics, with greater preference for some image types when edges are more evenly distributed across orientations (e.g., Grebenkina et al., 2018). Here, we investigated whether the developing visual system is sensitive to EOE, by measuring infants’ visual preferences for stimuli previously used to identify the relationship between EOE and adult aesthetics (e.g., Grebenkina et al, 2018). Stimuli were greyscale photos of simple and ornamental building facades equalised for luminance. A set of 24 oriented Gabor filters were used to extract the edge orientations and to calculate EOE. Every stimulus was paired with every other stimulus, and each participant saw a random selection of 50 image pairs whilst eye-movements were recorded. Twenty-nine 4-9-month old infants took part, and 29 adults were also asked to look freely at the images. Infants looked significantly longer the greater the EOE, with over half of the variance in infant looking time predicted. Infants and adults both looked longest at images where all edge orientations are about equally likely to occur (high 1st-order EOE), and at images with low correlation of edge orientations across the image (high 2nd-order EOE). Infant looking time explained over half of the variance in adult looking time and adult pleasantness ratings taken from Grebenkina et al. (2018). Our results suggest that even as young as 4-months of age, infants’ spatial vision is sensitive to EOE and is biased to natural scenes where edges are more evenly distributed across orientations. We also tentatively suggest that high EOE is a ‘perceptual primitive’ of aesthetics, at least for some types of stimuli.
StyleDiff: Attribute comparison between unlabeled datasets in latent disentangled space

One major challenge in machine learning applications is coping with mismatches between the datasets used in the development and those obtained in real-world applications. These mismatches may lead to inaccurate predictions and errors, resulting in poor product quality and unreliable systems. To address the mismatches, it is important to understand in what sense the two datasets differ. In this study, we propose StyleDiff to inform developers about the types of mismatches between the two datasets in an unsupervised manner. Given two unlabeled image datasets, StyleDiff automatically extracts latent attributes that are distributed differently between the given datasets and visualizes the differences in a human-understandable manner. For example, for an object detection dataset, latent attributes might include the time of day, weather, and traffic congestion of an image that are not explicitly labeled. StyleDiff helps developers understand the differences between the datasets with respect to such latent attribute distributions. Developers can then, for example, collect additional development data with these attributes and conduct additional tests for these attributes to enhance reliability. We demonstrate that StyleDiff accurately detects differences between datasets and presents them in an understandable format using, for example, driving scene datasets.
A Trustworthiness Score to Evaluate DNN Predictions

Due to the black box nature of deep neural networks (DNN), the continuous validation of DNN during operation is challenging with the absence of a human monitor. As a result this makes it difficult for developers and regulators to gain confidence in the deployment of autonomous systems employing DNN. It is critical for safety during operation to know when DNN’s predictions are trustworthy or suspicious. With the absence of a human monitor, the basic approach is to use the model’s output confidence score to assess if predictions are trustworthy or suspicious. However, the model’s confidence score is a result of computations coming from a black box, therefore lacks transparency and makes it challenging to automatedly credit trustworthiness to predictions. We introduce the trustworthiness score (TS), a simple metric that provides a more transparent and effective way of providing confidence in DNN predictions compared to model’s confidence score. The metric quantifies the trustworthiness in a prediction by checking for the existence of certain features in the predictions made by the DNN. We also use the underlying idea of the TS metric, to provide a suspiciousness score (SS) in the overall input frame to help in the detection of suspicious frames where false negatives exist. We conduct a case study using YOLOv5 on persons detection to demonstrate our method and usage of TS and SS. The case study shows that using our method consistently improves the precision of predictions compared to relying on model confidence score alone, for both 1) approving of trustworthy predictions (~ 20% improvement) and 2) detecting suspicious frames (~ 5% improvement).