Miracle-dz/UniFAR-FTUs huggingface dataset:
Initiative and Materiality: Exploring Mixed-Initiative Calculators with the Tangible Human-A.I. Interaction Framework

How can interactions with A.I. systems be designed? This paper explores the design space for A.I. interaction to develop tools for designers to think about tangible and physical A.I. interactions. Our proposed framework consists of two dimensions: initiative (human, mixed, or machine) and materiality (physical, combined, or digital form). A particularly interesting area of interactions we identify is the quadrant of physical, machine-initiated interactions. With our framework, we examine calculator interactions and attempt to expand these to the tangible, mixed-initiative space. We illustrate each area in our proposed framework with one representative example of a calculator -- a common and well-known example of a computing device. We discuss existing examples of calculators and speculative future interactions with mixed-initiative and physical calculator systems. We reflect on the implications of our framework for the larger task of designing human-A.I. collaborative systems. Designers can also apply this framework as a guideline for analogous solutions to problems in the same domain.
Atlas: Hybrid Cloud Migration Advisor for Interactive Microservices

Hybrid cloud provides an attractive solution to microservices for better resource elasticity. A subset of application components can be offloaded from the on-premises cluster to the cloud, where they can readily access additional resources. However, the selection of this subset is challenging because of the large number of possible combinations. A poor choice degrades the application performance, disrupts the critical services, and increases the cost to the extent of making the use of hybrid cloud unviable. This paper presents Atlas, a hybrid cloud migration advisor. Atlas uses a data-driven approach to learn how each user-facing API utilizes different components and their network footprints to drive the migration decision. It learns to accelerate the discovery of high-quality migration plans from millions and offers recommendations with customizable trade-offs among three quality indicators: end-to-end latency of user-facing APIs representing application performance, service availability, and cloud hosting costs. Atlas continuously monitors the application even after the migration for proactive recommendations. Our evaluation shows that Atlas can achieve 21% better API performance (latency) and 11% cheaper cost with less service disruption than widely used solutions.
Exploration via linearly perturbed loss minimisation

We introduce exploration via linear loss perturbations (EVILL), a randomised exploration method for structured stochastic bandit problems that works by solving for the minimiser of a linearly perturbed regularised negative log-likelihood function. We show that, for the case of generalised linear bandits, EVILL reduces to perturbed history exploration (PHE), a method where exploration is done by training on randomly perturbed rewards. In doing so, we provide a simple and clean explanation of when and why random reward perturbations give rise to good bandit algorithms. We propose data-dependent perturbations not present in previous PHE-type methods that allow EVILL to match the performance of Thompson-sampling-style parameter-perturbation methods, both in theory and in practice. Moreover, we show an example outside generalised linear bandits where PHE leads to inconsistent estimates, and thus linear regret, while EVILL remains performant. Like PHE, EVILL can be implemented in just a few lines of code.
Personalized Federated Learning via ADMM with Moreau Envelope

Personalized federated learning (PFL) is an approach proposed to address the issue of poor convergence on heterogeneous data. However, most existing PFL frameworks require strong assumptions for convergence. In this paper, we propose an alternating direction method of multipliers (ADMM) for training PFL models with Moreau envelope (FLAME), which achieves a sublinear convergence rate, relying on the relatively weak assumption of gradient Lipschitz continuity. Moreover, due to the gradient-free nature of ADMM, FLAME alleviates the need for hyperparameter tuning, particularly in avoiding the adjustment of the learning rate when training the global model. In addition, we propose a biased client selection strategy to expedite the convergence of training of PFL models. Our theoretical analysis establishes the global convergence under both unbiased and biased client selection strategies. Our experiments validate that FLAME, when trained on heterogeneous data, outperforms state-of-the-art methods in terms of model performance. Regarding communication efficiency, it exhibits an average speedup of 3.75x compared to the baselines. Furthermore, experimental results validate that the biased client selection strategy speeds up the convergence of both personalized and global models.
Spotting Fake Profiles in Social Networks via Keystroke Dynamics

Spotting and removing fake profiles could curb the menace of fake news in society. This paper, thus, investigates fake profile detection in social networks via users' typing patterns. We created a novel dataset of 468 posts from 26 users on three social networks: Facebook, Instagram, and $X$ (previously Twitter) over six sessions. Then, we extract a series of features from keystroke timings and use them to predict whether two posts originated from the same users using three prominent statistical methods and their score-level fusion. The models' performance is evaluated under same, cross, and combined-cross-platform scenarios. We report the performance using k-rank accuracy for $k$ varying from 1 to 5. The best-performing model obtained accuracies between 91.6%-100% on Facebook (Fusion), 70.8 - 87.5% on Instagram (Fusion), and 75% - 87.5% on X (Fusion) for $k$ from 1 to 5. Under a cross-platform scenario, the fusion model achieved mean accuracies of 79.1% - 91.6%, 87.5% - 91.6%, and 83.3% - 87.5% when trained on Facebook, Instagram, and Twitter posts, respectively. In combined cross-platform, which involved mixing two platforms' data for model training while testing happened on the third platform's data, the best model achieved accuracy ranges of 75% - 95.8% across different scenarios. The results highlight the potential of the presented method in uncovering fake profiles across social network platforms.11©2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. To appear in 2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)
A Comprehensive Survey On Client Selections in Federated Learning

Federated Learning (FL) is a rapidly growing field in machine learning that allows data to be trained across multiple decentralized devices. The selection of clients to participate in the training process is a critical factor for the performance of the overall system. In this survey, we provide a comprehensive overview of the state-of-the-art client selection techniques in FL, including their strengths and limitations, as well as the challenges and open issues that need to be addressed. We cover conventional selection techniques such as random selection where all or partial random of clients is used for the trained. We also cover performance-aware selections and as well as resource-aware selections for resource-constrained networks and heterogeneous networks. We also discuss the usage of client selection in model security enhancement. Lastly, we discuss open issues and challenges related to clients selection in dynamic constrained, and heterogeneous networks.
Understanding Path Planning Explanations

Navigation is a must-have skill for any mobile robot. A core challenge in navigation is the need to account for an ample number of possible configurations of environment and navigation contexts. We claim that a mobile robot should be able to explain its navigational choices making its decisions understandable to humans. In this paper, we briefly present our approach to explaining navigational decisions of a robot through visual and textual explanations. We propose a user study to test the understandability and simplicity of the robot explanations and outline our further research agenda.
Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study

The brittleness of finetuned language model performance on out-of-distribution (OOD) test samples in unseen domains has been well-studied for English, yet is unexplored for multi-lingual models. Therefore, we study generalization to OOD test data specifically in zero-shot cross-lingual transfer settings, analyzing performance impacts of both language and domain shifts between train and test data. We further assess the effectiveness of counterfactually augmented data (CAD) in improving OOD generalization for the cross-lingual setting, since CAD has been shown to benefit in a monolingual English setting. Finally, we propose two new approaches for OOD generalization that avoid the costly annotation process associated with CAD, by exploiting the power of recent large language models (LLMs). We experiment with 3 multilingual models, LaBSE, mBERT, and XLM-R trained on English IMDb movie reviews, and evaluate on OOD test sets in 13 languages: Amazon product reviews, Tweets, and Restaurant reviews. Results echo the OOD performance decline observed in the monolingual English setting. Further, (i) counterfactuals from the original high-resource language do improve OOD generalization in the low-resource language, and (ii) our newly proposed cost-effective approaches reach similar or up to +3.1% better accuracy than CAD for Amazon and Restaurant reviews.
IndexMAC: A Custom RISC-V Vector Instruction to Accelerate Structured-Sparse Matrix Multiplications

Structured sparsity has been proposed as an efficient way to prune the complexity of modern Machine Learning (ML) applications and to simplify the handling of sparse data in hardware. The acceleration of ML models - for both training and inference - relies primarily on equivalent matrix multiplications that can be executed efficiently on vector processors or custom matrix engines. The goal of this work is to incorporate the simplicity of structured sparsity into vector execution, thereby accelerating the corresponding matrix multiplications. Toward this objective, a new vector index-multiply-accumulate instruction is proposed, which enables the implementation of lowcost indirect reads from the vector register file. This reduces unnecessary memory traffic and increases data locality. The proposed new instruction was integrated in a decoupled RISCV vector processor with negligible hardware cost. Extensive evaluation demonstrates significant speedups of 1.80x-2.14x, as compared to state-of-the-art vectorized kernels, when executing layers of varying sparsity from state-of-the-art Convolutional Neural Networks (CNNs).
C-Procgen: Empowering Procgen with Controllable Contexts

We present C-Procgen, an enhanced suite of environments on top of the Procgen benchmark. C-Procgen provides access to over 200 unique game contexts across 16 games. It allows for detailed configuration of environments, ranging from game mechanics to agent attributes. This makes the procedural generation process, previously a black-box in Procgen, more transparent and adaptable for various research needs.The upgrade enhances dynamic context management and individualized assignments, while maintaining computational efficiency. C-Procgen's controllable contexts make it applicable in diverse reinforcement learning research areas, such as learning dynamics analysis, curriculum learning, and transfer learning. We believe that C-Procgen will fill a gap in the current literature and offer a valuable toolkit for future works.
Towards Automatic Honey Bee Flower-Patch Assays with Paint Marking Re-Identification

In this paper, we show that paint markings are a feasible approach to automatize the analysis of behavioral assays involving honey bees in the field where marking has to be as lightweight as possible. We contribute a novel dataset for bees re-identification with paint-markings with 4392 images and 27 identities. Contrastive learning with a ResNet backbone and triplet loss led to identity representation features with almost perfect recognition in closed setting where identities are known in advance. Diverse experiments evaluate the capability to generalize to separate IDs, and show the impact of using different body parts for identification, such as using the unmarked abdomen only. In addition, we show the potential to fully automate the visit detection and provide preliminary results of compute time for future real-time deployment in the field on an edge device.
Managing Large Enclaves in a Data Center

Live migration of an application or VM is a well-known technique for load balancing, performance optimization, and resource management. To minimize the total downtime during migration, two popular methods -- pre-copy or post-copy -- are used in practice. These methods scale to large VMs and applications since the downtime is independent of the memory footprint of an application. However, in a secure, trusted execution environment (TEE) like Intel's scalable SGX, the state-of-the-art still uses the decade-old stop-and-copy method, where the total downtime is proportional to the application's memory footprint. This is primarily due to the fact that TEEs like Intel SGX do not expose memory and page table accesses to the OS, quite unlike unsecure applications. However, with modern TEE solutions that efficiently support large applications, such as Intel's Scalable SGX and AMD's Epyc, it is high time that TEE migration methods also evolve to enable live migration of large TEE applications with minimal downtime (stop-and-copy cannot be used any more). We present OptMig, an end-to-end solution for live migrating large memory footprints in TEE-enabled applications. Our approach does not require a developer to modify the application; however, we need a short, separate compilation pass and specialized software library support. Our optimizations reduce the total downtime by 98% for a representative microbenchmark that uses 20GB of secure memory and by 90 -- 96% for a suite of Intel SGX applications that have multi-GB memory footprints.
CARBON SEQUESTRATION POTENTIAL OF URBAN TREES

Climate change is widely recognized not just as an environmental issue but one with severe socioeconomic implications across the globe. The impact of higher temperature, erratic precipitation, extreme weather events and rise in sea level are felt at local, regional and global levels. As the living space in urban areas are becoming rapidly cluttered and haphazard and as concrete jungles continue to degrade the natural and aesthetic environment and turning it into heat islands, there is a need to revive urban green efforts. Trees are considered to be major capital assets in cities as they provide myriad of benefits. They provide shade, filter air pollutants, absorb greenhouse gases, improve property value and contribute to aesthetic beauty. In recognition of the importance of urban trees and in an effort to build the native biodiversity and generate ecosystem services such as carbon sequestration within the urban landscape, an attempt has been made to address the potential of trees in carbon sequestration. Assessment of carbon sequestration of urban trees is carried out through biomass estimation and quantification. Trees are identified to species level, and their diameter at breast height (DBH) and height are recorded using ground measurements. Such studies highlight the role of urban tree cover in carbon sequestration and emphasize the need for greater attention to be paid to the selection of trees in cities, not just with a view to easy maintenance as is currently the case, but to select an appropriate mix of trees that supports biodiversity and maximizes environmental services.
Discrete Nonparametric Causal Discovery Under Latent Class Confounding

Directed acyclic graphs are used to model the causal structure of a system. ``Causal discovery'' describes the problem of learning this structure from data. When data is an aggregate from multiple sources (populations or environments), global confounding obscures conditional independence properties that drive many causal discovery algorithms. This setting is sometimes known as a mixture model or a latent class. While some modern methods for causal discovery are able to work around unobserved confounding in specific cases, the only known ways to deal with a global confounder involve parametric assumptions. that are unsuitable for discrete distributions.Focusing on discrete and non-parametric observed variables, we demonstrate that causal discovery can still be identifiable under bounded latent classes. The feasibility of this problem is governed by a trade-off between the cardinality of the global confounder, the cardinalities of the observed variables, and the sparsity of the causal structure.
Locality properties for discrete and continuum Widom--Rowlinson models in random environments

We consider the Widom--Rowlinson model in which hard disks of two possible colors are constrained to a hard-core repulsion between particles of different colors, in quenched random environments. These random environments model spatially dependent preferences for the attachment of disks. We investigate the possibility to represent the joint process of environment and infinite-volume Widom--Rowlinson measure in terms of continuous (quasilocal) Papangelou intensities. We show that this is not always possible: In the case of the symmetric Widom-Rowlinson model on a non-percolating environment, we can explicitly construct a discontinuity coming from the environment. This is a new phenomenon for systems of continuous particles, but it can be understood as a continuous-space echo of a simpler non-locality phenomenon known to appear for the diluted Ising model (Griffiths singularity random field) on the lattice, as we explain in the course of the proof.
Morphology of pyrite in particulate matter from shallow submarine hydrothermal vents

Abstract Here we present the first study of pyrite textures observed in particulate matter from an active shallow submarine vent site, which is characterized by large spatial and temporal variations in the physico-chemical conditions caused by a combination of geochemical processes and microbial activity. Morphologic characterization of pyrite crystals from suspended particulate matter in the discharged fluid of a shallow submarine vent system near Punta Mita, Nayarit, Mexico, showed diverse crystal morphologies and aggregates; well-defined framboids are only observed within the deposits of finelayered calcite (calcareous tuff) formed around the vents, whereas particulate matter contains diverse pyrite crystal forms as globular, cubic, octahedral, and pyritohedral but no framboids. Available sulfur isotopic data indicate that pyrite is formed as a consequence of microbial sulfate reduction in a hydrothermal reducing environment. The results of our study in a natural system provide evidence of the effect of variations in key parameters, such as redox state, on the pyrite morphologies and framboid size distribution, and support the conclusions of numerous studies that have attempted to explain pyrite crystallization processes that generate different morphologies as a result of large variations in the physico-chemical conditions.
Speech-based Slot Filling using Large Language Models

Recently, advancements in large language models (LLMs) have shown an unprecedented ability across various language tasks. This paper investigates the potential application of LLMs to slot filling with noisy ASR transcriptions, via both in-context learning and task-specific fine-tuning. Dedicated prompt designs and fine-tuning approaches are proposed to improve the robustness of LLMs for slot filling with noisy ASR transcriptions. Moreover, a linearised knowledge injection (LKI) scheme is also proposed to integrate dynamic external knowledge into LLMs. Experiments were performed on SLURP to quantify the performance of LLMs, including GPT-3.5-turbo, GPT-4, LLaMA-13B and Vicuna-13B (v1.1 and v1.5) with different ASR error rates. The use of the proposed fine-tuning together with the LKI scheme for LLaMA-13B achieved an 8.3% absolute SLU-F1 improvement compared to the strong Flan-T5-base baseline system on a limited data setup.
Controllability-Constrained Deep Network Models for Enhanced Control of Dynamical Systems

Control of a dynamical system without the knowledge of dynamics is an important and challenging task. Modern machine learning approaches, such as deep neural networks (DNNs), allow for the estimation of a dynamics model from control inputs and corresponding state observation outputs. Such data-driven models are often utilized for the derivation of model-based controllers. However, in general, there are no guarantees that a model represented by DNNs will be controllable according to the formal control-theoretical meaning of controllability, which is crucial for the design of effective controllers. This often precludes the use of DNN-estimated models in applications, where formal controllability guarantees are required. In this proof-of-the-concept work, we propose a control-theoretical method that explicitly enhances models estimated from data with controllability. That is achieved by augmenting the model estimation objective with a controllability constraint, which penalizes models with a low degree of controllability. As a result, the models estimated with the proposed controllability constraint allow for the derivation of more efficient controllers, they are interpretable by the control-theoretical quantities and have a lower long-term prediction error. The proposed method provides new insights on the connection between the DNN-based estimation of unknown dynamics and the control-theoretical guarantees of the solution properties. We demonstrate the superiority of the proposed method in two standard classical control systems with state observation given by low resolution high-dimensional images.
To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuning

Existing visual instruction tuning methods typically prompt large language models with textual descriptions to generate instruction-following data. Despite the promising performance achieved, these descriptions are derived from image annotations, which are oftentimes coarse-grained. Furthermore, the instructions might even contradict the visual content without observing the entire visual context. To address this challenge, we introduce a fine-grained visual instruction dataset, LVIS-Instruct4V, which contains 220K visually aligned and context-aware instructions produced by prompting the powerful GPT-4V with images from LVIS. Through experimental validation and case studies, we demonstrate that high-quality visual instructional data could improve the performance of LLaVA-1.5, a state-of-the-art large multimodal model, across a wide spectrum of benchmarks by clear margins. Notably, by simply replacing the LLaVA-Instruct with our LVIS-Instruct4V, we achieve better results than LLaVA on most challenging LMM benchmarks, e.g., LLaVA$^w$ (76.7 vs. 70.7) and MM-Vet (40.2 vs. 35.4). We release our data and model at https://github.com/X2FD/LVIS-INSTRUCT4V.