Reviewerly/exHarmony huggingface dataset:
Measuring the Impact of Gradient Accumulation on Cloud-based Distributed Training

Gradient accumulation (GA) is a commonly adopted technique for addressing the GPU memory shortage problem in model training. It reduces memory consumption at the cost of increased computation time. Although widely used, its benefits to model training have not been systematically studied. Our work evaluates and summarizes the benefits of GA, especially in cloud-based distributed training scenarios, where training cost is determined by both execution time and resource consumption. We focus on how GA can be utilized to balance execution time and resource consumption to achieve the lowest bills. Through empirical evaluations on AliCloud platforms, we observe that the total training cost can be reduced by 31.2% on average with a 17.3% increase in training time, when GA is introduced in the large-model and small-bandwidth scenarios with data-parallel training strategies. Besides, taking micro-batch size into optimization can further decrease training time and cost by 21.2% and 24.8% on average, respectively, for hybrid-parallel strategies in large-model and GPU training scenarios.
BRAMAC: Compute-in-BRAM Architectures for Multiply-Accumulate on FPGAs

Deep neural network (DNN) inference using reduced integer precision has been shown to achieve significant improvements in memory utilization and compute throughput with little or no accuracy loss compared to full-precision floating-point. Modern FPGA-based DNN inference relies heavily on the on-chip block RAM (BRAM) for model storage and the digital signal processing (DSP) unit for implementing the multiply-accumulate (MAC) operation, a fundamental DNN primitive. In this paper, we enhance the existing BRAM to also compute MAC by proposing BRAMAC (Compute-in-BRAM Architectures for Multiply-Accumulate). BRAMAC supports 2's complement 2- to 8-bit MAC in a small dummy BRAM array using a hybrid bit-serial & bit-parallel data flow. Unlike previous compute-in-BRAM architectures, BRAMAC allows read/write access to the main BRAM array while computing in the dummy BRAM array, enabling both persistent and tiling-based DNN inference. We explore two BRAMAC variants: BRAMAC-2SA (with 2 synchronous dummy arrays) and BRAMAC-1DA (with 1 double-pumped dummy array). BRAMAC-2SA/BRAMAC-1DA can boost the peak MAC throughput of a large Arria-10 FPGA by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$2.6\times/2.1\times,2.3\times/2.0\times$</tex> , and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.9\times/1.7$</tex> × for 2-bit, 4-bit, and 8-bit precisions, respectively at the cost of 6.8%/3.4% increase in the FPGA core area. By adding BRAMAC-2SA/BRAMAC-1DA to a state-of-the-art tiling-based DNN accelerator, an average speedup of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$2.05\times/1.7\times$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.33\times/1.52\times$</tex> can be achieved for AlexNet and ResNet-34, respectively across different model precisions. Our code is available at: https://github.com/abdelfattah-lab/BRAMAC.
A Cloud-Fog Architecture for Video Analytics on Large Scale Camera Networks Using Semantic Scene Analysis

This paper proposes a scalable distributed video analytics framework that can process thousands of video streams from sources such as CCTV cameras using semantic scene analysis. The main idea is to deploy deep learning pipelines on the fog nodes and generate semantic scene description records (SDRs) of video feeds from the associated CCTV cameras. These SDRs are transmitted to the cloud instead of video frames saving on network bandwidth. Using these SDRs stored on the cloud database, we can answer many complex queries and perform rich video analytics, within extremely low latencies. There is no need to scan and process the video streams again on a per query basis. The software architecture on the fog nodes allows for integrating new deep learning pipelines dynamically into the existing system, thereby supporting novel analytics and queries. We demonstrate the effectiveness of the system by proposing a novel distributed algorithm for real-time vehicle pursuit. The proposed algorithm involves asking multiple spatio-temporal queries in an adaptive fashion to reduce the query processing time and is robust to inaccuracies in the deployed deep learning pipelines and camera failures.
HDFL: A Heterogeneity and Client Dropout-Aware Federated Learning Framework

Cross-device Federated Learning (FL) enables training machine learning (ML) models on private data that is heterogeneously distributed over many IoT end devices without violating privacy requirements. Clients typically vary significantly in data quality, hardware resources and stability, which results in challenges such as increased training times, higher resource costs, sub-par model performance and biased training. Existing works tend to address each of these challenges in isolation, but overlook how they might impact each other holistically. We perform a first of its kind characterization study that empirically demonstrates how these properties interact with each other to impact important performance metrics such as model error, fairness, resource cost and training time. We then propose a method called HDFL based on our observations, which is the first framework to our knowledge that comprehensively considers the multiple aforementioned important challenges of practical FL systems. We implement HDFL on a real distributed system and evaluate it on multiple benchmark datasets which show that HDFL achieves better Pareto frontier compared to both the state-of-the-practice and state-of-the-art systems with up to 4-10% better model accuracy, 33% improved good-intent fairness, 63% lower cost, and 17% faster training time.
Blockchain Proportional Governance Reconfiguration: Mitigating a Governance Oligarchy

Blockchain governance is paramount to lead securely a large group of users towards the same decisions without disputes about the legitimacy of a blockchain instance over another. As of today, there is no efficient way of protecting this governance against an oligarchy. This paper aims to offer a new dimension to the security of blockchains by proposing a solution known as proportional governance reconfiguration. This solution mitigates the formation of an oligarchy by (1) electing governors proportionally using a proportional multi-winner election protocol (2) reconfiguring the governance automatically and periodically. The proportional governance reconfiguration relies on a Solidity based implementation making it compatible and usable in many smart contract supported blockchains. We prove our solution solves the proportional governance problem and we evaluate our solution on two smart contract supporting blockchains Ethereum-PoA and Smart Redbelly Blockchain. Our results indicate that our proportional governance can elect 200 governors within 6–12 minutes when 1000 voters from 5 continents vote for 500 candidates.
Hardware/Software Co-design for Machine Learning Accelerators

This abstract highlights challenges in machine learning accelerator design and proposes solutions through software/hardware co-design techniques. To optimize single object detection, we introduce Mask-Net, a lightweight network that eliminates redundant computation. To address hardware limitations in Dynamic Graph Neural Networks (DGNNs), we present DGNN-Booster, a graph-agnostic FPGA accelerator. Our designs are open-source, generic, and applicable to real-world scenarios.
Attribution-based Personas in Virtual Software Engineering Education

The COVID-19 pandemic and the consequent introduction of virtual collaboration have introduced educators to unexpected situations and challenges. One of these challenges is social distance, which minimizes knowledge of another person’s character, and leaves room for misconceptions. Perceptions of a person’s personality are also referred to as dispositional attributions and, when misplaced, impact the educator-student dynamics. This paper studies dispositional attributions exhibited by software engineering educators in higher education and aims to raise awareness of potential misconceptions affecting the educator-student relationship caused by the virtual setting. We performed an exploratory case study in a practical university course with twelve distributed software engineering teams, each led by one or two educators. The course was conducted entirely virtually during the COVID-19 pandemic. The research process included discovering, categorizing, and modeling attribution-based personas, followed by qualitative and quantitative research methods of semi-structured interviews and survey questionnaires. These personas represent the subjects of potential misconceptions and encapsulate typical behaviors and attributions. Our research created seven personas: the Unprofessional, Ego is the Enemy, The Detached, the Loner, the Underperformer, Hiding but not Seeking, and Distraction Monster. These personas differ primarily in terms of character traits and motivation attributed to them. The results provide evidence that the virtual setting of the course can lead to several dispositional attributions. Educators in virtual software engineering settings should be aware of these attributions and their potential impact on the educator-student relationship.
Chronica: A Data-Imbalance-Aware Scheduler for Distributed Deep Learning

One of the major challenges in distributed deep learning is attenuating straggler problem. The straggler increases synchronization latency and significantly inhibits the convergence of deep learning model. We empirically observe that the imbal-anced data samples worsen the straggler problem and make the convergence of the deep learning model slower. However, existing approaches such as BOA and EP4DDL have not addressed data imbalance issues while solving the straggler problem. To overcome the straggler and data imbalance problems, we propose Chronica,a new data-imbalance-aware scheduler. Based on the size of the data samples and the configuration of each worker, Chronicaelaborately predicts the training time required for each worker. Chronicathen provides equivalent training time to each of the workers, alleviating both step- and epoch-level straggler problems. Furthermore, Chronicasuggests a new parameter synchronization scheme to achieve fast convergence based on the weighted average of the training workload on each worker. Our extensive evaluation using four deep learning models on 32 Amazon EC2 GPU instances showed that the new Chronicaachieves up to 3.19 times speedup over the state-of-the-art systems.
Cognitive Reflection in Software Verification and Testing

Verifying whether code meets its specifications requires critical thinking and analysis. However, cognitive biases may influence how well software engineers verify and test code. In this paper, I explore cognitive reflection and its association with accurately verifying code via both manual inspection and unit testing. In a two-phase exploratory study of Software Engineering undergraduate students (n=140), I examined their performance on Cognitive Reflection Tests (CRT), manual verification of function implementations, and the accuracy of unit test suites. The first phase found no relationship between CRT and unit test accuracy. However, the higher a student’s CRT score, the more likely they were to reject a defective implementation when inspecting it manually (p<0.0001, 95% CI 1.56-4.50). The second phase replicated the outcomes on an alternate version of the CRT and on a different unit test assignment, revealing a positive correlation between CRT and recognizing defects manually ($\rho$=0.478, p<0.01). I conclude that cognitive reflection is associated with software engineering students’ aptitude at identifying defects, but is not associated with their affirmation of algorithms without defects, nor with the accuracy of their unit tests.
XFaaS: Cross-platform Orchestration of FaaS Workflows on Hybrid Clouds

Functions as a Service (FaaS) have gained popularity for programming public clouds due to their simple abstraction, ease of deployment, effortless scaling and granular billing. Cloud providers also offer basic capabilities to compose these functions into workflows. FaaS and FaaS workflow models, however, are proprietary to each cloud provider. This prevents their portability across cloud providers, and requires effort to design workflows that run on different cloud providers or data centers. Such requirements are increasingly important to meet regulatory requirements, leverage cost arbitrage and avoid vendor lock-in. Further, the FaaS execution models are also different, and the overheads of FaaS workflows due to message indirection and cold-starts need custom optimizations for different platforms. In this paper, we propose XFaaS, a cross-platform deployment and orchestration engine for FaaS workflows to operate on multiple clouds. XFaaS allows “zero touch” deployment of functions and workflows across AWS and Azure clouds by automatically generating the necessary code wrappers, cloud queues, and coordinating with the native FaaS engine of the cloud providers. It also uses intelligent function fusion and placement logic to reduce the workflow execution latency in a hybrid cloud while mitigating costs, using performance and billing models specific to the providers based in detailed benchmarks. Our empirical results indicate that fusion offers up to ≈75 % benefits in latency and ≈57% reduction in cost, while placement strategies reduce the latency by ≈ 24%, compared to baselines in the best cases.
ScaMP: Scalable Meta-Parallelism for Deep Learning Search

Deep Learning (DL) models are growing exponentially and require increasingly powerful High Performance Computing (HPC) systems to train them. Achieving state-of-the-art results requires carefully tuning the DL model architecture and training settings, which is a time-consuming process commonly relegated to distributed search frameworks and trial-and-error. However, search frameworks don't provide a flexible parallelism scheme within and among the chosen DL framework for modern out-of-core DL models. In this paper, we propose Scalable Meta-Parallelism for Deep Learning Search (ScaMP): a distributed Hyperparameter Optimization (HPO) and Neural Architecture Search (NAS) framework that supports out-of-core models with flexible parallelism schemes. SCaMP is integrated into the modern DL ecosystem, and enables both efficient parallel training of concurrent candidate architectures and aggregate device memory saturation via a powerful load balancing engine. SCaMP estimates the memory requirements of each candidate architecture and automatically applies the appropriate model-parallel degree and maximum batch size supported for the given candidate. Further, HPO and NAS with SCaMP are highly customizable via flexible configuration options. We evaluate the benefits of our designs on synthetic training benchmarks and in training a state-of-the-art vision transformer model. We select transformers as a candidate DL model type and demonstrate a 29% improvement in end-to-end HPO time on 32 V100 GPUs on the Lassen and ThetaGPU HPC systems. Further, we demonstrate a reduction in the proportion of NAS time spent in communication from 28% to 15%. Finally, we thoroughly verify the correctness of SCaMP by training a state-of-the-art SwinIR model.
HyQ: Hybrid I/O Queue Architecture for NVMe over Fabrics to Enable High- Performance Hardware Offloading

NVMe over Fabrics (NVMe-oF) has been widely applied as a remote storage protocol in cloud computing. The existing NVMe-oF software stack consumes a large number of CPU resources. Emerging devices, such as Smart NICs and DPUs, have supported hardware offloading of NVMe-oF to free these valuable CPU cores. However, NVMe-oF offloading capacity is always compromised because of limited hardware resources on design. Additionally, from thorough evaluations, we found that NVMe-oF inevitably suffers from severe performance degradation on complex application I/O patterns when using hardware offloading. It is challenging to achieve high performance and fully utilize NVMe-oF offloading simultaneously. In this paper, we propose HyQ, a novel hybrid I/O queue architecture for NVMe-oF, to achieve high performance while gaining the advantages of hardware offloading. HyQ realizes the coexistence of hardware offloading and software non-offloading queues, thus enabling the dynamic dispatching of I/O requests to appropriate processing queues according to user-defined I/O scheduling policies. Additionally, HyQ provides a request scheduling framework to support customized schedulers that select appropriate queues for I/O requests. In our evaluation, HyQ achieves up to 1.91x IOPS and 8.36x bandwidth performance improvement over the original hardware offloading scheme.
農作業を考慮した複数移動ロボットの経路計画と運搬車への応用

In automatic harvesting of vegetables such as cabbages which are loaded in containers, autonomous driving of carriers is needed to carry containers. In order to operate multiple harvesters in a wide field such as in Hokkaido, it is necessary to run multiple carriers at the same time. The carrier needs to switch from forward to reverse for connection with the harvester. Moreover, the crop area decreases and the drivable area changes as the harvest progresses. In this paper, we propose a path planning method for multiple carriers that considers these features and shortens the total waiting time of harvesters after they finish harvesting in order to maximize farm work. In this method, the optimal paths are decided by Conflict Tree expansion. A path planning method of single agent considering the switching to reverse from forward is also proposed. In the actual driving, speed control is performed to avoid collision considering time lag from expectations of path planning. Finally, we show experimental results and explain the effectiveness of the proposed method. Sliding mode control is used for autonomous driving of the harvester and carrier. We confirmed that our method allows vehicles to travel simultaneously without collision or being stuck.
Chaotic Properties of New 1-D and 2-D Maps

The application of chaotic systems in several engineering domains becomes a major trend in last decades. Moreover, the cryptography domain has been attracted by such kind of systems due to their characteristics. Hence, through this paper, we propose a new chaotic discrete time systems. First, we introduce the mathematical description respectively for the proposed one dimensional (1-D) and two dimensional (2-D) maps. Then, we present the main properties of the proposed chaotic systems based on specific guidelines such as bifurcation, Lyapunov exponents, trajectory, sensitivity and key space. Simulation results prove the application of these new maps in the domain of the cryptography.
Quantum Algorithm for Position Weight Matrix Matching

We propose two quantum algorithms for a problem in bioinformatics, position weight matrix (PWM) matching, which aims to find segments (sequence motifs) in a biological sequence such as DNA and protein that have high scores defined by the PWM and are thus of informational importance related to biological function. The two proposed algorithms, the naive iteration method and the Monte-Carlo-based method, output matched segments, given the oracular accesses to the entries in the biological sequence and the PWM. The former uses quantum amplitude amplification (QAA) for sequence motif search, resulting in the query complexity scaling on the sequence length <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> , the sequence motif length <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</i> and the number of the PWMs <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(m\sqrt{Kn})$</tex-math></inline-formula> , which means speedup over existing classical algorithms with respect to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> . The latter also uses QAA, and further, quantum Monte Carlo integration for segment score calculation, instead of iteratively operating quantum circuits for arithmetic in the naive iteration method; then it provides the additional speedup with respect to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</i> in some situation. As a drawback, these algorithms use quantum random access memories and their initialization takes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> ) time. Nevertheless, our algorithms keep the advantage especially when we search matches in a sequence for many PWMs in parallel.
Public Feedback on the Programs and Services of Municipal Health Office of Padada Through Sentiment Analysis Using Convolutional Neural Network

This project goal is to assist the Municipal Heath Office of Padada in its daily operation in terms of collecting and facilitating data through its client satisfactory survey forms. There are two systems deployed which is the Web portal and Mobile application. Web portal is the medium to assist municipal health office administrators in reviewing the submitted data and its results coming from the clients who are using the mobile application, modification of the existing details of the programs and services and posting important announcements. The mobile application is utilized by the clients, in medium of submission, the clients can receive announcements posted by the administrators for information dissemination, review the programs and services that is being offered in the health office and the easy access on giving its satisfactory rating and comments. The app was created with through Java (React Native Framework) and JavaScript(ReactS, NodeJS) and with Firebase product and its tools. To obtain final feedback and validate whether or not the desired output was successfully materialized, various testing methods were used. Test cases and results were made and disseminated. The findings indicate that the system passed the testing phase and met the users requirements. Moreover, using the Web portal and Mobile Application, these applications provide user with a quick, easy-to-use and efficiently facilitating results from the collected data coming from different users and automated sending of satisfactory rating and comments
What Does Balloon Mean? Quantifying Language Frictions in the Mortgage Market

This paper studies how language frictions translate into differences in borrower outcomes in the mortgage market. Using newly available survey data, I document significant descriptive differences in perceptions of mortgages, application experiences, and interest rates between limited English proficient (LEP) and non-LEP borrowers. To measure the causal effects of language frictions, I exploit a Federal Housing Finance Agency policy that provided translated mortgage documents to lenders. After the policy change, LEP borrowers had a more streamlined application process, contacted more lenders, understood mortgage contracts better, and had lower borrowing costs. Reducing language frictions led to an expansion in credit supply, reduced loan risks, and a more competitive mortgage market for LEP borrowers in areas with a larger share of LEP people. Overall, my findings highlight a cost-effective way to expand financial inclusion, facilitate assimilation, and accumulate wealth for well-qualified LEP borrowers.