Reviewerly/exHarmony huggingface dataset:
Vector Commitments over Rings and Compressed $$\varSigma $$-Protocols

Compressed $$\varSigma $$ -Protocol Theory (CRYPTO 2020) presents an "alternative" to Bulletproofs that achieves the same communication complexity while adhering more elegantly to existing $$\varSigma $$ -protocol theory, which enables their techniques to be directly applicable to other widely used settings in the context of "plug & play" algorithmics. Unfortunately, their techniques are restricted to arithmetic circuits over prime fields, which rules out the possibility of using more machine-friendly moduli such as powers of 2, which have proven to improve efficiency in applications. In this work we show that such techniques can be generalized to the case of arithmetic circuits modulo any number. This enables the use of powers of 2, which can prove to be beneficial for efficiency, but it also facilitates the use of other moduli that might prove useful in different applications. In order to achieve this, we first present an instantiation of the main building block of the theory of compressed $$\varSigma $$ -protocols, namely compact vector commitments. Our construction, which may be of independent interest, is homomorphic modulo any positive integer m, a result that was not known in the literature before. Second, we generalize Compressed $$\varSigma $$ -Protocol Theory from finite fields to $$\mathbb {Z}_m$$ . The main challenge here is ensuring that there are large enough challenge sets as to fulfill the necessary soundness requirements, which is achieved by considering certain ring extensions. Our techniques have direct application for example to verifiable computation on homomorphically encrypted data.
Video-Text Representation Learning via Differentiable Weak Temporal Alignment

Learning generic joint representations for video and text by a supervised method requires a prohibitively substantial amount of manually annotated video datasets. As a practical alternative, a large-scale but uncurated and narrated video dataset, HowTo100M, has recently been introduced. But it is still challenging to learn joint embeddings of video and text in a self-supervised manner, due to its ambiguity and non-sequential alignment. In this paper, we propose a novel multi-modal self-supervised framework Video-Text Temporally Weak Alignment-based Contrastive Learning (VT-TWINS) to capture significant information from noisy and weakly correlated data using a variant of Dynamic Time Warping (DTW). We observe that the standard DTW inherently cannot handle weakly correlated data and only considers the globally optimal alignment path. To address these problems, we develop a differentiable DTW which also reflects local information with weak temporal alignment. Moreover, our proposed model applies a contrastive learning scheme to learn feature representations on weakly correlated data. Our extensive experiments demonstrate that VT-TWINS attains significant improvements in multi-modal representation learning and outperforms various challenging downstream tasks. Code is available at https://github.com/mlvlab/VT-Twins.
Power-Performance Characterization of TinyML Systems

TinyML systems are enabling machine learning (ML) inference at the edge. However, there exists little quantitative analysis of such systems. This paper presents a systematic performance and power characterization of diverse TinyML applications on micro-controllers (MCUs), spanning neural network models, software libraries, operating systems, and hardware architectures. We focus on the impact of the multiple layers of abstractions that provide higher programmability at the expense of performance and energy efficiency. We propose a model to estimate the costs of different abstraction layers and make recommendations for minimizing those costs. Our findings can help designers with Neural Architecture Search (NAS) and CNN inference optimization on edge devices.
Quaternion-Valued Correlation Learning for Few-Shot Semantic Segmentation

Few-shot segmentation (FSS) aims to segment unseen classes given only a few annotated samples. Encouraging progress has been made for FSS by leveraging semantic features learned from base classes with sufficient training samples to represent novel classes. The correlation-based methods lack the ability to consider interaction of the two subspace matching scores due to the inherent nature of the real-valued 2D convolutions. In this paper, we introduce a quaternion perspective on correlation learning and propose a novel Quaternion-valued Correlation Learning Network (QCLNet), with the aim to alleviate the computational burden of high-dimensional correlation tensor and explore internal latent interaction between query and support images by leveraging operations defined by the established quaternion algebra. Specifically, our QCLNet is formulated as a hyper-complex valued network and represents correlation tensors in the quaternion domain, which uses quaternion-valued convolution to explore the external relations of query subspace when considering the hidden relationship of the support sub-dimension in the quaternion space. Extensive experiments on the PASCAL-5i and COCO-20i datasets demonstrate that our method outperforms the existing state-of-the-art methods effectively. Our code is available at https://github.com/zwzheng98/QCLNet and our article "Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation" was published in IEEE Transactions on Circuits and Systems for Video Technology, vol. 33,no.5,pp.2102-2115,May 2023,doi: 10.1109/TCSVT.2022.3223150.
CreativeBot: a Creative Storyteller Agent Developed by Leveraging Pre-trained Language Models

In an attempt to nurture children's creativity, we developed a creative conversational agent to be used in a collaborative storytelling context with a child. We presented a novel approach to develop creative Artificial Intelligence (AI). Our approach uses the four creativity measures: fluency, flexi-bility, elaboration and originality in order to generate creative behavior. We analyzed and annotated our previously collected storytelling data sets -collected with children- according to our four creativity measures. We then used the extracted and annotated data (636 statements) in order to fine-tune two pre-trained language models (Open AI GPT-3). The two models were aimed at generating creative versus non-creative behavior in a collaborative storytelling scenario. We developed the two models to be able to assess the results and compare them together. We conducted an evaluation to assess stories generated collaboratively between a human and both agents separately (n = 26). Adult Users rated the creativity of the agent according to the stories generated. Results showed that the creative agent was perceived as significantly more creative than the non-creative agent. With the experiment results confirming the validity of our system, we may therefore proceed with testing the effects of the creative behavior of the agent on children's creativity skills.
Butsukusa: A Conversational Mobile Robot Describing Its Own Observations and Internal States

This paper presents an autonomous conversational mobile robot Butsukusa that can describe its own observations and internal states during patrolling tasks. The proposed robot can observe the surrounding environment using the recognition module for objects, humans, environment, localization, and speech and then move autonomously around an indoor living space. Interaction skills via language are required for the robot to perform in such human-centered spaces. To investigate a better communication protocol with users, we evaluate various language generation patterns based on different observations and interaction patterns. The evaluation results indicate that the importance of describing the robot's observation results and internal states, as well as the necessity of an appropriate description, depends on the situation.
GUSOT: Green and Unsupervised Single Object Tracking for Long Video Sequences

Supervised and unsupervised deep trackers that rely on deep learning technologies are popular in recent years. Yet, they demand high computational complexity and a high memory cost. A green unsupervised single-object tracker, called GUSOT, that aims at object tracking for long videos under a resource-constrained environment is proposed in this work. Built upon a baseline tracker, UHP-SOT++, which works well for short-term tracking, GUSOT contains two additional new modules: 1) lost object recovery, and 2) color-saliency-based shape proposal. They help resolve the tracking loss problem and offer a more flexible object proposal, respectively. Thus, they enable GUSOT to achieve higher tracking accuracy in the long run. We conduct experiments on the large-scale dataset LaSOT with long video sequences, and show that GUSOT offers a lightweight high-performance tracking solution that finds applications in mobile and edge computing platforms.
A Software Aging-Related Bug Prediction Framework Based on Deep Learning and Weakly Supervised Oversampling

Software aging refers to the phenomenon of sys-tem performance degradation and eventual failure caused by Aging-Related Bugs (ARBs). Software aging seriously affects the reliability and availability of software systems. To discover and remove ARBs, ARBs prediction is presented, and most of them only employed static code metrics to predict those buggy codes. However, static code metrics do not capture the syntactic and semantic features of the code, which are important to building accurate prediction models. To address this problem, we design a deep neural network by combining the bidirectional long short-term memory (BLSTM) and the attention mechanism to extract context-sensitive semantic features of the code. In addition, we apply a weakly supervised oversampling (WSO) method to alleviate class imbalance problems in datasets. We named our framework ABLSTM-WSO. We conduct experiments with five classifiers on two widely used open-source projects(MySQL and Linux) and use AUC, Balance, and F1-score as the evaluation metrics. Experimental results show that ABLSTM-WSO can significantly improve the ARBs prediction performance.
Modeling Multi-View Interactions with Contrastive Graph Learning for Collaborative Filtering

Graph Neural Networks (GNNs) and its many variants have recently been successfully utilized to tackle various recommendation tasks. Although effective, existing methods still face several limitations. First, supervision signal sparsity makes it difficult for them to learn high-quality representations and optimize the model parameters. Second, the learned representations are vulnerable to noisy interactions, as the neighborhood aggregation scheme increases the impact of observed edges. To alleviate these issues, we exploit self-supervised learning (SSL) to learn more accurate and robust representations from the user-item interaction graph. Particularly, we propose a novel SSL model that effectively integrates contrastive multi-view learning and pseudo-siamese network to construct a pre-training and post-training framework. Moreover, we present three data augmentation techniques during the pre-training stage and explore the effects of combining different augmentations, which allow us to learn more general and robust representations for the recommendation. Experimental evaluations on two real-world datasets show that the proposed solution significantly improves the recommendation accuracy, especially for sparse supervision signal, and is also noise resistant.
Posture and Appearance Fusion Network for Driver Distraction Recognition

Distracted driving is the act of driving while engaged in other activities, such as using a cell phone, texting, eating, or reading, which takes the driver’ attention away from the road. Nowadays, the distracted driving detection models based on deep learning can extract critical information from video data to characterize the driving behavior process. But the distraction driving method based solely on appearance features cannot essentially eliminate the noise impact of the complex environment on the model, and the distracted driving recognition method based solely on skeletal information is unable to recognize the joint action of the human body and the objects. Therefore, the development of an accurate distracted driving detection model has become challenging. In this paper, we propose a distracted driving recognition model MFD-former based on the fusion of posture and appearance. First, a feature extraction module is proposed to extract skeleton data(i.e., posture) and appearance features(i.e., descriptors), which are merged by a graph neural network. Then, the two kinds of information are input into the MFD-former encoder module, and the self-attention mechanism quickly extracts the sparse data. Finally, the classification results of distracted driving are obtained by extracting the classification labels through the MLP Head. The MFD-former model outperforms existing models. It achieved $$95.1\%$$ accuracy on the State Farm dataset and $$90.24\%$$ accuracy on the self-built Train Drivers dataset.
Transportation Object Counting With Graph-Based Adaptive Auxiliary Learning

This paper proposes an adaptive auxiliary task learning-based approach for transport object counting problems such as humans and vehicles. These problems are essential in many real-world tasks such as video surveillance, traffic monitoring, public security, and urban planning, to aid intelligent transportation systems. Unlike existing auxiliary task learning-based methods, we develop an attention-enhanced adaptively shared backbone network to enable both task-shared and task-tailored features that are learned in an end-to-end manner. The network seamlessly combines a standard Convolution Neural Network (CNN) and a Graph Convolution Network (GCN) for feature extraction and feature reasoning among different domains of tasks. Our approach gains enriched contextual information by iteratively and hierarchically fusing features across different task branches of the adaptive CNN backbone. The whole framework pays special attention to objects’ spatial locations and varied density levels, informed by object (or crowd) segmentation and density level segmentation auxiliary tasks. In particular, thanks to the proposed dilated contrastive density loss function, our network benefits from individual and regional context supervision, along with strengthened robustness. Experiments on six challenging multi-domain datasets demonstrate that our method achieves superior performance compared with state-of-the-art auxiliary task learning-based counting methods. Our code is publicly available.
Multiscale depthwise separable convolution based network for high-resolution image segmentation

ABSTRACTABSTRACTDeep learning-based segmentation methods have demonstrated significant performance over their traditional counterparts. However, striving for better accuracy with such networks usually leads to the deterioration of the network's computational efficiency, thereby rendering them inefficient for deployment on resource constraint devices. Establishing the required tradeoff between the accuracy of pixel prediction and computational efficiency remains challenging. In this article, a lightweight multiscale segmentation framework is proposed. We leverage the representation power of different receptive fields to attain optimal accuracy while maintaining computational efficiency by embedding the sparse network architecture with the depthwise separable convolution at the multiscale level. Experimental results from two challenging remote sensing segmentation datasets show that the proposed network can achieve substantial pixel prediction accuracy at relatively low computational overhead compared to state-of-the-art networks.KEYWORDS: Multiscalesegmentationremote sensingdepthwise separable convolution Disclosure statementNo potential conflict of interest was reported by the author(s).
SAMKR: Bottom-up Keypoint Regression Pose Estimation Method Based On Subspace Attention Module

As a hot research issue in computer vision, 2D human pose estimation plays an important role in human-computer interaction, intelligent monitoring, 3D human pose estimation and so on. Aiming at the problem of human scale inconsistency in the situation of multi-person, a keypoint regression method based on subspace attention module (SAMKR) is proposed in this paper for the 2D human pose estimation. Firstly, each keypoint is divided into independent regression branches, and then the feature mapping in each keypoint regression branch is evenly divided into a specified number of feature mapping subspaces, and different attention mappings are derived for each feature mapping subspace. By learning different attention maps in each feature subspace, multi-scale feature representation can be effectively improved. The experimental results show that SAMKR achieved 74.4 AP score on the CrowdPose test set, which may lead to an improvement of + 7.1AP, and reached 70.4 AP score on the COCO test-dev data set, which was 0.4AP higher than the baseline.
Toward Understanding Task Complexity in Maintenance-Based Studies of Programming Tools

Researchers conducting studies on programming tools often make use of maintenance tasks. The complexity of these tasks can influence the behavior of participants significantly. At the same time, the complexity of tasks is difficult to pinpoint due to the many sources of complexity for maintenance tasks. As a result, researchers may struggle to deliberately decide in which regard their tasks should be complex and in which regard they should be simple.
An approach to automatically assess method names

An approach is presented to automatically assess the quality of method names by providing a score and feedback. The approach implements ten method naming standards to evaluate the names. The naming standards are taken from work that validated the standards via a large survey of software professionals. Natural language processing techniques such as part-of-speech tagging, identifier splitting, and dictionary lookup are required to implement the standards. The approach is evaluated by first manually constructing a large golden set of method names. Each method name is rated by several developers and labeled as conforming to each standard or not. These ratings allow for comparing the results of the approach against expert assessment. Additionally, the approach is applied to several systems and the results are manually inspected for accuracy.
AdaFocusV3: On Unified Spatial-Temporal Dynamic Video Recognition

Recent research has revealed that reducing the temporal and spatial redundancy are both effective approaches towards efficient video recognition, e.g., allocating the majority of computation to a task-relevant subset of frames or the most valuable image regions of each frame. However, in most existing works, either type of redundancy is typically modeled with another absent. This paper explores the unified formulation of spatial-temporal dynamic computation on top of the recently proposed AdaFocusV2 algorithm, contributing to an improved AdaFocusV3 framework. Our method reduces the computational cost by activating the expensive high-capacity network only on some small but informative 3D video cubes. These cubes are cropped from the space formed by frame height, width, and video duration, while their locations are adaptively determined with a light-weighted policy network on a per-sample basis. At test time, the number of the cubes corresponding to each video is dynamically configured, i.e., video cubes are processed sequentially until a sufficiently reliable prediction is produced. Notably, AdaFocusV3 can be effectively trained by approximating the non-differentiable cropping operation with the interpolation of deep features. Extensive empirical results on six benchmark datasets (i.e., ActivityNet, FCVID, Mini-Kinetics, Something-Something V1 &V2 and Diving48) demonstrate that our model is considerably more efficient than competitive baselines.
Investigating Cyber Attacker Team Cognition

Cyber attackers commonly operate in teams, which may process information collectively and thus, may be best understood when the team is treated as the unit of analysis. Future research in Oppositional Human Factors (OHF) should consider the impact of team-influencing and team-level biases and the impact that defensive interventions have on team cognition in general. Existing measurement approaches using team interactions may be well suited for studying red teams, and how OHF interventions impact cyber attackers.
Deep Learning for Automatic Road Marking Detection with Yolov5

One of the most important responsibilities of a visual driver aid system is recognizing and tracking road signs. In recent years, tremendous progress has been made in both deep learning and the identification of road markings. Pedestrian crossings, directional arrows, zebra crossings, speed limit signs, and similar signs and text are all road surface markings. These markings are painted directly onto the surface of the road. This paper implements YOLOv5s and YOLOv5m to identify the road marking sign. We built a dataset and focused on the Taiwan road marking sign. According to the findings of our experiments, YOLOv5m contains eleven categories of whose training accuracy is superior to that of YOLOv5s. It has been discovered that the YOLOv5m model is the most accurate, scoring 87.30 percent overall throughout testing, while the YOLOv5s model scores an average of 83.60 percent.
FitGAN: Fit- and Shape-Realistic Generative Adversarial Networks for Fashion

Amidst the rapid growth of fashion e-commerce, remote fitting of fashion articles remains a complex and challenging problem and a main driver of customers’ frustration. Despite the recent advances in 3D virtual try-on solutions, such approaches still remain limited to a very narrow – if not only a handful – selection of articles, and often for only one size of those fashion items. Other state-of-the-art approaches that aim to support customers find what fits them online mostly require a high level of customer engagement and privacy-sensitive data (such as height, weight, age, gender, belly shape, etc.), or alternatively need images of customers’ bodies in tight clothing. They also often lack the ability to produce fit and shape aware visual guidance at scale, coming up short by simply advising which size to order that would best match a customer’s physical body attributes, without providing any information on how the garment may fit and look. Contributing towards taking a leap forward and surpassing the limitations of current approaches, we present FitGAN, a generative adversarial model that explicitly accounts for garments’ entangled size and fit characteristics of online fashion at scale. Conditioned on the fit and shape of the articles, our model learns disentangled item representations and generates realistic images reflecting the true fit and shape properties of fashion articles. Through experiments on real world data at scale, we demonstrate how our approach is capable of synthesizing visually realistic and diverse fits of fashion items and explore its ability to control fit and shape of images for thousands of online garments.