Reviewerly/exHarmony huggingface dataset:
MCCN

Cross-modal retrieval is an important multimedia research area which aims to take one type of data as the query to retrieve relevant data of another type. Most of the existing methods follow the paradigm of pair-wise learning and class-level learning to generate a common embedding space, where the similarity of heterogeneous multimodal samples can be calculated. However, in contrast to large-scale cross-modal retrieval applications which often need to tackle multiple modalities, previous studies on cross-modal retrieval mainly focus on two modalities (i.e., text-image or text-video). In addition, for large-scale cross-modal retrieval with modality diversity, another important problem is that the available training data are considerably modality-imbalanced. In this paper, we focus on the challenging problem of modality-imbalanced cross-modal retrieval, and propose a Multimodal Coordinated Clustering Network (MCCN) which consists of two modules, Multimodal Coordinated Embedding (MCE) module to alleviate the imbalanced training data and Multimodal Contrastive Clustering (MCC) module to tackle the imbalanced optimization. The MCE module develops a data-driven approach to coordinate multiple modalities via multimodal semantic graph for the generation of modality-balanced training samples. The MCC module learns class prototypes as anchors to preserve the pair-wise and class-level similarities across modalities for intra-class compactness and inter-class separation, and further introduces intra-class and inter-class margins to enhance optimization flexibility. We conduct experiments on the benchmark multimodal datasets to verify the effectiveness of our proposed method.
JokerGAN

Collecting labeled data for training of models for image recognition problems, including handwritten text recognition (HTR), is a tedious and expensive task. Recent work on handwritten text generation shows that generative models can be used as a data augmentation method to improve the performance of HTR systems.
Generalized Hate: Bias Victimization against Non-Asian Racial/Ethnic Minorities during the COVID-19 Pandemic

While much attention has been focused on the impact of the COVID-19 pandemic on hate crime victimization among the Asian population, there is reason to expect that other racial/ethnic minorities may also be at risk of bias-motivated victimization. The current research examines the prevalence of discrimination, hate crime victimization, and fear of victimization among Black and Hispanic individuals during the pandemic. The results, obtained from a survey administered in May 2020 to roughly 1,400 non-Hispanic Black and Hispanic respondents, indicate substantial bias victimization among both groups during the pandemic. Additionally, results reveal important associations between victimization and pandemic-related circumstances.
Snappy Wheelchair: An IoT-Based Flex Controlled Robotic Wheel Chair for Disabled People

According to the Bangladesh Bureau of Statistics 2011, almost 8 lac people are physically challenged. Among them, about 1.5 lac are from urban areas and the rest of the people from rural areas. Lots of challenged people need a wheelchair to move. This work results in a Snappy Wheelchair which will help physically challenged persons. It has two different controlling systems: first one joystick control system and second one flex sensor control system. Challenged person who can move only two fingers of their whole body also can control the Snappy Wheelchair with flex sensor control system. If anyone can move hand or single leg, they can control Snappy Wheelchair with joystick control system. If anyone wants twice controlling system at a time, he/she also can do it by pressing controlling switch to change one controlling system to another. Snappy Wheelchair can move forward, forward-left, forward-right, backward, backward-left, and backward-right. Snappy Wheelchair saves 50% among all of those automated and manual control wheelchairs which are available in the market. This is designed for those people with middle-class families who also can buy and it comes with helpful features. This work is already running in real-time environment.
Robust Logo Detection in E-Commerce Images by Data Augmentation

Logo detection is an important task in the intellectual property protection in e-commerce. In the paper, we introduce our solution for the ACM MM2021 Robust Logo Detection Grand Challenge. The competition requires the detection of logos (515 categories) in e-commerce images. This competition is challenged by long-tail distribution, small objects, and different types of noises. To overcome these challenges, we built a highly optimized and robust detector. We first tested many effective techniques for general object detection and then focused on data augmentation. We found that data augmentation was effective in improving the performance and robustness of logo detection. Based on the combination of these techniques, we achieved APs of 64.6% and 61.3% on the clean and noisy datasets respectively, which were improved by 8.1% and 19.5% relative to the official baseline. We ranked 5th among 36489 teams in the competition.
Accuracy evaluation of a new generic Trajectory Prediction model for Unmanned Aerial Vehicles

Unmanned Aerial Vehicles (UAVs) attract much attention, and they require trajectory information for planning and tactical operations. Based on the current trajectory data obtained by navigation systems, appropriate Trajectory Prediction (TP) models are required to determine future UAV trajectories. Regardless of the disparity of the models, the knowledge of TP accuracy and its performance evaluation are of paramount importance. After the review of extensive literature on current TP models, this paper develops a more accurate TP model by creating valid methodologies for error budgeting and mitigation. This is followed by the specification for quantifying TP accuracy and sub-functional testing employed in the process of model development. The new model is tested and validated by three credible case studies in relation to the most stringent UAV applications. The validation evidence demonstrates that the new model improves TP accuracy based on previous models.
Multi-Agent Distributed Deep Deterministic Policy Gradient for Partially Observable Tracking

In many existing multi-agent reinforcement learning tasks, each agent observes all the other agents from its own perspective. In addition, the training process is centralized, namely the critic of each agent can access the policies of all the agents. This scheme has certain limitations since every single agent can only obtain the information of its neighbor agents due to the communication range in practical applications. Therefore, in this paper, a multi-agent distributed deep deterministic policy gradient (MAD3PG) approach is presented with decentralized actors and distributed critics to realize multi-agent distributed tracking. The distinguishing feature of the proposed framework is that we adopted the multi-agent distributed training with decentralized execution, where each critic only takes the agent’s and the neighbor agents’ policies into account. Experiments were conducted in the distributed tracking tasks based on multi-agent particle environments where N(N=3,N=5) agents track a target agent with partial observation. The results showed that the proposed method achieves a higher reward with a shorter training time compared to other methods, including MADDPG, DDPG, PPO, and DQN. The proposed novel method leads to a more efficient and effective multi-agent tracking.
An infinite family of antiprimitive cyclic codes supporting Steiner systems $$S(3,8, 7^m+1)$$

Coding theory and combinatorial t-designs have close connections and interesting interplay. One of the major approaches to the construction of combinatorial t-designs is the employment of error-correcting codes. As we all known, some t-designs have been constructed with this approach by using certain linear codes in recent years. However, only a small number of infinite families of cyclic codes holding an infinite family of 3-designs are reported in the literature. The objective of this paper is to study an infinite family of antiprimitive cyclic codes and determine their parameters. By the parameters of these codes and their dual, some infinite family of 3-designs are presented and their parameters are also explicitly determined. In particular, the complements of the supports of the minimum weight codewords in the studied cyclic code form a Steiner system. Furthermore, we show that the infinite family of cyclic codes admit 3-transitive automorphism groups.
Towards Exploring the Limitations of Active Learning: An Empirical Study

Deep neural networks (DNNs) are increasingly deployed as integral parts of software systems. However, due to the complex interconnections among hidden layers and massive hyperparameters, DNNs must be trained using a large number of labeled inputs, which calls for extensive human effort for collecting and labeling data. Spontaneously, to alleviate this growing demand, multiple state-of-the-art studies have developed different metrics to select a small yet informative dataset for the model training. These research works have demonstrated that DNN models can achieve competitive performance using a carefully selected small set of data. However, the literature lacks proper investigation of the limitations of data selection metrics, which is crucial to apply them in practice. In this paper, we fill this gap and conduct an extensive empirical study to explore the limits of data selection metrics. Our study involves 15 data selection metrics evaluated over 5 datasets (2 image classification tasks and 3 text classification tasks), 10 DNN architectures, and 20 labeling budgets (ratio of training data being labeled). Our findings reveal that, while data selection metrics are usually effective in producing accurate models, they may induce a loss of model robustness (against adversarial examples) and resilience to compression. Overall, we demonstrate the existence of a trade-off between labeling effort and different model qualities. This paves the way for future research in devising data selection metrics considering multiple quality criteria.
Postdigital education in a biotech future

This paper explores a possible future of postdigital education in 2050 using the means of social science fiction. The first part of the paper introduces the shift from 20 th century primacy of physics to 21 st century primacy of biology with an accent to new postdigital–biodigital reconfigurations and challenges in and after the COVID-19 pandemic. The second part of the paper presents a fictional speech at the graduation ceremony of a fictional military academy in a fictional East Asian country in 2050. This fictional world is marked by global warfare and militarization, and addressed graduates are the first generation of artificially evolved graduates in human history. The third part of the paper interprets the fictional narrative, contextualizes it into educational challenges of today, and argues for a dialogical, humanistic conception of new postdigital education in a biotech future.
Have Best of Both Worlds: Two-Pass Hybrid and E2E Cascading Framework for Speech Recognition

Hybrid and end-to-end (E2E) systems have their individual advantages, with different error patterns in the speech recognition results. By jointly modeling audio and text, the E2E model performs better in matched scenarios and scales well with a large amount of paired audio-text training data. The modularized hybrid model is easier for customization, and better to make use of a massive amount of unpaired text data. This paper proposes a two-pass hybrid and E2E cascading (HEC) framework to combine the hybrid and E2E model in order to take advantage of both sides, with hybrid in the first pass and E2E in the second pass. We show that the proposed system achieves 8-10% relative word error rate reduction with respect to each individual system. More importantly, compared with the pure E2E system, we show the proposed system has the potential to keep the advantages of hybrid system, e.g., customization and segmentation capabilities. We also show the second pass E2E model in HEC is robust with respect to the change in the first pass hybrid model.
From Superficial to Deep: Language Bias driven Curriculum Learning for Visual Question Answering

Most Visual Question Answering (VQA) models are faced with language bias when learning to answer a given question, thereby failing to understand multimodal knowledge simultaneously. Based on the fact that VQA samples with different levels of language bias contribute differently for answer prediction, in this paper, we overcome the language prior problem by proposing a novel Language Bias driven Curriculum Learning (LBCL) approach, which employs an easy-to-hard learning strategy with a novel difficulty metric Visual Sensitive Coefficient (VSC). Specifically, in the initial training stage, the VQA model mainly learns the superficial textual correlations between questions and answers (easy concept) from more-biased examples, and then progressively focuses on learning the multimodal reasoning (hard concept) from less-biased examples in the following stages. The curriculum selection of examples on different stages is according to our proposed difficulty metric VSC, which is to evaluate the difficulty driven by the language bias of each VQA sample. Furthermore, to avoid the catastrophic forgetting of the learned concept during the multi-stage learning procedure, we propose to integrate knowledge distillation into the curriculum learning framework. Extensive experiments show that our LBCL can be generally applied to common VQA baseline models, and achieves remarkably better performance on the VQA-CP v1 and v2 datasets, with an overall 20% accuracy boost over baseline models.
Scalable Recursive Distributed Collaborative State Estimation for Aided Inertial Navigation

This paper presents a novel approach to recover outdated cross-covariance between correlated agents at the moment they perform joint observations. This allows to render Collaborative State Estimation (CSE) fully distributed, with communication only required for the moment of joint observation and most importantly, it significantly reduces the maintenance effort in case of high frequent propagation sensors. These properties make the approach suitable to a wide range of multi-robot applications. In our evaluation on a Quaternion-based Error-State Extended Kalman Filter (Q-ESEKF) using an Inertial Measurement Unit (IMU) as propagation sensor at a rate of 200Hz, we showed a significant speedup against our previous approach for maintaining a couple of interdependence. We compared the approach in total against four different approaches on both, a simulation and on a real-world dataset for Micro Aerial Vehicles (MAVs). Video: https://youtu.be/xkljfwbhMP0
Hardware-Efficient Residual Neural Network Execution in Line-Buffer Depth-First Processing

Deep convolutional neural networks (CNNs) provide State-of-the-Art (SotA) results in a variety of machine learning (ML) applications, such as image classification and object detection. The trends towards higher-resolution input images and large-scale model structures have brought drastic performance improvements for these tasks. Yet, these trends also pose increased stress on hardware resources. Recent work proposed an advanced layer fusion method relying on the line-buffer depth-first (LBDF) processing, to reduce memory usage and off-chip memory accesses in high resolution CNN processing. However, when deploying such LBDF approach for residual networks, the required external memory access or on-chip memory capacity drastically increases again, hence canceling many benefits of the depth-first approach. This paper, therefore, outlines and analyzes the possible methods for handling residual connections (RC) in combination with LBDF. With the proposed ReLU-based compression and tiling techniques, we minimize the overhead of RC on on-chip memory requirements and external memory accesses. Moreover, a proposed hybrid strategy, optimally combining the different methods across different residual blocks of a residual network, yields for ResNet20 and SqueezeNet a reduction of 24.7% and 45.1% respectively in inference energy cost. Also the execution latency is reduced by 19.4% and 46.7% respectively in comparison to the best single method under certain on-chip memory budgets. In addition, compared to the SotAs, in the best case, our hybrid strategy only consumes 43% energy and 29.9% latency while use the same on-chip memories; only requires 27.4% on-chip memory to achieve the same performance.
Sentiment Analysis on Hindi–English Code-Mixed Social Media Text

Social media has been experiencing an enormous amount of activity from millions of people across the globe over last few years. This resulted in the accumulation of substantial amount of textual data and increased several opportunities of analysis. Sentiment analysis and classification is one such task where the opinion expressed in the text is identified and classified accordingly. This becomes even more trickier in code-mixed text due to free style of writing which does not have a proper syntactic structure. In this paper, we worked on such Hind–English code-mixed texts obtained from SentiMix shared task of SemEval-2020. We created a novel customized embedding model for feature generation from Hindi–English code-mixed texts to classify them to various sentiments like positive, neutral and negative using deep learning techniques. It is observed that attention-based CNN-Bi-LSTM model has achieved better performance out of all models with 70.32% F1-score.
SLA-DQTS: SLA Constrained Adaptive Online Task Scheduling Based on DDQN in Cloud Computing

Task scheduling is key to performance optimization and resource management in cloud computing systems. Because of its complexity, it has been defined as an NP problem. We introduce an online scheme to solve the problem of task scheduling under a dynamic load in the cloud environment. After analyzing the process, we propose a server level agreement constraint adaptive online task scheduling algorithm based on double deep Q-learning (SLA-DQTS) to reduce the makespan, cost, and average overdue time under the constraints of virtual machine (VM) resources and deadlines. In the algorithm, we prevent the change of the model input dimension with the number of VMs by taking the Gaussian distribution of related parameters as a part of the state space. Through the design of the reward function, the model can be optimized for different goals and task loads. We evaluate the performance of the algorithm by comparing it with three heuristic algorithms (Min-Min, random, and round robin) under different loads. The results show that the algorithm in this paper can achieve similar or better results than the comparison algorithms at a lower cost.
Dispersion-Minimizing Motion Primitives for Search-Based Motion Planning

Search-based planning with motion primitives is a powerful motion planning technique that can provide dynamic feasibility, optimality, and real-time computation times on size, weight, and power-constrained platforms in unstructured environments. However, optimal design of the motion planning graph, while crucial to the performance of the planner, has not been a main focus of prior work. This paper proposes to address this by introducing a method of choosing vertices and edges in a motion primitive graph that is grounded in sampling theory and leads to theoretical guarantees on planner completeness. By minimizing dispersion of the graph vertices in the metric space induced by trajectory cost, we optimally cover the space of feasible trajectories with our motion primitive graph. In comparison with baseline motion primitives defined by uniform input space sampling, our motion primitive graphs have lower dispersion, find a plan with fewer iterations of the graph search, and have only one parameter to tune.
Spatiotemporal Video Highlight by Neural Network Considering Gaze and Hands of Surgeon in Egocentric Surgical Videos

In the medical field, surgical videos can be used to introduce surgical skills. Medical students and residents watch the videos to study the surgical skills and increase learning speed by compensating for the lack of experience in surgical rooms due to limited opportunity to join in surgery. To record egocentric surgical videos by a wearable camera is a solution to record surgical skills of a surgeon in detail. However, most egocentric surgical videos are of quite long duration. For example, in the case of tumor removal in breast surgery, a video recording time often reaches 2[Formula: see text]h. With that length, it is time consuming to see important scenes in the video, particularly because many surgical videos include nonessential scenes such as sterilization and preparation of tools. For extracting specific scenes from a long video, we can apply scene estimation by machine learning. Furthermore, it is important to know where the surgeon is looking to observe the area of the incision in detail. In particular, it is vital to be able to zoom in on key elements, allowing viewers to see the incision area and the fine details of the necessary surgical skills. In this study, we aimed to highlight incision scenes from egocentric surgical videos in the spatiotemporal domain by utilizing two neural networks for the temporal and spatial highlights. For the temporal highlights, we designed a neural network that estimates the incision scenes by learning gaze speed, hand movements, number of hands, and background movements in egocentric surgical videos. For the spatial highlights, in order to estimate the important area to zoom in, we designed a neural network that learns the surgeon’s gaze on natural features of surgical scenes to form a probability map as a representation of the estimated gaze area. The estimated gaze area was also used to calculate the appropriate zoom-in position and zoom-in ratio. To control the highlighted parameters in accord with user preferences, we also made a user interface that allows for the selection of playback speed gain and zoom ratio gain. For the evaluation, we verified the performance of the networks by a quantitative assessment and conducted a user study with medical doctors by showing an actual surgical video to obtain a qualitative assessment on the proposed system.
Adversarial Robustness Evaluation of Deep Convolutional Neural Network Based SAR ATR Algorithm

Robustness, both to accident and to malevolent perturbations, is a crucial determinant of the successful deployment of deep convolutional neural network based SAR ATR systems in various security-sensitive applications. This paper performs a detailed adversarial robustness evaluation of deep convolutional neural network based SAR ATR models across two public available SAR target recognition datasets. For each model, seven different adversarial perturbations, ranging from gradient based optimization to self-supervised feature distortion, are generated for each testing image. Besides adversarial average recognition accuracy, feature attribution techniques have also been adopted to analyze the feature diffusion effect of adversarial attacks, which promotes the understanding of vulnerability of deep learning models.