Reviewerly/exHarmony huggingface dataset:
Online Domain Adaptation for Semantic Segmentation in Ever-Changing Conditions

Unsupervised Domain Adaptation (UDA) aims at reducing the domain gap between training and testing data and is, in most cases, carried out in offline manner. However, domain changes may occur continuously and unpredictably during deployment (e.g. sudden weather changes). In such conditions, deep neural networks witness dramatic drops in accuracy and offline adaptation may not be enough to contrast it. In this paper, we tackle Online Domain Adaptation (OnDA) for semantic segmentation. We design a pipeline that is robust to continuous domain shifts, either gradual or sudden, and we evaluate it in the case of rainy and foggy scenarios. Our experiments show that our framework can effectively adapt to new domains during deployment, while not being affected by catastrophic forgetting of the previous domains.
Unsupervised Pre-training for 3D Object Detection with Transformer

Transformer improve the performance of 3D object detection with few hyperparameters. Inspired by the recent success of the pre-training Transformer in 2D object detection and natural language processing, we propose a pretext task named random block detection to unsupervisedly pre-train 3DETR (UP3DETR). Specifically, we sample random blocks from original point clouds and feed them into the Transformer decoder. Then, the whole Transformer is trained by detecting the locations of these blocks. The pretext task can pre-train the Transformer-based 3D object detector without any manual annotations. In our experiments, UP3DETR performs 6.2 $$\%$$ better than 3DETR baseline on challenging ScanNetV2 datasets and has a faster convergence speed on object detection tasks.
Recent Challenges and Opportunities in Video Summarization With Machine Learning Algorithms

The fast progress in digital technology has sparked generation of large number of voluminous data from different social media platforms like Instagram, Facebook, YouTube, etc. There are other platforms as well which generate large data like News, CCTV videos, Sports, Entertainment, etc. Lengthy Videos typically contain a significant number of duplicate occurrences that are uninteresting to the viewer. Eliminating this unnecessary information and concentrating only on the crucial events will be far more advantageous. This produces a summary of lengthy films, which can save viewers time and enable better memory management. The highlights of a lengthy video are condensed into a video summary. Video summarization is an essential topic today since many industries have CCTV cameras installed for various reasons such as monitoring, security, and tracking. Because surveillance videos are taken 24 hours a day, enormous amounts of memory and time are required if one wish to trace any incident or person from the full day’s video. The Summary generated from multiple view is far more challenging so more study and advancement in MVS is required. The conceptual basis of video summarizing approaches is thoroughly addressed in this paper. This paper addresses applications, technology challenges in Single view and Multi View summarization.
Multiclass Classification for GvHD Prognosis Prior to Allogeneic Stem Cell Transplantation

Hematopoietic Stem Cell Transplantation (HSCT) is an effective treatment for a variety of blood diseases, including hematologic and lymphoid malignancies, along with numerous other conditions. One of the most prevalent adverse effects of Allogeneic Stem Cell Transplantation (ASCT) in patients is Graft versus Host Disease (GvHD). Inflammation brought on by the donor's stem cells attacking the patient's body can lead to GvHD. Moreover, Acute Graft versus Host Disease (aGvHD) and Chronic Graft versus Host Disease (cGvHD) are the two possible manifestations. The patient has a chance of developing this disease even if the donor and the recipient are a perfect match. Therefore, early diagnosis of the forms of GvHD before a patient receives ASCT treatment is essential. However, it is still necessary to identify the type of GvHD even if the patient has already undergone a transplant to advise any clinical decision. In this research, the types of the GvHD are precisely predicted using a variety of multi-class classification models. The techniques utilized in this study include Random Forest, Decision Tree, K-Nearest Neighbor, Gradient Boosting, XG Boosting, LG Boosting and a feed forward Artificial Neural Network named Multilayer Perceptron. This study revealed that the Random Forest algorithm demonstrated state-of-the-art performance in multi-class classification, with an accuracy of 98.62% along with 96.38% F1-Score and an area under the ROC curve (AUC) score of 98.02%. In terms of accuracy and reduced feature dependence for predicting the multi-class target feature, this study offers a useful prognosis tool for medical experts.
Information Security: Safeguarding Resources and Building Trust

Security is a complex issue which is related to infrastructure, software, people, and processes. There are threats against all of these dimensions. In this chapter we will talk about security aspects of building modern applications and understand which most common solutions are applied to protect software from common threats.
Investigating Customers' Preferences of Robot's Serving Styles

This work investigated how robots with different service style in food industries are perceived by customers. We analysed responses from 53 participants who evaluated the video interaction of a robot waiter while taking orders and that delivered a customer service experience based on different styles of a service robot interaction (service-focused vs. social-focused). Results indicated that social-focused robots are perceived to have more natural and pleasant behaviours compared with service-focused robots, and therefore better accepted by customers. Social-focused robots were also perceived as having a more persuasive effect on the users' choices. In contrast, service-focused robots were perceived as hasty, and participants did not feel in charge of the food choices to order.
Asymmetry-Aware Load Balancing With Adaptive Switching Granularity in Data Center

Datacenter networks provide large bisection bandwidth by load balancing traffic over rich parallel paths in multi-rooted tree topologies. Nevertheless, production datacenters operate under various path diversities caused by traffic dynamics, hardware failures and heterogeneous switching equipment. Therefore, the load balancing schemes in data center should be resilient to network asymmetry. Prior fine-grained schemes such as RPS and Presto are prone to experience packet reordering problem under asymmetric topology since they split flows into small units which are spread across all parallel paths. The coarse-grained solutions such as ECMP and LetFlow effectively avoid packet reordering, but easily leading to under-utilization of multiple paths. To solve these problems, we propose a load balancing mechanism called AG, which adaptively adjusts switching granularity according to the asymmetric degree of multiple paths. AG increases switching granularity to alleviate packet reordering under large degrees of topology asymmetry, while reducing switching granularity to obtain high link utilization under small degrees of topology asymmetry. Moreover, we design a switch-based scheme which measures the difference of one-way delay of multiple paths to obtain accurate state of topology asymmetry with low overhead. AG is a practical switch-based solution without modification at end hosts. The experimental results of NS2 simulations and real implementation show that AG reduces the average and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$99^{th}$ </tex-math></inline-formula> flow completion time by up to 54% and 65% compared with the state-of-the-art load balancing schemes, respectively.
Aspect-guided Syntax Graph Learning for Explainable Recommendation

Explainable recommendation systems provide explanations for recommendation results to improve their transparency and persuasiveness. The existing explainable recommendation methods generate textual explanations without explicitly considering the user's preferences on different aspects of the item. In this paper, we propose a novel explanation generation framework, namely <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>A</u></b> spect-guided <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>E</u></b> xplanation generation with <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>S</u></b> yntax <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>G</u></b> raph ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AESG</b> ), for explainable recommendation. Specifically, AESG employs a review-based syntax graph to provide a unified view of the user/item details. An aspect-guided graph pooling operator is proposed to extract the aspect-relevant information from the review-based syntax graphs to model the user's preferences on an item at the aspect level. Then, an aspect-guided explanation decoder is developed to generate aspects and aspect-relevant explanations based on the attention mechanism. The experimental results on three real datasets indicate that AESG outperforms state-of-the-art explanation generation methods in both single-aspect and multi-aspect explanation generation tasks, and also achieves comparable or even better preference prediction accuracy than strong baseline methods.
Active Transfer Learning for Handwriting Recognition

With the advent of deep neural networks, handwriting recognition systems have recently achieved remarkable performance. Unfortunately, to achieve high-quality results, these models require large amounts of labeled training data, which is difficult to obtain. Various methods have been proposed to reduce the volume of training data required. We propose a framework for fitting new handwriting recognition models that joins both active and transfer learning into a unified framework. Empirical results show that our method performs better than traditional active learning, transfer learning, and standard supervised training methods.
Semantic VL-BERT: Visual Grounding via Attribute Learning

In recent years, Smart Home Assistants have expanded into tens of thousands of devices and transformed from a voice only assistant to a much more versatile smart assistant, that uses a connected display to provide a multi-modal customer experience. In order to further improve on the multi-modality experience, comprehension systems need models that can work with multisensory inputs. We focus on the problem of visual grounding, which allows customers to interact with and manipulate items displayed on a screen via voice. We propose a novel learning approach that improves upon a lightweight single stream transformer architecture by adjusting it to better align the visual input features with the referring expressions. Our approach learns to cluster parts of the image along spatial and channel dimensions based on descriptive attributes in the query, and takes advantage of the information in separate clusters more efficiently, as demonstrated by a 1.32% absolute accuracy improvement on a public dataset over the baseline. Given that modern-day Smart Home Assistants have very stringent memory and latency requirements, we restrict our focus to a family of lightweight single stream transformer architectures - our focus is not to beat the ever improving state-of-the-art in visual grounding but to improve upon a lightweight transformer architecture which leads to a model that is easy to train and deploy while having improved semantic awareness.
GraphMapper: Efficient Visual Navigation by Scene Graph Generation

Understanding the geometric relationships between objects in a scene is a core capability in enabling both humans and autonomous agents to navigate in new environments. A sparse, unified representation of the scene topology will allow agents to act efficiently to move through their environment, communicate the environment state with others, and utilize the representation for diverse downstream tasks. To this end, we propose a method to train an autonomous agent to learn to accumulate a 3D scene graph representation of its environment by simultaneously learning to navigate through said environment. We demonstrate that our approach, GraphMapper, enables the learning of effective navigation policies through fewer interactions with the environment than vision-based systems alone. Further, we show that GraphMapper can act as a modular scene encoder to operate alongside existing Learning-based solutions to not only increase navigational efficiency but also generate intermediate scene representations that are useful for other future tasks.
A Neighborhood Re-Ranking Model With Relation Constraint for Knowledge Graph Completion

Knowledge graph completion (KGC) aims to predict missing links based on observed triples. However, current KGC models are still limited by the following two aspects. (1) the entity semantics is implicitly learned by neural network and merely depends on existing facts, which mostly suffers from less additional specific knowledge. Although previous studies have noticed that entity type information can effectively improve KGC task, most of them rely on labeled type-specific data. (2) the recent graph-based models mainly concentrate on Graph Neural Network (GNN) to update source entity representation, regardless of the separate role that neighborhood information plays and may mix noisy neighbor features for target prediction. To address the above two issues, we propose a neighborhood re-ranking model with relation constraint for KGC task. We suggest that both relation constraint and structured information located in triples can boost the model performance. More importantly, we automatically generate explicit constraints as additional type feature to enrich entity representation instead of depending on human annotated labels. Meanwhile, we construct a neighborhood completion module to re-rank candidate entities for full use of the neighbor structure rather than traditional GNN updating manner. Extensive experiments on seven benchmarks demonstrate that our model achieves the competitive results in comparison to the recent advanced baselines.
Structural Identity Representation Learning of Blockchain Transaction Network for Metaverse

Both the metaverse and its underlying blockchain technology have attracted extensive attention in the past few years. It becomes a natural problem to extract, process, and analyze the tremendous data generated by the blockchain systems for various metaverse applications though it also poses diverse challenges. Amongst those challenges, this paper mainly focuses on modeling and understanding the blockchain transaction network from a structural identity perspective, which represents the entire network structure and reveals the relations among multiple entities. In this paper, we propose a novel representation learning method named Structure-to-Vector with Random Pace (SVRP) for learning both latent representation and structural identity of blockchain transaction networks. We then conduct node classification and link prediction tasks with integration with Graph Neural Networks (GNNs). Empirical results on three representative blockchain data sets, namely Non-fungible token (NFT), Ethereum (ETH), and Bitcoin (BTC), demonstrate that our proposed SVRP outperforms other existing methods in multiple tasks. In particular, our SVRP achieves the highest node classification accuracy (99.3%) while only requiring original non-attributed graphs (i.e., graphs without node features).
Precise Vertex Regression and Feature Decoupling for Oriented Object Detection

Oriented object detection is a key task in the field of remote sensing image interpretation. Although extensive efforts have been made over the past few years, accurate oriented object detection remains a big challenge due to the dense arrangement and diverse orientations of objects. In this paper, we propose an oriented object detector based on the Faster R-CNN, which mainly consists of a Precise Vertex Regression (PVR) module and a Feature Decoupling (FD) module. Specifically, the PVR module predicts the arbitrary quadrilaterals of oriented objects with the precise vertex regression manner, which discretizes the regression range of vertex into several bins and applies a classification network to predict which bin the vertex belongs to. The FD module decouples the RoI features for classification and regression tasks by lightweight affine transformation. Experimental results on DOTA and DIOR-R datasets validate the effectiveness of our proposed method. Code is available at https://github.com/ShichengMiao16/VRDet.
Novel Class Discovery Without Forgetting

Humans possess an innate ability to identify and differentiate instances that they are not familiar with, by leveraging and adapting the knowledge that they have acquired so far. Importantly, they achieve this without deteriorating the performance on their earlier learning. Inspired by this, we identify and formulate a new, pragmatic problem setting of NCDwF: Novel Class Discovery without Forgetting, which tasks a machine learning model to incrementally discover novel categories of instances from unlabeled data, while maintaining its performance on the previously seen categories. We propose 1) a method to generate pseudo-latent representations which act as a proxy for (no longer available) labeled data, thereby alleviating forgetting, 2) a mutual-information based regularizer which enhances unsupervised discovery of novel classes, and 3) a simple Known Class Identifier which aids generalized inference when the testing data contains instances form both seen and unseen categories. We introduce experimental protocols based on CIFAR-10, CIFAR-100 and ImageNet-1000 to measure the trade-off between knowledge retention and novel class discovery. Our extensive evaluations reveal that existing models catastrophically forget previously seen categories while identifying novel categories, while our method is able to effectively balance between the competing objectives. We hope our work will attract further research into this newly identified pragmatic problem setting.
CES-KD: Curriculum-based Expert Selection for Guided Knowledge Distillation

Knowledge distillation (KD) is an effective tool for compressing deep classification models for edge devices. However, the performance of KD is affected by the large capacity gap between the teacher and student networks. Recent methods have resorted to a multiple teacher assistant (TA) setting for KD, which sequentially decreases the size of the teacher model to relatively bridge the size gap between these models. This paper proposes a new technique called Curriculum Expert Selection for Knowledge Distillation (CES-KD) to efficiently enhance the learning of a compact student under the capacity gap problem. This technique is built upon the hypothesis that a student network should be guided gradually using stratified teaching curriculum as it learns easy (hard) data samples better and faster from a lower (higher) capacity teacher network. Specifically, our method is a gradual TA-based KD technique that selects a single teacher per input image based on a curriculum driven by the difficulty in classifying the image. In this work, we empirically verify our hypothesis and rigorously experiment with CIFAR-10, CIFAR-100, CINIC-10, and ImageNet datasets and show improved accuracy on VGG-like models, ResNets, and WideResNets architectures.
Solving basic and advanced human activities using LSTM

Because of the increasing use of numerous sensors, human activity recognition has grown popular in a variety of fields. It has been noticed that when the requirement for protection, defense, and human activity classification and recognition grows, more research in these areas becomes necessary. Such technologies are capable of detecting terrorists and assisting in disaster relief. The goal of this research study is to look at human activity recognition systems that use deep learning. In addition, the integration of such deep learning mechanisms has been assessed. It has been seen that in previous studies, data augmentation mechanisms have been utilized to perform human activity recognition, which has resulted in a significant increase in accuracy. This accuracy can be obtained with the support of multilayer LSTM network. An approach for assessing individual activity based on electro-magnetic radiation and deep learning algorithms has been reported in a number of papers. Time and space complexity are two major hurdles in such study. As a result, the goal of this analysis review is to research the technological and economic practicability of applying deep-learning for human action recognition systems. In this study, LSTM is applied for basic and advanced human activities identification.
Real-world Underwater Image Enhancement via Degradation-aware Dynamic Network

Underwater degradations are various and sometimes scene-dependent such as different kinds of color distortion and different extent of fuzziness, attributed to complicated underwater environments. Recently methods produce promising restoration results, however, they are time-consuming and introduce suboptimal results when facing complicated scenes. Therefore, the two main challenges are how to maintain fast and generic processing when tackling complicated scenes. To address these problems, we develop a degradation-aware dynamic network named DD-Net by introducing the Koschmieder’s light scattering model. Specifically, to handle various color distortions, we propose a scene-adaptive convolution, the basic element of which is no longer a float point number, but a function. The function outputs scene-adaptive parameters of the convolutions by taking the estimated scene factor of the underwater image as input, accordingly. In addition, to handle different extents of fuzziness degradation, we proposed a novel haze-adaptive regularization. The regularization uses medium transmission maps to guide the model to implicitly assign different attention to regions suffering from various degrees of degradation. By fully utilizing the global information (i.e., the global background light) and the local information (i.e., medium transmission map), our proposed method outperforms previous methods in terms of quantitative metrics. More importantly, our model is implemented in a lightweight manner, providing the possibility of real-time restoration.