Reviewerly/exHarmony huggingface dataset:
Cloud-Based E-Learning: Development of Conceptual Model for Adaptive E-Learning Ecosystem Based on Cloud Computing Infrastructure

The purpose of this article is to discuss researches done on the efficacy of cloud computing in e-learning. It summarises the present state of cloud-based e-learning and the consequences for educators. Thus, the article begins with an overview of e-learning and cloud computing architectures and a discussion of their key characteristics. The article discusses the potential challenges associated with implementing e-learning systems. The benefits of cloud computing are promoted as a possible solution to these issues. Additionally, there are answers to problems that arise when e-learning makes use of cloud computing, as well as an overview of the most commonly used architectural design patterns of cloud-based e-learning. Additionally, the challenges associated with implementing cloud-based e-learning systems and potential solutions are discussed. This article further proposes a paradigmatic model for cloud-based e-learning. The model is created using Diffusion of Innovation and Fit-Viability model, along with factors influencing information culture. The principal objective of the proposed model is to discover the most important variables that influence cloud computing for the purpose of improving e-learning.
A Systematic Literature Review on Machine Learning and Deep Learning Methods for Semantic Segmentation

Machine learning and deep learning algorithms are widely used in computer science domains. These algorithms are mostly used for classification and regression problems in almost every field of life. Semantic segmentation is an instantly growing research topic in the last few decades that refers to the association of each pixel in the image to the class it belongs. This paper illustrates the systematic survey of advanced research in the field of semantic segmentation till date. This study provides the brief knowledge about the latest proposed methods in the domain of semantic segmentation. The proposed study comprehends the concepts, techniques, tool, and results of different research frameworks proposed in the context of semantic segmentation. This study discusses the latest research papers in which machine learning and deep learning techniques are exploited for semantic segmentation and published between 2016 and 2021. The systematic literature review collected from seven different article libraries including ACM digital Library, Google Scholar, IEEE Xplore, Science Direct, Google Books, Refseek and Worldwide Science. For assuring the quality of the paper those papers are selected which have several citations on standardized platforms. Most of the studies used COCO, PASCAL, Cityscapes and CamVid dataset for training and validation of the machine learning and deep learning models. The results of the selected research articles are collected in the form of accuracy, mIoU value, F1 score, precision, and recall. In this study, we also conclude that most of the semantic segmentation studies use ResNet as the backbone of the architecture and none of the researchers used ensemble learning methods for semantic segmentation that is the loophole of the selected studies.
Fine-tuning Image Transformers using Learnable Memory

In this paper we propose augmenting Vision Transformer models with learnable memory tokens. Our approach allows the model to adapt to new tasks, using few parameters, while optionally preserving its capabilities on previously learned tasks. At each layer we introduce a set of learnable embedding vectors that provide contextual information useful for specific datasets. We call these “memory tokens”. We show that augmenting a model with just a handful of such tokens per layer significantly improves accuracy when compared to conventional head-only fine-tuning, and performs only slightly below the significantly more expensive full fine-tuning. We then propose an attention-masking approach that enables extension to new downstream tasks, with a computation reuse. In this setup in addition to being parameters efficient, models can execute both old and new tasks as a part of single inference at a small incremental cost.
Modeling Image Composition for Complex Scene Generation

We present a method that achieves state-of-the-art results on challenging (few-shot) layout-to-image generation tasks by accurately modeling textures, structures and relationships contained in a complex scene. After compressing RGB images into patch tokens, we propose the Transformer with Focal Attention (TwFA) for exploring dependencies of object-to-object, object-to-patch and patch-to-patch. Compared to existing CNN-based and Transformer-based generation models that entangled modeling on pixel-level&patch-level and object-level&patch-level respectively, the proposed focal attention predicts the current patch token by only focusing on its highly-related tokens that specified by the spatial layout, thereby achieving disambiguation during training. Furthermore, the proposed TwFA largely increases the data efficiency during training, therefore we propose the first few-shot complex scene generation strategy based on the well-trained TwFA. Comprehensive experiments show the superiority of our method, which significantly increases both quantitative metrics and qualitative visual realism with respect to state-of-the-art CNN-based and transformer-based methods. Code is available at https://github.com/JohnDreamer/TwFA.
Future Transformer for Long-term Action Anticipation

The task of predicting future actions from a video is crucial for a real-world agent interacting with others. When anticipating actions in the distant future, we humans typically consider long-term relations over the whole sequence of actions, i.e., not only observed actions in the past but also potential actions in the future. In a similar spirit, we propose an end-to-end attention model for action anticipation, dubbed Future Transformer (FUTR), that leverages global attention over all input frames and output tokens to predict a minutes-long sequence of future actions. Unlike the previous autoregressive models, the proposed method learns to predict the whole sequence of future actions in parallel decoding, enabling more accurate and fast inference for long-term anticipation. We evaluate our method on two standard benchmarks for long-term action anticipation, Breakfast and 50 Salads, achieving state-of-the-art results.
Sar Ship Detection Based on Swin Transformer and Feature Enhancement Feature Pyramid Network

With the booming of Convolutional Neural Networks (CNNs), CNNs such as VGG-16 and ResNet-50 widely serve as backbone in SAR ship detection. However, CNN based backbone is hard to model long-range dependencies, and causes the lack of enough high-quality semantic infor-mation in feature maps of shallow layers, which leads to poor detection performance in complicated background and small-sized ships cases. To address these problems, we pro-pose a SAR ship detection method based on Swin Trans-former and Feature Enhancement Feature Pyramid Network (FEFPN). Swin Transformer serves as backbone to model long-range dependencies and generates hierarchical features maps. FEFPN is proposed to further improve the quality of feature maps by gradually enhencing the semantic infor-mation of feature maps at all levels, especially feature maps in shallow layers. Experiments conducted on SAR ship de-tection dataset (SSDD) reveal the advantage of our pro-posed methods.
Task Decoupled Framework for Reference-based Super-Resolution

Reference-based super-resolution(RefSR) has achieved impressive progress on the recovery of high-frequency details thanks to an additional reference high-resolution(HR) image input. Although the superiority compared with Single-Image Super-Resolution(SISR), existing RefSR methods easily result in the reference-underuse issue and the reference-misuse as shown in Fig. I. In this work, we deeply investigate the cause of the two issues and further propose a novel framework to mitigate them. Our studies find that the issues are mostly due to the improper coupled framework design of current methods. Those methods conduct the super-resolution task of the input low-resolution(LR) image and the texture transfer task from the reference image together in one module, easily introducing the interference between LR and reference features. Inspired by this finding, we propose a novel framework, which decouples the two tasks of RefSR, eliminating the interference between the LR image and the reference image. The super-resolution task upsamples the LR image leveraging only the LR image itself. The texture transfer task extracts and transfers abundant textures from the reference image to the coarsely upsampled result of the super-resolution task. Extensive experiments demonstrate clear improvements in both quantitative and qualitative evaluations over state-of-the-art methods.
Distributing Collaborative Multi-Robot Planning With Gaussian Belief Propagation

Precise coordinated planning over a forward time window enables safe and highly efficient motion when many robots must work together in tight spaces, but this would normally require centralised control of all devices which is difficult to scale. We demonstrate GBP Planning, a new purely distributed technique based on Gaussian Belief Propagation for multi-robot planning problems, formulated by a generic factor graph defining dynamics and collision constraints over a forward time window. In simulations, we show that our method allows high performance collaborative planning where robots are able to cross each other in busy, intricate scenarios. They maintain shorter, quicker and smoother trajectories than alternative distributed planning techniques even in cases of communication failure. We encourage the reader to view the accompanying video demonstration at https://youtu.be/8VSrEUjH610.
Code review practices for refactoring changes

Modern code review is a widely used technique employed in both industrial and open-source projects to improve software quality, share knowledge, and ensure adherence to coding standards and guidelines. During code review, developers may discuss refactoring activities before merging code changes in the code base. To date, code review has been extensively studied to explore its general challenges, best practices and outcomes, and socio-technical aspects. However, little is known about how refactoring is being reviewed and what developers care about when they review refactored code. Hence, in this work, we present a quantitative and qualitative study to understand what are the main criteria developers rely on to develop a decision about accepting or rejecting a submitted refactored code, and what makes this process challenging. Through a case study of 11,010 refactoring and non-refactoring reviews spread across OpenStack open-source projects, we find that refactoring-related code reviews take significantly longer to be resolved in terms of code review efforts. Moreover, upon performing a thematic analysis on a significant sample of the refactoring code review discussions, we built a comprehensive taxonomy consisting of 28 refactoring review criteria. We envision our findings reaffirming the necessity of developing accurate and efficient tools and techniques that can assist developers in the review process in the presence of refactorings.
A Tertiary Review on Blockchain and Sustainability With Focus on Sustainable Development Goals

Sustainable development is crucial to securing the future of humanity. Blockchain as a disruptive technology and a driver for social change has exhibited great potential to promote sustainable practices and help organizations and governments achieve the United Nations’ Sustainable Development Goals (SDGs). Existing literature reviews on blockchain and sustainability often focus only on topics related to a few SDGs. There is a need to consolidate existing results in terms of SDGs and provide a comprehensive overview of the impacts that blockchain technology may have on each SDG. This paper intends to bridge this gap, presenting a tertiary review based on 42 literature reviews, to investigate the relationship between blockchain and sustainability in light of SDGs. The method used is a consensus-based expert elicitation with thematic analysis. The findings include a novel and comprehensive mapping of impact-based interlinkage of blockchain and SDGs and a systematic overview of drivers and barriers to adopting blockchain for sustainability. The findings reveal that blockchain can have a positive impact on all 17 SDGs though some negative effects can occur and impede the achievement of certain objectives. 76 positive and 10 negative linkages between blockchain adoption and the 17 SDGs as well as 45 factors that drive or hinder blockchain adoption for the achievement of SDGs have been identified. Research gaps to overcome the barriers and enhance blockchain’s positive impacts have also been identified. The findings may help managers in evaluating the applicability and tradeoffs, and policymakers in making supportive measures to facilitate sustainability using blockchain.
Self-supervised Sparse Representation for Video Anomaly Detection

Video anomaly detection (VAD) aims at localizing unexpected actions or activities in a video sequence. Existing mainstream VAD techniques are based on either the one-class formulation, which assumes all training data are normal, or weakly-supervised, which requires only video-level normal/anomaly labels. To establish a unified approach to solving the two VAD settings, we introduce a self-supervised sparse representation (S3R) framework that models the concept of anomaly at feature level by exploring the synergy between dictionary-based representation and self-supervised learning. With the learned dictionary, S3R facilitates two coupled modules, en-Normal and de-Normal, to reconstruct snippet-level features and filter out normal-event features. The self-supervised techniques also enable generating samples of pseudo normal/anomaly to train the anomaly detector. We demonstrate with extensive experiments that S3R achieves new state-of-the-art performances on popular benchmark datasets for both one-class and weakly-supervised VAD tasks. Our code is publicly available at https://github.com/louisYen/S3R .
Sequential Manipulation Planning on Scene Graph

We devise a 3D scene graph representation, contact graph <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> (cg <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> ), for efficient sequential manipulation planning. Augmented with predicate-like attributes, this contact graph-based representation abstracts scene layouts with succinct geometric information and valid robot-scene interactions. Goal configurations, naturally specified on contact graphs, can be produced by a genetic algorithm with a stochastic optimization method. A task plan is then initialized by computing the Graph Editing Distance (GED) between the initial contact graph and the goal configuration, which generates graph edit operations corresponding to possible robot actions. We finalize the task plan by imposing constraints to regulate the temporal feasibility of graph edit operations, ensuring valid task and motion correspondences. In a series of simulated and real experiments, robots successfully complete complex sequential object rearrangement tasks that are difficult to specify using conventional planning language like Planning Domain Definition Language (PDDL), demonstrating high potential of planning sequential manipulation tasks on cg <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> .
Deep Fourier-Based Exposure Correction Network with Spatial-Frequency Interaction

Images captured under incorrect exposures unavoidably suffer from mixed degradations of lightness and structures. Most existing deep learning-based exposure correction methods separately restore such degradations in the spatial domain. In this paper, we present a new perspective for exposure correction with spatial-frequency interaction. Specifically, we first revisit the frequency properties of different exposure images via Fourier transform where the amplitude component contains most lightness information and the phase component is relevant to structure information. To this end, we propose a deep Fourier-based Exposure Correction Network (FECNet) consisting of an amplitude sub-network and a phase sub-network to progressively reconstruct the representation of lightness and structure components. To facilitate learning these two representations, we introduce a Spatial-Frequency Interaction (SFI) block in two formats tailored to these two sub-networks, which interactively process the local spatial features and the global frequency information to encourage the complementary learning. Extensive experiments demonstrate that our method achieves superior results than other approaches with fewer parameters and can be extended to other image enhancement tasks, validating its potential in wide-range applications. Code will be available at https://github.com/KevinJ-Huang/FECNet.
Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation

Automatic generation of ophthalmic reports using datadriven neural networks has great potential in clinical practice.When writing a report, ophthalmologists make inferences with prior clinical knowledge.This knowledge has been neglected in prior medical report generation methods.To endow models with the capability of incorporating expert knowledge, we propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG), in which clinical relation triples are injected into the visual features as prior knowledge to drive the decoding procedure.However, two major common Knowledge Noise (KN) issues may affect models' effectiveness. 1) Existing general biomedical knowledge bases such as the UMLS may not align meaningfully to the specific context and language of the report, limiting their utility for knowledge injection.2) Incorporating too much knowledge may divert the visual features from their correct meaning.To overcome these limitations, we design an automatic information extraction scheme based on natural language processing to obtain clinical entities and relations directly from in-domain training reports.Given a set of ophthalmic images, our CGT first restores a sub-graph from the clinical graph and injects the restored triples into visual features.Then visible matrix is employed during the encoding procedure to limit the impact of knowledge.Finally, reports are predicted by the encoded cross-modal features via a Transformer decoder.Extensive experiments on the large-scale FFA-IR benchmark demonstrate that the proposed CGT is able to outperform previous benchmark methods and achieve state-of-the-art performances.
Lightweight Attentional Feature Fusion: A New Baseline for Text-to-Video Retrieval

In this paper we revisit feature fusion, an old-fashioned topic, in the new context of text-to-video retrieval. Different from previous research that considers feature fusion only at one end, let it be video or text, we aim for feature fusion for both ends within a unified framework. We hypothesize that optimizing the convex combination of the features is preferred to modeling their correlations by computationally heavy multi-head self attention. We propose Lightweight Attentional Feature Fusion (LAFF). LAFF performs feature fusion at both early and late stages and at both video and text ends, making it a powerful method for exploiting diverse (off-the-shelf) features. The interpretability of LAFF can be used for feature selection. Extensive experiments on five public benchmark sets (MSR-VTT, MSVD, TGIF, VATEX and TRECVID AVS 2016–2020) justify LAFF as a new baseline for text-to-video retrieval.
LSVC: A Learning-based Stereo Video Compression Framework

In this work, we propose the first end-to-end optimized framework for compressing automotive stereo videos (i.e., stereo videos from autonomous driving applications) from both left and right views. Specifically, when compressing the current frame from each view, our framework reduces temporal redundancy by performing motion compensation using the reconstructed intra-view adjacent frame and at the same time exploits binocular redundancy by conducting disparity compensation using the latest reconstructed cross-view frame. Moreover, to effectively compress the introduced motion and disparity offsets for better compensation, we further propose two novel schemes called motion residual compression and disparity residual compression to respectively generate the predicted motion offset and disparity offset from the previously compressed motion offset and disparity offset, such that we can more effectively compress residual offset information for better bit-rate saving. Overall, the entire framework is implemented by the fully-differentiable modules and can be optimized in an end-to-end manner. Our comprehensive experiments on three automotive stereo video benchmarks Cityscapes, KITTI 2012 and KITTI 2015 demonstrate that our proposed framework outperforms the learning-based single-view video codec and the traditional hand-crafted multi-view video codec.
Levenshtein OCR

A novel scene text recognizer based on Vision-Language Transformer (VLT) is presented. Inspired by Levenshtein Transformer in the area of NLP, the proposed method (named Levenshtein OCR, and LevOCR for short) explores an alternative way for automatically transcribing textual content from cropped natural images. Specifically, we cast the problem of scene text recognition as an iterative sequence refinement process. The initial prediction sequence produced by a pure vision model is encoded and fed into a cross-modal transformer to interact and fuse with the visual features, to progressively approximate the ground truth. The refinement process is accomplished via two basic character-level operations: deletion and insertion, which are learned with imitation learning and allow for parallel decoding, dynamic length change and good interpretability. The quantitative experiments clearly demonstrate that LevOCR achieves state-of-the-art performances on standard benchmarks and the qualitative analyses verify the effectiveness and advantage of the proposed LevOCR algorithm.
3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image

Visual content creation has spurred a soaring interest given its applications in mobile photography and AR / VR. Style transfer and single-image 3D photography as two representative tasks have so far evolved independently. In this paper, we make a connection between the two, and address the challenging task of 3D photo stylization - generating stylized novel views from a single image given an arbitrary style. Our key intuition is that style transfer and view synthesis have to be jointly modeled. To this end, we propose a deep model that learns geometry-aware content features for stylization from a point cloud representation of the scene, resulting in high-quality stylized images that are consistent across views. Further, we introduce a novel training protocol to enable the learning using only 2D images. We demonstrate the superiority of our method via extensive qualitative and quantitative studies, and showcase key applications of our method in light of the growing demand for 3D content creation from 2D image assets. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Project page: http://pages.es.wise.edu/-fmu/style3d