Reviewerly/exHarmony huggingface dataset:
A multi-scale semantic attention representation for multi-label image recognition with graph networks

Multi-label image recognition is a basic and challenging task in computer vision and multimedia fields. Graph Convolutional Networks (GCNs) are often used to learn the multi-label semantic features and multi-label dependency. Although the label semantic features in GCNs can learn the global image visual representation well, they are rarely used on the local image regions. Therefore, we try to use GCNs to learn global and local features at the same time, and make a balance between them. In this paper, we give a multi-scale semantic attention model MS-SGA-GCN including three main modules (i.e., MS, SGA and GCN) for multi-label image recognition. The Multi-Scale module (MS) utilizes feature maps of different sizes to obtain global features and have strong generalization capabilities. Semantic Guide Attention module (SGA) applies the label embeddings learned by GCNs to guide the generation of the cross-modality class-specific attention maps, which can discover the locations of semantically related regions for each label. Experiments show that our model on two datasets MS-COCO and PASCAL VOC2007 separately achieves the classification accuracy by 83.4% and 94.2%, which has a competitive advantage over other mainstream models.
Situating Search

Search systems, like many other applications of machine learning, have become increasingly complex and opaque. The notions of relevance, usefulness, and trustworthiness with respect to information were already overloaded and often difficult to articulate, study, or implement. Newly surfaced proposals that aim to use large language models to generate relevant information for a user's needs pose even greater threat to transparency, provenance, and user interactions in a search system. In this perspective paper we revisit the problem of search in the larger context of information seeking and argue that removing or reducing interactions in an effort to retrieve presumably more relevant information can be detrimental to many fundamental aspects of search, including information verification, information literacy, and serendipity. In addition to providing suggestions for counteracting some of the potential problems posed by such models, we present a vision for search systems that are intelligent and effective, while also providing greater transparency and accountability.
Solar Irradiance Forecasting Using Dynamic Ensemble Selection

Solar irradiance forecasting has been an essential topic in renewable energy generation. Forecasting is an important task because it can improve the planning and operation of photovoltaic systems, resulting in economic advantages. Traditionally, single models are employed in this task. However, issues regarding the selection of an inappropriate model, misspecification, or the presence of random fluctuations in the solar irradiance series can result in this approach underperforming. This paper proposes a heterogeneous ensemble dynamic selection model, named HetDS, to forecast solar irradiance. For each unseen test pattern, HetDS chooses the most suitable forecasting model based on a pool of seven well-known literature methods: ARIMA, support vector regression (SVR), multilayer perceptron neural network (MLP), extreme learning machine (ELM), deep belief network (DBN), random forest (RF), and gradient boosting (GB). The experimental evaluation was performed with four data sets of hourly solar irradiance measurements in Brazil. The proposed model attained an overall accuracy that is superior to the single models in terms of five well-known error metrics.
A Taxonomy of Inter-Team Coordination Mechanisms in Large-Scale Agile

In large-scale agile software development, many teams work together to achieve overarching project goals. The more teams, the greater the coordination requirements. Despite the growing popularity of large-scale agile, inter-team coordination is challenging to practice and research. We conducted a case study over 1.5 years in a large-scale software development firm to better understand which inter-team coordination mechanisms are used in large-scale agile and how they support inter-team coordination. Based on a thematic analysis of 31 interviews, 113 hours of observations, and supplemental material, we identified 27 inter-team coordination mechanisms. From this, we offer the following contributions. First, we propose a taxonomy of inter-team coordination with three categories: coordination meetings, such as communities of practice, inter-team stand-ups, and retrospectives; coordination roles, such as the program architects and the platform team; and coordination tools and artefacts, such as Slack and JIRA as well as inter-team task boards, product backlogs, and roadmaps. Second, the coordination mechanisms displayed combinations of four key characteristics, technical, organizational, physical, and social (TOPS), which form the basis of the TOPS framework to capture the multifaceted characteristics of coordination mechanisms. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Technical</i> relates to the software product and/or technical tools supporting software development. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Organizational</i> pertains to the structural aspects of the organization. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Physical</i> refers to tangible or spatial characteristics. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Social</i> captures interpersonal and community-based characteristics. Finally, the taxonomy and the TOPS framework provide a knowledge base and a structured approach for researchers to study as well as for software practitioners to understand and improve inter-team coordination in large-scale agile.
Industrial internet of things and unsupervised deep learning enabled real-time occupational safety monitoring in cold storage warehouse

Occupational safety and health (OSH) has always been a big concern in the labor-intensive warehouse industry, especially under peculiar circumstances like a low temperature. Accordingly, this paper aims to propose a framework of a smart system using the Industrial Internet of Things (IIoT) and digital twin (DT) technologies to realize real-time occupational safety monitoring in the warehouse and ensure synchronized cyber-physical spaces for information traceability and visibility. The unsupervised deep neural structure of stacked auto-encoder (SAE) is designed to identify abnormal stationary from human motion status, which is perceived as a sign of potential accident. The model is developed to automatically update online by cooperating with calibration samples so as to keep in accordance with the evolution of surroundings. The Bluetooth low energy (BLE) and a log-distance path loss model are used to fulfill indoor localization in order for managers to promptly respond to an incident on site. Besides, some intelligent services are enabled to promote the efficiency of safety management. A real-life case study is carried out in an air cargo cold storage warehouse to illustrate the viability and rationality of the proposed system and methods. The elaboration of the implementation is envisioned to facilitate replication and reproduction effectively. The impact of learning features concerned with distance and vibration on the performance of anomaly detection has also been analyzed by experiments. The insights and lessons gained in this study hold the promise of providing a reference or sparking new ideas for researchers and practitioners to meet similar needs in practice.
Progressive End-to-End Object Detection in Crowded Scenes

In this paper, we propose a new query-based detection framework for crowd detection. Previous query-based detectors suffer from two drawbacks: first, multiple predictions will be inferred for a single object, typically in crowded scenes; second, the performance saturates as the depth of the decoding stage increases. Benefiting from the nature of the one-to-one label assignment rule, we propose a progressive predicting method to address the above issues. Specifically, we first select accepted queries prone to generate true positive predictions, then refine the rest noisy queries according to the previously accepted predictions. Experiments show that our method can significantly boost the performance of query-based detectors in crowded scenes. Equipped with our approach, Sparse RCNN achieves 92.0% AP, 41.4% MR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−2</sup> and 83.2% JI on the challenging CrowdHuman [35] dataset, outperforming the box-based method MIP [8] that specifies in handling crowded scenarios. Moreover, the proposed method, robust to crowdedness, can still obtain consistent improvements on moderately and slightly crowded datasets like CityPersons [47] and COCO [26]. Code will be made publicly available at https://github.com/megvii-model/Iter-E2EDET.
Rethinking Graph Convolutional Networks in Knowledge Graph Completion

Graph convolutional networks (GCNs)—which are effective in modeling graph structures—have been increasingly popular in knowledge graph completion (KGC). GCN-based KGC models first use GCNs to generate expressive entity representations and then use knowledge graph embedding (KGE) models to capture the interactions among entities and relations. However, many GCN-based KGC models fail to outperform state-of-the-art KGE models though introducing additional computational complexity. This phenomenon motivates us to explore the real effect of GCNs in KGC. Therefore, in this paper, we build upon representative GCN-based KGC models and introduce variants to find which factor of GCNs is critical in KGC. Surprisingly, we observe from experiments that the graph structure modeling in GCNs does not have a significant impact on the performance of KGC models, which is in contrast to the common belief. Instead, the transformations for entity representations are responsible for the performance improvements. Based on the observation, we propose a simple yet effective framework named LTE-KGE, which equips existing KGE models with linearly transformed entity embeddings. Experiments demonstrate that LTE-KGE models lead to similar performance improvements with GCN-based KGC methods, while being more computationally efficient. These results suggest that existing GCNs are unnecessary for KGC, and novel GCN-based KGC models should count on more ablation studies to validate their effectiveness. The code of all the experiments is available on GitHub at https://github.com/MIRALab-USTC/GCN4KGC.
Prospects of Structural Similarity Index for Medical Image Analysis

An image quality matrix provides a significant principle for objectively observing an image based on an alteration between the original and distorted images. During the past two decades, a novel universal image quality assessment has been developed with the ability of adaptation with human visual perception for measuring the difference of a degraded image from the reference image, namely a structural similarity index. Structural similarity has since been widely used in various sectors, including medical image evaluation. Although numerous studies have reported the use of structural similarity as an evaluation strategy for computer-based medical images, reviews on the prospects of using structural similarity for medical imaging applications have been rare. This paper presents previous studies implementing structural similarity in analyzing medical images from various imaging modalities. In addition, this review describes structural similarity from the perspective of a family’s historical background, as well as progress made from the original to the recent structural similarity, and its strengths and drawbacks. Additionally, potential research directions in applying such similarities related to medical image analyses are described. This review will be beneficial in guiding researchers toward the discovery of potential medical image examination methods that can be improved through structural similarity index.
Ultrafast low-pump fluence all-optical modulation based on graphene-metal hybrid metasurfaces

Graphene is an attractive material for all-optical modulation because of its ultrafast optical response and broad spectral coverage. However, all-optical graphene modulators reported so far require high pump fluence due to the ultrashort photo-carrier lifetime and limited absorption in graphene. We present modulator designs based on graphene-metal hybrid plasmonic metasurfaces with highly enhanced light-graphene interaction in the nanoscale hot spots at pump and probe (signal) wavelengths. Based on this design concept, we have demonstrated high-speed all-optical modulators at near and mid-infrared wavelengths (1.56 μm and above 6 μm) with significantly reduced pump fluence (1-2 orders of magnitude) and enhanced optical modulation. Ultrafast near-infrared pump-probe measurement results suggest that the modulators' response times are ultimately determined by graphene's ultrafast photocarrier relaxation times on the picosecond scale. The proposed designs hold the promise to address the challenges in the realization of ultrafast all-optical modulators for mid-and far-infrared wavelengths.
Detection in Adverse Weather Conditions for Autonomous Vehicles via Deep Learning

Weather detection systems (WDS) have an indispensable role in supporting the decisions of autonomous vehicles, especially in severe and adverse circumstances. With deep learning techniques, autonomous vehicles can effectively identify outdoor weather conditions and thus make appropriate decisions to easily adapt to new conditions and environments. This paper proposes a deep learning (DL)-based detection framework to categorize weather conditions for autonomous vehicles in adverse or normal situations. The proposed framework leverages the power of transfer learning techniques along with the powerful Nvidia GPU to characterize the performance of three deep convolutional neural networks (CNNs): SqueezeNet, ResNet-50, and EfficientNet. The developed models have been evaluated on two up-to-date weather imaging datasets, namely, DAWN2020 and MCWRD2018. The combined dataset has been used to provide six weather classes: cloudy, rainy, snowy, sandy, shine, and sunrise. Experimentally, all models demonstrated superior classification capacity, with the best experimental performance metrics recorded for the weather-detection-based ResNet-50 CNN model scoring 98.48%, 98.51%, and 98.41% for detection accuracy, precision, and sensitivity. In addition to this, a short detection time has been noted for the weather-detection-based ResNet-50 CNN model, involving an average of 5 (ms) for the time-per-inference step using the GPU component. Finally, comparison with other related state-of-art models showed the superiority of our model which improved the classification accuracy for the six weather conditions classifiers by a factor of 0.5–21%. Consequently, the proposed framework can be effectively implemented in real-time environments to provide decisions on demand for autonomous vehicles with quick, precise detection capacity.
BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment

This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. To over-come the challenges in BurstSR, we propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. To achieve this goal, we propose a Pyramid Flow-Guided Deformable Convolution Network (Pyramid FG-DCN) and incorporate Swin Trans-former Blocks and Groups as our main backbone. More specifically, we combine optical flows and deformable convolutions, hence our BSRT can handle misalignment and aggregate the potential texture information in multi-frames more efficiently. In addition, our Transformer-based structure can capture long-range dependency to further improve the performance. The evaluation on both synthetic and real-world tracks demonstrates that our approach achieves a new state-of-the-art in BurstSR task. Further, our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.
Fluid Micelle Network for Image Super-Resolution Reconstruction

Most existing convolutional neural-network-based super-resolution (SR) methods focus on designing effective neural blocks but rarely describe the image SR mechanism from the perspective of image evolution in the SR process. In this study, we explore a new research routine by abstracting the movement of pixels in the reconstruction process as the flow of fluid in the field of fluid dynamics (FD), where explicit motion laws of particles have been discovered. Specifically, a novel fluid micelle network is devised for image SR based on the theory of FD that follows the residual learning scheme but learns the residual structure by solving the finite difference equation in FD. The pixel motion equation in the SR process is derived from the Navier-Stokes (N-S) FD equation, establishing a guided branch that is aware of edge information. Thus, the second-order residual drives the network for feature extraction, and the guided branch corrects the direction of the pixel stream to supplement the details. Experiments on popular benchmarks and a real-world microscope chip image dataset demonstrate that the proposed method outperforms other modern methods in terms of both objective metrics and visual quality. The proposed method can also reconstruct clear geometric structures, offering the potential for real-world applications.
Zero-Shot Stance Detection via Contrastive Learning

Zero-shot stance detection (ZSSD) is challenging as it requires detecting the stance of previously unseen targets during the inference stage. Being able to detect the target-related transferable stance features from the training data is arguably an important step in ZSSD. Generally speaking, stance features can be grouped into target-invariant and target-specific categories. Target-invariant stance features carry the same stance regardless of the targets they are associated with. On the contrary, target-specific stance features only co-occur with certain targets. As such, it is important to distinguish these two types of stance features when learning stance features of unseen targets. To this end, in this paper, we revisit ZSSD from a novel perspective by developing an effective approach to distinguish the types (target-invariant/-specific) of stance features, so as to better learn transferable stance features. To be specific, inspired by self-supervised learning, we frame the stance-feature-type identification as a pretext task in ZSSD. Furthermore, we devise a novel hierarchical contrastive learning strategy to capture the correlation and difference between target-invariant and -specific features and further among different stance labels. This essentially allows the model to exploit transferable stance features more effectively for representing the stance of previously unseen targets. Extensive experiments on three benchmark datasets show that the proposed framework achieves the state-of-the-art performance in ZSSD.
Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentation

The key challenge for few-shot semantic segmentation (FSS) is how to tailor a desirable interaction among sup-port and query features and/or their prototypes, under the episodic training scenario. Most existing FSS methods im-plement such support/query interactions by solely leveraging plain operations - e.g., cosine similarity and feature concatenation - for segmenting the query objects. How-ever, these interaction approaches usually cannot well capture the intrinsic object details in the query images that are widely encountered in FSS, e.g., if the query object to be segmented has holes and slots, inaccurate segmentation al-most always happens. To this end, we propose a dynamic prototype convolution network (DPCN) to fully capture the aforementioned intrinsic details for accurate FSS. Specifi-cally, in DPCN, a dynamic convolution module (DCM) is firstly proposed to generate dynamic kernels from support foreground, then information interaction is achieved by con-volution operations over query features using these kernels. Moreover, we equip DPCN with a support activation mod-ule (SAM) and a feature filtering module (FFM) to generate pseudo mask and filter out background information for the query images, respectively. SAM and FFM together can mine enriched context information from the query features. Our DPCN is also flexible and efficient under the k-shot FSS setting. Extensive experiments on PASCAL-5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> and COCO 20 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> show that DPCN yields superior performances under both 1-shot and 5-shot settings.
A lightweight hand gesture recognition in complex backgrounds

Hand Gesture Recognition (HGR) is widely used in human–computer interaction due to its convenience. However, there are still some challenges in real-world scenarios, such as recognizing hand gestures in the complex backgrounds. To this end, the paper proposes a two-stage HGR system to solve the above issue. Specifically, the first stage performs accurate segmentation to segment the hand from the background. The segmentation network combines dilated residual network, atrous spatial pyramid pooling module and a simplified decoder. The segmentation network can effectively determine hand region even in challenging backgrounds. In the second stage, the double-channel Convolutional Neural Networks (CNNs) are presented to improve the recognition performance. The double-channel CNNs can learn features from the RGB input images and the segmented hand images separately. Experiment results show that the proposed method has an accuracy of 91.17% with the model size of 1.8 MB, both of which are better than other state-of-arts in hand gesture recognition. The method successfully constructed a lightweight model while keeping a high gesture recognition accuracy at final.
From human-centred to life-centred design: Considering environmental and ethical concerns in the design of interactive products

Over the past decades, the field of interaction design has shaped how people interact with digital technology, both through research and practice. Interaction designers adopted human-centred design to ensure that the interactive products they design meet the needs and desires of end consumers. However, there is surmounting evidence that placing the end consumer at the centre of the design process creates unintended consequences, damaging global systems that are essential to human well-being. This article reviews emerging paradigms that provide a more holistic perspective, such as value-sensitive design, more-than-human participation and life-centred design. Based on this review, the article introduces a practical framework for life-centred design consisting of principles, actionable methods and a model for responsible innovation. The article discusses how interaction designers can use the framework to balance human-centred considerations with environmental and ethical concerns when designing interactive products.