Reviewerly/exHarmony huggingface dataset:
How deep convolutional neural networks lose spatial information with training

Abstract A central question of machine learning is how deep nets manage to learn tasks in high dimensions. An appealing hypothesis is that they achieve this feat by building a representation of the data where information irrelevant to the task is lost. For image datasets, this view is supported by the observation that after (and not before) training, the neural representation becomes less and less sensitive to diffeomorphisms acting on images as the signal propagates through the network. This loss of sensitivity correlates with performance and surprisingly correlates with a gain of sensitivity to white noise acquired during training. Which are the mechanisms learned by convolutional neural networks (CNNs) responsible for the these phenomena? In particular, why is the sensitivity to noise heightened with training? Our approach consists of two steps. (1) Analyzing the layer-wise representations of trained CNNs, we disentangle the role of spatial pooling in contrast to channel pooling in decreasing their sensitivity to image diffeomorphisms while increasing their sensitivity to noise. (2) We introduce model scale-detection tasks, which qualitatively reproduce the phenomena reported in our empirical analysis. In these models we can assess quantitatively how spatial pooling affects these sensitivities. We find that the increased sensitivity to noise observed in deep ReLU networks is a mechanistic consequence of the perturbing noise piling up during spatial pooling, after being rectified by ReLU units. Using odd activation functions like tanh drastically reduces the CNNs’ sensitivity to noise.
Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Images

Computer graphics images (CGIs) are artificially generated by means of computer programs and are widely perceived under various scenarios, such as games, streaming media, etc. In practice, the quality of CGIs consistently suffers from poor rendering during production, inevitable compression artifacts during the transmission of multimedia applications, and low aesthetic quality resulting from poor composition and design. However, few works have been dedicated to dealing with the challenge of computer graphics image quality assessment (CGIQA). Most image quality assessment (IQA) metrics are developed for natural scene images (NSIs) and validated on databases consisting of NSIs with synthetic distortions, which are not suitable for in-the-wild CGIs. To bridge the gap between evaluating the quality of NSIs and CGIs, we construct a large-scale in-the-wild CGIQA database consisting of 6,000 CGIs (CGIQA-6k) and carry out the subjective experiment in a well-controlled laboratory environment to obtain the accurate perceptual ratings of the CGIs. Then, we propose an effective deep learning–based no-reference (NR) IQA model by utilizing both distortion and aesthetic quality representation. Experimental results show that the proposed method outperforms all other state-of-the-art NR IQA methods on the constructed CGIQA-6k database and other CGIQA-related databases. The database is released at https://github.com/zzc-1998/CGIQA6K .
Smart Objects and Replicas: A Survey of Tangible and Embodied Interactions in Museums and Cultural Heritage Sites

Tangible and Embodied interactions are areas of research within HCI and Interaction design. They refer to a way in which interacting with computer systems is closer to the way we interact with the real world. Instead of using devices such as a computer or a phone, we interact using seemingly non-technological objects, by moving our bodies or by using gestures. Since the early 2000s, tangible and embodied interactions have been applied and researched also in Cultural Heritage and museums, in an attempt to overcome issues induced by screen-based devices that may disengage visitors from the objects, their materiality and the physicality of the visit. This article surveys tangible and embodied interactions in museums, over a period of two decades since 2000. Over 120 projects have been researched and analysed thematically to provide a categorization based on cultural communication, interaction features and museological aspects. This categorization offers a conceptualization of tangible and embodied interactions in museums and Cultural Heritage; it suggests a terminology to describe the design characteristics of tangible and embodied interaction interventions, therefore facilitating the orientation of future research efforts in the field.
Adaptive Adversarial Contrastive Learning for Cross-Domain Recommendation

Graph-based cross-domain recommendations (CDRs) are useful for suggesting appropriate items because of their promising ability to extract features from user–item interactions and transfer knowledge across domains. Thus, the model can effectively alleviate cold start and data sparsity issues. Although the graph-based CDRs can capture valuable information, they still have some limitations. First, embeddings are highly vulnerable to noisy interactions, because the message aggregation in the graph convolutional network can further enlarge the impact. Second, because of the property of graph-structured data, the influence of high-degree nodes on representation learning is more than that of the long-tail items, and this can cause a poor recommendation performance. In this study, we devised a novel A daptive A dversarial C ontrastive L earning framework for graph-based C ross- D omain R ecommendation ( ACLCDR ). The ACLCDR introduces reinforcement learning to generate adaptive augmented samples for contrastive learning tasks. Then, we leveraged a multitask training strategy to jointly optimize the model with auxiliary tasks. Finally, we verified the effectiveness of the ACLCDR through nine real-world cross-domain tasks adopted from Amazon and Douban. We observed that ACLCDR exceeded the best state-of-the-art baseline by 25%, 42.5%, 16.3%, and 23.8% in terms of HR@ 10 and NDCG@10 for the Music &amp; Movie task from Amazon.
An optimized GRT model with blockchain digital smart contracts for power generation enterprises

The traditional power generation rights trading (GRT) market is faced with the problems of weak interconnection of electricity‑carbon market and low security. Using smart contracts in the blockchain, the idea of establishing a weakly centralized GRT structure is proposed in this paper. The carbon emission factor was introduced to improve the GRT model, and carbon emission market is used to further stimulate the emission reduction vitality of generating units. The empirical results show that compared with the benchmark model and improved model 1, the improved GRT model proposed by us has the best emission reduction effect. The contribution of this paper is to make up for the existing research that cannot fully consider the impact of carbon peak and carbon neutralization on the GRT market, as well as the information security issues brought by big data trading on the GRT platform. This paper puts forward some policy implications for the decarbonization and green development of the electricity market advocated by the Chinese government.
How Agile Practices Can Improve Student Research

Agile, Lean, and Scrum (collectively referred to as Agile) are frameworks that help teams collaborate more effectively and transparently on complex projects. Originally developed in business and information technology, these methods have since been adapted for use in a variety of disciplines. Agile is based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams (i.e. workgroups), end users, and stakeholders. By promoting open communication and allowing teams to adapt quickly, Agile can help meet timeline expectations more efficiently. This manuscript introduces the use of Agile practices for managing research projects in kinesiology disciplines involving student contributors and discusses the challenges and benefits of this approach. Using Agile frameworks in student research can enhance students’ sense of project accountability. The framework allows faculty to lead from anywhere, whether in face-to-face, hybrid, or fully online settings. Additionally, Agile improves communication between faculty and students by increasing the visibility of work, making the project more adaptable, reducing time to completion, and improving chances of project completion. The visible work and clear communication reduce assumptions between the student and mentor, while also shifting the focus from process/project-oriented (e.g. emphasis on check-list timetables) to people-oriented (e.g. emphasis on the interpersonal skill acquisition and iterative, solution-focused communication). This shift allows students to flourish in the discovery process, which also ensures that the project is completed successfully.
Automated deep learning system for power line inspection image analysis and processing: Architecture and design issues

The continuous growth in the scale of unmanned aerial vehicle (UAV) applications in transmission line inspection has resulted in a corresponding increase in the demand for UAV inspection image processing. Owing to its excellent performance in computer vision, deep learning has been applied to UAV inspection image processing tasks such as power line identification and insulator defect detection. Despite their excellent performance, electric power UAV inspection image processing models based on deep learning face several problems such as a small application scope, the need for constant retraining and optimization, and high R&D monetary and time costs due to the black-box and scene data-driven characteristics of deep learning. In this study, an automated deep learning system for electric power UAV inspection image analysis and processing is proposed as a solution to the aforementioned problems. This system design is based on the three critical design principles of generalizability, extensibility, and automation. Pre-trained models, fine-tuning (downstream task adaptation), and automated machine learning, which are closely related to these design principles, are reviewed. In addition, an automated deep learning system architecture for electric power UAV inspection image analysis and processing is presented. A prototype system was constructed and experiments were conducted on the two electric power UAV inspection image analysis and processing tasks of insulator self-detonation and bird nest recognition. The models constructed using the prototype system achieved 91.36% and 86.13% mAP for insulator self-detonation and bird nest recognition, respectively. This demonstrates that the system design concept is reasonable and the system architecture feasible
Test Case Prioritization Based on Neural Network Classification with Artifacts Traceability

Regression testing is an important factor in ensuring software system reliability once new changes are introduced, but maintaining complex testing suites in continuous integration environments is challenging. Test case prioritization techniques are a potential solution to this problem by computing a reordered testing suite that can provide better fault detection capabilities. However, current methods rely on manually providing artifact dependencies (requirements to code, code to test cases, test cases to faults) as input. The purpose of this paper is to minimize the gap between automatic dependency computation and test case prioritization by analyzing how Behavior-Driven Development (BDD) practices affect the two tasks. Thus, the first contribution of this paper is related to the design and implementation of an automatic traceability component to retrieve dependencies based on BDD artifacts (requirements, source code, test cases, and faults). The second contribution refers to the integration of the discovered traces as features in a neural network classification model for test cases for further prioritization. Various architectures were used for the neural network classification model. Two real-world BDD projects were used for the validation of the models, comparing the best-performing models with a baseline test case prioritization technique to assess their fault-detection capabilities. Our approach achieved promising fault detection rates that demonstrate the efficiency of automatic traceability and may lead to future applicability to large-scale projects.
Test Case Recommendations with Distributed Representation of Code Syntactic Features

Frequent modifications of unit test cases are inevitable due to software's continuous underlying changes in source code, design, and requirements. Since manually maintaining software test suites is tedious, timely, and costly, automating the process of generation and maintenance of test units will significantly impact the effectiveness and efficiency of software testing processes. To this end, we propose an automated approach which exploits both structural and semantic properties of source code methods and test cases to recommend the most relevant and useful unit tests to the developers. The proposed approach initially trains a neural network to transform method-level source code, as well as unit tests, into distributed representations (embedded vectors) while preserving the importance of the structure in the code. Retrieving the semantic and structural properties of a given method, the approach computes cosine similarity between the method's embedding and the previously-embedded training instances. Further, according to the similarity scores between the embedding vectors, the model identifies the closest methods of embedding and the associated unit tests as the most similar recommendations. The results on the Methods2Test dataset showed that, while there is no guarantee to have similar relevant test cases for the group of similar methods, the proposed approach extracts the most similar existing test cases for a given method in the dataset, and evaluations show that recommended test cases decrease the developers' effort to generating expected test cases.
ICO: A Platform for Optimizing Highly Configurable Systems

Dealing with large configuration spaces is a complex task for developers, especially when manually searching for the configuration that best suits both their functional and performance requirements. Indeed, a well-performing configuration may not fit developers' needs because of conflicting functional requirements, or vice-versa. In this paper, we propose ICO, a lightweight, domain-agnostic platform that supports multi-objective optimization for configurable software. The purpose of ICO is to provide the developer with the best-performing configuration by altering as little as possible the initial one, in order to remain as close as possible to the developer's functional requirements. We explain the foundations of ICO, describe its architecture, and explain how it can be used either through a command-line client or an Eclipse plugin. Finally, we assess ICO by evaluating its execution time and the time saved to users compared to a manual optimization.
MetaML: Automating Customizable Cross-Stage Design-Flow for Deep Learning Acceleration

This paper introduces a novel optimization framework for deep neural network (DNN) hardware accelerators, enabling the rapid development of customized and automated design flows.More specifically, our approach aims to automate the selection and configuration of low-level optimization techniques, encompassing DNN and FPGA low-level optimizations.We introduce novel optimization and transformation tasks for building design-flow architectures, which are highly customizable and flexible, thereby enhancing the performance and efficiency of DNN accelerators.Our results demonstrate considerable reductions of up to 92% in DSP usage and 89% in LUT usage for two networks, while maintaining accuracy and eliminating the need for human effort or domain expertise.In comparison to state-of-the-art approaches, our design achieves higher accuracy and utilizes three times fewer DSP resources, underscoring the advantages of our proposed framework.
Multi-label learning based on instance correlation and feature redundancy

Multi-label classification is a machine-learning task that simultaneously processes instances associated with multiple labels. Label-specific feature learning selects each label's most discriminative feature subset, effectively reducing the feature dimension and improving the classification performance. However, most methods only consider label correlation, ignoring the correlation between instances and feature redundancy. To solve this problem, a multi-label classification method based on instance correlation and feature redundancy is proposed. The proposed method merges instance correlation by updating the data set and removes redundant features by calculating mutual information. By jointly considering label correlation, instance correlation, and feature redundancy, our method promotes effective multi-label feature selection. The experimental results on ten data sets demonstrate the effectiveness of the proposed method.
Preference Based Multi-Objective Optimization Techniques for Coverage Path Planning of a Mobile Robot

This research aims to solve a coverage path planning (CPP) problem for an autonomous mobile robot in offline static environments. Unlike the existing approaches, multi-objective optimizations (MOO) methods provide flexibility in terms of choice for a user who may want to operate with a path that is optimized for more than one criterion for example the coverage area and energy consumption. The problem is solved by determining the collision-free path that meets the minimum energy consumption and maximum area coverage criteria. Three preference-based MOO methods are used to solve the CPP problem: the weighted sum method (WSM), the weighted metric method (WMM), and Benson's method (BM). The working environment is discretized into square cells using a grid-based method. The proposed methods are compared to the conventional constraint method (CCM) in two rectilinear simulation environments. The results show that the proposed algorithms outperformed the CCM in both working environments.
[WORKSHOP] Designerly ways of engaging with nature

In this workshop, we will bring together designers and researchers working with, for, and around nature to facilitate a transversal conversation around how to engage nature as a key part of our design processes. By deliberately adopting an open and ambiguous idea of what we mean by ‘nature’, we hope to embrace diverse kinds of more-than-human entanglements, including (but not only): farming, companion species, microbiomes, body ecologies, forests and other large-scale landscapes (e.g. oceans), or cohabitation in houses. We argue for the importance of taking such an open-ended perspective, to embrace all possible relevant vectors of nature-related design: multispecies, cohabitation, posthuman sustainability, posthuman care… The workshop is set as a as a platform for shared methodological reflection through the lenses of a more-than-human approach to posthuman research. It will primarily be in-person, given our aim of bringing researchers together and co-experiencing each others’ methods and techniques.
Strategic Analysis of Participants in BCFL-Enabled Decentralized IoT Data Sharing

Blockchain-based federated learning (BCFL) has been regarded as an emerging data sharing paradigm in edge networks of internet-of-things (IoT) because of its advantages, such as decentralization, collaborative model training, and privacy protection. However, there have been few studies focusing on strategic analysis in the BCFL system, which is important for establishing a robust and sustainable BCFL system in an untrustworthy and profit-driven environment. In this paper, we first propose a self-organizing data sharing system supported by BCFL to deeply analyze the data sharing logic. Then, a mathematical model based on evolutionary game theory is established to analyze the interaction between model owners and data providers, aiming at exploring the stability of user strategies under different considerations. According to the strategic analysis, we designed and further discussed a dynamic system control mechanism based on smart contracts to adaptively maintain a robust and sustainable BCFL system. We conducted numerical analysis and experiments to verify our work.
A Motion Deblurring Network for Enhancing UAV Image Quality in Bridge Inspection

Unmanned aerial vehicles (UAVs) have been increasingly utilized for facility safety inspections due to their superior safety, cost effectiveness, and inspection accuracy compared to traditional manpower-based methods. High-resolution images captured by UAVs directly contribute to identifying and quantifying structural defects on facility exteriors, making image quality a critical factor in achieving accurate results. However, motion blur induced by external factors such as vibration, low light conditions, and wind during UAV operation significantly degrades image quality, leading to inaccurate defect detection and quantification. To address this issue, this research proposes a deblurring network using a Generative Adversarial Network (GAN) to eliminate the motion blur effect in UAV images. The GAN-based motion deblur network represents an image inpainting method that leverages generative models to correct blurry artifacts, thereby generating clear images. Unlike previous studies, this proposed approach incorporates deblur and blur learning modules to realistically generate blur images required for training the generative models. The UAV images processed using the motion deblur network are evaluated using a quality assessment method based on local blur map and other well-known image quality assessment (IQA) metrics. Moreover, in the experiment of crack detection utilizing the object detection system, improved detection results are observed when using enhanced images. Overall, this research contributes to improving the quality and accuracy of facility safety inspections conducted with UAV-based inspections by effectively addressing the challenges associated with motion blur effects in UAV-captured images.
Leveraging response times in learning environments: opportunities and challenges

Abstract Computer-based learning environments can easily collect student response times. These can be used for multiple purposes, such as modeling student knowledge and affect, domain modeling, and cheating detection. However, to fully leverage them, it is essential to understand the properties of response times and associated caveats. In this study, we delve into the properties of response time distributions, including the influence of aberrant student behavior on response times. We then provide an overview of modeling approaches that use response times and discuss potential applications of response times for guiding the adaptive behavior of learning environments.