Reviewerly/exHarmony huggingface dataset:
PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks

The increased importance of mobile photography created a need for fast and performant RAW image processing pipelines capable of producing good visual results in spite of the mobile camera sensor limitations. While deep learning-based approaches can efficiently solve this problem, their computational requirements usually remain too large for high-resolution on-device image processing. To address this limitation, we propose a novel PyNET-V2 Mobile CNN architecture designed specifically for edge devices, being able to process RAW 12MP photos directly on mobile phones under 1.5 second and producing high perceptual photo quality. To train and to evaluate the performance of the proposed solution, we use the real-world Fujifilm UltraISP dataset consisting on thousands of RAW-RGB image pairs captured with a professional medium-format 102MP Fujifilm camera and a popular Sony mobile camera sensor. The results demonstrate that the PyNET-V2 Mobile model can substantially surpass the quality of tradition ISP pipelines, while outperforming the previously introduced neural network-based solutions designed for fast image processing. Furthermore, we show that the proposed architecture is also compatible with the latest mobile AI accelerators such as NPUs or APUs that can be used to further reduce the latency of the model to as little as 0.5 second. The dataset, code and pre-trained models used in this paper are available on the project website: https://github.com/gmalivenko/PyNET-v2
R2YOLOX: A Lightweight Refined Anchor-Free Rotated Detector for Object Detection in Aerial Images

Existing anchor-based rotated object detection methods have achieved some amazing results, but these methods require some manual preset anchors, which not only introduce additional hyperparameters but also introduce extra computational burdens. Due to the above drawbacks, anchor-free methods have been rapidly developed in recent years. However, the existing high-performance anchor-free rotated object detection methods are relatively complex and the inference speed is also slow. And Yolo series models not only maintain high-efficiency inference but also keep competitive performance detection performance in the general object detection tasks. Hence, we propose an anchor-free rotated detector based on the YOLOX method for object detection in aerial images. Our methods consist of two improvements: a Refined Rotated Module (RRM) and a new assigner method which is called the Gaussian distribution Sampling Optimal Transport Assignment method (GSOTA). The RRM can align features and get more useful priors for final detector heads. The GSOTA uses Gaussian Distribution to model the oriented bounding box (OBB) firstly, and a Gaussian Center Sampling method (GCS) with maximum classification center mean (MCCM) is proposed to simplify the label Assignment Optimal Transport problem, finally using an improved dynamic top-k strategy to get an approximate solution. Extensive experiments demonstrate that our models can achieve competitive performance in several challenging aerial object detection datasets while keeping the best efficiency. Our R2YOLOX-X model achieves 79.33%, 97.4%, 97.7%, and 92.5% mAP on the DOTA, HRSC2016, UCAS-AOD, and FGSD2021, respectively, while R2YOLOX-S can reach the fastest 58.2 FPS when inferencing on aerial datasets and R2YOLOX-L gets the best speed-accuracy trade-off.
BNUDC: A Two-Branched Deep Neural Network for Restoring Images from Under-Display Cameras

The images captured by under-display cameras (UDCs) are degraded by the screen in front of them. We model this degradation in terms of a) diffraction by the pixel grid, which attenuates high-spatial-frequency components of the image; and b) diffuse intensity and color changes caused by the multiple thin-film layers in an OLED, which modulate the low-spatial-frequency components of the image. We introduce a deep neural network with two branches to reverse each type of degradation, which is more effective than performing both restorations in a single forward network. We also propose an affine transform connection to replace the skip connection used in most existing DNNs for restoring UDC images. Confining the solution space to the linear transform domain reduces the blurring caused by convolution; and any gross color shift in the training images is eliminated by inverse color filtering. Trained on three datasets of UDC images, our network outperformed existing methods in terms of measures of distortion and of perceived image quality.
TRUCON: Blockchain-Based Trusted Data Sharing With Congestion Control in Internet of Vehicles

The Internet of vehicles (IoV) has a substantial impact on traffic efficiency improvement and accidents avoidance. Due to restricted resources, vehicles must share observed data with RSUs and other vehicles to execute some time-tolerant computing tasks. However, data provided by vehicles cannot always be trusted due to the presence of attackers. Fake messages could have catastrophic ramifications, such as vehicle collisions. Furthermore, extensive data sharing might cause channel congestion, resulting in the loss of vital messages during delivery. To overcome the aforementioned issues, we propose TRUCON, a blockchain-based trusted data sharing mechanism with congestion control in IoV. Firstly, we propose a Kademlia algorithm-based traffic data forwarding method to control channel congestion state. By adjusting the bucket size and distance threshold, source vehicles can limit the number of reference vehicles forwarded. Secondly, we present a cuckoo filter-based traffic data deduplication and discrimination approach. To avoid repetitive sharing, vehicles and RSUs can check their local filters to verify if the current data report has been shared. Based on the foregoing, we propose a blockchain-based trust management mechanism with congestion control. RSUs serve as full nodes while vehicles are light nodes in the blockchain. Finally, we develop a trust management prototype system with congestion control that incorporates both on-chain and off-chain parts. It signifies that our scheme is both feasible and effective.
Multilevel Spatial–Temporal Excited Graph Network for Skeleton-Based Action Recognition

The ability to capture joint connections in complicated motion is essential for skeleton-based action recognition. However, earlier approaches may not be able to fully explore this connection in either the spatial or temporal dimension due to fixed or single-level topological structures and insufficient temporal modeling. In this paper, we propose a novel multilevel spatial-temporal excited graph network (ML-STGNet) to address the above problems. In the spatial configuration, we decouple the learning of the human skeleton into general and individual graphs by designing a multilevel graph convolution (ML-GCN) network and a spatial data-driven excitation (SDE) module, respectively. ML-GCN leverages joint-level, part-level, and body-level graphs to comprehensively model the hierarchical relations of a human body. Based on this, SDE is further introduced to handle the diverse joint relations of different samples in a data-dependent way. This decoupling approach not only increases the flexibility of the model for graph construction but also enables the generality to adapt to various data samples. In the temporal configuration, we apply the concept of temporal difference to the human skeleton and design an efficient temporal motion excitation (TME) module to highlight the motion-sensitive features. Furthermore, a simplified multiscale temporal convolution (MS-TCN) network is introduced to enrich the expression ability of temporal features. Extensive experiments on the four popular datasets NTU-RGB+D, NTU-RGB+D 120, Kinetics Skeleton 400, and Toyota Smarthome demonstrate that ML-STGNet gains considerable improvements over the existing state of the art.
Cryptocurrencies and Decentralized Finance (DeFi)

The paper provides an overview of cryptocurrencies and decentralized finance (DeFi). The discussion lays out potential benefits and challenges of the new system and presents a comparison to the traditional system of financial intermediation. Our analysis highlights that while the DeFi architecture might have the potential to reduce transaction costs, similar to the traditional financial system, there are several layers where rents can accumulate due to endogenous constraints to competition. We show that the permissionless and pseudonymous design of DeFi generates challenges for enforcing tax compliance and anti–money laundering laws and preventing financial malfeasance. We highlight ways to regulate the DeFi system which would preserve a majority of benefits of the underlying blockchain architecture but support accountability and regulatory compliance.
Sentiment Analysis Using Pre-Trained Language Model With No Fine-Tuning and Less Resource

Sentiment analysis has become popular when Natural Language Processing algorithms were proven to be able to process complex sentences with good accuracy. Recently, pre-trained language models such as BERT and mBERT, have been shown to be effective for improving language tasks. Most of the work in implementing the models focuses on fine-tuning BERT to achieve desirable results. However, this approach is resource-intensive and requires a long training time, up to a few hours on a GPU, depending on the dataset. Hence, this paper proposes a less complex system with less training time using the BERT model without the fine-tuning process and adopting a feature reduction algorithm to reduce sentence embeddings. The experimental results show that with 50% fewer sentence embeddings, the proposed system improves the accuracy by 1-2% with 71% less training time and 89% less memory usage. The proposed approach has also been proven to work for multilingual tasks by using a single mBERT model.
An Environmentally-sustainable Dimensioning Workbench towards Dynamic Resource Allocation in Cloud-computing Environments

With the exponential growth in data generated every year, Big Data has become one of the core research subjects in the overall computing domain. But when considering big data scenarios in a cloud centric environment, the need for a resource management mechanism is of vital importance. Under those circumstances, intelligent allocation of resources can have a direct and noticeable impact on application performance. The aim of this paper is to present a solution on dynamic resource allocation for efficient cloud scalability. This is made possible by using machine learning algorithms as well as user feedback, in order to generate an adequate resource forecasting model. The efficiency of the tool is evaluated by repeatedly executing extensive analysis of various datasets provided by the end users, exploiting the cloud computing paradigm for their analytic purposes. The given solution is able to learn and enhance its knowledge graph considering user feedback, as well as previously executed processes in our cloud environment. To this extent, the forecasting model will attempt to estimate optimal resource allocation for each user scenario.
Towards Forecasting the Onset of Cybersickness by Fusing Physiological, Head-tracking and Eye-tracking with Multimodal Deep Fusion Network

A plethora of studies has been conducted to detect and reduce cybersickness in real-time. However, prior attempts to detect and minimize cybersickness after its onset may be ineffective as the onset tends to persist beyond its first occurrence. By forecasting the onset of cybersickness, it may be possible to mitigate the severity of cybersickness through earlier interventions. This research proposed a multimodal deep fusion approach to forecast cybersickness from the user’s physiological, head-tracking, and eye-tracking data. We proposed several hybrid multimodal deep fusion neural networks with Long short-term memory (LSTMs), Neural basis expansion analysis for interpretable time series forecasting(NBEATs) and Deep Temporal Convolutional Networks(DeepTCN) neural models to forecast cybersickness 30-60s in advance to its onset. To validate our proposed approach, we recruited 30 participants who were immersed in five virtual reality simulations. We collected eye-tracking, head-tracking, heart rate, and galvanic skin response data and used the fast-motion scale as ground truth. Our results suggest that the DeepTCN model with our proposed multimodal fusion network can forecast cybersickness onset 60 seconds in advance with a root-mean-square error of 0.49 (on a scale from 0-10). Furthermore, our results demonstrated that fusing eye tracking, heart rate, and galvanic skin response data outperformed other data fusion approaches. This research clarifies how early cybersickness can be forecast, paving the way for future research on early cybersickness mitigation approaches.
Class-Incremental Semantic Segmentation of Aerial Images via Pixel-Level Feature Generation and Task-Wise Distillation

Deep neural networks achieve significant progress in semantic segmentation but still suffer from the catastrophic forgetting problem, i.e., networks will forget old classes as they learn new ones. In this article, we propose an effective class-incremental segmentation framework without storing old data. Specifically, to alleviate the issue of catastrophic forgetting, we present two important modules, i.e., the Pixel-level Feature Generation (PFG) module, and the Task-wise Knowledge distillation (TKD) module. The PFG module is designed to constantly generate any number of features of the old classes to keep the old memory. The PFG module is the first attempt to use the generative method in class-incremental segmentation of aerial images, and it abandons the previous image generation approach but to generate pixel-level features, which is more suitable for the segmentation task. Meanwhile, the proposed TKD module is specially designed for class incremental tasks, and it only compares classes in the same learning step (task), thus avoiding the squeezing of new classes to old classes when the output is normalized (softmax), making distillation more effective. Sufficient experiments show that our method is remarkably effective and achieves more than 4.5% gains compared with state-of-the-art methods, and more than 13% compared with baselines, on all learning conditions. The ablation studies show that the PFG module and the TKD module are both indispensables. Besides, the proposed framework can be well combined with any existing class incremental learning method to achieve better performance.
Towards Efficient and Scale-Robust Ultra-High-Definition Image Demoiréing

With the rapid development of mobile devices, modern widely-used mobile phones typically allow users to capture 4K resolution (i.e., ultra-high-definition) images. However, for image demoiréing, a challenging task in low-level vision, existing works are generally carried out on low-resolution or synthetic images. Hence, the effectiveness of these methods on 4K resolution images is still unknown. In this paper, we explore moiré pattern removal for ultra-high-definition images. To this end, we propose the first ultra-high-definition demoiréing dataset (UHDM), which contains 5,000 real-world 4K resolution image pairs, and conduct a benchmark study on current state-of-the-art methods. Further, we present an efficient baseline model ESDNet for tackling 4K moiré images, wherein we build a semantic-aligned scale-aware module to address the scale variation of moiré patterns. Extensive experiments manifest the effectiveness of our approach, which outperforms state-of-the-art methods by a large margin while being much more lightweight. Code and dataset are available at https://xinyu-andy.github.io/uhdm-page .
Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classification

Hierarchical multi-granularity classification (HMC) assigns hierarchical multi-granularity labels to each object and focuses on encoding the label hierarchy, e.g., [“Albatross”, “Laysan Albatross”] from coarse-to-fine levels. However, the definition of what is fine-grained is subjective, and the image quality may affect the identification. Thus, samples could be observed at any level of the hierarchy, e.g., [“Albatross”] or [“Albatross”, “Laysan Albatross”], and examples discerned at coarse categories are often neglected in the conventional setting of HMC. In this paper, we study the HMC problem in which objects are labeled at any level of the hierarchy. The essential designs of the proposed method are derived from two motivations: (1) learning with objects labeled at various levels should transfer hierarchical knowledge between levels; (2) lower-level classes should inherit attributes related to upper-level superclasses. The proposed combinatorial loss maximizes the marginal probability of the observed ground truth label by aggregating information from related labels defined in the tree hierarchy. If the observed label is at the leaf level, the combinatorial loss further imposes the multi-class cross-entropy loss to increase the weight of fine-grained classification loss. Considering the hierarchical feature interaction, we propose a hierarchical residual network (HRN), in which granularity-specific features from parent levels acting as residual connections are added to features of children levels. Experiments on three commonly used datasets demonstrate the effectiveness of our approach compared to the state-of-the-art HMC approaches. The code will be available at https://github.com/MonsterZhZh/HRN.
Algorithmic failure as a humanities methodology: Machine learning's mispredictions identify rich cases for qualitative analysis

This commentary tests a methodology proposed by Munk et al. (2022) for using failed predictions in machine learning as a method to identify ambiguous and rich cases for qualitative analysis. Using a dataset describing actions performed by fictional characters interacting with machine vision technologies in 500 artworks, movies, novels and videogames, I trained a simple machine learning algorithm (using the kNN algorithm in R) to predict whether or not an action was active or passive using only information about the fictional characters. Predictable actions were generally unemotional and unambiguous activities where machine vision technologies were treated as simple tools. Unpredictable actions, that is, actions that the algorithm could not correctly predict, were more ambivalent and emotionally loaded, with more complex power relationships between characters and technologies. The results thus support Munk et al.'s theory that failed predictions can be productively used to identify rich cases for qualitative analysis. This test goes beyond simply replicating Munk et al.'s results by demonstrating that the method can be applied to a broader humanities domain, and that it does not require complex neural networks but can also work with a simpler machine learning algorithm. Further research is needed to develop an understanding of what kinds of data the method is useful for and which kinds of machine learning are most generative. To support this, the R code required to produce the results is included so the test can be replicated. The code can also be reused or adapted to test the method on other datasets.
Engineering education and the mission of public disciplines in the educational process

The article examines the impact of progress in scientific, technical, social development on the transformation processes associated with the value orientations of specialists in engineering, which are changing and the main features of engineering education. Today transformational technology raises a number of important problems of world outlook and methodological orientation, and engineering education can address these problems by forming specialists of critical thinking. Consideration of the influence of the role of general humanitarian disciplines in engineering education can be used as a basis for improving the methodology and methodology of technical education. The study of general humanitarian disciplines contributes to the formation of skills and abilities to navigate in advanced conceptual methods in the study of the problems of modern society, the formation of an integral, highly cultured progressive personality, capable of subsequently using knowledge, skills and personal properties in a professional environment.
CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution

Despite breakthrough advances in image super-resolution (SR) with convolutional neural networks (CNNs), SR has yet to enjoy ubiquitous applications due to the high computational complexity of SR networks. Quantization is one of the promising approaches to solve this problem. However, existing methods fail to quantize SR models with a bit-width lower than 8 bits, suffering from severe accuracy loss due to fixed bit-width quantization applied everywhere. In this work, to achieve high average bit-reduction with less accuracy loss, we propose a novel Content-Aware Dynamic Quantization (CADyQ) method for SR networks that allocates optimal bits to local regions and layers adaptively based on the local contents of an input image. To this end, a trainable bit selector module is introduced to determine the proper bit-width and quantization level for each layer and a given local image patch. This module is governed by the quantization sensitivity that is estimated by using both the average magnitude of image gradient of the patch and the standard deviation of the input feature of the layer. The proposed quantization pipeline has been tested on various SR networks and evaluated on several standard benchmarks extensively. Significant reduction in computational complexity and the elevated restoration accuracy clearly demonstrate the effectiveness of the proposed CADyQ framework for SR. Codes are available at https://github.com/Cheeun/CADyQ.
Digital Transformation in Higher Education: A Framework for Maturity Assessment

Literature in digital transformation maturity is scarce. Digital transformation in higher education, especially after COVID-19 is seen as inevitable. This research explores digital transformation maturity and challenges within Higher Education. The significance of this study stems from the role digital transformation plays in today's knowledge economy. This study proposes a new framework based on Deloitte's 2019 digital transformation assessment framework with Petkovic 2014 mega and major higher education process mapping. The study triangulates the findings of multiple research instruments, including survey, interviews, case study, and direct observation. The research findings show a significant variance between the respondents' perception of digital transformations maturity levels, and the core requirements of digital transformation maturity. The findings also show the lack of holistic vision, digital transformation competency, and data structure and processing as the leading challenges of digital transformation.