Reviewerly/exHarmony huggingface dataset:
SSTP: Social and Spatial-Temporal Aware Next Point-of-Interest Recommendation

Abstract The expansion of available information in location-based social networks (LBSNs) has led to information overload, making it urgent to discover users’ next point-of-interest (POI). Some existing works only consider certain modal information in LBSNs and do not transform them into high-dimensional structures, which hinders the alleviation of the data sparsity problem. Moreover, many approaches rely solely on social relationships, making it difficult to recommend POIs to new users without association information. To tackle these challenges, we propose a social- and spatial–temporal-aware next point-of-Interest (SSTP) recommendation model. SSTP uses two feature encoders based on self-attention mechanism and gate recurrent unit to model users’ check-in enhancement sequence hierarchically. We also design a random neighborhood sampling approach to mine user social relationships, thus alleviating the user cold start problem. Finally, we propose a geographical-aware graph attention network to learn the sensitivity of users to distance. Extensive experiments on two real-world datasets show that SSTP outperforms state-of-the-art models, improving Hit@k by 2.26–6.55 $$\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>%</mml:mo> </mml:math> and MAP@k by 3.49–6.55 $$\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>%</mml:mo> </mml:math> . Moreover, SSTP has better performance on sparse data, with an average improvement of 6.09 $$\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>%</mml:mo> </mml:math> on the Hit@k. The code can be downloaded at https://github.com/Rih0/sstp .
The Technology of Aerospace Monitoring of Changes in the Ecological Condition of the Environment Using the Statistical Criteria Method with Virtual Standards, on the Example of Mining and Industrial Territories of Ukraine

The article describes the technology of determining changes in geosystems of the environment based on the satellite images using the method of statistical criteria with virtual standards on the example of the study of mining areas. The modification and adaptation of the statistical criterion was carried out, virtual benchmarks were formed for four categories of ecological changes of the territory (natural, anthropogenically modified, anthropogenic and technogenic). The developed technology was tested on the mining and industrial areas of Nikopol and Kryvyi Rih of Ukraine.
Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features

Given a query composed of a reference image and a relative caption, the Composed Image Retrieval goal is to retrieve images visually similar to the reference one that integrates the modifications expressed by the caption. Given that recent research has demonstrated the efficacy of large-scale vision and language pre-trained (VLP) models in various tasks, we rely on features from the OpenAI CLIP model to tackle the considered task. We initially perform a task-oriented fine-tuning of both CLIP encoders using the element-wise sum of visual and textual features. Then, in the second stage, we train a Combiner network that learns to combine the image-text features integrating the bimodal information and providing combined features used to perform the retrieval. We use contrastive learning in both stages of training. Starting from the bare CLIP features as a baseline, experimental results show that the task-oriented fine-tuning and the carefully crafted Combiner network are highly effective and outperform more complex state-of-the-art approaches on FashionIQ and CIRR, two popular and challenging datasets for composed image retrieval. Code and pre-trained models are available at https://github.com/ABaldrati/CLIP4Cir .
RACE: An Efficient Redundancy-aware Accelerator for Dynamic Graph Neural Network

Dynamic Graph Neural Network (DGNN) has recently attracted a significant amount of research attention from various domains, because most real-world graphs are inherently dynamic. Despite many research efforts, for DGNN, existing hardware/software solutions still suffer significantly from redundant computation and memory access overhead, because they need to irregularly access and recompute all graph data of each graph snapshot. To address these issues, we propose an efficient redundancy-aware accelerator, RACE , which enables energy-efficient execution of DGNN models. Specifically, we propose a redundancy-aware incremental execution approach into the accelerator design for DGNN to instantly achieve the output features of the latest graph snapshot by correctly and incrementally refining the output features of the previous graph snapshot and also enable regular accesses of vertices’ input features. Through traversing the graph on the fly, RACE identifies the vertices that are not affected by graph updates between successive snapshots to reuse these vertices’ states (i.e., their output features) of the previous snapshot for the processing of the latest snapshot. The vertices affected by graph updates are also tracked to incrementally recompute their new states using their neighbors’ input features of the latest snapshot for correctness. In this way, the processing and accessing of many graph data that are not affected by graph updates can be correctly eliminated, enabling smaller redundant computation and memory access overhead. Besides, the input features, which are accessed more frequently, are dynamically identified according to graph topology and are preferentially resident in the on-chip memory for less off-chip communications. Experimental results show that RACE achieves on average 1139× and 84.7× speedups for DGNN inference, with average 2242× and 234.2× energy savings, in comparison with the state-of-the-art software DGNN running on Intel Xeon CPU and NVIDIA A100 GPU, respectively. Moreover, for DGNN inference, RACE obtains on average 13.1×, 11.7×, 10.4×, and 7.9× speedup and 14.8×, 12.9×, 11.5×, and 8.9× energy savings over the state-of-the-art Graph Neural Network accelerators, i.e., AWB-GCN, GCNAX, ReGNN, and I-GCN, respectively.
Smart-DNN+: A Memory-efficient Neural Networks Compression Framework for the Model Inference

Deep Neural Networks (DNNs) have achieved remarkable success in various real-world applications. However, running a Deep Neural Network (DNN) typically requires hundreds of megabytes of memory footprints, making it challenging to deploy on resource-constrained platforms such as mobile devices and IoT. Although mainstream DNNs compression techniques such as pruning, distillation, and quantization can reduce the memory overhead of model parameters during DNN inference, they suffer from three limitations: (i) low model compression ratio for the lightweight DNN structures with little redundancy, (ii) potential degradation in model inference accuracy, and (iii) inadequate memory compression ratio is attributable to ignoring the layering property of DNN inference. To address these issues, we propose a lightweight memory-efficient DNN inference framework called Smart-DNN+, which significantly reduces the memory costs of DNN inference without degrading the model quality. Specifically, ① Smart-DNN+ applies a layerwise binary-quantizer with a remapping mechanism to greatly reduce the model size by quantizing the typical floating-point DNN weights of 32-bit to the 1-bit signs layer by layer. To maintain model quality, ② Smart-DNN+ employs a bucket-encoder to keep the compressed quantization error by encoding the multiple similar floating-point residuals into the same integer bucket IDs. When running the compressed DNN in the user’s device, ③ Smart-DNN+ utilizes a partially decompressing strategy to greatly reduce the required memory overhead by first loading the compressed DNNs in memory and then dynamically decompressing the required materials for model inference layer by layer. Experimental results on popular DNNs and datasets demonstrate that Smart-DNN+ achieves lower 0.17%–0.92% memory costs at lower runtime overheads compared with the states of the art without degrading the inference accuracy. Moreover, Smart-DNN+ potentially reduces the inference runtime up to 2.04× that of conventional DNN inference workflow.
A study towards contextual understanding of toxicity in online conversations

Abstract Identifying and annotating toxic online content on social media platforms is an extremely challenging problem. Work that studies toxicity in online content has predominantly focused on comments as independent entities. However, comments on social media are inherently conversational, and therefore, understanding and judging the comments fundamentally requires access to the context in which they are made. We introduce a study and resulting annotated dataset where we devise a number of controlled experiments on the importance of context and other observable confounders – namely gender, age and political orientation – towards the perception of toxicity in online content. Our analysis clearly shows the significance of context and the effect of observable confounders on annotations. Namely, we observe that the ratio of toxic to non-toxic judgements can be very different for each control group, and a higher proportion of samples are judged toxic in the presence of contextual information.
HPRoP: Hierarchical Privacy-preserving Route Planning for Smart Cities

Route Planning Systems (RPS) are a core component of autonomous personal transport systems essential for safe and efficient navigation of dynamic urban environments with the support of edge-based smart city infrastructure, but they also raise concerns about user route privacy in the context of both privately owned and commercial vehicles. Numerous high-profile data breaches in recent years have fortunately motivated research on privacy-preserving RPS, but most of them are rendered impractical by greatly increased communication and processing overhead. We address this by proposing an approach called Hierarchical Privacy-Preserving Route Planning (HPRoP), which divides and distributes the route-planning task across multiple levels and protects locations along the entire route. This is done by combining Inertial Flow partitioning, Private Information Retrieval (PIR), and Edge Computing techniques with our novel route-planning heuristic algorithm. Normalized metrics were also formulated to quantify the privacy of the source/destination points ( endpoint location privacy ) and the route itself ( route privacy ). Evaluation on a simulated road network showed that HPRoP reliably produces routes differing only by ≤ 20% in length from optimal shortest paths, with completion times within ∼ 25 seconds, which is reasonable for a PIR-based approach. On top of this, more than half of the produced routes achieved near-optimal endpoint location privacy (∼ 1.0) and good route privacy (≥ 0.8).
Gleaning Emotions from Virtual Stickers: An Intercultural Study

Virtual stickers—as a newer generation of graphic icons—possess the potential to facilitate online intercultural communication through emotional expression. However, people from different cultural backgrounds rely on varied display and decoding rules, which lead to disagreement on the emotions communicated by stickers. Drawing from the dialect theory of communicating emotion, the current study employs an online survey to examine whether culture-, emotion-, and technology-related factors impact American college students’ recognition accuracy of 6 basic emotions expressed through 36 Chinese animated and static stickers. Results demonstrate that sticker use frequency, intercultural communication competence, and empathy are positively related to cross-cultural sticker emotion recognition accuracy. However, neither prior intercultural experience with Chinese students nor social media use frequency influences cross-cultural sticker emotion recognition accuracy. No gender differences were identified. This study represents one of the first to systematically examine predictors of sticker interpretation accuracy in a cross-cultural context.
P‐14: Image Quality Study of Super Retina Resolution AMOLED Display Device

A 6.42‐inch super retina resolution LTPS AMOLED panel have been presented by using one pixel contains three sub‐pixels arrangement structure. The evaluation of image quality shows that super retina is better than QHD SPR (Sub‐pixel rendering) OLED module, under high refresh rate mode, super retina display uniformity is much better than QHD SPR. What's more, super retina has 20% IC power consumption reduction compared with QHD SPR and over 1 hour increase of mobile phone operation time.
P‐141: <i>Late‐News Poster:</i> Chroma Remapping Power Saving Technique for OLEDs

This paper proposes a novel image processing algorithm that maintains perceivable luminance as much as possible while it significantly reduces EL power consumption of OLED based upon a well‐known Helmholtz‐Kohl effect which is one of the representative theories based on human visual system's (HVS) characteristics of recognizing higher saturation images to be brighter. Proposed method effectively resolves various image artifacts incurred by the modification of ‘saturation’ value in each pixel, by carefully analyzing various image artifacts in each of problematic cases. Experimental results show that we have achieved a score of 4.18 based upon a subjective image quality evaluation methodology (Mean Opinion Score) and an average of 15.3% of EL power consumption savings were achieved in 20 evaluation images at the same time.
P‐17: All Weather‐Robust Image Quality Enhancement based on Image Feature Fusion and Multiscale Degradation Profile

Severe weather conditions often induce degraded camera output images in automotive applications, which decreases the visibility of drivers. In order to address this problem, this paper proposes a multi‐scale degradation profile‐based image enhancement framework. The proposed framework is composed of a feature fusion module, a multi‐scale degradation profile module, and an image enhancement module. The qualitative evaluation, ablation study, and quantitative evaluation proved the advantages and practicability of the proposed framework.
46‐2: <i>Invited Paper:</i> Outdoor Displays for Industrial and Digital Signage Applications

Outdoor applications integrating displays have expanded significantly during the past few years, as people want information in real‐time regardless of location or weather conditions. However, outdoor use comes with its own set of challenges.
P‐100: Consistent Low‐Light Image Restoration for Under‐Display Camera

Under‐display camera (UDC) allows viewers to interact with the camera and watch the display at the same time. This paper introduces the UDC for Mini‐LED/Micro‐LED displays embedding the camera by drilling a pinhole between LED dies. Based on such an application, the main issues of UDC images are low‐light and noise. We propose a CNN‐based image restoration approach to enhance brightness and remove noise. Moreover, the input of the proposed neural network uses three consecutive frames to reduce the temporal flickering of videos. The processing speed on GPU can reach 1080p@30fps, which is feasible for real‐time streaming applications.
43‐1: <i>Invited Paper:</i> The Application and Future Development Trend of Oxide Technology in the Meta‐Universe

With the rise of concept “meta‐universe”, VR/AR products get more attention currently. This paper introduces the applied oxide technology in ultra‐high resolution VR products and its future development trend. It was investigated how oxide technology become the choice of ultra‐high resolution due to its transparent property. It was also highly researched how to ensure the device characteristics with short channel through adjustment of process route. In the end, the future trend, such as higher mobility and removal of light shield layer, is simply introduces.
Machine Vision and Tagging Aesthetics: Assembling Socio-Technical Subjects through New Media Art

This paper builds on the concept of ‘tagging aesthetics’ (Bozzi, 2020b) to discuss new media art projects that combine machine vision and social media to address how different kinds of socio-technical subjects are assembled through AI. The premise outlines how the naturalisation of machine vision involves a range of subjects, juxtaposed along different conflictual lines: ontological (human-machine), biopolitical (classifier-classified), socio-technical (tech worker-data cleaner), political (AI-viewing public). Embracing the ambiguity inherent in the shifting boundaries of these subjects, I analyse works by different new media artists who approach one or more of these juxtapositions by engaging with diverse forms of tagging. The practice of tagging is often discussed through data-driven analyses of hashtags and how related publics can be mapped, but in my framework, tagging can encompass a wider spectrum of techno-social practices of connection (e.g. geotagging, tagging users). I discuss artworks by Kate Crawford and Trevor Paglen, Dries Depoorter and Max Dovey to illustrate how these practices can be leveraged artistically to make visible and even ‘stitch together’ the manifold subjects of machine vision. I explain how those taggings denaturalise processes of socio-technical classification by activating awareness, if not agency, through the sheer proximity they enact. Far from being a tool to map knowledge and essentialised identities, tagging aesthetics are ways to perform the techno-social and shape future cultural encounters with various forms of others. By exploring different approaches to tagging aesthetics – (dis)identification, semi-automated assembly and embodied encounter – this paper illustrates how tagging can be used to culturally negotiate the impact of machine vision in terms of issues such as surveillance and the performance of digital identity.
Truthful meta-explanations for local interpretability of machine learning models

Abstract Automated Machine Learning-based systems’ integration into a wide range of tasks has expanded as a result of their performance and speed. Although there are numerous advantages to employing ML-based systems, if they are not interpretable, they should not be used in critical or high-risk applications. To address this issue, researchers and businesses have been focusing on finding ways to improve the explainability of complex ML systems, and several such methods have been developed. Indeed, there are so many developed techniques that it is difficult for practitioners to choose the best among them for their applications, even when using evaluation metrics. As a result, the demand for a selection tool, a meta-explanation technique based on a high-quality evaluation metric, is apparent. In this paper, we present a local meta-explanation technique which builds on top of the truthfulness metric, which is a faithfulness-based metric. We demonstrate the effectiveness of both the technique and the metric by concretely defining all the concepts and through experimentation.
15‐1: A Novel Compression Algorithm Application for De‐Mura Compensation of AMOLED Panel

A novel compression algorithm was adopted to compress De‐mura compensation data, and the compression process was described in detail. The compensation performance of 2x2 compression case is close to the original 2x2 case, and performs better than 4x4 compensation case with the same file size as it. Apart from that, the combination of De‐mura and compression also performs the good compensation results at low gray as well as low display brightness. The better performance, small space occupancy and low power consumption were reached simultaneously, indicating the effective application of combination of compression algorithm and De‐mura algorithm.
38‐2: Ubiquitous Display with High Transparency Phosphor Dot Projection Film

A see‐through projection system with phosphor dot film has been proposed and demonstrated to provide clear and sharp image on any kind of object surface. The attached projection film will not disturb the original feature of the surface and in turn make a display with no clearly defined display frame.
71‐1: Data‐Driven and Optics‐Inspired Decomposition of Global Pupil Swim in VR/AR for an Improved Perception Model of Motion Discomfort

Users can observe dynamic distortion (or global pupil swim) during the usage of VR/AR headsets. Our earlier study correlated pupil swim to selected optic flow patterns corresponding to motion and mathematically modeled discomfort. This study decomposes global pupil swim as a linear sum of orthogonal basis patterns based on Zernike polynomials for improved applicability and for an improved perception model.
42‐2: Automotive Image Enhancement

Automotive display image enhancement may be utilized to enhance display visibility under various lighting environments typically encountered in the automotive cockpit. Image enhancement methods may be combined with automatic luminance control methods to yield a new class of display visibility product.
22‐4: <i>Invited Paper:</i> Deep Learning‐based Image Deblurring for Display Vision Inspection

Image quality acts as a major factor in determining its performance in a vision inspection task. The moire pattern caused by frequency aliasing severely degrades the visual quality in display devices, where such high‐quality images are required. To remove these quality undermining patterns, the images are acquired by intentional defocusing. Then, to restore the details lost during the image acquisition, image deblurring is used. The existing deblurring methods fail to output satisfactory results for low contrast Mura images. To solve this problem, we present a novel approach using a generalized Gaussian kernel for real‐world vision inspection tasks. We evaluated the performance and experimented under different settings to validate the robustness of the proposed method. The performance for the proposed method has improved in no‐reference image quality assessment metrics.
Multi-distribution fitting for multi-view stereo

We propose a multi-view stereo network based on multi-distribution fitting (MDF-Net), which achieves high-resolution depth map prediction with low memory and high efficiency. This method adopts a four-stage cascade structure, which mainly has the following three contributions. First, view cost regularization is proposed to weaken the influence of matching noise on building the cost volume. Second, it is suggested to adaptively calculate the depth refinement interval using multi-distribution fitting (MDF). Gaussian distribution fitting is used to refine and correct depth within a large interval, and then Laplace distribution fitting is used to accurately estimate depth within a small interval. Third, the lightweight image super-resolution network is applied to upsample the depth map in the fourth stage to reduce running time and memory requirements. The experimental results on the DTU dataset indicate that MDF-Net has achieved the most advanced results. It has the lowest memory consumption and running time among the high-resolution reconstruction methods, requiring only approximately 4.29G memory for predicting a depth map with the resolution of 1600 × 1184. In addition, we validate the generalization ability on Tanks and Temples dataset, achieving very competitive performance. The code has been released at https://github.com/zongh5a/MDF-Net .
71‐2: Effect of Compensation Method on Visually Induced Motion Sickness for Transparent Window Display

An information compensation method to reduce motion sickness caused by reading information on the transparent window display has been studied and proposed. In this research, Visual Analogue Scale (VAS), Center of Pressure(CoP), Simulator sickness questionnaire (SSQ) and Heart Rate Variability (HRV) were conducted and the data analysis revealed that the information compensation based on the degree of vehicle vibration improved the comfort of reading the text information for passengers.