Reviewerly/exHarmony huggingface dataset:
Neural Rate Estimator and Unsupervised Learning for Efficient Distributed Image Analytics in Split-DNN models

Thanks to advances in computer vision and AI, there has been a large growth in the demand for cloud-based visual analytics in which images captured by a low-powered edge device are transmitted to the cloud for analytics. Use of conventional codecs (JPEG, MPEG, HEVC, etc.) for compressing such data introduces artifacts that can seriously degrade the performance of the downstream analytic tasks. Split-DNN computing has emerged as a paradigm to address such usages, in which a DNN is partitioned into a client-side portion and a server side portion. Low-complexity neural networks called ‘bottleneck units' are introduced at the split point to transform the intermediate layer features into a lower-dimensional representation better suited for compression and transmission. Optimizing the pipeline for both compression and task-performance requires high-quality estimates of the information-theoretic rate of the intermediate features. Most works on compression for image analytics use heuristic approaches to estimate the rate, leading to suboptimal performance. We propose a high-quality ‘neural rateestimator’ to address this gap. We interpret the lower-dimensional bottleneck output as a latent representation of the intermediate feature and cast the rate-distortion optimization problem as one of training an equivalent variational auto-encoder with an appropriate loss function. We show that this leads to improved rate-distortion outcomes. We further show that replacing supervised loss terms (such as cross-entropy loss) by distillation-based losses in a teacher-student framework allows for unsupervised training of bottleneck units without the need for explicit training labels. This makes our method very attractive for real world deployments where access to labeled training data is difficult or expensive. We demonstrate that our method outperforms several state-of-the-art methods by obtaining improved task accuracy at lower bi-trates on image classification and semantic segmentation tasks.
Distribution Shift Inversion for Out-of-Distribution Prediction

Machine learning society has witnessed the emergence of a myriad of Out-of-Distribution (OoD) algorithms, which address the distribution shift between the training and the testing distribution by searching for a unified predictor or invariant feature representation. However, the task of directly mitigating the distribution shift in the unseen testing set is rarely investigated, due to the unavailability of the testing distribution during the training phase and thus the impossibility of training a distribution translator mapping between the training and testing distribution. In this paper, we explore how to bypass the requirement of testing distribution for distribution translator training and make the distribution translation useful for OoD prediction. We propose a portable Distribution Shift Inversion (DSI) algorithm, in which, before being fed into the prediction model, the OoD testing samples are first linearly combined with additional Gaussian noise and then transferred back towards the training distribution using a diffusion model trained only on the source distribution. Theoretical analysis reveals the feasibility of our method. Experimental results, on both multiple-domain generalization datasets and single-domain generalization datasets, show that our method provides a general performance gain when plugged into a wide range of commonly used OoD algorithms. Our code is available at https://github.com/yu-rp/Distribution-Shift-Iverson.
Overcoming the TradeOff between Accuracy and Plausibility in 3D Hand Shape Reconstruction

Direct mesh fitting for 3D hand shape reconstruction is highly accurate. However, the reconstructed meshes are prone to artifacts and do not appear as plausible hand shapes. Conversely, parametric models like MANO ensure plausible hand shapes but are not as accurate as the non-parametric methods. In this work, we introduce a novel weakly-supervised hand shape estimation framework that integrates non-parametric mesh fitting with MANO model in an end-to-end fashion. Our joint model overcomes the tradeoff in accuracy and plausibility to yield well-aligned and high-quality 3D meshes, especially in challenging two-hand and hand-object interaction scenarios.
OSAN: A One-Stage Alignment Network to Unify Multimodal Alignment and Unsupervised Domain Adaptation

Extending from unimodal to multimodal is a critical challenge for unsupervised domain adaptation (UDA). Two major problems emerge in unsupervised multimodal domain adaptation: domain adaptation and modality alignment. An intuitive way to handle these two problems is to fulfill these tasks in two separate stages: aligning modalities followed by domain adaptation, or vice versa. However, domains and modalities are not associated in most existing two-stage studies, and the relationship between them is not leveraged which can provide complementary information to each other. In this paper, we unify these two stages into one to align domains and modalities simultaneously. In our model, a tensor-based alignment module (TAL) is presented to explore the relationship between domains and modalities. By this means, domains and modalities can interact sufficiently and guide them to utilize complementary information for better results. Furthermore, to establish a bridge between domains, a dynamic domain generator (DDG) module is proposed to build transitional samples by mixing the shared information of two domains in a self-supervised manner, which helps our model learn a domain-invariant common representation space. Extensive experiments prove that our method can achieve superior performance in two real-world applications. The code will be publicly available.
Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation

Fully test-time adaptation aims to adapt the network model based on sequential analysis of input samples during the inference stage to address the cross-domain performance degradation problem of deep neural networks. We take inspiration from the biological plausibility learning where the neuron responses are tuned based on a local synapse-change procedure and activated by competitive lateral inhibition rules. Based on these feed-forward learning rules, we design a soft Hebbian learning process which provides an unsupervised and effective mechanism for online adaptation. We observe that the performance of this feed-forward Hebbian learning for fully test-time adaptation can be significantly improved by incorporating a feedback neuromodulation layer. It is able to fine-tune the neuron responses based on the external feedback generated by the error backpropagation from the top inference layers. This leads to our proposed neuro-modulated Hebbian learning (NHL) method for fully test-time adaptation. With the unsupervised feed-forward soft Hebbian learning being combined with a learned neuromodulator to capture feedback from external responses, the source model can be effectively adapted during the testing process. Experimental results on benchmark datasets demonstrate that our proposed method can significantly improve the adaptation performance of network models and outperforms existing state-of-the-art methods.
ShadowNeuS: Neural SDF Reconstruction by Shadow Ray Supervision

By supervising camera rays between a scene and multi-view image planes, NeRF reconstructs a neural scene representation for the task of novel view synthesis. On the other hand, shadow rays between the light source and the scene have yet to be considered. Therefore, we propose a novel shadow ray supervision scheme that optimizes both the samples along the ray and the ray location. By supervising shadow rays, we successfully reconstruct a neural SDF of the scene from single-view images under multiple lighting conditions. Given single-view binary shadows, we train a neural network to reconstruct a complete scene not limited by the camera's line of sight. By further modeling the correlation between the image colors and the shadow rays, our technique can also be effectively extended to RGB inputs. We compare our method with previous works on challenging tasks of shape reconstruction from single-view binary shadow or RGB images and observe significant improvements. The code and data are available at https://github.com/gerwang/ShadowNeuS.
The Best Defense is a Good Offense: Adversarial Augmentation Against Adversarial Attacks

Many defenses against adversarial attacks (e.g. robust classifiers, randomization, or image purification) use countermeasures put to work only after the attack has been crafted. We adopt a different perspective to introduce A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> (Adversarial Augmentation Against Adversarial Attacks), a novel framework including the first certified preemptive defense against adversarial attacks. The main idea is to craft a defensive perturbation to guarantee that any attack (up to a given magnitude) towards the input in hand will fail. To this aim, we leverage existing automatic perturbation analysis tools for neural networks. We study the conditions to apply A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> effectively, analyze the importance of the robustness of the to-be-defended classifier, and inspect the appearance of the robustified images. We show effective on-the-fly defensive augmentation with a robustifier network that ignores the ground truth label, and demonstrate the benefits of robustifier and classifier co-training. In our tests, A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> consistently beats state of the art certified defenses on MNIST, CIFAR10, FashionMNIST and Tinyimagenet. We also show how to apply A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> to create certifiably robust physical objects. Our code at https://github.com/NVlabs/A5 allows experimenting on a wide range of scenarios beyond the man-in-the-middle attack tested here, including the case of physical attacks.
Digital discourse in the realm of related phenomena

The article is devoted to the study of digital discourse as the most diverse socio-cultural background of the contemporaries’ interaction. This work outlines the theoretical background of digital-oriented discourse in a harmonious symbiosis with the practical manifestation of its existence. This paper distinguishes between the concepts of “computer discourse”, “virtual discourse” and “digital discourse”. The genres of virtual discourse, such as online learning classes, instant messaging, and video game genres are represented as related, but not equal phenomena. These cutting-edge sociolinguistic phenomena manifest the pragmatic intentions of clients and agents of digital discourse, which are discussed in the matrix of their characteristics. In the key focus of the study, one can find common and distinctive properties of related types of discourse, their parameters, and multilevel means that arrange them. Institutional and institutional-typological characteristics of digital discourse, conditions of communication and ways of its organization are revealed; its socializing and communicative purpose is outlined; hypero- and hyponymic dimensions of discourse are considered; strategies for sending and receiving information are presented; micro-, meso-, and macro-levels of digital discourse are identified; the concept of “digital discourse” is developed.
3D Human Mesh Estimation from Virtual Markers

Inspired by the success of volumetric 3D pose estimation, some recent human mesh estimators propose to estimate 3D skeletons as intermediate representations, from which, the dense 3D meshes are regressed by exploiting the mesh topology. However, body shape information is lost in extracting skeletons, leading to mediocre performance. The advanced motion capture systems solve the problem by placing dense physical markers on the body surface, which allows to extract realistic meshes from their non-rigid motions. However, they cannot be applied to wild images without markers. In this work, we present an intermediate representation, named virtual markers, which learns 64 landmark keypoints on the body surface based on the large-scale mocap data in a generative style, mimicking the effects of physical markers. The virtual markers can be accurately detected from wild images and can reconstruct the intact meshes with realistic shapes by simple interpolation. Our approach outperforms the state-of-the-art methods on three datasets. In particular, it surpasses the existing methods by a notable margin on the SURREAL dataset, which has diverse body shapes. Code is available at https://github.com/ShirleyMaxx/VirtualMarker
Frequency-Revealing Attacks against Frequency-Hiding Order-Preserving Encryption

Order-preserving encryption (OPE) allows efficient comparison operations over encrypted data and thus is popular in encrypted databases. However, most existing OPE schemes are vulnerable to inference attacks as they leak plaintext frequency. To this end, some frequency-hiding order-preserving encryption (FH-OPE) schemes are proposed and claim to prevent the leakage of frequency. FH-OPE schemes are considered an important step towards mitigating inference attacks. Unfortunately, there are still vulnerabilities in all existing FH-OPE schemes. In this work, we revisit the security of all existing FH-OPE schemes. We are the first to demonstrate that plaintext frequency hidden by them is recoverable. We present three ciphertext-only attacks named frequency-revealing attacks to recover plaintext frequency. We evaluate our attacks in three real-world datasets. They recover over 90% of plaintext frequency hidden by any existing FH-OPE scheme. With frequency revealed, we also show the potentiality to apply inference attacks on existing FH-OPE schemes. Our findings highlight the limitations of current FH-OPE schemes. Our attacks demonstrate that achieving frequency-hiding requires addressing the leakages of both non-uniform ciphertext distribution and insertion orders of ciphertexts, even though the leakage of insertion orders is always ignored in OPE.
From Gestures to Audio: A Dataset Building Approach for Egyptian Sign Language Translation to Arabic Speech

The communication barriers faced by people with disabilities, particularly the deaf or hard of hearing, nonverbal, deaf-mute, and blind have a significant impact on their quality of life and social inclusion. Our research aims to provide real-time translation from sign language to speech and vice versa. The ability to provide real-time speech-to-text and text-to-sign language translation will help alleviate these barriers, improve communication, and increase social inclusivity for this community ensuring they are not left out in conversations and social interactions. A significant amount of data is required to develop a deep learning model that automatically translates Egyptian sign language to audio. Due to the unavailability of large-scale datasets for Egyptian sign language, we present a dataset-building approach to address this issue. In this paper, we discuss collecting and preprocessing the dataset, which includes extracting audio, video scrapping, cropping the required part of the video containing the sign language translator, and applying human and hand recognition models to identify the translators and their gestures, respectively. The resulting dataset will be used to train deep learning models to translate Egyptian sign language to audio as an end-to-end approach. The dataset will be available in our research GitHub.
Football activities classification

Football, also known as soccer in some regions, is a popular sport that requires teams to score more goals than each other to win. As the sport gains more popularity, people are increasingly interested in learning its techniques. This study proposes a system that can help users learn football techniques such as juggling, inside-of-the-foot pass, and fake shot double tap (drill), and this move is divided into two parts. The system offers real-time feedback on users' actions, allowing them to learn more effectively. To achieve this, the system uses Mediapipe, an open-source library, to extract the coordinates of the human skeleton from the video recorded by the user. Besides, four classification algorithms, namely dollarpy, KNN, RFE, and SVM, to classify the actions performed by the user. If the user performs any of the specified actions correctly, they are awarded one point, and at the end of the session, they receive a report indicating how many times they performed each action. System results show that the model was able to predict the actions with high accuracy. The accuracies reached by the models used RFE, SVM, KNN, and dollarpy were 80%, 90%, 75&, and 55% respectively.
Comparative Analysis of Pre-Trained CNN Architectures for Multi-Class Weather Classification: A Study on Deep Learning Techniques

In recent years, convolutional neural networks (CNNs) have proven to be highly effective in a wide range of computer vision tasks, including image classification. In this study, we compare the performance of several pretrained CNN models on the task of weather classification from single images. We evaluate the models on a dataset of weather images, consisting of five classes: cloudy, foggy, rainy, shine and sunrise; We consider several popular CNN models, including MobileNet V2, Inception V3, Densenets 121, 169 and 201 and finally the Xception model which were all trained on the ImageNet dataset.We conduct experiments to evaluate the performance of each model on the weather classification task.
Geometric bloch vector solution to minimum-error discriminations of mixed qubit states

Abstract We investigate minimum-error (ME) discrimination for mixed qubit states using a geometric approach. By analyzing positive operator-valued measure (POVM) solutions and introducing Lagrange operator $$\Gamma $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>Γ</mml:mi> </mml:math> , we develop a four-step structured instruction to find $$\Gamma $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>Γ</mml:mi> </mml:math> for N mixed qubit states. Our method covers optimal solutions for two, three, and four mixed qubit states, including a novel result for four qubit states. We introduce geometric-based POVM classes and non-decomposable subsets for constructing optimal solutions, enabling us to find all possible answers for the general problem of minimum-error discrimination for N mixed qubit states with arbitrary a priori probabilities.
Study on Lane-Change Replanning and Trajectory Tracking for Intelligent Vehicles Based on Model Predictive Control

When an intelligent vehicle changes lanes, the state of other vehicles may change, which increases the risk of collision. Therefore, real-time local path replanning is needed at this time. Based on model predictive control (MPC), a lane-change trajectory replanning strategy was proposed, which was divided into a lane-change trajectory correction strategy, a lane-change switchback strategy and forward active collision avoidance strategy according to collision risk. Based on the collision risk function of the rectangular safety neighborhood, the objective functions were designed according to the specific requirements of different strategies. The vehicle lateral controller based on MPC and the vehicle longitudinal motion controller were established. The longitudinal velocity was taken as the joint point to establish the lateral and longitudinal integrated controller. The trajectory planning module, trajectory replanning module and trajectory tracking module were integrated in layers, and the three trajectory replanning strategies of lane-change trajectory correction, lane-change switchback and forward active collision avoidance were respectively simulated and verified. The simulation results showed the trajectory replanning strategy achieves collision avoidance under different scenarios and ensures the vehicle’s driving stability. The trajectory tracking layer achieves accurate tracking of the conventional lane-change trajectory and has good driving stability and comfort.
Promoting Elderly Care Sustainability by Smart Village Facilities Integration—Construction of a Public Service Field with Introduction of Fall Posture Monitoring

In recent years, rural communities in China have been actively exploring a novel approach to elderly care that integrates treatment with health preservation. This study aims to contribute to the sustainable development of rural elderly care by introducing smart technology, specifically fall posture monitoring, into public services and facilities within rural communities. The focus of this study is on addressing a critical issue in elderly medical care: the timely feedback and treatment of falls. To begin, we conduct a comprehensive review of the current status and challenges associated with the application of fall posture monitoring technology. Additionally, we examine the environmental factors that contribute to the risk of falls in public spaces for the elderly. These assessments serve as the technical and environmental foundation for developing the proposed service framework. Our research is conducted from two primary perspectives: the supply of service resources that combine treatment with health preservation and the identification of risk factors associated with outdoor public spaces in the community where falls are likely to occur. Data for this study are collected through behavior mapping and field interviews. In conclusion, we present a constructive logic for the development of a public service field that effectively combines treatment with health preservation. This logic encompasses the integration of technology applications, resource coordination, and improvements to the physical environment. The findings from this study provide a scientific basis for the construction of public service fields in “smart villages” and serve as practical references for similar villages striving to adopt this innovative model. By leveraging the insights gained from this research, it is expected that rural communities will be better equipped to address the challenges of elderly care and facilitate the widespread adoption of this integrated care model.
Coined quantum walks on the line: Disorder, entanglement, and localization

Disorder in coined quantum walks generally leads to localization. We investigate the influence of the localization on the entanglement properties of coined quantum walks. Specifically, we consider quantum walks on the line and explore the effects of quenched disorder in the coin operations. After confirming that our choice of disorder localizes the walker, we study how the localization affects the properties of the coined quantum walk. We find that the mixing properties of the walk are altered nontrivially with mixing being improved at short time scales. Special focus is given to the influence of coin disorder on the properties of the quantum state and the coin-walker entanglement. We find that disorder alters the quantum state significantly even when the walker probability distribution is still close to the nondisordered case. We observe that, generically, coin disorder decreases the coin-walker entanglement and that the localization leaves distinct traces in the entanglement entropy and the entanglement negativity of the coined quantum walk.