Reviewerly/exHarmony huggingface dataset:
Confidential Truth Finding with Multi-Party Computation

Federated knowledge discovery and data mining are challenged to assess the trustworthiness of data originating from autonomous sources while protecting confidentiality and privacy. Truth-finding algorithms help corroborate data from disagreeing sources. For each query it receives, a truth-finding algorithm predicts a truth value of the answer, possibly updating the trustworthiness factor of each source. Few works, however, address the issues of confidentiality and privacy. We devise and present a secure secret-sharing-based multi-party computation protocol for pseudo-equality tests that are used in truth-finding algorithms to compute additions depending on a condition. The protocol guarantees confidentiality of the data and privacy of the sources. We also present a variants of a truth-finding algorithm that would make the computation faster when executed using secure multi-party computation. We empirically evaluate the performance of the proposed protocol on a state-of-the-art truth-finding algorithm, 3-Estimates, and compare it with that of the baseline plain algorithm. The results confirm that the secret-sharing-based secure multi-party algorithms are as accurate as the corresponding baselines but for proposed numerical approximations that significantly reduce the efficiency loss incurred.
Multi-view semi-supervised learning with adaptive graph fusion

Multi-view Semi-supervised Learning (MSL) is effective in using limited labels and considerable label-free data to improve learning performance. It has been successfully applied to a lot of real scenarios. In this study, we propose a model, termed MSL with Adaptive Graph Fusion (MSLAGF), which provides a novel solution for MSL. Unlike most existing methods propagating label information through the linear combination of pre-built fixed view-based similarity graphs, MSLAGF merges view-based graph construction, graph fusion, and label propagation. It adaptively learns view-specific graphs and automatically assigns weight coefficients to them. A multi-view fusion optimal graph is cleverly learned depending not only on the raw feature space but also on the dynamically predicted label space. Moreover, we present an efficient optimization algorithm to solve the formulated model. The view-specific graphs, the weight coefficients, the optimal graph, and the predicted labels are mutually negotiated and optimized in the optimization procedure. Extensive experimental results on six benchmark datasets validate the superiority.
Time-Optimal Handover Trajectory Planning for Aerial Manipulators Based on Discrete Mechanics and Complementarity Constraints

Planning a time-optimal trajectory for aerial robots is critical in many drone applications, such as rescue missions and package delivery, which have been widely researched in recent years. However, it still involves several challenges, particularly when it comes to incorporating special task requirements as well as the aerial robot's dynamics into the planning. In this work, we study a case where an aerial manipulator shall pick up a parcel from a moving mobile robot in a time-optimal manner. Rather than setting up the approach trajectory manually, which makes it difficult to determine the optimal total travel time to accomplish the desired task within dynamic limits, we propose an optimization framework, which utilizes the framework of discrete mechanics and complementarity constraints. In the proposed approach, the system dynamics is considered via discrete variational Lagrangian mechanics that provides reliable estimation results according to our experiments. The handover opportunities are automatically determined and arranged based on the desired complementarity constraints. Finally, the performance of the proposed framework is verified with numerical simulations and hardware experiments with our self-designed aerial manipulator.
Mitigating bias in facial analysis systems by incorporating label diversity

Facial analysis models are increasingly applied in real-world applications that significantly impact people’s lives. However, as literature has shown, models that automatically classify facial attributes might exhibit algorithmic discrimination behavior concerning protected groups, potentially negatively impacting individuals and society. It is, therefore, critical to develop techniques that can mitigate unintended biases in facial classifiers. Hence, this work introduces a novel learning method combining subjective human-based labels and objective annotations based on mathematical definitions of facial traits. Specifically, we generate new objective annotations from two large-scale human-annotated datasets, each capturing a different perspective of the analyzed facial trait. We then propose an ensemble learning method, which combines individual models trained on different types of annotations. We provide an in-depth analysis of the annotation procedure as well as the datasets distribution. We next show how much each individual model, each trained on a different set of annotations, contributes to the final ensemble. Finally, we empirically show that, by incorporating label diversity in the model pipeline, our method successfully mitigates unintended biases while maintaining significant accuracy on the downstream tasks. Our method is generic and can be applied to different types of facial tasks.
Diffusion-Based Document Layout Generation

We develop a diffusion-based approach for various document layout sequence generation. Layout sequences specify the contents of a document design in an explicit format. Our novel diffusion-based approach works in the sequence domain rather than the image domain in order to permit more complex and realistic layouts. We also introduce a new metric, Document Earth Mover’s Distance (Doc-EMD). By considering similarity between heterogeneous categories document designs, we handle the shortcomings of prior document metrics that only evaluate the same category of layouts. Our empirical analysis shows that our diffusion-based approach is comparable to or outperforming other previous methods for layout generation across various document datasets. Moreover, our metric is capable of differentiating documents better than previous metrics for specific cases.
Evaluating the influence of security considerations on information dissemination via social networks

This study investigates the factors that influence the sharing of information on social media platforms and examines the effects of perceived security, perceived privacy, and user awareness on users' trust in social media platforms, as well as the moderating effects of age, gender, educational attainment, and internet proficiency on information sharing. The study collected data from 837 social media users in Jordan and analyzed them using structural equation modeling (SEM), confirmatory factor analysis (CFA), and machine learning (ML) methods. The findings of the study indicate that perceived security, perceived privacy, and user awareness all have a significant impact on users' trust in social media platforms. Trust, in turn, has a significant impact on the amount of information shared on these platforms. Also, the findings of this study provide valuable insights into the dynamics of information sharing on social networks. This knowledge will be of interest to managers, policymakers, and developers of social media platforms. In addition, the findings of the study also have implications for the privacy and security of social media users. For example, social media users can be more careful about the information they share on social media platforms, and they can take steps to protect their privacy.
Visual Quality, Motility Behavior, and Retinal Changes Associated with Reading Tasks Performed on Electronic Devices

Background: The purpose of this study was to objectively evaluate visual discomfort using an eye tracker and aberrometer after a 21-min reading session on an iPad and an Ebook. Additionally, retinal changes were analyzed using optical coherence tomography (OCT). Methods: A total of 31 young subjects (24 ± 4 years) participated in this study. They read for 21 min on an Ebook and for another 21 min on an iPad under controlled lighting conditions while their eye movements were monitored using an eye tracker. Aberrometry and retinal OCT measurements were taken before and after each reading session. Parameters such as pupil diameter, fixations, saccades, blinks, total aberration, high-order aberration, low-order aberration, and central and peripheral retinal thickness in the nine early treatment diabetic retinopathy study (ETDRS) areas were measured for each reading situation. Statistical analysis was performed on the collected data. Results: No statistically significant differences (p &gt; 0.05) between the two devices were observed in terms of the different types of eye movements or the changes in retinal thickness. However, the aberrometric analysis showed variations in post-reading situations depending on the device used. Conclusion: Reading speed and visual discomfort resulting from electronic device usage can be objectively assessed using an eye tracker and aberrometer. Additionally, changes found in central and peripheral retinal thickness between the two devices and the baseline measurements were not significant and remained relatively stable.
Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!

Finetuning a large vision language model (VLM) on a target dataset after large scale pretraining is a dominant paradigm in visual question answering (VQA). Datasets for specialized tasks such as knowledge-based VQA or VQA in non natural-image domains are orders of magnitude smaller than those for general-purpose VQA. While collecting additional labels for specialized tasks or domains can be challenging, unlabeled images are often available. We introduce SelTDA (Self-Taught Data Augmentation), a strategy for finetuning large VLMs on small-scale VQA datasets. SelTDA uses the VLM and target dataset to build a teacher model that can generate question-answer pseudolabels directly conditioned on an image alone, allowing us to pseudolabel unlabeled images. SelTDA then finetunes the initial VLM on the original dataset augmented with freshly pseudolabeled images. We describe a series of experiments showing that our self-taught data augmentation increases robustness to adversarially searched questions, counterfactual examples and rephrasings, improves domain generalization, and results in greater retention of numerical reasoning skills. The proposed strategy requires no additional annotations or architectural modifications, and is compatible with any modern encoder-decoder multimodal transformer. Code available at https://github.com/codezakh/SelTDA.
Boosting Accuracy and Robustness of Student Models via Adaptive Adversarial Distillation

Distilled student models in teacher-student architectures are widely considered for computational-effective deployment in real-time applications and edge devices. However, there is a higher risk of student models to encounter adversarial attacks at the edge. Popular enhancing schemes such as adversarial training have limited performance on compressed networks. Thus, recent studies concern about adversarial distillation (AD) that aims to inherit not only prediction accuracy but also adversarial robustness of a robust teacher model under the paradigm of robust optimization. In the min-max framework of AD, existing AD methods generally use fixed supervision information from the teacher model to guide the inner optimization for knowledge distillation which often leads to an overcorrection towards model smoothness. In this paper, we propose an adaptive adversarial distillation (AdaAD) that involves the teacher model in the knowledge optimization process in a way interacting with the student model to adaptively search for the inner results. Comparing with state-of-the-art methods, the proposed AdaAD can significantly boost both the prediction accuracy and adversarial robustness of student models in most scenarios. In particular, the ResNet-18 model trained by AdaAD achieves top-rank performance (54.23% robust accuracy) on RobustBench under AutoAttack.
Full-Decentralized Federated Learning-Based Edge Computing Peer Offloading Towards Industry 5.0

This paper gives a generic architecture and system modeling for full-decentralized on-demand edge computing-based peer offloading. Compared with the conventional centralized peer offloading strategies, the proposed full-decentralized peer offloading, based on federated learning, physically decentralizes peer offloading algorithm into edge computing, fully eliminating the rely on centralized servers (e.g., cloud). Meanwhile, compared with the other previous decentralized offloading schemes (blockchain-based, game theory-based, etc.), edge computing servers in this paper does not require global information to be shared, when they reach consensus of optimal peer offloading. In particular, the adjacent edge computing servers only share property-sensitive data (for the service providers of the edge computing servers) among each other, relying on which the whole edge computing network can reach global optimal peer offloading. In this paper, we consider energy efficiency as a use case to analyze the feasibility and efficiency of the proposed full-decentralized peer offloading architecture.
OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis

We present OmniAvatar, a novel geometry-guided 3D head synthesis model trained from in-the-wild unstructured images that is capable of synthesizing diverse identity-preserved 3D heads with compelling dynamic details under full disentangled control over camera poses, facial expressions, head shapes, articulated neck and jaw poses. To achieve such high level of disentangled control, we first explicitly define a novel semantic signed distance function (SDF) around a head geometry (FLAME) conditioned on the control parameters. This semantic SDF allows us to build a differentiable volumetric correspondence map from the observation space to a disentangled canonical space from all the control parameters. We then leverage the 3D-aware GAN framework (EG3D) to synthesize detailed shape and appearance of 3D full heads in the canonical space, followed by a volume rendering step guided by the volumetric correspondence map to output into the observation space. To ensure the control accuracy on the synthesized head shapes and expressions, we introduce a geometry prior loss to conform to head SDF and a control loss to conform to the expression code. Further, we enhance the temporal realism with dynamic details conditioned upon varying expressions and joint poses. Our model can synthesize more preferable identity-preserved 3D heads with compelling dynamic details compared to the state-of-the-art methods both qualitatively and quantitatively. We also provide an ablation study to justify many of our system design choices.
Resource-Efficient RGBD Aerial Tracking

Aerial robots are now able to fly in complex environments, and drone-captured data gains lots of attention in object tracking. However, current research on aerial perception has mainly focused on limited categories, such as pedestrian or vehicle, and most scenes are captured in urban environments from a birds-eye view. Recently, UAVs equipped with depth cameras have been also deployed for more complex applications, while RGBD aerial tracking is still unexplored. Compared with traditional RGB object tracking, adding depth information can more effectively deal with more challenging scenes such as target and background interference. To this end, in this paper, we explore RGBD aerial tracking in an overhead space, which can greatly enlarge the development of drone-based visual perception. To boost the research, we first propose a large-scale benchmark for RGBD aerial tracking, containing 1,000 drone-captured RGBD videos with dense annotations. Then, as drone-based applications require for real-time processing with limited computational resources, we also propose an efficient RGBD tracker named EMT. Our tracker runs at over 100 fps on GPU, and 25 fps on the edge platform of NVidia Jetson NX Xavier, benefiting from its efficient multimodal fusion and feature matching. Extensive experiments show that our EMT achieves promising tracking performance. All resources are available at https://github.com/yjybuaa/RGBDAerialTracking.
PA&amp;DA: Jointly Sampling PAth and DAta for Consistent NAS

Based on the weight-sharing mechanism, one-shot NAS methods train a supernet and then inherit the pre-trained weights to evaluate sub-models, largely reducing the search cost. However, several works have pointed out that the shared weights suffer from different gradient descent directions during training. And we further find that large gradient variance occurs during supernet training, which degrades the supernet ranking consistency. To mitigate this issue, we propose to explicitly minimize the gradient variance of the supernet training by jointly optimizing the sampling distributions of PAth and DAta (PA&DA). We theoretically derive the relationship between the gradient variance and the sampling distributions, and reveal that the optimal sampling probability is proportional to the normalized gradient norm of path and training data. Hence, we use the normalized gradient norm as the importance indicator for path and training data, and adopt an importance sampling strategy for the supernet training. Our method only requires negligible computation cost for optimizing the sampling distributions of path and data, but achieves lower gradient variance during supernet training and better generalization performance for the supernet, resulting in a more consistent NAS. We conduct comprehensive comparisons with other improved approaches in various search spaces. Results show that our method surpasses others with more reliable ranking performance and higher accuracy of searched architectures, showing the effectiveness of our method. Code is available at https://github.com/ShunLu91/PA-DA.
Exactly Tight Information-Theoretic Generalization Error Bound for the Quadratic Gaussian Problem

We provide a new information-theoretic generalization error bound that is exactly tight (i.e., matching even the constant) for the canonical quadratic Gaussian mean estimation problem. Despite considerable existing efforts in deriving information-theoretic generalization error bounds, applying them to this simple setting where sample average is used as the estimate of the mean value of Gaussian data has not yielded satisfying results. In fact, most existing bounds are order-wise loose in this setting, which has raised concerns about the fundamental capability of information-theoretic bounds in reasoning the generalization behavior for machine learning. The proposed new bound adopts the individual-sample-based approach proposed by Bu et al., but also has several key new ingredients. Firstly, instead of applying the change of measure inequality on the loss function, we apply it to the generalization error function itself; secondly, the bound is derived in a conditional manner; lastly, a reference distribution, which bears a certain similarity to the prior distribution in the Bayesian setting, is introduced. The combination of these components produces a general KL-divergence-based generalization error bound. We further show that although the conditional bounding and the reference distribution can make the bound exactly tight, removing them does not significantly degrade the bound, which leads to a mutual-information-based bound that is also asymptotically tight in this setting.
Learning with Noisy labels via Self-supervised Adversarial Noisy Masking

Collecting large-scale datasets is crucial for training deep models, annotating the data, however, inevitably yields noisy labels, which poses challenges to deep learning algorithms. Previous efforts tend to mitigate this problem via identifying and removing noisy samples or correcting their labels according to the statistical properties (e.g., loss values) among training samples. In this paper, we aim to tackle this problem from a new perspective, delving into the deep feature maps, we empirically find that models trained with clean and mislabeled samples manifest distinguishable activation feature distributions. From this observation, a novel robust training approach termed adversarial noisy masking is proposed. The idea is to regularize deep features with a label quality guided masking scheme, which adaptively modulates the input data and label simultaneously, preventing the model to overfit noisy samples. Further, an auxiliary task is designed to reconstruct input data, it naturally provides noise-free self-supervised signals to rein-force the generalization ability of models. The proposed method is simple yet effective, it is tested on synthetic and real-world noisy datasets, where significant improvements are obtained over previous methods. Code is available at https://github.com/yuanpengtu/SANM.
Selective Structured State-Spaces for Long-Form Video Understanding

Effective modeling of complex spatiotemporal dependencies in long-form videos remains an open problem. The recently proposed Structured State-Space Sequence ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S4$</tex> ) model with its linear complexity offers a promising direction in this space. However, we demonstrate that treating all imagetokens equally as done by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S4$</tex> model can adversely affect its efficiency and accuracy. To address this limitation, we present a novel Selective <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S4$</tex> (i.e., <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S5)$</tex> model that employs a lightweight mask generator to adaptively select informative image tokens resulting in more efficient and accurate modeling of long-term spatiotemporal dependencies in videos. Unlike previous mask-based token reduction methods used in transformers, our <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S5$</tex> model avoids the dense self-attention calculation by making use of the guidance of the momentum-updated <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S4$</tex> model. This enables our model to efficiently discard less informative tokens and adapt to various long-form video understanding tasks more effectively. However, as is the case for most token reduction methods, the informative image tokens could be dropped incorrectly. To improve the robustness and the temporal horizon of our model, we propose a novel long-short masked contrastive learning (LSMCL) approach that enables our model to predict longer temporal context using shorter input videos. We present extensive comparative results using three challenging long-form video understanding datasets (LVU, COIN and Breakfast), demonstrating that our approach consistently outperforms the previous state-of-the-art S4 model by up to 9.6% accuracy while reducing its memory footprint by 23%.