Reviewerly/exHarmony huggingface dataset:
HoW-3D: Holistic 3D Wireframe Perception from a Single Image

This paper studies the problem of holistic 3D wireframe perception (HoW-3D), a new task of perceiving both the visible 3D wireframes and the invisible ones from single-view 2D images. As the non-front surfaces of an object cannot be directly observed in a single view, estimating the nonline-of-sight (NLOS) geometries in HoW-3D is a fundamentally challenging problem and remains open in computer vision. We study the problem of HoW-3D by proposing an ABC-HoW benchmark, which is created on top of CAD models sourced from the ABC-dataset with 12k single-view images and the corresponding holistic 3D wireframe models. With our large-scale ABC-HoW benchmark available, we present a novel Deep Spatial Gestalt (DSG) model to learn the visible junctions and line segments as the basis and then infer the NLOS 3D structures from the visible cues by following the Gestalt principles of human vision systems. In our experiments, we demonstrate that our DSG model performs very well in inferring the holistic 3D wireframes from single-view images. Compared with the strong baseline methods, our DSG model outperforms the previous wire-frame detectors in detecting the invisible line geometry in single-view images and is even very competitive with prior arts that take high-fidelity PointCloud as inputs on reconstructing 3D wireframes.
Dual-Space NeRF: Learning Animatable Avatars and Scene Lighting in Separate Spaces

Modeling the human body in a canonical space is a common practice for capturing and animation. But when involving the neural radiance field (NeRF), learning a static NeRF in the canonical space is not enough because the lighting of the body changes when the person moves even though the scene lighting is constant. Previous methods alleviate the inconsistency of lighting by learning a per-frame em-bedding, but this operation does not generalize to unseen poses. Given that the lighting condition is static in the world space while the human body is consistent in the canonical space, we propose a dual-space NeRF that models the scene lighting and the human body with two MLPs in two separate spaces. To bridge these two spaces, previous methods mostly rely on the linear blend skinning (LBS) algorithm. However, the blending weights for LBS of a dynamic neural field are intractable and thus are usually memorized with another MLP, which does not generalize to novel poses. Although it is possible to borrow the blending weights of a parametric mesh such as SMPL, the interpolation operation introduces more artifacts. In this paper, we propose to use the barycentric mapping, which can directly generalize to unseen poses and surprisingly achieves superior results than LBS with neural blending weights. Quantitative and qualitative results on the Human3.6M and the ZJU-MoCap datasets show the effectiveness of our method. Our code is available at: https://github.com/zyhbili/Dual-Space-NeRF.
Privacy in the Time of Language Models

Pretrained large language models (LLMs) have consistently shown state-of-the-art performance across multiple natural language processing (NLP) tasks. These models are of much interest for a variety of industrial applications that use NLP as a core component. However, LLMs have also been shown to memorize portions of their training data, which can contain private information. Therefore, when building and deploying LLMs, it is of value to apply privacy-preserving techniques that protect sensitive data.
A efficient and robust privacy-preserving framework for cross-device federated learning

Abstract To ensure no private information is leaked in the aggregation phase in federated learning (FL), many frameworks use homomorphic encryption (HE) to mask local model updates. However, the heavy overheads of these frameworks make them unsuitable for cross-device FL, where the clients are a huge number of mobile and edge devices with limited computing resources. Even worse, some of them also fail to manage the dynamic changes of clients. To overcome these shortcomings, we propose a threshold multi-key HE scheme tMK-CKKS and design an efficient and robust privacy-preserving FL framework. Robustness means that our framework allows clients to join in or drop out during the training process. Besides, because our tMK-CKKS scheme can pack multiple messages in a single ciphertext, our framework significantly reduces the computation and communication overhead. Moreover, the threshold mechanism in tMK-CKKS ensures that our framework can resist collusion attacks between the server and no more than t (threshold value) curious internal clients. Finally, we implement our framework in FedML and conduct extensive experiments to evaluate our framework. Utility evaluations on 6 benchmark datasets show that our framework can protect privacy without sacrificing the model accuracy. Efficiency evaluations on 4 typical deep learning models demonstrate that: our framework can speed up the computation by at least 1.21 $$\times $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>×</mml:mo> </mml:math> over xMK-CKKS-based framework, 15.84 $$\times $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>×</mml:mo> </mml:math> over Batchcrypt-based framework, and 20.30 $$\times $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>×</mml:mo> </mml:math> over CRT-Paillier-based framework. Our framework can reduce the communication burden by at least 8.61 MB over Batchcrypt-based framework, 35.36 MB over xMK-CKKS-based framework and 42.58 MB over CRT-Paillier-based framework. The advantages in both computation and communication expand with the size of deep learning models.
NP-hard but no longer hard to solve? Using quantum computing to tackle optimization problems

In the last decade, public and industrial research funding has moved quantum computing from the early promises of Shor's algorithm through experiments to the era of noisy intermediate scale quantum devices (NISQ) for solving real-world problems. It is likely that quantum methods can efficiently solve certain (NP-)hard optimization problems where classical approaches fail. In our perspective, we examine the field of quantum optimization where we solve optimisation problems using quantum computers. We demonstrate this through a proper use case and discuss the current quality of quantum computers, their solver capabilities, and benchmarking difficulties. Although we show a proof-of-concept rather than a full benchmark, we use the results to emphasize the importance of using appropriate metrics when comparing quantum and classical methods. We conclude with discussion on some recent quantum optimization breakthroughs and the current status and future directions.
A Cost-Effective Fall-Detection Framework for the Elderly Using Sensor-Based Technologies

Falls are critical events among the elderly living alone in their rooms and can have intense consequences, such as the elderly person being left to lie for a long time after the fall. Elderly falling is one of the serious healthcare issues that have been investigated by researchers for over a decade, and several techniques and methods have been proposed to detect fall events. To overcome and mitigate elderly fall issues, such as being left to lie for a long time after a fall, this project presents a low-cost, motion-based technique for detecting all events. In this study, we used IRA-E700ST0 pyroelectric infrared sensors (PIR) that are mounted on walls around or near the patient bed in a horizontal field of view to detect regular motions and patient fall events; we used PIR sensors along with Arduino Uno to detect patient falls and save the collected data in Arduino SD for classification. For data collection, 20 persons contributed as patients performing fall events. When a patient or elderly person falls, a signal of different intensity (high) is produced, which certainly differs from the signals generated due to normal motion. A set of parameters was extracted from the signals generated by the PIR sensors during falling and regular motions to build the dataset. When the system detects a fall event and turns on the green signal, an alarm is generated, and a message is sent to inform the family members or caregivers of the individual. Furthermore, we classified the elderly fall event dataset using five machine learning (ML) classifiers, namely: random forest (RF), decision tree (DT), support vector machine (SVM), naïve Bayes (NB), and AdaBoost (AB). Our result reveals that the RF and AB algorithms achieved almost 99% accuracy in elderly fall-d\detection.
Approaches That Use Domain-Specific Expertise: Behavioral-Cloning-Based Advantage Actor-Critic in Basketball Games

Research on the application of artificial intelligence (AI) in games has recently gained momentum. Most commercial games still use AI based on a finite state machine (FSM) due to complexity and cost considerations. However, FSM-based AI decreases user satisfaction given that it performs the same patterns of consecutive actions in the same situations. This necessitates a new AI approach that applies domain-specific expertise to existing reinforcement learning algorithms. We propose a behavioral-cloning-based advantage actor-critic (A2C) that improves learning performance by applying a behavioral cloning algorithm to an A2C algorithm in basketball games. The state normalization, reward function, and episode classification approaches are used with the behavioral-cloning-based A2C. The results of the comparative experiments with the traditional A2C algorithms validated the proposed method. Our proposed method using existing approaches solved the difficulty of learning in basketball games.
Fine-Grained Facial Expression Recognition in Multiple Smiles

Smiling has often been incorrectly interpreted as “happy” in the popular facial expression datasets (AffectNet, RAF-DB, FERPlus). Smiling is the most complex human expression, with positive, neutral, and negative smiles. We focused on fine-grained facial expression recognition (FER) and built a new smiling face dataset, named Facial Expression Emotions. This dataset categorizes smiles into six classes of smiles, containing a total of 11,000 images labeled with corresponding fine-grained facial expression classes. We propose Smile Transformer, a network architecture for FER based on the Swin Transformer, to enhance the local perception capability of the model and improve the accuracy of fine-grained face recognition. Moreover, a convolutional block attention module (CBAM) was designed, to focus on important features of the face image and suppress unnecessary regional responses. For better classification results, an image quality evaluation module was used to assign different labels to images with different qualities. Additionally, a dynamic weight loss function was designed, to assign different learning strategies according to the labels during training, focusing on hard yet recognizable samples and discarding unidentifiable samples, to achieve better recognition. Overall, we focused on (a) creating a novel dataset of smiling facial images from online annotated images, and (b) developing a method for improved FER in smiling images. Facial Expression Emotions achieved an accuracy of 88.56% and could serve as a new benchmark dataset for future research on fine-grained FER.
Simulation of Grover Search with Polar CH<sub>3</sub>CN Molecules by Optimal Control Fields

Abstract Theoretical schemes for the implementation of the Grover search algorithm are proposed based on ultracold polar molecules in an electric field. The molecular qubits and qudit are chosen as the field‐dressed states formed by the rotational modes of CH 3 CN. With the help of multi‐target optimal control theory, the microwave pulses for the elementary logic operations including the one‐qubit Hadamard gates and the conditional phase gates in a dipole‐dipole system are designed and the factorized Grover algorithm demonstrated using two coupled CH 3 CN molecules. To reduce the accumulation of imprecision and decoherence resulting from the combination of multiple elementary gates, the optimal pulses that can achieve the two‐qubit Hadamard gate and diffusion operation in one step are designed successfully. In this way, the probabilities to find the states corresponding to the desired elements in an unsorted database are enhanced to some extent. Moreover, the optimal pulse sequence suitable for a four‐level molecular qudit, which can be used to simulate the quantum search on a single CH 3 CN molecule with high fidelity, is designed. These results can shed some light on the physical realization of quantum computing based on ultracold polyatomic molecules.
Federated learning using game strategies: State-of-the-art and future trends

Federated learning (FL) is a new and promising paradigm that allows devices to learn without sharing data with the centralized server. It is often built on decentralized data where edge nodes use the internet of everything to mitigate the malicious attacks. The server gives incentive to all the participants according to their individual contributions. For profit maximization, each participating node balances between training rewards and costs. Game theory (GT) is a mathematical optimization technique that can be used to solve problems in wireless communication including security, resource allocation, power management, node rewards and punishments, and balancing numerous trade-offs. The complicated interactions between the server and the edge devices are interpreted using GT to maximize their utility. In this review article, we present an overview of the latest research on GT-based FL models for profit maximization, authentication, privacy management, trust management, and threat detection. This study also investigates the bibliometric analysis covering the period from 2019 to 2022 with an emphasis on various mechanisms of FL for GT applications. This article seeks to fill the gap by exploring the significant works highlighting the authors, citations, algorithms used, findings, and applications in this field. Based on the findings, we conclude this article with several key challenges and future approaches for researchers to implement an efficient GT-based FL model.
ICCL: Independent and Correlative Correspondence Learning for few-shot image classification

Few-shot learning, which aims to transfer knowledge from past experiences to recognize novel categories with limited samples, is a challenging task in computer vision. However, existing few-shot works tend to focus on determining the baseline model independently and ignoring the correlation learning among instances. In light of this, in this paper, we propose a novel approach, termed Independent and Correlative Correspondence Learning (ICCL), to deal with the few-shot image classification problem. To build independent learning, we supervise the different geometry directions of each instance under data augmentation and the category of the instance in the source domain space, which enhance the discrimination of features. For constructing the correlation learning, we utilize the transformer unit to yield the long-range relationship between the global and local representation and the N-pair metric to learn the intra-class and inter-class variations, which generate credible decision boundaries. By performing the operations of independent learning and correlation learning, the generalization ability of the model can be substantially enhanced for the unseen category. Comprehensive experiments on public few-shot image classification benchmarks demonstrate that our proposed ICCL gains state-of-the-art results, and different ingredients in ICCL reflect their effectiveness.
Double-counting in software engineering tertiary studies — An overlooked threat to validity

Double-counting in a literature review occurs when the same data, population, or evidence is erroneously counted multiple times during synthesis. Detecting and mitigating the threat of double-counting is particularly challenging in tertiary studies. Although this topic has received much attention in the health sciences, it seems to have been overlooked in software engineering. We describe issues with double-counting in tertiary studies, investigate the prevalence of the issue in software engineering, and propose ways to identify and address the issue. We analyze 47 tertiary studies in software engineering to investigate in which ways they address double-counting and whether double-counting might be a threat to validity in them. In 19 of the 47 tertiary studies, double-counting might bias their results. Of those 19 tertiary studies, only 5 consider double-counting a threat to their validity, and 7 suggest strategies to address the issue. Overall, only 9 of the 47 tertiary studies, acknowledge double-counting as a potential general threat to validity for tertiary studies. Double-counting is an overlooked issue in tertiary studies in software engineering, and existing design and evaluation guidelines do not address it sufficiently. Therefore, we propose recommendations that may help to identify and mitigate double-counting in tertiary studies.
A Systematic Literature Review on Robust Swarm Intelligence Algorithms in Search-Based Software Engineering

Swarm intelligence algorithms are metaheuristics inspired by the collective behavior of species such as birds, fish, bees, and ants. They are used in many optimization problems due to their simplicity, flexibility, and scalability. These algorithms get the desired convergence during the search by balancing the exploration and exploitation processes. These metaheuristics have applications in various domains such as global optimization, bioinformatics, power engineering, networking, machine learning, image processing, and environmental applications. This paper presents a systematic literature review (SLR) on applications of four swarm intelligence algorithms i.e., grey wolf optimization (GWO), whale optimization algorithms (WOA), Harris hawks optimizer (HHO), and moth-flame optimizer (MFO) in the field of software engineering. It presents an in-depth study of these metaheuristics’ adoption in the field of software engineering. This SLR is mainly comprised of three phases such as planning, conducting, and reporting. This study covers all related studies published from 2014 up to 2022. The study shows that applications of the selected metaheuristics have been utilized in various fields of software engineering especially software testing, software defect prediction, and software reliability. The study also points out some of the areas where applications of these swarm intelligence algorithms can be utilized. This study may act as a guideline for researchers in improving the current state-of-the-art on generally adopting these metaheuristics in software engineering.
Fast Real-Time Video Object Segmentation with a Tangled Memory Network

In this article, we present a fast real-time tangled memory network that segments the objects effectively and efficiently for semi-supervised video object segmentation (VOS). We propose a tangled reference encoder and a memory bank organization mechanism based on a state estimator to fully utilize the mask features and alleviate memory overhead and computational burden brought by the unlimited memory bank used in many memory-based methods. First, the tangled memory network exploits the mask features that uncover abundant object information like edges and contours but are not fully explored in existing methods. Specifically, a tangled two-stream reference encoder is designed to extract and fuse the features from both RGB frames and the predicted masks. Second, to indicate the quality of the predicted mask and feedback the online prediction state for organizing the memory bank, we devise a target state estimator to learn the IoU score between the predicted mask and ground truth. Moreover, to accelerate the forward process and avoid memory overflow, we use a memory bank of fixed size to store historical features by designing a new efficient memory bank organization mechanism based on the mask state score provided by the state estimator. We conduct comprehensive experiments on the public benchmarks DAVIS and YouTube-VOS, demonstrating that our method obtains competitive results while running at high speed (66 FPS on the DAVIS16-val set).
Smart contract-enabled consortium blockchains for the control of supply chain information distortion

Conflicts between supply chain members emerge because individual strategic actions may not be jointly optimal. Efforts to forecast consumer demand represent a source of conflict. The coordination of forecasts requires a powerful incentive alignment approach. This work proposes a smart contract equipped consortium blockchain system that creates an incentive structure that makes coordination with respect to forecasts economically appealing. Distortions of demand information due to uncoordinated forecasting are captured by a bullwhip measure that factors both forecast error and variance. Cooperation under the system is shown to help minimize this bullwhip measure, thus generating new outcomes for the participants that allow for a higher reward. Under a fixed payout structure, the system achieves credibility of continued cooperation, thus promoting an optimally coordinated equilibrium between the retailer and supplier. Blockchain technology represents a novel information system and consensus formation mechanism that can intermediate the behavior of a supply chain network.
Dual-Stream CNN-BiLSTM Model with Attention Layer for Automatic Modulation Classification

The Automatic Modulation Classification (AMC) approach is utilized to determine the nature of modulation. It is a significant job for intelligent receivers, essential components of future wireless communication technologies, including adaptive modulation and cognitive radio. Deep learning is often utilized for automatic modulation classification because of its superior performance and learning capability. A dual-stream CNN and BiLSTM architecture are used in this paper to classify seven different forms of digital modulation (8PSK, CPFSK, GFSK, QAM64, QPSK, BPSK, QAM16). Additionally, an attention layer is added to the model to enhance performance. Moreover, using dual streams rather than a single one improves the quality of the output. The proposed model is compared with earlier works. In comparison to the current research, it is observed that the proposed model provides superior classification accuracy with less complexity. The time needed for classification is also decreased due to the reduction in complexity.