Reviewerly/exHarmony huggingface dataset:
NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds

This paper proposes NeuralEditor that enables neural radiance fields (NeRFs) natively editable for general shape editing tasks. Despite their impressive results on novel-view synthesis, it remains a fundamental challenge for NeRFs to edit the shape of the scene. Our key insight is to exploit the explicit point cloud representation as the underlying structure to construct NeRFs, inspired by the intuitive interpretation of NeRF rendering as a process that projects or “plots” the associated 3D point cloud to a 2D image plane. To this end, NeuralEditor introduces a novel rendering scheme based on deterministic integration within K-D tree-guided density-adaptive voxels, which produces both high-quality rendering results and precise point clouds through optimization. NeuralEditor then performs shape editing via mapping associated points between point clouds. Extensive evaluation shows that NeuralEditor achieves state-of-the-art performance in both shape deformation and scene morphing tasks. Notably, NeuralEditor supports both zeroshot inference and further fine-tuning over the edited scene. Our code, benchmark, and demo video are available at immortalco.github.io/NeuralEditor.
Global-to-Local Modeling for Video-Based 3D Human Pose and Shape Estimation

Video-based 3D human pose and shape estimations are evaluated by intra-frame accuracy and inter-frame smoothness. Although these two metrics are responsible for different ranges of temporal consistency, existing state-of-the-art methods treat them as a unified problem and use monotonous modeling structures (e.g., RNN or attention-based block) to design their networks. However, using a single kind of modeling structure is difficult to balance the learning of short-term and long-term temporal correlations, and may bias the network to one of them, leading to undesirable predictions like global location shift, temporal inconsistency, and insufficient local details. To solve these problems, we propose to structurally decouple the modeling of long-term and short-term correlations in an end-to-end framework, Global-to-Local Transformer (GLoT). First, a global transformer is introduced with a Masked Pose and Shape Estimation strategy for long-term modeling. The strategy stimulates the global transformer to learn more inter-frame correlations by randomly masking the features of several frames. Second, a local transformer is responsible for exploiting local details on the human mesh and interacting with the global transformer by leveraging cross-attention. Moreover, a Hierarchical Spatial Correlation Regressor is further introduced to refine intra-frame estimations by decoupled global-local representation and implicit kinematic constraints. Our GLoT surpasses previous state-of-the-art methods with the lowest model parameters on popular benchmarks, i.e., 3DPW, MPI-INF-3DHP, and Human3.6M. Codes are available at https://github.com/sxl142/GLoT.
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning

Computer vision models suffer from a phenomenon known as catastrophic forgetting when learning novel concepts from continuously shifting training data. Typical solutions for this continual learning problem require extensive rehearsal of previously seen data, which increases memory costs and may violate data privacy. Recently, the emergence of large-scale pre-trained vision transformer models has enabled prompting approaches as an alternative to data-rehearsal. These approaches rely on a key-query mechanism to generate prompts and have been found to be highly resistant to catastrophic forgetting in the well-established rehearsal-free continual learning setting. However, the key mechanism of these methods is not trained end-to-end with the task sequence. Our experiments show that this leads to a reduction in their plasticity, hence sacrificing new task accuracy, and inability to benefit from expanded parameter capacity. We instead propose to learn a set of prompt components which are assembled with input-conditioned weights to produce input-conditioned prompts, resulting in a novel attention-based end-to-end key-query scheme. Our experiments show that we outperform the current SOTA method DualPrompt on established benchmarks by as much as 4.5% in average final accuracy. We also outperform the state of art by as much as 4.4% accuracy on a continual learning benchmark which contains both class-incremental and domain-incremental task shifts, corresponding to many practical settings. Our code is available at https://github.com/GT-RIPL/CODA-Prompt
Collaboration Helps Camera Overtake LiDAR in 3D Detection

Camera-only 3D detection provides an economical solution with a simple configuration for localizing objects in 3D space compared to LiDAR-based detection systems. However, a major challenge lies in precise depth estimation due to the lack of direct 3D measurements in the input. Many previous methods attempt to improve depth estimation through network designs, e.g., deformable layers and larger receptive fields. This work proposes an orthogonal direction, improving the camera-only 3D detection by introducing multi-agent collaborations. Our proposed collaborative camera-only 3D detection (CoCa3D) enables agents to share complementary information with each other through communication. Meanwhile, we optimize communication efficiency by selecting the most informative cues. The shared messages from multiple view-points disambiguate the single-agent estimated depth and complement the occluded and long-range regions in the single-agent view. We evaluate CoCa3D in one real-world dataset and two new simulation datasets. Results show that CoCa3D improves previous SOTA performances by 44.21% on DAIR-V2X, 30.60% on OPV2V+, 12.59% on CoPerception-UAVs+ for AP@70. Our preliminary results show a potential that with sufficient collaboration, the camera might overtake LiDAR in some practical scenarios. We released the dataset and code.
Identifying and preventing fraudulent responses in online public health surveys: Lessons learned during the COVID-19 pandemic

Web-based survey data collection has become increasingly popular, and limitations on in-person data collection during the COVID-19 pandemic have fueled this growth. However, the anonymity of the online environment increases the risk of fraudulent responses provided by bots or those who complete surveys to receive incentives, a major risk to data integrity. As part of a study of COVID-19 and the return to in-person school, we implemented a web-based survey of parents in Maryland between December 2021 and July 2022. Recruitment relied, in part, on social media advertisements. Despite implementing many existing best practices, we found the survey challenged by sophisticated fraudsters. In response, we iteratively improved survey security. In this paper, we describe efforts to identify and prevent fraudulent online survey responses. Informed by this experience, we provide specific, actionable recommendations for identifying and preventing online survey fraud in future research. Some strategies can be deployed within the data collection platform such as careful crafting of survey links, Internet Protocol address logging to identify duplicate responses, and comparison of client-side and server-side time stamps to identify responses that may have been completed by respondents outside of the survey's target geography. Other strategies can be implemented during the survey design phase. These approaches include the use of a 2-stage design in which respondents must be eligible on a preliminary screener before receiving a personalized link. Other design-based strategies include within-survey and cross-survey validation questions, the addition of "speed bump" questions to thwart careless or computerized responders, and the use of optional open-ended survey questions to identify fraudsters. We describe best practices for ongoing monitoring and post-completion survey data review and verification, including algorithms to expedite some aspects of data review and quality assurance. Such strategies are increasingly critical to safeguarding survey-based public health research.
Interpretable Multimodal Sentiment Classification Using Deep Multi-View Attentive Network of Image and Text Data

Multimodal data can convey user emotions and feelings more effectively and interactively than unimodal content. Thus, multimodal sentiment analysis (MSA) research has recently acquired great significance as a field of study. However, most current approaches either acquire sentimental features independently for each modality or simply combine multiple modal features. Thus, semantic details pertinent to sentiment analysis and the relationship between visual and textual content have been neglected. Furthermore, most available multimodal datasets are sentiment-annotated, although user emotions are usually rich and unlimited. Motivated by these observations, this paper proposes a novel deep multi-view attentive network (DMVAN) for robust multimodal sentiment and emotion classification. The DMVAN model has three phases: feature learning, attentive interaction learning, and cross-modal fusion learning. During the feature learning phase, visual features from a multi-view perspective (region and scene) and textual features from various levels of analysis (word, sentence, and document) are extracted to capture information effectively for accurate classification. In the attentive interaction learning phase, we employed the image-text interaction learning mechanism to enhance visual and textual information interaction by extracting sentimental and discriminative visual features and utilizing the textual information to guide the learning process of image features. Moreover, a cross-modal fusion learning module has been developed to integrate the various features into a comprehensive framework that leverages the complementary nature of multiple modalities, followed by a multi-head attention mechanism that is intended to extract sufficient information from the intermediate features, thereby enabling the creation of a robust joint representation. Finally, a multi-layer perceptron with multiple stacking-fully connected layers is used to deeply fuse the modal features, thereby enhancing sentiment classification performance. An interpretable multimodal sentiment classification model is further developed utilizing the local interpretable model-agnostic explanation model (LIME) to ensure the model’s explainability and strength. To perform a multimodal emotion classification, an image-text emotion dataset named emotion-Getty (EMO-G) was constructed from Getty Images and labeled by distinct emotions. The proposed model was tested on three real-world datasets, attaining 99.801% accuracy on binary_Getty (BG), 96.867% on Twitter, and 96.174% on the EMO-G dataset. These results show that the suggested model outperforms single-model techniques and current state-of-the-art methodologies based on model evaluation criteria.
A quantum homomorphic signature scheme with verifiable identity based on four-particle Cluster states

Abstract With the development of cloud computing in recent years, homomorphic signatures have become a research hotspot in modern cryptography. This article proposes a quantum homomorphic signature scheme with verifiable identity based on four-particle Cluster states, starting from the identity verification of the signers. Compared with Shang’s scheme (Shang et al 2015 Quantum Inf. Process. 14 393–410), our scheme uses four-particle Cluster states as quantum channels and achieves identity verification of the signers through the use of X operation and quantum measurement techniques. Meanwhile, our scheme extends the length of message M to 2 n bits and uses random numbers during the signature process, which makes the scheme more secure. Security analysis shows that our scheme can ensure the security of the keys and the unforgeability of the signatures.
On the Benefits of 3D Pose and Tracking for Human Action Recognition

In this work we study the benefits of using tracking and 3D poses for action recognition. To achieve this, we take the Lagrangian view on analysing actions over a trajectory of human motion rather than at a fixed point in space. Taking this stand allows us to use the tracklets of people to predict their actions. In this spirit, first we show the benefits of using 3D pose to infer actions, and study person-person in-teractions. Subsequently, we propose a Lagrangian Action Recognition model by fusing 3D pose and contextualized appearance over tracklets. To this end, our method achieves state-of-the-art performance on the AVA v2.2 dataset on both pose only settings and on standard benchmark settings. When reasoning about the action using only pose cues, our pose model achieves +10.0 mAP gain over the corresponding state-of-the-art while our fused model has a gain of +2.8 mAP over the best state-of-the-art model. Code and results are available at: https://brjathu.github.io/LART
Federated Incremental Semantic Segmentation

Federated learning-based semantic segmentation (FSS) has drawn widespread attention via decentralized training on local clients. However, most FSS models assume categories are fixed in advance, thus heavily undergoing forgetting on old categories in practical applications where local clients receive new categories incrementally while have no memory storage to access old classes. Moreover, new clients collecting novel classes may join in the global training of FSS, which further exacerbates catastrophic forgetting. To surmount the above challenges, we propose a Forgetting-Balanced Learning (FBL) model to address heterogeneous forgetting on old classes from both intra-client and interclient aspects. Specifically, under the guidance of pseudo labels generated via adaptive class-balanced pseudo labeling, we develop a forgetting-balanced semantic compensation loss and a forgetting-balanced relation consistency loss to rectify intra-client heterogeneous forgetting of old categories with background shift. It performs balanced gradient propagation and relation consistency distillation within local clients. Moreover, to tackle heterogeneous forgetting from inter-client aspect, we propose a task transition monitor. It can identify new classes under privacy protection and store the latest old global model for relation distillation. Qualitative experiments reveal large improvement of our model against comparison methods. The code is available at https://github.com/JiahuaDong/FISS.
Extracting Class Activation Maps from Non-Discriminative Features as well

Extracting class activation maps (CAM) from a classification model often results in poor coverage on foreground objects, i.e., only the discriminative region (e.g., the “head” of “sheep”) is recognized and the rest (e.g., the “leg” of “sheep”) mistakenly as background. The crux behind is that the weight of the classifier (used to compute CAM) captures only the discriminative features of objects. We tackle this by introducing a new computation method for CAM that explicitly captures non-discriminative features as well, thereby expanding CAM to cover whole objects. Specifically, we omit the last pooling layer of the classification model, and perform clustering on all local features of an object class, where “local” means “at a spatial pixel position”. We call the resultant K cluster centers local prototypes — represent local semantics like the “head”, “leg”, and “body” of “sheep”. Given a new image of the class, we compare its unpooled features to every prototype, derive K similarity matrices, and then aggregate them into a heatmap (i.e., our CAM). Our CAM thus captures all local features of the class without discrimination. We evaluate it in the challenging tasks of weakly-supervised semantic segmentation (WSSS), and plug it in multiple state-of-the-art WSSS methods, such as MCTformer [45] and AMN [26], by simply replacing their original CAM with ours. Our extensive experiments on standard WSSS benchmarks (PASCAL VOC and MS COCO) show the superiority of our method: consistent improvements with little computational overhead. Our code is provided at https://github.com/zhaozhengChen/LPCAM.
A New Framework for Fraud Detection in Bitcoin Transactions Through Ensemble Stacking Model in Smart Cities

Bitcoin has a reputation of being used for unlawful activities, such as money laundering, dark web transactions, and payments for ransomware in the context of smart cities. Blockchain technology prevents illegal transactions, but cannot detect these transactions. Anomaly detection is a fundamental technique for recognizing potential fraud. The heuristic and signature-based approaches were the foundation of earlier detection techniques, but tragically, these methods were insufficient to explore the entire complexity of anomaly detection. Machine Learning (ML) is a promising approach to anomaly detection, as it can be trained on large datasets of known malware samples to identify patterns and features of the transactions. Researchers are focusing on determining an efficient fraud and security threat detection model that overcomes the drawbacks of the existing methods. Therefore, ensemble learning can be applied to anomaly detection in Bitcoin by combining multiple ML classifiers. In the proposed model, the ADASYN-TL (Adaptive Synthetic + Tomek Link) balancing technique is used for data balancing. Random search, grid search and Bayesian optimization are used for hyperparameter tuning. The hyperparameters have a great impact on the performance of the model. For classification, we used the stacking model by combining Decision Tree, Naive Bayes, K-Nearest Neighbors, and Random Forest. We used SHapley Additive exPlanation (SHAP) to interpret the predictions of the stacking model. The model also explores the performance of different classifiers using accuracy, F1-score, Area Under Curve-Receiver Operating Characteristic (AUC-ROC), precision, recall, False Positive Rate (FPR) and execution time, and ultimately selects the ideal model. The proposed model contributes to the development of effective fraud detection models that address the limitations of the existing algorithms. Our stacking model, which combines the prediction of multiple classifiers, achieved the highest F1-score of 97%, precision of 96%, recall of 98%, accuracy of 97%, AUC-ROC of 99% and FPR of 3%.
Reveal the correlation between randomness and Lyapunov exponent of n-dimensional non-degenerate hyper chaotic map

In this paper, we investigate the correlation between the Lyapunov exponent (LE) and the randomness in chaotic maps. The recently constructed chaotic map has certain limitations when used in cryptography. These limitations include the lack of ergodicity in phase space, weak positive LE, and poor randomness. To overcome these limitations and reveal the correlation between LE and randomness, we first constructed a generic n-dimensional non-degenerate hyper chaotic map (nD-NDCM) model with adjustable LE through the eigenvalues of the Jacobian matrix. Furthermore, we instantiated 2D-NDCM and 3D-NDCM based on this model, and then we used the bifurcation diagrams, phase diagrams, and LE to analyze their dynamic performance. Additionally, we used NIST SP800-22 and TestU01 to analyze the iterative sequence and determine the corresponding LE interval with good randomness. Utilizing the established correlation, we can directly and feasibly construct any multi-dimensional chaotic map with good randomness.
A methodology to determine the optimal train-set size for autoencoders applied to energy systems

In the latest years, deep learning has been massively used to face problems that have not been solved by means of classical approaches. In particular, an autoencoder is a popular unsupervised artificial neural network that learns efficient data representations (encoding) by training the network to ignore features with a small content of information. Even though autoencoders over-perform classical techniques in several applications like anomaly detection, dimensionality reduction, features denoising, and missing values imputation, the literature does not provide a commonly accepted methodology to define the optimal amount of data needed to train the model. This paper proposes a procedure to determine the optimal train-set size to minimize the reconstruction error of an autoencoder with pre-defined structure and hyper-parameters that will be trained to encode the normal behavior of energy generation systems. This procedure exploits the outcome of learning curves, a powerful tool to track algorithms performance while the train-set dimension varies. Afterward, the procedure is applied to three real case studies where two types of autoencoders are trained to learn the normal behavior of a YANMAR combined heat and power unit with the scope of detecting incoming anomalies. In the end, the outcomes of the procedure are explained and, under the constraint of a daily retraining frequency, 6 weeks are identified as the optimal train-set size for both autoencoders.
Entanglement Parallelization via Quantum Fourier Transform

Abstract In this study, a technique based on the quantum Fourier transform (QFT) that allows the generation of disjoint sets of entangled particles is presented, in such a way that particles of the same set are entangled with each other, while particles of different sets are completely independent. Several applications of this technique are implemented on three physical platforms, of 5 (Belem), 7 (Oslo), and 14 (Melbourne) qubits, of the international business machine (IBM Q) quantum experience program, where all these applications are specially selected due to their particular commitment to the future Quantum Internet.
Marie and BERT─A Knowledge Graph Embedding Based Question Answering System for Chemistry

This paper presents a novel knowledge graph question answering (KGQA) system for chemistry, which is implemented on hybrid knowledge graph embeddings, aiming to provide fact-oriented information retrieval for chemistry-related research and industrial applications. Unlike other existing designs, the system operates on multiple embedding spaces, which use various embedding methods and queries the embedding spaces in parallel. With the answers returned from multiple embedding spaces, the system leverages a score alignment model to adjust the answer scores and rerank the answers. Further, the system implements an algorithm to derive implicit multihop relations to handle the complexities of deep ontologies and improve multihop question answering. The system also implements a BERT-based bidirectional entity-linking model to enhance the robustness and accuracy of the entity-linking module. The system uses a joint numerical embedding model to efficiently handle numerical filtering questions. Further, it can invoke semantic agents to perform dynamic calculations autonomously. Finally, the KGQA system handles numerous chemical reaction mechanisms using semantic parsing supported by a Linked Data Fragment server. This paper evaluates the accuracy of each module within the KGQA system with a chemistry question data set.