Reviewerly/exHarmony huggingface dataset:
BUS : Efficient and Effective Vision-language Pre-training with Bottom-Up Patch Summarization

Vision Transformer (ViT) based Vision-Language Pre-training (VLP) models have demonstrated impressive performance in various tasks. However, the lengthy visual token sequences fed into ViT can lead to training inefficiency and ineffectiveness. Existing efforts address the challenge by either bottom-level patch extraction in the ViT backbone or top-level patch abstraction outside, not balancing training efficiency and effectiveness well. Inspired by text summarization in natural language processing, we propose a Bottom-Up Patch Summarization approach named BUS, coordinating bottom-level extraction and top-level abstraction to learn a concise summary of lengthy visual token sequences efficiently. Specifically, We incorporate a Text-Semantics-Aware Patch Selector (TSPS) into the ViT backbone to perform a coarse-grained visual token extraction and then attach a flexible Transformer-based Patch Abstraction Decoder (PAD) upon the backbone for top-level visual abstraction. This bottom-up collaboration enables our BUS to yield high training efficiency while maintaining or even improving effectiveness. We evaluate our approach on various visual-language understanding and generation tasks and show competitive downstream task performance while boosting the training efficiency by 50%. Additionally, our model achieves state-of-the-art performance on many downstream tasks by increasing input image resolution without increasing computational costs over baselines.
3DPPE: 3D Point Positional Encoding for Transformer-based Multi-Camera 3D Object Detection

Transformer-based methods have swept the benchmarks on 2D and 3D detection on images. Because tokenization before the attention mechanism drops the spatial information, positional encoding becomes critical for those methods. Recent works found that encodings based on samples of the 3D viewing rays can significantly improve the quality of multi-camera 3D object detection. We hypothesize that 3D point locations can provide more information than rays. Therefore, we introduce 3D point positional encoding, 3DPPE, to the 3D detection Transformer decoder. Although 3D measurements are not available at the inference time of monocular 3D object detection, 3DPPE uses predicted depth to approximate the real point positions. Our hybrid-depth module combines direct and categorical depth to estimate the refined depth of each pixel. Despite the approximation, 3DPPE achieves 46.0 mAP and 51.4 NDS on the competitive nuScenes dataset, significantly outperforming encodings based on ray samples. The code is available at https://github.com/drilistbox/3DPPE.
Task Agnostic Restoration of Natural Video Dynamics

In many video restoration/translation tasks, image processing operations are naïvely extended to the video domain by processing each frame independently, disregarding the temporal connection of the video frames. This disregard for the temporal connection often leads to severe temporal inconsistencies. State-Of-The-Art (SOTA) techniques that address these inconsistencies rely on the availability of unprocessed videos to implicitly siphon and utilize consistent video dynamics to restore the temporal consistency of frame-wise processed videos which often jeopardizes the translation effect. We propose a general framework for this task that learns to infer and utilize consistent motion dynamics from inconsistent videos to mitigate the temporal flicker while preserving the perceptual quality for both the temporally neighboring and relatively distant frames without requiring the raw videos at test time. The proposed framework produces SOTA results on two benchmark datasets, DAVIS and videvo.net, processed by numerous image processing applications. The code and the trained models are available at https://github.com/MKashifAli/TARONVD.
H3WB: Human3.6M 3D WholeBody Dataset and Benchmark

We present a benchmark for 3D human whole-body pose estimation, which involves identifying accurate 3D keypoints on the entire human body, including face, hands, body, and feet. Currently, the lack of a fully annotated and accurate 3D whole-body dataset results in deep networks being trained separately on specific body parts, which are combined during inference. Or they rely on pseudo-groundtruth provided by parametric body models which are not as accurate as detection based methods. To overcome these issues, we introduce the Human3.6M 3D WholeBody (H3WB) dataset, which provides whole-body annotations for the Human3.6M dataset using the COCO Wholebody layout. H3WB comprises 133 whole-body keypoint annotations on 100K images, made possible by our new multi-view pipeline. We also propose three tasks: i) 3D whole-body pose lifting from 2D complete whole-body pose, ii) 3D whole-body pose lifting from 2D incomplete whole-body pose, and iii) 3D whole-body pose estimation from a single RGB image. Additionally, we report several baselines from popular methods for these tasks. Furthermore, we also provide automated 3D whole-body annotations of TotalCapture and experimentally show that when used with H3WB it helps to improve the performance.
Beyond Object Recognition: A New Benchmark towards Object Concept Learning

Understanding objects is a central building block of AI, especially for embodied AI. Even though object recognition excels with deep learning, current machines struggle to learn higher-level knowledge, e.g., what attributes an object has, and what we can do with it. Here, we propose a challenging Object Concept Learning (OCL) task to push the envelope of object understanding. It requires machines to reason out affordances and simultaneously give the reason: what attributes make an object possess these affordances. To support OCL, we build a densely annotated knowledge base including extensive annotations for three levels of object concept (category, attribute, affordance), and the clear causal relations of three levels. By analyzing the causal structure of OCL, we present a baseline, Object Concept Reasoning Network (OCRN). It leverages concept instantiation and causal intervention to infer the three levels. In experiments, OCRN effectively infers the object knowledge while following the causalities well. Our data and code are available at https://mvig-rhos.com/ocl.
Blockchain and Deep Learning-Based Models for Crypto Price Forecasting

Blockchain technology is becoming more and more popular because of its decentralized, unchangeable, safe, and anonymous characteristics. Cryptocurrencies, which have seen tremendous development and market capitalization in recent years, are where they are most widely used. Individuals, organizations, and businesses are all making significant investments in this area. In contrast to conventional commodities markets, the cryptocurrency market is more volatile, unstable, and unpredictable. It can be affected by a variety of technological, emotional, and legal variables. Although there has been a lot of research towards predicting cryptocurrency prices, many of the current methods are not appropriate for use in real time. To solve this, we suggest using the Python-based LGBM Neural network model to forecast market changes in real-time and reduce investor losses. The outcomes of our evaluation of this strategy using reference datasets show its dependability. Thus, over the past ten years, researchers from a variety of domains have used neural networks as an efficient data mining method.
Federated Learning Over Images: Vertical Decompositions and Pre-Trained Backbones Are Difficult to Beat

We carefully evaluate a number of algorithms for learning in a federated environment, and test their utility for a variety of image classification tasks. We consider many issues that have not been adequately considered before: whether learning over data sets that do not have diverse sets of images affects the results; whether to use a pre-trained feature extraction "backbone"; how to evaluate learner performance (we argue that classification accuracy is not enough), among others. Overall, across a wide variety of settings, we find that vertically decomposing a neural network seems to give the best results, and outperforms more standard reconciliation-used methods.
On the Study of Partial Qubit Hamiltonian for Efficient Molecular Simulation Using Variational Quantum Eigensolvers

Quantum computing is being extensively used in quantum chemistry, especially in simulating simple molecules and evaluating properties like the ground state energy, dipole moment, etc. The transformation of a molecular Hamiltonian from the fermionic space to the qubit space provides us with a series of Pauli strings and the energy calculation involves the evaluation of the expectation values of all these individual strings. This introduces a major bottleneck for applications of VQEs in quantum chemistry. Unlike the fermionic Hamiltonian, the terms in a qubit Hamiltonian are additive and the present paper exploits this property to describe a new approach for extracting information from the partial qubit Hamiltonian of simple molecules to design more efficient variational quantum eigensolvers. In the partial (qubit) Hamiltonian approach (PHA), the qubit Hamiltonian is studied term-by-term to understand their relative contributions to the overall energy, and a partial Hamiltonian is constructed with fewer Pauli strings that can resolve the entire Hamiltonian. With PHA, we can simulate molecules at a much lower computational cost with a truncated Hamiltonian. Additionally, the outcomes of the measurements with PHA quench the error due to noise introduced by the quantum circuits. We have also demonstrated the application of PHA as an initialization technique, where the simple partial Hamiltonian can be used to find an optimal starting point for a more complex system.
Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigation

Cross-modal alignment is one key challenge for Vision-and-Language Navigation (VLN). Most existing studies concentrate on mapping the global instruction or single sub-instruction to the corresponding trajectory. However, another critical problem of achieving fine-grained alignment at the entity level is seldom considered. To address this problem, we propose a novel Grounded Entity-Landmark Adaptive (GELA) pre-training paradigm for VLN tasks. To achieve the adaptive pre-training paradigm, we first introduce grounded entity-landmark human annotations into the Room-to-Room (R2R) dataset, named GEL-R2R. Additionally, we adopt three grounded entity-landmark adaptive pre-training objectives: 1) entity phrase prediction, 2) landmark bounding box prediction, and 3) entity-landmark semantic alignment, which explicitly supervise the learning of fine-grained cross-modal alignment between entity phrases and environment landmarks. Finally, we validate our model on two downstream benchmarks: VLN with descriptive instructions (R2R) and dialogue instructions (CVDN). The comprehensive experiments show that our GELA model achieves state-of-the-art results on both tasks, demonstrating its effectiveness and generalizability.
Towards Attack-tolerant Federated Learning via Critical Parameter Analysis

Federated learning is used to train a shared model in a decentralized way without clients sharing private data with each other. Federated learning systems are susceptible to poisoning attacks when malicious clients send false updates to the central server. Existing defense strategies are ineffective under non-IID data settings. This paper proposes a new defense strategy, FedCPA (Federated learning with Critical Parameter Analysis). Our attack-tolerant aggregation method is based on the observation that benign local models have similar sets of top-k and bottom-k critical parameters, whereas poisoned local models do not. Experiments with different attack scenarios on multiple datasets demonstrate that our model outperforms existing defense strategies in defending against poisoning attacks.
Improving Representation Learning for Histopathologic Images with Cluster Constraints

Recent advances in whole-slide image (WSI) scanners and computational capabilities have significantly propelled the application of artificial intelligence in histopathology slide analysis. While these strides are promising, current supervised learning approaches for WSI analysis come with the challenge of exhaustively labeling high-resolution slides—a process that is both labor-intensive and time-consuming. In contrast, self-supervised learning (SSL) pretraining strategies are emerging as a viable alternative, given that they don’t rely on explicit data annotations. These SSL strategies are quickly bridging the performance disparity with their supervised counterparts. In this context, we introduce an SSL framework. This framework aims for transferable representation learning and semantically meaningful clustering by synergizing invariance loss and clustering loss in WSI analysis. Notably, our approach outperforms common SSL methods in downstream classification and clustering tasks, as evidenced by tests on the Came-lyon16 and a pancreatic cancer dataset. The code and additional details are accessible at https://github.com/wwyi1828/CluSiam.
Testing the Accuracy of Surface Code Decoders

Large-scale, fault-tolerant quantum computations will be enabled by quantum error-correcting codes (QECC). This work presents the first systematic technique to test the accuracy and effectiveness of different QECC decoding schemes by comparing a look-up table decoder to solutions generated using algorithmic decoders. Specifically, we examine the results of minimum-weight-perfect-matching and belief-propagation decoders against exhaustive look-up tables for surface codes up to distance seven and categorise where errors are accurately corrected in both decoding schemes. While our results are preliminary, we show that significant quantitative results can be generated, comparing how actual error channels are successfully or unsuccessfully decoded. We show that different decoding schemes perform very differently under the same QECC scheme and error model, and detail how decoders can be tested and classified with respect to errors that are successfully decodable. This work paves the way to the data driven tuning of decoder ensembles and will enable tailored design of hybrid decoding schemes that allow for real-time decoding, while maintaining the high theoretical thresholds allowed by specific quantum error correction codes.
Quantum Array Multiplier

This paper proposes a new and improved implementation of a quantum integer multiplier. Performing arithmetic computations is sometimes a necessary step in the implementation of quantum algorithms. In this work, Quantum Fourier Transform is used in order to perform scalable arithmetic in a generic bit-width quantum system. In the phase domain, addition can be implemented through accumulated controlled rotations on the qubits’ state. Leveraging this, and inspired by the classical implementation of an array multiplier, a new integer multiplier is fully designed and tested in a quantum environment.The depth of a quantum circuit is the number of computational steps necessary to completion, and it is a key parameter that reflects on the performance of the design. The new design reduces the quantum depth of the design from the exponential order of the previously proposed designs to polynomial order.
Optimal Clifford Initial States for Ising Hamiltonians

Modern day quantum devices are in the NISQ era, meaning that the effects of size restrictions and noise are essential considerations when developing practical quantum algorithms. Recent developments have demonstrated that Variational Quantum Algorithms (VQAs) are an appropriate choice for current era quantum devices. VQAs make use of classical computation to iteratively optimize parameters of a quantum circuit, referred to as the ansatz. These parameters are usually chosen such that a given objective function is minimized. Generally, the cost function to be minimized is computed on a quantum device, and then the parameters are updated using a classical optimizer. One of the most promising VQAs for practical hardware is the Variational Quantum Eigensolver (VQE). VQE uses the expectation value of a given Hamiltonian as the cost function, and the minimal value of this particular cost function corresponds to the minimal eigenvalue of the Hamiltonian.Because evaluating quantum circuits is currently very noisy, developing classical bootstraps that help minimize the number of times a given circuit has to be evaluated is a powerful technique for improving the practicality of VQE. One possible such bootstrapping method is creating an ansatz which can be efficiently simulated on classical computers for restricted parameter values. Once the optimal set of restricted parameters is determined, they can be used as initial parameters for a VQE optimization which has access to the full parameter space. Stabilizer states are states which are generated by a particular group of operators called the Clifford Group. Because of the underlying structure of these operators, circuits consisting of only Clifford operators can be simulated efficiently on classical computers. Clifford Ansatz For Quantum Algorithms (CAFQA) is a proposed classical bootstrap for VQAs that uses ansatzes which reduce to clifford operators for restricted parameter values [1].CAFQA has been shown to produce fairly accurate initialization for VQE applied to molecular Hamiltonians. Motivated by this result, in this paper we seek to analyze the Clifford states that optimize the cost function for a new type of Hamiltonian, namely Transverse Field Ising Hamiltonians. Our primary result is contained in theorem IV.1 which connects the problem of finding the optimal CAFQA initialization to a submodular minimization problem which in turn can be solved in polynomial time.
Optimization of automatic entry function of financial accounting system based on GA algorithm

In view of the low efficiency and low accuracy of paper reimbursement documents in the financial accounting system, the paper proposes the design of paper reimbursement documents based on image recognition technology to be automatically input into the financial system. By scanning and identifying the image information of paper bills, the system realizes the automatic input function of the financial accounting system. In practical work, the system consumes a computer greatly based on the distributed hash table, so it is proposed to improve the genetic algorithm. It is found that the advantage of the DHT load algorithm improved with the GA algorithm is more obvious in terms of the average response time and throughput of the system. When the number of connections increased to 100, the data throughput of the improved DHT load algorithm of the GA algorithm reached about 7.78 MB/s, while the original DHT load algorithm was only 5.46 MB / s, and the throughput efficiency of the improved algorithm increased by 42.49%. In the process of identifying target search, the improved recognition algorithm has the least number and time consuming of pixel search compared with other algorithms. In terms of financial personnel satisfaction, 90.62% of financial personnel are satisfied with the response speed and work efficiency of the system, and 87.74% of financial personnel are satisfied with the humanized operation of the system. It shows that the improved financial system improves the efficiency and accuracy of data processing, and has relatively obvious advantages, which can make the financial processing process of large enterprises more convenient.
A deep learning-based multi-language document layout analysis method

Layout analysis of complex layout documents is significant for document understanding systems and is still a challenging task. A deep learning-based layout analysis method is proposed for contemporary printed documents in this paper. The network is developed from Mask-RCNN and the standard convolution structure in Mask-RCNN is replaced by deformable convolution network (DCN) to improve the location accuracy of the segmentation contour. Transfer learning is employed to solve the few-shot problem and the fine-tune mode is investigated. The network is pre-trained on the PubLayNet dataset and fine-tuned on the ICDAR2009 dataset. Experiments show that the proposed method outperforms the tested traditional method and is insensitive to language.
Self-Supervised Burst Super-Resolution

We introduce a self-supervised training strategy for burst super-resolution that only uses noisy low-resolution bursts during training. Our approach eliminates the need to carefully tune synthetic data simulation pipelines, which often do not match real-world image statistics. Compared to weakly-paired training strategies, which require noisy smartphone burst photos of static scenes, paired with a clean reference obtained from a tripod-mounted DSLR camera, our approach is more scalable, and avoids the color mismatch between the smartphone and DSLR. To achieve this, we propose a new self-supervised objective that uses a forward imaging model to recover a high-resolution image from aliased high frequencies in the burst. Our approach does not require any manual tuning of the forward model's parameters; we learn them from data. Furthermore, we show our training strategy is robust to dynamic scene motion in the burst, which enables training burst super-resolution models using in-the-wild data. Extensive experiments on real and synthetic data show that, despite only using noisy bursts during training, models trained with our self-supervised strategy match, and sometimes surpass, the quality of fully-supervised baselines trained with synthetic data or weakly-paired ground-truth. Finally, we show our training strategy is general using four different burst super-resolution architectures.
Transformation of questioning subjects in high school English classroom under the thinking quality

Classroom questioning is an important way to develop students' thinking ability. For a long time, high school English teachers have occupied the right to ask questions in the classroom, preventing students from becoming the main body of classroom questioning, making it difficult to cultivate and improve the students' thinking quality. It is of great significance to change the subject of classroom questioning from teachers to students. Under the perspective of classroom questioning, various strategies should be adopted to realize the transformation of classroom subject and the implementation of the core literacy of the English discipline.
Determining Students' Nutritional Status Using Mamdani Fuzzy Logic Method

Nutritional status is a body condition as a result of food consumption or a measure of the success of nutritional fulfillment; there is a balance between the amount of nutrient intake and the amount required by the body for a variety of biological functions such as physical growth, development, activity or productivity, health maintenance, and others. Children’s nutritional status is indicated by weight and height [6]. Theoretically, nutritional status can be determined based on Anthropometric standards. Fuzzy logic is a method to show problems from input to expected output [4]. The Mamdani fuzzy method is one of the methods of the fuzzy inference system. This research uses Matlab R2021b software to apply the fuzzy method