Reviewerly/exHarmony huggingface dataset:
An Efficient Data Storage Technique for User Files in Cloud

A remote data storage service provided by cloud providers is known as cloud storage. It substitutes an online-accessible central storage repository for the need for nearby physical storage devices. Cloud storage is efficient due to its wide range of accessibility. An efficient approach for managing file storage in cloud computing environments is presented in this paper. The proposed method utilizes the concept of dividing files into blocks and combining them to generate file hashes using Merkle hash tree (MHT). These file hashes facilitate quick and reliable data integrity verification. Upon transmission of multiple files to the Cloud Service Provider (CSP), a secure storage process is implemented within the cloud environment. The data packets are organized and managed through a $B+$ tree structure. The proposed scheme also supports dynamic file operations. Where users can modify files, and for each modification, a new file hash is generated by merging the block hashes through MHT. By combining the efficiency of Merkle hash tree and B+ tree structures, this proposed system offers a dependable solution for effectively managing files in cloud storage. This method makes a significant contribution to the fields of cloud computing and file management by improving data security, integrity, and accessibility.
Attention U-Net: A Deep Learning Approach for Breast Cancer Segmentation

One of the main causes of death for women globally is breast cancer. Early discovery can reduce the frequency of early deaths. Breast ultrasound images can help in diagnosis and treatment planning by helping to categories different types of breast cancer. To segment breast cancer pictures in this study, we employed an Attention U-Net. A deep learning model for picture segmentation is called the Attention U-Net. We used a dataset of 780 breast ultrasound pictures divided into three classes-normal, benign, and malignant-to train the Attention U-Net. The mean IoU, precision, recall, and F1 scores for the Attention U-Net were 0.840,0.815,0.848, and 0.831 respectively. These outcomes are comparable to those of other cutting-edge techniques for segmenting breast cancer. The findings of this study indicate that the Attention U-Net is a potential technique for segmenting breast cancer. The great accuracy that the Attention U-Net can attain can assist to increase the precision of diagnosis and treatment planning
A Fast Image Encryption Technique Based on Chaotic Maps and Image Divisional Techniques for Multiple Applications

The proliferation of communication networks has led to a surge in the exchange of multimedia information through unsecured channels. Therefore, it is necessary to safeguard multimedia images during transmission through unsecured communication channels. The paper aims to propose a scheme for fast encryption of images that utilizes chaos-based maps and image divisional techniques. The method involves the initial division of the image into four sub-images. Next, a pixel-permutation process is executed on the pixels of each sub-image utilising four distinct indexing sequences based on chaotic maps. The permutated outputs are subjected to a pixel diffusion operation utilising four distinct key images based on chaotic maps. Finally, the encrypted output is produced by joining all the dispersed outputs. Based on the computer simulations and analysis on security, it has been determined that the suggested method is competent of securely and efficiently encrypting images.
Robust Mixture-of-Expert Training for Convolutional Neural Networks

Sparsely-gated Mixture of Expert (MoE), an emerging deep model architecture, has demonstrated a great promise to enable high-accuracy and ultra-efficient model inference. Despite the growing popularity of MoE, little work investigated its potential to advance convolutional neural networks (CNNs), especially in the plane of adversarial robustness. Since the lack of robustness has become one of the main hurdles for CNNs, in this paper we ask: How to adversarially robustify a CNN-based MoE model? Can we robustly train it like an ordinary CNN model? Our pilot study shows that the conventional adversarial training (AT) mechanism (developed for vanilla CNNs) no longer remains effective to robustify an MoE-CNN. To better understand this phenomenon, we dissect the robustness of an MoE-CNN into two dimensions: Robustness of routers (i.e., gating functions to select data-specific experts) and robustness of experts (i.e., the router-guided pathways defined by the subnetworks of the backbone CNN). Our analyses show that routers and experts are hard to adapt to each other in the vanilla AT. Thus, we propose a new router-expert alternating Adversarial training framework for MoE, termed AdvMoE. The effectiveness of our proposal is justified across 4 commonly-used CNN model architectures over 4 benchmark datasets. We find that AdvMoE achieves 1% ~ 4% adversarial robustness improvement over the original dense CNN, and enjoys the efficiency merit of sparsity-gated MoE, leading to more than 50% inference cost reduction. Codes are available at https://github.com/OPTML-Group/Robust-MoE-CNN.
A Diffusion-Only Image Encryption System for High-Speed Applications

The widespread and growing utilisation of digital images has made image encryption a crucial concern, garnering significant attention from researchers. Regrettably, the development and study of fast encryption schemes for colour images are currently lacking. This paper proposes a specifically designed colour image encryption system for high-speed applications, utilising only diffusion techniques. The proposed technique employs a set of five Piece-wise Linear Chaotic Maps (PWLCM) with different first values and control factors to execute the image encryption operation. The SHA-256 hashing is employed for the initialization of first values and control factors in PWLCM system. The technique involves initially separating the colour image into its three colour components. Subsequently, the three components are horizontally concatenated. Next, the generation of three key images is achieved by employing the three PWLCM systems, each with distinct key parameters. Subsequently, these three key images are concatenated together. Furthermore, the concatenated outputs are partitioned into blocks with dimensions of $(1 \times 3)$. The process involves performing a bit-XORing diffusion process on the concatenated outputs of the selected blocks of the colour component and the key image. The block selection operation is performed using two distinct PWLCM systems. The diffused blocks are ultimately merged to produce the ciphertext. Based on the analysis on security and experimental outputs, it has been determined, the described method successfully encrypts the color photos of different sizes while ensuring their security.
Parametric Depth Based Feature Representation Learning for Object Detection and Segmentation in Bird’s-Eye View

Recent vision-only perception models for autonomous driving achieved promising results by encoding multi-view image features into Bird’s-Eye-View (BEV) space. A critical step and the main bottleneck of these methods is transforming image features into the BEV coordinate frame. This paper focuses on leveraging geometry information, such as depth, to model such feature transformation. Existing works rely on non-parametric depth distribution modeling leading to significant memory consumption, or ignore the geometry information to address this problem. In contrast, we propose to use parametric depth distribution modeling for feature transformation. We first lift the 2D image features to the 3D space defined for the ego vehicle via a predicted parametric depth distribution for each pixel in each view. Then, we aggregate the 3D feature volume based on the 3D space occupancy derived from depth to the BEV frame. Finally, we use the transformed features for downstream tasks such as object detection and semantic segmentation. Existing semantic segmentation methods do also suffer from an hallucination problem as they do not take visibility information into account. This hallucination can be particularly problematic for subsequent modules such as control and planning. To mitigate the issue, our method provides depth uncertainty and reliable visibility-aware estimations. We further leverage our parametric depth modeling to present a novel visibility-aware evaluation metric that, when taken into account, can mitigate the hallucination problem. Ex tensive experiments on object detection and semantic segmentation on the nuScenes datasets demonstrate that our method outperforms existing methods on both tasks.
Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Video

Synthesizing realistic videos according to a given speech is still an open challenge. Previous works have been plagued by issues such as inaccurate lip shape generation and poor image quality. The key reason is that only motions and appearances on limited facial areas (e.g., lip area) are mainly driven by the input speech. Therefore, directly learning a mapping function from speech to the entire head image is prone to ambiguity, particularly when using a short video for training. We thus propose a decomposition-synthesis-composition framework named Speech to Lip (Speech2Lip) that disentangles speech-sensitive and speech-insensitive motion/appearance to facilitate effective learning from limited training data, resulting in the generation of natural-looking videos. First, given a fixed head pose (i.e., canonical space), we present a speech-driven implicit model for lip image generation which concentrates on learning speech-sensitive motion and appearance. Next, to model the major speech-insensitive motion (i.e., head movement), we introduce a geometry-aware mutual explicit mapping (GAMEM) module that establishes geometric mappings between different head poses. This allows us to paste generated lip images at the canonical space onto head images with arbitrary poses and synthesize talking videos with natural head movements. In addition, a Blend-Net and a contrastive sync loss are introduced to enhance the overall synthesis performance. Quantitative and qualitative results on three benchmarks demonstrate that our model can be trained by a video of just a few minutes in length and achieve state-of-the-art performance in both visual quality and speechvisual synchronization. Code: https://github.com/CVMILab/Speech2Lip.
Forward Flow for Novel View Synthesis of Dynamic Scenes

This paper proposes a neural radiance field (NeRF) approach for novel view synthesis of dynamic scenes using forward warping. Existing methods often adopt a static NeRF to represent the canonical space, and render dynamic images at other time steps by mapping the sampled 3D points back to the canonical space with the learned backward flow field. However, this backward flow field is non-smooth and discontinuous, which is difficult to be fitted by commonly used smooth motion models. To address this problem, we propose to estimate the forward flow field and directly warp the canonical radiance field to other time steps. Such forward flow field is smooth and continuous within the object region, which benefits the motion model learning. To achieve this goal, we represent the canonical radiance field with voxel grids to enable efficient forward warping, and propose a differentiable warping process, including an average splatting operation and an inpaint network, to resolve the many-to-one and one-to-many mapping issues. Thorough experiments show that our method outperforms existing methods in both novel view rendering and motion modeling, demonstrating the effectiveness of our forward flow motion modeling. Project page: https://npucvr.github.io/ForwardFlowDNeRF.
Identity-Consistent Aggregation for Video Object Detection

In Video Object Detection (VID), a common practice is to leverage the rich temporal contexts from the video to enhance the object representations in each frame. Existing methods treat the temporal contexts obtained from different objects indiscriminately and ignore their different identities. While intuitively, aggregating local views of the same object in different frames may facilitate a better understanding of the object. Thus, in this paper, we aim to enable the model to focus on the identity-consistent temporal contexts of each object to obtain more comprehensive object representations and handle the rapid object appearance variations such as occlusion, motion blur, etc. However, realizing this goal on top of existing VID models faces low-efficiency problems due to their redundant region proposals and nonparallel frame-wise prediction manner. To aid this, we propose ClipVID, a VID model equipped with Identity-Consistent Aggregation (ICA) layers specifically designed for mining fine-grained and identity-consistent temporal contexts. It effectively reduces the redundancies through the set prediction strategy, making the ICA layers very efficient and further allowing us to design an architecture that makes parallel clip-wise predictions for the whole video clip. Extensive experimental results demonstrate the superiority of our method: a state-of-the-art (SOTA) performance (84.7% mAP) on the ImageNet VID dataset while running at a speed about 7× faster (39.3 fps) than previous SOTAs.
Secure Key Generation Schemes for Cloud Data Storage

The secure generation of cryptographic keys holds significant importance in safeguarding the confidentiality, authenticity, and integrity of data stored in the cloud. Traditionally, Key Generation Centers have been trusted with the task of generating partial private keys or private keys to the user, thereby requiring a user to trust a centralized third party. The primary objective focuses on identifying effective methods to enhance the security of key generation processes within existing systems. Additionally, the research takes a deeper look into the possible alternative approaches that eliminate the need for a centralized key generation authority. This study makes a significant contribution to the field of secure key generation by thoroughly examining security advancements within conventional systems. Moreover, it highlights the feasibility of trustless key generation, empowering organizations and individuals to implement more secure key generation mechanisms while reducing their reliance on centralized authorities.
Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval

In text-video retrieval, recent works have benefited from the powerful learning capabilities of pre-trained text-image foundation models (e.g., CLIP) by adapting them to the video domain. A critical problem for them is how to effectively capture the rich semantics inside the video using the image encoder of CLIP. To tackle this, state-of-the-art methods adopt complex cross-modal modeling techniques to fuse the text information into video frame representations, which, however, incurs severe efficiency issues in large-scale retrieval systems as the video representations must be recomputed online for every text query. In this paper, we discard this problematic cross-modal fusion process and aim to learn semantically-enhanced representations purely from the video, so that the video representations can be computed offline and reused for different texts. Concretely, we first introduce a spatial-temporal "Prompt Cube" into the CLIP image encoder and iteratively switch it within the encoder layers to efficiently incorporate the global video semantics into frame representations. We then propose to apply an auxiliary video captioning objective to train the frame representations, which facilitates the learning of detailed video semantics by providing fine-grained guidance in the semantic space. With a naive temporal fusion strategy (i.e., mean-pooling) on the enhanced frame representations, we obtain state-of-the-art performances on three benchmark datasets, i.e., MSR-VTT, MSVD, and LSMDC.
COCO-O: A Benchmark for Object Detectors under Natural Distribution Shifts

Practical object detection application can lose its effectiveness on image inputs with natural distribution shifts. This problem leads the research community to pay more attention on the robustness of detectors under Out-Of-Distribution (OOD) inputs. Existing works construct datasets to benchmark the detector’s OOD robustness for a specific application scenario, e.g., Autonomous Driving. However, these datasets lack universality and are hard to benchmark general detectors built on common tasks such as COCO. To give a more comprehensive robustness assessment, we introduce COCO-O(ut-of-distribution), a test dataset based on COCO with 6 types of natural distribution shifts. COCO-O has a large distribution gap with training data and results in a significant 55.7% relative performance drop on a Faster R-CNN detector. We leverage COCO-O to conduct experiments on more than 100 modern object detectors to investigate if their improvements are credible or just over-fitting to the COCO test set. Unfortunately, most classic detectors in early years do not exhibit strong OOD generalization. We further study the robustness effect on recent breakthroughs of detector’s architecture design, augmentation and pre-training techniques. Some empirical findings are revealed: 1) Compared with detection head or neck, backbone is the most important part for robustness; 2) An end-to-end detection transformer design brings no enhancement, and may even reduce robustness; 3) Large-scale foundation models have made a great leap on robust object detection. We hope our COCO-O could provide a rich testbed for robustness study of object detection. The dataset will be available at https://github.com/alibaba/easyrobust/tree/main/benchmarks/coco_o.
TopoSeg: Topology-Aware Nuclear Instance Segmentation

Nuclear instance segmentation has been critical for pathology image analysis in medical science, e.g., cancer diagnosis. Current methods typically adopt pixel-wise optimization for nuclei boundary exploration, where rich structural information could be lost for subsequent quantitative morphology assessment. To address this issue, we develop a topology-aware segmentation approach, termed TopoSeg, which exploits topological structure information to keep the predictions rational, especially in common situations with densely touching and overlapping nucleus instances. Concretely, TopoSeg builds on a topology-aware module (TAM), which encodes dynamic changes of different topology structures within the three-class probability maps (inside, boundary, and background) of the nuclei to persistence barcodes and makes the topology-aware loss function. To efficiently focus on regions with high topological errors, we propose an adaptive topology-aware selection (ATS) strategy to enhance the topology-aware optimization procedure further. Experiments on three nuclear instance segmentation datasets justify the superiority of TopoSeg, which achieves state-of-the-art performance. The code is available at https://github.com/hhlisme/toposeg.
CROSSFIRE: Camera Relocalization On Self-Supervised Features from an Implicit Representation

Beyond novel view synthesis, Neural Radiance Fields (NeRF) are useful for applications that interact with the real world. In this paper, we use them as an implicit map of a given scene and propose a camera relocalization algorithm tailored for this representation. The proposed method enables to compute in real-time the precise position of a device using a single RGB camera, during its navigation. In contrast with previous work, we do not rely on pose regression or photometric alignment but rather use dense local features obtained through volumetric rendering which are specialized on the scene with a self-supervised objective. As a result, our algorithm is more accurate than competitors, able to operate in dynamic outdoor environments with changing lightning conditions and can be readily integrated in any volumetric neural renderer.
Cybersecurity in the Age of Blockchain

The frequency and complexity of cyber assaults have risen considerably in recent years, resulting in major financial losses and reputational harm for both corporations and people. Traditional cybersecurity measures, such as firewalls and encryption, have been insufficient in protecting against these threats. Due to a rise in cyber assaults and data breaches, there is an increased need for effective cybersecurity solutions. With its decentralized nature blockchain technology has come out as a promising option for enhancing cybersecurity. This article investigates how blockchain may be adapted to enhance cybersecurity in areas such as decentralized identity management, safe data storage and sharing, and smart contract security. The article also investigates the potential benefits and barriers of using blockchain in cybersecurity including some cases of successful blockchain implementations in the field of cybersecurity. The report contends that blockchain technology has been able to enhance cybersecurity by offering a secure and transparent platform for digital transactions and data sharing.
Contextual Emotion Estimation from Image Captions

Emotion estimation in images is a challenging task, typically using computer vision methods to directly estimate people’s emotions using face, body pose and contextual cues. In this paper, we explore whether Large Language Models (LLMs) can support the contextual emotion estimation task, by first captioning images, then using an LLM for inference. First, we must understand: how well do LLMs perceive human emotions? And which parts of the information enable them to determine emotions? One initial challenge is to construct a caption that describes a person within a scene with information relevant for emotion perception. Towards this goal, we propose a set of natural language descriptors for faces, bodies, interactions, and environments. We use them to manually generate captions and emotion annotations for a subset of 331 images from the EMOTIC dataset. These captions offer an interpretable representation for emotion estimation, towards understanding how elements of a scene affect emotion perception in LLMs and beyond. Secondly, we test the capability of a large language model to infer an emotion from the resulting image captions. We find that GPT3.5, specifically the text-davinci-003 model, provides surprisingly reasonable emotion predictions consistent with human annotations, but accuracy can depend on the emotion concept. Overall, the results suggest promise in the image captioning and LLM approach.
An Effective, Permutation-only, Bit-Level Image Encryption System Using Chaotic Maps

The widespread and growing utilization of digital images has elevated the significance of image encryption as a crucial concern, garnering significant attention from researchers. Regrettably, the current state of image encryption schemes lacks comprehensive development and research. This article presents a new image encryption system that uses only permutations and works at the bit level. The system incorporates chaotic maps to enhance its effectiveness. The method begins by executing the bit-plane generation process, which is then followed by the bit-plane concatenation operation. The bit-planes are individually subjected to bit-shuffling using distinct Logistic chaotic maps (LM) with varying key values. The shuffled outputs are vertically concatenated and then horizontally concatenated to execute the final stage of the bit-level shuffling operation. The output obtained from concatenation is subjected to a two-step shuffling process. First, a row-shuffling operation is performed, followed by a column-shuffling operation. Both shuffling operations are carried out using a separate LM system. The shuffled output is subjected to bit-plane separation and subsequently combined to produce the cipher image. The hash-based key values of LM systems enhance the method’s resistance towards the selected-plaintext and selected-ciphertext attacks. Based on the computer simulations and security analysis conducted, it has been determined that the proposed method has been demonstrated to be effective in encrypting images securely and efficiently.