Reviewerly/exHarmony huggingface dataset:
Reversible Data Hiding With Hierarchical Block Variable Length Coding for Cloud Security

Reversible data hiding in encrypted images (RDHEI) can serve as a technical solution to secure data in applications that rely on cloud storage. The key features of an RDHEI scheme are reversibility, security, and data embedding rate. To enlarge the embedding rate, this paper proposes a novel RDHEI scheme based on the median edge detector (MED) and a new proposed hierarchical block variable length coding (HBVLC) technique. In our scheme, the image owner first predicts the pixel values of the carrier image with MED. Then, the prediction error array is sliced into bit-planes and encoded plane by plane. By leveraging the inherent features of the prediction error bit-planes, the image owner adaptively decomposes a bit-plane into blocks of different hierarchical levels based on its local smoothness and encodes the blocks with a variable length coding method. As a result, the carrier image is efficiently compressed to provide spare room for data embedding. The encoded carrier image is then processed with the conventional steps of an RDHEI technique. Experimental results show that the proposed scheme not only can restore the secret data and the carrier image without loss but also outperforms state-of-the-art methods in the embedding rate for images with various features.
Multi-scale and Cross-scale Contrastive Learning for Semantic Segmentation

This work considers supervised contrastive learning for semantic segmentation. We apply contrastive learning to enhance the discriminative power of the multi-scale features extracted by semantic segmentation networks. Our key methodological insight is to leverage samples from the feature spaces emanating from multiple stages of a model’s encoder itself requiring neither data augmentation nor online memory banks to obtain a diverse set of samples. To allow for such an extension we introduce an efficient and effective sampling process, that enables applying contrastive losses over the encoder’s features at multiple scales. Furthermore, by first mapping the encoder’s multi-scale representations to a common feature space, we instantiate a novel form of supervised local-global constraint by introducing cross-scale contrastive learning linking high-resolution local features to low-resolution global features. Combined, our multi-scale and cross-scale contrastive losses boost performance of various models (DeepLabv $$3$$ , HRNet, OCRNet, UPerNet) with both CNN and Transformer backbones, when evaluated on 4 diverse datasets from natural (Cityscapes, PascalContext, ADE20K) but also surgical (CaDIS) domains. Our code is available at https://github.com/RViMLab/MS_CS_ContrSeg .
Filter Pruning via Feature Discrimination in Deep Neural Networks

Filter pruning is one of the most effective methods to compress deep convolutional networks (CNNs). In this paper, as a key component in filter pruning, We first propose a feature discrimination based filter importance criterion, namely Receptive Field Criterion (RFC). It turns the maximum activation responses that characterize the receptive field into probabilities, then measure the filter importance by the distribution of these probabilities from a new perspective of feature discrimination. However, directly applying RFC to global threshold pruning may lead to some problems, because global threshold pruning neglects the differences between different layers. Hence, we propose Distinguishing Layer Pruning based on RFC (DLRFC), i.e., discriminately prune the filters in different layers, which avoids measuring filters between different layers directly against filter criteria. Specifically, our method first selects relatively redundant layers by hard and soft changes of the network output, and then prunes only at these layers. The whole process dynamically adjusts redundant layers through iterations. Extensive experiments conducted on CIFAR-10/100 and ImageNet show that our method achieves state-of-the-art performance in several benchmarks.
The 5-th Generation Stage-Gate Idea-to-Launch Process

If your organization faces new-product challenges—development projects taking too long, new products not hitting sales targets, or an overloaded development pipeline—then it is time to move to a fifth-generation system. In the 40 years since Stage-Gate was introduced, leading firms have made major improvements to their NPD gating systems; some changes are relatively simple in concepts, such as using <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Value Steam Mapping</i> to remove bureaucracy and waste, or adopting <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">concurrency</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">parallel processing. Iterations</i> can be built in throughout the development process, a series of build-and-test steps to get both feedback from customers and technical validation. Another major improvement is more <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">focus on the development pipeline</i> by toughening the gates—making them <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">gates with teeth</i> or <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">investment decision points</i> . Finally, integrating Agile into your gating system <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">is a major step,</i> far larger than simply introducing some new methods and tools but has yielded very positive results for some physical-product firms.
Brain Tumor Segmentation Using Partial Depthwise Separable Convolutions

Gliomas are the most common and aggressive form of all brain tumors, with medial survival rates of less than two years for the highest grade.While accurate and reproducible segmentation of brain tumors is paramount for an effective treatment plan and diagnosis, automatic brain tumor segmentation is challenging because the lesion can appear anywhere in the brain with varying shapes and sizes from one patient to another.Moreover, segmentation is only done by analyzing pixel intensity values of surrounding tissues, and the diffusing nature of aggressive brain tumors makes it even more challenging to delineate tumor boundaries.Nevertheless, deep learning methods have superior performance in automatic brain tumor segmentation.However, their boost in performance comes at the cost of high computational complexity.This paper proposes efficient network architecture for 3D brain tumor segmentation, partially utilizing depthwise separable convolutions to reduce computational costs.The experimental results on the BraTS 2020 dataset show that our methods could achieve comparable results with the state-of-the-art methods with minimum computational complexity.Furthermore, we provide a critical analysis of the current efficient model designs.The code for this project is available at https://github.com/tmagadza/partialDepthwiseNet.
A Sketch is Worth a Thousand Words: Image Retrieval with Text and Sketch

We address the problem of retrieving in-the-wild images with both a sketch and a text query. We present TASK-former (Text And SKetch transformer), an end-to-end trainable model for image retrieval using a text description and a sketch as input. We argue that both input modalities complement each other in a manner that cannot be achieved easily by either one alone. TASK-former follows the late-fusion dual-encoder approach, similar to CLIP [35], which allows efficient and scalable retrieval since the retrieval set can be indexed independently of the queries. We empirically demonstrate that using an input sketch (even a poorly drawn one) in addition to text considerably increases retrieval recall compared to traditional text-based image retrieval. To evaluate our approach, we collect 5,000 hand-drawn sketches for images in the test set of the COCO dataset. The collected sketches are available a https://janesjanes.github.io/tsbir/ .
Expanding Large Pre-trained Unimodal Models with Multimodal Information Injection for Image-Text Multimodal Classification

Fine-tuning pre-trained models for downstream tasks is mainstream in deep learning. However, the pre-trained models are limited to be fine-tuned by data from a specific modality. For example, as a visual model, DenseNet cannot directly take the textual data as its input. Hence, although the large pre-trained models such as DenseNet or BERT have a great potential for the downstream recognition tasks, they have weaknesses in leveraging multimodal information, which is a new trend of deep learning. This work focuses on fine-tuning pre-trained unimodal models with multimodal inputs of image-text pairs and expanding them for image-text multimodal recognition. To this end, we propose the Multimodal Information Injection Plug-in (MI2P) which is attached to different layers of the unimodal models (e.g., DenseNet and BERT). The proposed MI2P unit provides the path to integrate the information of other modalities into the unimodal models. Specifically, MI2P performs cross-modal feature transformation by learning the fine-grained correlations between the visual and textual features. Through the proposed MI2P unit, we can inject the language information into the vision backbone by attending the word-wise textual features to different visual channels, as well as inject the visual information into the language backbone by attending the channel-wise visual features to different textual words. Armed with the MI2P attachments, the pre-trained unimodal models can be expanded to process multimodal data without the need to change the network structures.
DisCo: Remedying Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning

While Self-Supervised Learning (SSL) has received widespread attention from the community, recent researches argue that its performance often suffers a cliff fall when the model size decreases. Since current SSL methods mainly rely on contrastive learning to train the network, we propose a simple yet effective method termed Distilled Contrastive Learning (DisCo) to ease this issue. Specifically, we find that the final inherent embedding of the mainstream SSL methods contains the most important information, and propose to distill the final embedding to maximally transmit a teacher’s knowledge to a lightweight model by constraining the last embedding of the student to be consistent with that of the teacher. In addition, we find that there exists a phenomenon termed Distilling BottleNeck and propose to enlarge the embedding dimension to alleviate this problem. Since the MLP only exists during the SSL phase, our method does not introduce any extra parameters to lightweight models for the downstream task deployment. Experimental results demonstrate that our method surpasses the state-of-the-art on many lightweight models by a large margin. Particularly, when ResNet-101/ResNet-50 is used respectively as a teacher to teach EfficientNet-B0, the linear result of EfficientNet-B0 on ImageNet is improved by 22.1% and 19.7%, respectively, which is very close to ResNet-101/ResNet-50 with much fewer parameters. Code is available at https://github.com/Yuting-Gao/DisCo-pytorch .
Benchmarking library recognition in tweets

Software developers often use social media (such as Twitter) to share programming knowledge such as new tools, sample code snippets, and tips on programming. One of the topics they talk about is the software library. The tweets may contain useful information about a library. A good understanding of this information, e.g., on the developer's views regarding a library can be beneficial to weigh the pros and cons of using the library as well as the general sentiments towards the library. However, it is not trivial to recognize whether a word actually refers to a library or other meanings. For example, a tweet mentioning the word "pandas" may refer to the Python pandas library or to the animal. In this work, we created the first benchmark dataset and investigated the task to distinguish whether a tweet refers to a programming library or something else. Recently, the pre-trained Transformer models (PTMs) have achieved great success in the fields of natural language processing and computer vision. Therefore, we extensively evaluated a broad set of modern PTMs, including both general-purpose and domain-specific ones, to solve this programming library recognition task in tweets. Experimental results show that the use of PTM can outperform the best-performing baseline methods by 5% - 12% in terms of F1-score under within-, cross-, and mixed-library settings.
IVLBC: An Involutive Lightweight Block Cipher for Internet of Things

Nowadays, the use of the Internet of Things has reached a commanding height in a new round of economic and technological upsurge. Its data transmission security has attracted much attention. It is well known that substitution permutation networks (SPNs) ciphers with high diffusion are not advantageous in unified encryption and decryption circuits with extremely resources constrained. Although some research has been carried out to address this issue, there are still insufficiencies. In this article, we propose a new 64-bit lightweight block cipher based on SPN named IVLBC, whose key allows 80 and 128 bits. The components of IVLBC are involutions. In particular, we propose a Feistel with tree structure to obtain a compact and involutive S-box. Also, the nibble-based involutive permutation is proposed to obtain the involutive permutation. Decryption can reuse encrypted code and circuitry in both software and hardware implementations. We prove that the costs of IVLBC are less than PRESENT, PRINCE, Midori, I-PRESENTTM, CRAFT, etc., in unified encryption and decryption circuits. In addition, we conduct other performance tests on IVLBC such as the differential attack, linear attack, integral attack, algebraic attack, invariant attacks, etc.
Class-Incremental Learning with Cross-Space Clustering and Controlled Transfer

In class-incremental learning, the model is expected to learn new classes continually while maintaining knowledge on previous classes. The challenge here lies in preserving the model’s ability to effectively represent prior classes in the feature space, while adapting it to represent incoming new classes. We propose two distillation-based objectives for class incremental learning that leverage the structure of the feature space to maintain accuracy on previous classes, as well as enable learning the new classes. In our first objective, termed cross-space clustering (CSC), we propose to use the feature space structure of the previous model to characterize directions of optimization that maximally preserve the class - directions that all instances of a specific class should collectively optimize towards, and those directions that they should collectively optimize away from. Apart from minimizing forgetting, such a class-level constraint indirectly encourages the model to reliably cluster all instances of a class in the current feature space, and further gives rise to a sense of “herd-immunity”, allowing all samples of a class to jointly combat the model from forgetting the class. Our second objective termed controlled transfer (CT) tackles incremental learning from an important and understudied perspective of inter-class transfer. CT explicitly approximates and conditions the current model on the semantic similarities between incrementally arriving classes and prior classes. This allows the model to learn the incoming classes in such a way that it maximizes positive forward transfer from similar prior classes, thus increasing plasticity, and minimizes negative backward transfer on dissimilar prior classes, whereby strengthening stability. We perform extensive experiments on two benchmark datasets, adding our method (CSCCT) on top of three prominent class-incremental learning methods. We observe consistent performance improvement on a variety of experimental settings.
ABOVE &amp; BELOW: Investigating Ceiling and Floor for Augmented Reality Content Placement

Augmented Reality (AR) interfaces support users by providing access to digital content within real-world environments. However, displaying content at the users’ eye level might result in the occlusion of the real world. Therefore, it requires finding AR content placement areas that free the users’ field of vision. In this work, we systematically investigate two content placement areas beyond the users’ eye level: the ceiling and floor. To understand how potential users perceive virtual content on the ceiling and floor and how the content should be placed on these areas, we conducted two user studies. While the first exploratory study showed the general usefulness of either area, the second quantitative study allowed us to define optimal placement parameters regarding visibility and comfort. With insights from our studies, we provide design recommendations for future AR applications that support 2D content presentation on the ceiling and the floor.
Multimodal Emotion-Cause Pair Extraction in Conversations

Conversation is an important form of human communication and contains a large number of emotions. It is interesting to discover emotions and their causes in conversations. Conversation in its natural form is multimodal. Many studies have been carried out on multimodal emotion recognition in conversations, yet there is still a lack of work on multimodal emotion cause analysis. In this article, we introduce a new task named Multimodal Emotion-Cause Pair Extraction in Conversations, aiming to jointly extract emotions and the corresponding causes from conversations reflected in multiple modalities (i.e., text, audio and video). We accordingly construct a multimodal conversational emotion cause dataset, Emotion-Cause-in-Friends, which contains 9,794 multimodal emotion-cause pairs among 13,619 utterances in the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Friends</i> sitcom. We benchmark the task by establishing two baseline systems including a heuristic approach considering inherent patterns in the location of causes and emotions and a deep learning approach that incorporates multimodal features for emotion-cause pair extraction, and conduct the human performance test for comparison. Furthermore, we investigate the effect of multimodal information, explore the potential of incorporating commonsense knowledge, and perform the task under both Static and Real-time settings.
Prediction Interval Estimation and Deterministic Forecasting Model Using Ground-Based Sky Image

Due to the explosive penetration of photovoltaic (PV) systems in the power grid, accurate and well-informed solar PV power forecasting has become exceptionally crucial, while accurate irradiance prediction is the basis of PV power forecasting. Yet most research on PV power/irradiance forecasting focus on deterministic forecasting, whereas it is difficult to express the credibility of the forecasting results, especially in the scene of instantaneous fluctuation of irradiance and photovoltaic power caused by cloud movement. Therefore, in order to accurately estimate and track the rapid fluctuations of solar irradiance, this paper takes the ground-based sky image information into consideration and obtains the probability-level prediction results along with the deterministic point forecasting, proposes a comprehensive evaluation function constructed with the average coverage and the average width as the fitness function to assist in the optimization of the deterministic forecasting results, and then constructs a three-output model, naming it DenLUBE. In addition, a framework for irradiance variation pattern prediction is proposed to provide pattern information for the forecasting time period to further improve the performance of the ultra-short-term forecasting model. The results show that the proposed model can achieve high quality and strongly robust deterministic forecasting results while producing tight prediction intervals independent of weather conditions. In addition, the performance of our model is greatly enhanced compared to the traditional approach of taking the middle position of the interval as the deterministic forecasting result, as well as other benchmark models.
Predicting Duplicate in Bug Report Using Topic-Based Duplicate Learning With Fine Tuning-Based BERT Algorithm

As the usage and coverage of software increase, various functional improvements and bugs are occurring. The Eclipse, Mozilla open-source projects receive more than about 300 bug reports per day. Usually, when a user finds a bug, they write a bug report. The developer assigned to the bug reads the content of the bug, and if it has already been fixed, the developer marks it as a duplicate bug report. However, if duplicate bug reports are submitted, the developer must manually identify the same bug, and this process requires a lot of effort by the developer. If redundancies in bug reports can be identified automatically, unnecessary effort on the part of the developer can be reduced. To resolve this problem, this paper predicts redundancy using the BERT (Bidirectional Encoder Representations from the Transformer) algorithm and topic-based duplicate/non-duplicate feature extraction. First, a bug report by bug status is extracted from the bug repository, and topic models are constructed by status by applying topic modeling to each status. In each topic, feature selection is performed using the non-duplicate status and the duplicate status. It learns the extracted features as inputs to the BERT algorithm and predicts duplicate bug reports. In this paper, Precision, Recall, F-measure, and Accuracy were used to evaluate the proposed model, and Eclipse, Mozilla, Apache, and KDE open sources were used. The proposed model shows about 87.67%, 89.85%, 87.03%, and 88.95% performance in Eclipse, Mozilla, Apache, and KDE, respectively. In addition, performance comparison with baselines (Naïve Bayes, Randomforest, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Convolutional Neural Networks-Long Short-Term Memory Networks (CNN-LSTM)) in Eclipse, Mozilla, Apache, and KDE about 36.33%, 44.46%, 47.77%, and 45.17%, improvement, respectively, showed that the proposed model is better at detecting duplicates than the baselines.