Reviewerly/exHarmony huggingface dataset:
BTS-Net: Bi-Directional Transfer-And-Selection Network for RGB-D Salient Object Detection

Depth information has been proved beneficial in RGB-D salient object detection (SOD). However, depth maps obtained often suffer from low quality and inaccuracy. Most existing RGB-D SOD models have no cross-modal interactions or only have unidirectional interactions from depth to RGB in their encoder stages, which may lead to inaccurate encoder features when facing low quality depth. To address this limitation, we propose to conduct progressive bidirectional interactions as early in the encoder stage, yielding a novel bi-directional transfer-and-selection network named BTS-Net, which adopts a set of bi-directional transfer-and-selection (BTS) modules to purify features during encoding. Based on the resulting robust encoder features, we also design an effective light-weight group decoder to achieve accurate final saliency prediction. Comprehensive experiments on six widely used datasets demonstrate that BTS-Net surpasses 16 latest state-of-the-art approaches in terms of four key metrics.
Human-like Controllable Image Captioning with Verb-specific Semantic Roles

Controllable Image Captioning (CIC) — generating image descriptions following designated control signals — has received unprecedented attention over the last few years. To emulate the human ability in controlling caption generation, current CIC studies focus exclusively on control signals concerning objective properties, such as contents of interest or descriptive patterns. However, we argue that almost all existing objective control signals have overlooked two indispensable characteristics of an ideal control signal: 1) Event-compatible: all visual contents referred to in a single sentence should be compatible with the described activity. 2) Sample-suitable: the control signals should be suitable for a specific image sample. To this end, we propose a new control signal for CIC: Verb-specific Semantic Roles (VSR). VSR consists of a verb and some semantic roles, which represents a targeted activity and the roles of entities involved in this activity. Given a designated VSR, we first train a grounded semantic role labeling (GSRL) model to identify and ground all entities for each role. Then, we propose a semantic structure planner (SSP) to learn human-like descriptive semantic structures. Lastly, we use a role-shift captioning model to generate the captions. Extensive experiments and ablations demonstrate that our framework can achieve better controllability than several strong base-lines on two challenging CIC benchmarks. Besides, we can generate multi-level diverse captions easily. The code is available at: https://github.com/mad-red/VSR-guided-CIC.
Every Annotation Counts: Multi-label Deep Supervision for Medical Image Segmentation

Pixel-wise segmentation is one of the most data and an-notation hungry tasks in our field. Providing representative and accurate annotations is often mission-critical especially for challenging medical applications. In this paper, we propose a semi-weakly supervised segmentation algorithm to overcome this barrier. Our approach is based on a new formulation of deep supervision and student-teacher model and allows for easy integration of different supervision signals. In contrast to previous work, we show that care has to be taken how deep supervision is integrated in lower layers and we present multi-label deep supervision as the most important secret ingredient for success. With our novel training regime for segmentation that flexibly makes use of images that are either fully labeled, marked with bounding boxes, just global labels, or not at all, we are able to cut the requirement for expensive labels by 94.22% – narrowing the gap to the best fully supervised baseline to only 5% mean IoU. Our approach is validated by extensive experiments on retinal fluid segmentation and we provide an in-depth analysis of the anticipated effect each annotation type can have in boosting segmentation performance.
Recent advances of single-object tracking methods: A brief survey

Single-object tracking is regarded as a challenging task in computer vision, especially in complex spatio-temporal contexts. The changes in the environment and object deformation make it difficult to track. In the last 10 years, the application of correlation filters and deep learning enhances the performance of trackers to a large extent. This paper summarizes single-object tracking algorithms based on correlation filters and deep learning. Firstly, we explain the definition of single-object tracking and analyze the components of general object tracking algorithms. Secondly, the single-object tracking algorithms proposed in the past decade are summarized according to different categories. Finally, this paper summarizes the achievements and problems of existing algorithms by analyzing experimental results and discusses the development trends.
OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning

Learning image representations without human supervision is an important and active research field. Several recent approaches have successfully leveraged the idea of making such a representation invariant under different types of perturbations, especially via contrastive-based instance discrimination training. Although effective visual representations should indeed exhibit such invariances, there are other important characteristics, such as encoding contextual reasoning skills, for which alternative reconstruction-based approaches might be better suited.With this in mind, we propose a teacher-student scheme to learn representations by training a convolutional net to reconstruct a bag-of-visual-words (BoW) representation of an image, given as input a perturbed version of that same image. Our strategy performs an online training of both the teacher network (whose role is to generate the BoW targets) and the student network (whose role is to learn representations), along with an online update of the visual-words vocabulary (used for the BoW targets). This idea effectively enables fully online BoW-guided unsupervised learning. Extensive experiments demonstrate the interest of our BoWbased strategy, which surpasses previous state-of-the-art methods (including contrastive-based ones) in several applications. For instance, in downstream tasks such Pascal object detection, Pascal classification and Places205 classification, our method improves over all prior unsupervised approaches, thus establishing new state-of-the-art results that are also significantly better even than those of supervised pre-training. We provide the implementation code at https://github.com/valeoai/obow.
appXchain: Application-Level Interoperability for Blockchain Networks

Blockchain technology has the potential to revolutionize industries by offering decentralized, transparent, data provenance, auditable, reliable, and trustworthy features. However, cross-chain interoperability is one of the crucial challenges preventing widespread adoption of blockchain applications. Cross-chain interoperability represents the ability for one blockchain network to interact and share data with another blockchain network. Contemporary cross-chain interoperability solutions are centralized and require re-engineering of the core blockchain stack to enable inter-communication and data sharing among heterogeneous blockchain networks. In this paper, we propose an application-based cross-chain interoperability solution named appXchain which allows blockchain networks of any architecture type and industrial focus to inter-communicate, share data, and make requests. Our solution utilizes the decentralized applications as a distributed translation layer that is capable of communicating and understanding multiple blockchain networks, thereby delegating requests and parameters among them. The architecture uses incentivized verifier nodes that maintain the integrity of shared data facilitating them to be readable by the entities of their network. We define and describe the roles and requirements of major entities of inter-operating blockchain networks in the context of healthcare. We present a detailed explanation of the sequence of interactions needed to share an Electronic Medical Record (EMR) document from one blockchain network to another along with the required algorithms. We implement the appXchain solution with Ethereum-based smart contracts for two hospitals and also present its cost and security analysis. We have made our smart contracts code and testing scripts publicly available.
Improving recommender systems via a Dual Training Error based Correction approach

We propose a method to improve the prediction performance of recommender systems via a Dual (user anditem) Training Error based Correction approach (DTEC). The proposed method is applied to the Synthetic Coordinate Recommendation system (SCoR) (Papadakis et al., 2017) and to other Ithree state-of-the-art systems. Initially, a recommender system is used Ito provide recommendations for users and items. Subsequently, we introduce a second stage, after initial execution of the recommender system, that improves its predictions taking into account the error in the training set between users and items and their similarity. These corrections can be performed from both user and item viewpoints, and finally a dual system is proposed that efficiently combines both corrections. DTEC computes a model that makes zero the recommendation error in the training set, and then applies it on the test set to improve the rating predictions. The proposed DTEC approach is applicable Ito any model-based recommender system with positive training error, potentially increasing the accuracy of the recommendations. The experimental results demonstrate the efficiency and high performance of DTEC on four well-known, real-world datasets.
Detecting cryptocurrency pump-and-dump frauds using market and social signals

The cryptocurrency market has gained significant traction in the last decade, becoming an alternative finance platform to traditional stock market trading. Despite its rapid evolution, legal regulations have not yet caught up to the cryptocurrency market’s progress, attracting the attention of scammers looking to exploit legal loopholes for profits. Pump-and-dump schemes, a well-worn fraud device, has regained relevance in this new territory. In a typical pump-and-dump scheme, scammers organize and leverage media channels to artificially inflate the price of an alternative cryptocurrency, only to quickly sell them to profit off unsuspecting buyers. The disruptive nature of pump-and-dump schemes necessitates a system to reliably forecast pump targets and the magnitude of its success. In this paper, we propose an approach to predict the target cryptocurrency for each pump before its announcement using market and social media signals using Neural Network-based architectures while offering interpretable insights into their black-box nature. Additionally, we construct models that are capable of forecasting the highest price induced by the pump after the cryptocurrency’s identity is revealed within 6.1% error margin. We examine the optimal temporal windows and describe the limitations of social data to predict the manipulations in cryptocurrency trade. Our experimental results serve as proof of a feasible forecasting expert system for identifying cryptocurrency pump-and-dump frauds using publicly available data.
ECKPN: Explicit Class Knowledge Propagation Network for Transductive Few-shot Learning

Recently, the transductive graph-based methods have achieved great success in the few-shot classification task. However, most existing methods ignore exploring the class-level knowledge that can be easily learned by humans from just a handful of samples. In this paper, we propose an Explicit Class Knowledge Propagation Network (ECKPN), which is composed of the comparison, squeeze and calibration modules, to address this problem. Specifically, we first employ the comparison module to explore the pairwise sample relations to learn rich sample representations in the instance-level graph. Then, we squeeze the instance-level graph to generate the class-level graph, which can help obtain the class-level visual knowledge and facilitate modeling the relations of different classes. Next, the calibration module is adopted to characterize the relations of the classes explicitly to obtain the more discriminative class-level knowledge representations. Finally, we combine the class-level knowledge with the instance-level sample representations to guide the inference of the query samples. We conduct extensive experiments on four few-shot classification benchmarks, and the experimental results show that the proposed ECKPN significantly outperforms the state-of-the art methods.
Understanding the Effect of Deplatforming on Social Networks

Aiming to enhance the safety of their users, social media platforms enforce terms of service by performing active moderation, including removing content or suspending users. Nevertheless, we do not have a clear understanding of how effective it is, ultimately, to suspend users who engage in toxic behavior, as that might actually draw users to alternative platforms where moderation is laxer. Moreover, this deplatforming efforts might end up nudging abusive users towards more extreme ideologies and potential radicalization risks. In this paper, we set to understand what happens when users get suspended on a social platform and move to an alternative one. We focus on accounts active on Gab that were suspended from Twitter and Reddit. We develop a method to identify accounts belonging to the same person on these platforms, and observe whether there was a measurable difference in the activity and toxicity of these accounts after suspension. We find that users who get banned on Twitter/Reddit exhibit an increased level of activity and toxicity on Gab, although the audience they potentially reach decreases. Overall, we argue that moderation efforts should go beyond ensuring the safety of users on a single platform, taking into account the potential adverse effects of banning users on major platforms.
ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring

Video deblurring models exploit consecutive frames to remove blurs from camera shakes and object motions. In order to utilize neighboring sharp patches, typical methods rely mainly on homography or optical flows to spatially align neighboring blurry frames. However, such explicit approaches are less effective in the presence of fast motions with large pixel displacements. In this work, we propose a novel implicit method to learn spatial correspondence among blurry frames in the feature space. To construct distant pixel correspondences, our model builds a correlation volume pyramid among all the pixel-pairs between neigh-boring frames. To enhance the features of the reference frame, we design a correlative aggregation module that maximizes the pixel-pair correlations with its neighbors based on the volume pyramid. Finally, we feed the aggregated features into a reconstruction module to obtain the restored frame. We design a generative adversarial paradigm to optimize the model progressively. Our proposed method is evaluated on the widely-adopted DVD dataset, along with a newly collected High-Frame-Rate (1000 fps) Dataset for Video Deblurring (HFR-DVD). Quantitative and qualitative experiments show that our model performs favorably on both datasets against previous state-of-the-art methods, confirming the benefit of modeling all-range spatial correspondence for video deblurring.
Multi-Scale Aligned Distillation for Low-Resolution Detection

In instance-level detection tasks (e.g., object detection), reducing input resolution is an easy option to improve runtime efficiency. However, this option traditionally hurts the detection performance much. This paper focuses on boosting performance of low-resolution models by distilling knowledge from a high- or multi-resolution model. We first identify the challenge of applying knowledge distillation (KD) to teacher and student networks that act on different input resolutions. To tackle it, we explore the idea of spatially aligning feature maps between models of varying input resolutions by shifting feature pyramid position and introduce aligned multi-scale training to train a multi-scale teacher that can distill its knowledge to a low-resolution student. Further, we propose crossing feature-level fusion to dynamically fuse teacher’s multi-resolution features to guide the student better. On several instance-level detection tasks and datasets, the low-resolution models trained via our approach perform competitively with high-resolution models trained via conventional multi-scale training, while outperforming the latter’s low-resolution models by 2.1% to 3.6% in terms of mAP. Our code is made publicly available at https://github.com/Jia-Research-Lab/MSAD.
Anticipating human actions by correlating past with the future with Jaccard similarity measures

We propose a framework for early action recognition and anticipation by correlating past features with the future using three novel similarity measures called Jaccard vector similarity, Jaccard cross-correlation and Jaccard Frobenius inner product over covariances. Using these combinations of novel losses and using our framework, we obtain state-of-the-art results for early action recognition in UCF101 and JHMDB datasets by obtaining 91.7 % and 83.5 % accuracy respectively for an observation percentage of 20. Similarly, we obtain state-of-the-art results for Epic-Kitchen55 and Breakfast datasets for action anticipation by obtaining 20.35 and 41.8 top-1 accuracy respectively.
Jigsaw Clustering for Unsupervised Visual Representation Learning

Unsupervised representation learning with contrastive learning achieved great success. This line of methods duplicate each training batch to construct contrastive pairs, making each training batch and its augmented version forwarded simultaneously and leading to additional computation. We propose a new jigsaw clustering pretext task in this paper, which only needs to forward each training batch itself, and reduces the training cost. Our method makes use of information from both intra- and inter-images, and outperforms previous single-batch based ones by a large margin. It is even comparable to the contrastive learning methods when only half of training batches are used.(p)(/p)Our method indicates that multiple batches during training are not necessary, and opens the door for future research of single-batch unsupervised methods. Our models trained on ImageNet datasets achieve state-of-the-art results with linear classification, outperforming previous single-batch methods by 2.6%. Models transferred to COCO datasets outperforms MoCo v2 by 0.4% with only half of the training batches. Our pretrained models outperform supervised ImageNet pretrained models on CIFAR-10 and CIFAR-100 datasets by 0.9% and 4.1% respectively.
Prototype-Guided Saliency Feature Learning for Person Search

Existing person search methods integrate person detection and re-identification (re-ID) module into a unified system. Though promising results have been achieved, the misalignment problem, which commonly occurs in person search, limits the discriminative feature representation for re-ID. To overcome this limitation, we introduce a novel framework to learn the discriminative representation by utilizing prototype in OIM loss. Unlike conventional methods using prototype as a representation of person identity, we utilize it as guidance to allow the attention network to consistently highlight multiple instances across different poses. Moreover, we propose a new prototype update scheme with adaptive momentum to increase the discriminative ability across different instances. Extensive ablation experiments demonstrate that our method can significantly enhance the feature discriminative power, outperforming the state-of-the-art results on two person search benchmarks including CUHK-SYSU and PRW.
Authenticated Keyword Search in Scalable Hybrid-Storage Blockchains

Blockchain has emerged as a promising solution for secure data storage and retrieval for decentralized applications. To scale blockchain systems, a prevailing approach is to employ a hybrid storage model, where only small meta-data are stored on-chain while the raw data are outsourced to an off-chain storage service provider. The key issue for query processing in such a system is the design of gas-efficient authenticated data structure (ADS) to authenticate the query results. In this paper, we study novel ADS schemes for authenticated keyword search in hybrid-storage blockchains. We first propose the Suppressed Merkle inverted (Merkle <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inv</sup> ) index, which maintains only a partial ADS structure on-chain that can be securely updated with a logarithm-sized cryptographic proof. Moreover, we propose a Chameleon inverted (Chameleon <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inv</sup> ) index that leverages the chameleon vector commitment to achieve a constant maintenance cost. It is further optimized with Bloom filters to enhance the query and verification performance. We prove the security of the proposed ADS schemes and evaluate their performance using real datasets on the Ethereum platform. Experimental results show that, compared to a baseline solution, the proposed Merkle <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inv</sup> and Chameleon <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inv</sup> indexes reduce the average on-chain maintenance cost from US$10.39 down to US$2.50 and US$0.24, respectively, without sacrificing much the query performance.
Grading of invasive breast carcinoma: the way forward

Histologic grading has been a simple and inexpensive method to assess tumor behavior and prognosis of invasive breast cancer grading, thereby identifying patients at risk for adverse outcomes, who may be eligible for (neo)adjuvant therapies. Histologic grading needs to be performed accurately, on properly fixed specimens, and by adequately trained dedicated pathologists that take the time to diligently follow the protocol methodology. In this paper, we review the history of histologic grading, describe the basics of grading, review prognostic value and reproducibility issues, compare performance of grading to gene expression profiles, and discuss how to move forward to improve reproducibility of grading by training, feedback and artificial intelligence algorithms, and special stains to better recognize mitoses. We conclude that histologic grading, when adequately carried out, remains to be of important prognostic value in breast cancer patients.