Reviewerly/exHarmony huggingface dataset:
Weakly Supervised Object Localization via Transformer with Implicit Spatial Calibration

Weakly Supervised Object Localization (WSOL), which aims to localize objects by only using image-level labels, has attracted much attention because of its low annotation cost in real applications. Recent studies leverage the advantage of self-attention in visual Transformer for long-range dependency to re-active semantic regions, aiming to avoid partial activation in traditional class activation mapping (CAM). However, the long-range modeling in Transformer neglects the inherent spatial coherence of the object, and it usually diffuses the semantic-aware regions far from the object boundary, making localization results significantly larger or far smaller. To address such an issue, we introduce a simple yet effective Spatial Calibration Module (SCM) for accurate WSOL, incorporating semantic similarities of patch tokens and their spatial relationships into a unified diffusion model. Specifically, we introduce a learnable parameter to dynamically adjust the semantic correlations and spatial context intensities for effective information propagation. In practice, SCM is designed as an external module of Transformer, and can be removed during inference to reduce the computation cost. The object-sensitive localization ability is implicitly embedded into the Transformer encoder through optimization in the training phase. It enables the generated attention maps to capture the sharper object boundaries and filter the object-irrelevant background area. Extensive experimental results demonstrate the effectiveness of the proposed method, which significantly outperforms its counterpart TS-CAM on both CUB-200 and ImageNet-1K benchmarks. The code is available at https://github.com/164140757/SCM.
Underwater Images Enhancement by Revised Underwater Images Formation Model

In this paper, we proposed the efficient and appealing technique for underwater images enhancement. Underwater images often suffer from haze, color distortion, low contrast and loss of the human acuity due to light scattering and absorption. To tackle these issues, proposed precise model is presented expressively showed that: (1) the estimation of the transmission map in atmospheric scatter model is divided into two cases, which are more applicable to underwater images in different situations, and make the enhanced results more robust; (2) model not only removes haze but also restores lost colors of underwater images in the image de-hazing step instead of color correction. First, we proposed a revised underwater dehazing model aiming to eliminate the color of water directly while solving the problem of haze in the underwater images. Then proposed color correction method can adaptively address the problem of color shifting without any additional information. Furthermore, we design a multi-scale illumination fusion to reveal more details and low illumination parts of the image. Experimental results demonstrate that our proposed method outperforms other methods significantly with 5%~77% quantitative improvement on all four evaluation performance indices and shows more obvious detailed underwater images. Our method can be applied to underwater detection and exploration as the pre-processing step.
Tackling Background Distraction in Video Object Segmentation

Semi-supervised video object segmentation (VOS) aims to densely track certain designated objects in videos. One of the main challenges in this task is the existence of background distractors that appear similar to the target objects. We propose three novel strategies to suppress such distractors: 1) a spatio-temporally diversified template construction scheme to obtain generalized properties of the target objects; 2) a learnable distance-scoring function to exclude spatially-distant distractors by exploiting the temporal consistency between two consecutive frames; 3) swap-and-attach augmentation to force each object to have unique features by providing training samples containing entangled objects. On all public benchmark datasets, our model achieves a comparable performance to contemporary state-of-the-art approaches, even with real-time performance. Qualitative results also demonstrate the superiority of our approach over existing methods. We believe our approach will be widely used for future VOS research. Code and models are available at https://github.com/suhwan-cho/TBD .
: A Traceable and Privacy-Preserving Data Exchange Scheme based on Non-Fungible Token and Zero-Knowledge

With the advent of the Big Data era, industry, business and academia have developed various data exchange schemes to make data more economically beneficial. Unfortunately, most of the existing systems provide only one-time data exchanges without the ability to track the provenance and transformations of datasets. In addition, existing systems encrypt the data to protect data privacy, which hinders demanders from verifying the correctness of the data and evaluating its value.To provide data traceability and privacy while ensuring fairness during data exchanges, we design and implement ZKDET, a traceable data exchange scheme based on non-fungible token and zero-knowledge, which is able to (i) track all transformations of data during their lifecycle and record them on the blockchain; (ii) provide zero-knowledge proofs to securely guarantee that all complex transformations and data contents are correct and meet specific requirements; and (iii) warrant exchange fairness and data privacy in public storage platforms. Security analysis and evaluations on ZKDET show that it can support traceable data exchange while preserving data privacy and maintaining high throughput despite large data volumes.
Densely Constrained Depth Estimator for Monocular 3D Object Detection

Estimating accurate 3D locations of objects from monocular images is a challenging problem because of lacking depth. Previous work shows that utilizing the object’s keypoint projection constraints to estimate multiple depth candidates boosts the detection performance. However, the existing methods can only utilize vertical edges as projection constraints for depth estimation. So these methods only use a small number of projection constraints and produce insufficient depth candidates, leading to inaccurate depth estimation. In this paper, we propose a method that utilizes dense projection constraints from edges of any direction. In this way, we employ much more projection constraints and produce considerable depth candidates. Besides, we present a graph matching weighting module to merge the depth candidates. The proposed method DCD (Densely Constrained Detector) achieves state-of-the-art performance on the KITTI and WOD benchmarks. Code is released at https://github.com/BraveGroup/DCD .
Exploring Efficiency of Vision Transformers for Self-Supervised Monocular Depth Estimation

Depth estimation is a crucial task for the creation of depth maps, one of the most important components for augmented reality (AR) and other applications. However, the most widely used hardware for AR and smartphones has only sparse depth sensors with different ground truth depth acquisition methods. Thus, depth estimation models that are robust for downstream AR tasks performance can only be trained reliably using self-supervised learning based on camera information. Previous works in the field mostly focus on self-supervised models with pure convolutional architectures, without taking global spatial context into account.In this paper, we utilize vision transformer architectures for self-supervised monocular depth estimation and propose VTDepth, a vision transformer-based model, which provides a solution to the problem of the global spatial context. We compare various combinations of convolutional and transformer architectures for self-supervised depth estimation and show that the best combination of models is an encoder with a transformer basis and convolutional decoder. Our experiments demonstrate the efficiency of VTDepth for self-supervised depth estimation. Our set of models achieves state-of-the-art performance for self-supervised learning on NYUv2 and KITTI datasets. Our code is available at https://github.com/ahbpp/VTDepth.
Cross-Architecture Self-supervised Video Representation Learning

In this paper, we present a new cross-architecture contrastive learning (CACL) framework for self-supervised video representation learning. CACL consists of a 3D CNN and a video transformer which are used in parallel to generate diverse positive pairs for contrastive learning. This allows the model to learn strong representations from such diverse yet meaningful pairs. Furthermore, we introduce a temporal self-supervised learning module able to predict an Edit distance explicitly between two video sequences in the temporal order. This enables the model to learn a rich temporal representation that compensates strongly to the video-level representation learned by the CACL. We evaluate our method on the tasks of video retrieval and action recognition on UCF101 and HMDB51 datasets, where our method achieves excellent performance, surpassing the state-of-the-art methods such as VideoMoCo [23] and MoCo+BE [34] by a large margin.
Securing Deployed Smart Contracts and DeFi With Distributed TEE Cluster

Smart contract technologies can be used to implement almost arbitrary business logic. They can revolutionize many businesses such as payments, insurance, and crowdfunding. The resulting birth of decentralized finance (DeFi) has gained significant momentum. Smart contracts and DeFi are now attractive targets for attacks. An important research question is how to protect deployed smart contracts and DeFi. Smart contracts cannot be modified once deployed, namely vulnerabilities cannot be fixed by patching. In this case, vulnerabilities in deployed contracts and DeFi might cause devastating consequences. In this paper, we put forward SolSaviour, a framework for protecting deployed smart contracts and DeFi. The core of SolSaviour is to build a smart contract protection mechanism based on democratic voting using a distributed trusted execution environment (TEE) cluster. Once a vulnerability in deployed contracts or DeFi is found, SolSaviour can destroy the defective contract and redeploy a patched contract via the distributed TEE cluster. Moreover, SolSaviour can migrate funds and state variables from the destroyed contract to the patched one. Compared with previous work, our approach can protect smart contracts and DeFi in a distributed manner, avoiding reliance on privileged users or trusted third parties. Our experiment results show that SolSaviour can protect smart contracts and complex DeFi protocols with feasible overhead.
The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting

We present the Caltech Fish Counting Dataset (CFC ), a large-scale dataset for detecting, tracking, and counting fish in sonar videos. We identify sonar videos as a rich source of data for advancing low signal-to-noise computer vision applications and tackling domain generalization in multiple-object tracking (MOT) and counting. In comparison to existing MOT and counting datasets, which are largely restricted to videos of people and vehicles in cities, CFC is sourced from a natural-world domain where targets are not easily resolvable and appearance features cannot be easily leveraged for target re-identification. With over half a million annotations in over 1,500 videos sourced from seven different sonar cameras, CFC allows researchers to train MOT and counting algorithms and evaluate generalization performance at unseen test locations. We perform extensive baseline experiments and identify key challenges and opportunities for advancing the state of the art in generalization in MOT and counting.
OIMNet++: Prototypical Normalization and Localization-Aware Learning for Person Search

We address the task of person search, that is, localizing and re-identifying query persons from a set of raw scene images. Recent approaches are typically built upon OIMNet, a pioneer work on person search, that learns joint person representations for performing both detection and person re-identification (reID) tasks. To obtain the representations, they extract features from pedestrian proposals, and then project them on a unit hypersphere with L2 normalization. These methods also incorporate all positive proposals, that sufficiently overlap with the ground truth, equally to learn person representations for reID. We have found that 1) the L2 normalization without considering feature distributions degenerates the discriminative power of person representations, and 2) positive proposals often also depict background clutter and person overlaps, which could encode noisy features to person representations. In this paper, we introduce OIMNet++ that addresses the aforementioned limitations. To this end, we introduce a novel normalization layer, dubbed ProtoNorm, that calibrates features from pedestrian proposals, while considering a long-tail distribution of person IDs, enabling L2 normalized person representations to be discriminative. We also propose a localization-aware feature learning scheme that encourages better-aligned proposals to contribute more in learning discriminative representations. Experimental results and analysis on standard person search benchmarks demonstrate the effectiveness of OIMNet++.
A Cyber Secure Medical Management System by Using Blockchain

In the pharmaceutical industry, problems like counterfeit drugs, including vaccines, and their supply chain management problems like transparency, immutability, and traceability exist. In the case of vaccines, it becomes more difficult to standardize and detect fake vaccines because the public has less awareness and knowledge about vaccines. Moreover, the increase in online pharmacies gives more opportunities for counterfeiting vaccines to enter the authentic supply chain management system. We present transparent, immutable and secure vaccine supply chain (TISVSchain), a framework based on blockchain to handle the issues of counterfeited vaccines and vaccine supply chain problems like transparency, immutability, and traceability. Our proposed framework can run both on the private and public blockchain. We have implemented the framework on public blockchain by using remix ide and the smart contracts designed by solidity language run on very low gas cost. We also carried out several experiments by changing the number of nodes and their block time to evaluate the performance of our framework in terms of transaction per second (TPS), gas cost, and propagation delay. Our proposed framework improves the security by using offline unique account addresses in blockchain-based frameworks and improves the overall efficiency of the framework by keeping the gas cost low, finding a way to decrease the number of lost blocks to keep low propagation delay, and keeping high TPS value. TISVSchain shows us promising results to improve vaccine supply chain management’s overall performance, security, and efficiency.
Low-Delay Analog Joint Source-Channel Coding With Deep Learning

We consider the design of low-delay joint source-channel coding (JSCC) schemes for the transmission of discrete-time analog sources over noisy channels based on deep neural networks. The design problem is addressed as optimization of an autoencoder model, and several scenarios are discussed. For point-to-point communication of independent and identically distributed (i.i.d) Gaussian sources and Gauss-Markov sources over additive-white Gaussian noise (AWGN) channels, the encoder and decoder are constructed using recurrent neural networks (RNNs). With minimum prior knowledge used for design, the performance of these RNNs-based models is optimized using fine tuning techniques during training. Sinusoidal representation networks (SIRENs)-based models are proposed and optimized for three JSCC problems namely, transmitting multivariate Gaussian sources over AWGN channels, transmitting i.i.d Gaussian sources with side information at the decoder, and for communicating correlated sources over orthogonal Gaussian channels. We show that these deep learning-based JSCC schemes perform comparably or better than state-of-the-art (SOTA) traditional schemes. The proposed scheme can extend flexibly to different pairs of source and channel dimensions. Moreover, the spontaneously learned encoder mappings exhibit structured patterns that are interpretable.
Zero-shot cost models for out-of-the-box learned cost prediction

In this paper, we introduce zero-shot cost models, which enable learned cost estimation that generalizes to unseen databases. In contrast to state-of-the-art workload-driven approaches, which require to execute a large set of training queries on every new database, zero-shot cost models thus allow to instantiate a learned cost model out-of-the-box without expensive training data collection. To enable such zero-shot cost models, we suggest a new learning paradigm based on pre-trained cost models. As core contributions to support the transfer of such a pre-trained cost model to unseen databases, we introduce a new model architecture and representation technique for encoding query workloads as input to those models. As we will show in our evaluation, zero-shot cost estimation can provide more accurate cost estimates than state-of-the-art models for a wide range of (real-world) databases without requiring any query executions on unseen databases. Furthermore, we show that zero-shot cost models can be used in a few-shot mode that further improves their quality by retraining them just with a small number of additional training queries on the unseen database.
ARARAT: A Collaborative Edge-Assisted Framework for HTTP Adaptive Video Streaming

With the ever-increasing demands for high-definition and low-latency video streaming applications, network-assisted video streaming schemes have become a promising complementary solution in the HTTP Adaptive Streaming (HAS) context to improve users’ Quality of Experience (QoE) as well as network utilization. Edge computing is considered one of the leading networking paradigms for designing such systems by providing video processing and caching close to the end-users. Despite the wide usage of this technology, designing network-assisted HAS architectures that support low-latency and high-quality video streaming, including edge collaboration is still a challenge. To address these issues, this article leverages the Software-Defined Networking (SDN), Network Function Virtualization (NFV), and edge computing paradigms to propose <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> collabo <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</u> ative edge- <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> ssisted framewo <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</u> k for HTTP <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> daptive video s <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</u> reaming ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ARARAT</monospace> ). Aiming at minimizing HAS clients’ serving time and network cost, besides considering available resources and all possible serving actions, we design a multi-layer architecture and formulate the problem as a centralized optimization model executed by the SDN controller. However, to cope with the high time complexity of the centralized model, we introduce three heuristic approaches that produce near-optimal solutions through efficient collaboration between the SDN controller and edge servers. Finally, we implement the <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ARARAT</monospace> framework, conduct our experiments on a large-scale cloud-based testbed including 250 HAS players, and compare its effectiveness with state-of-the-art systems within comprehensive scenarios. The experimental results illustrate that the proposed <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ARARAT</monospace> methods ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${i}$ </tex-math></inline-formula> ) improve users’ QoE by at least 47%, (ii) decrease the streaming cost, including bandwidth and computational costs, by at least 47%, and (iii) enhance network utilization, by at least 48% compared to state-of-the-art approaches.
FGC: GCN-Based Federated Learning Approach for Trust Industrial Service Recommendation

With the development of the Industrial Internet of Things system, the huge amount of devices, services, and continuous data, making it difficult to discover a trusted service in complex scenarios. To better leverage knowledge and historical behavior, recommendation systems are applied. However, the model accuracy closely depends on training data size; there is a great risk of data leaking by collecting from multiple departments. To solve these problems, we propose a graph-convolutional-neural-network-based federated approach, which accurately recommends proper service for participating clients without gathering the raw data. Specifically, each client trains locally and uploads the weights of their model to the server for aggregation. Besides, the potential overlapping services of different clients are leveraged to guide the embedding aggregation and sharing, which, in turn, optimize the local training results. Their sensitive scenarios' embedding is kept locally. Owing to the model aggregation, it also resists the poisoning attack to some degree. In addition, the comprehensive experiments on classic public recommendation datasets evaluate the feasibility, effectiveness, trustworthiness, and potential influences.
On the Feasibility of Unclonable Encryption, and More

Unclonable encryption, first introduced by Broadbent and Lord (TQC'20), is a one-time encryption scheme with the following security guarantee: any non-local adversary $$(\mathcal {A},\mathcal {B},\mathcal {C})$$ cannot simultaneously distinguish encryptions of two equal length messages. This notion is termed as unclonable indistinguishability. Prior works focused on achieving a weaker notion of unclonable encryption, where we required that any non-local adversary $$(\mathcal {A},\mathcal {B},\mathcal {C})$$ cannot simultaneously recover the entire message m. Seemingly innocuous, understanding the feasibility of encryption schemes satisfying unclonable indistinguishability (even for 1-bit messages) has remained elusive. We make progress towards establishing the feasibility of unclonable encryption.
SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted Expectation-Maximization

Matching-based methods, especially those based on space-time memory, are significantly ahead of other solutions in semi-supervised video object segmentation (VOS). However, continuously growing and redundant template features lead to an inefficient inference. To alleviate this, we propose a novel Sequential Weighted Expectation-Maximization (SWEM) network to greatly reduce the redundancy of memory features. Different from the previous methods which only detect feature redundancy between frames, SWEM merges both intra-frame and inter-frame similar features by leveraging the sequential weighted EM algorithm. Further, adaptive weights for frame features endow SWEM with the flexibility to represent hard samples, improving the discrimination of templates. Besides, the proposed method maintains a fixed number of template features in memory, which ensures the stable inference complexity of the VOS system. Extensive experiments on commonly used DAVIS and YouTube-VOS datasets verify the high efficiency (36 FPS) and high performance (84.3% J&F on DAVIS 2017 validation dataset) of SWEM.