Reviewerly/exHarmony huggingface dataset:
Blockchain Intelligence for Internet of Vehicles: Challenges and Solutions

With the development of communication and networking technologies, the Internet of Vehicles (IoV) has become the foundation of smart transportation. The development of blockchain and Machine Learning (ML) has contributed to the pervasiveness of the IoV, and they can effectively address the current issues of decentralisation, cyber security and data privacy in the IoV. In this article, blockchain and ML in the IoV are both reviewed, and corresponding technologies to support blockchain intelligence in the IoV are summarized. Importantly, blockchain intelligence is proposed as a key to integrate blockchain and ML, combining the advantages of both to drive the rapid development of the IoV. We discuss general frameworks, issuses, requirements and advantages for the implementation of blockchain intelligence in the IoV. Driven by its advantages, we summarize solutions of blockchain intelligence in the IoV from four aspects, including reliable interaction, network security and data privacy, trustworthy environment and scalability. Finally, a summary of current unresolved issues and challenges of blockchain intelligence in the IoV is presented, which provides guidelines for the future development of the IoV.
Applications of AI in pharmacy practice: a look at hospital and community settings

Keywords: Machine LearningPandemic responseDrug DiscoveryEpidemiologyClinical PracticeJEL codes: A11A1AA13
Iterative Geometry Encoding Volume for Stereo Matching

Recurrent All-Pairs Field Transforms (RAFT) has shown great potentials in matching tasks. However, all-pairs correlations lack non-local geometry knowledge and have difficulties tackling local ambiguities in ill-posed regions. In this paper, we propose Iterative Geometry Encoding Volume (IGEV-Stereo), a new deep network architecture for stereo matching. The proposed IGEV-Stereo builds a combined geometry encoding volume that encodes geometry and context information as well as local matching details, and iteratively indexes it to update the disparity map. To speed up the convergence, we exploit GEV to regress an accurate starting point for ConvGRUs iterations. Our IGEV-Stereo ranks 1st on KITTI 2015 and 2012 (Reflective) among all published methods and is the fastest among the top 10 methods. In addition, IGEV-Stereo has strong cross-dataset generalization as well as high inference efficiency. We also extend our IGEV to multi-view stereo (MVS), i.e. IGEV-MVS, which achieves competitive accuracy on DTU benchmark. Code is available at https://github.com/gangweiX/IGEV.
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting

Text-guided image editing can have a transformative impact in supporting creative applications. A key challenge is to generate edits that are faithful to input text prompts, while consistent with input images. We present Imagen Editor, a cascaded diffusion model built, by fine-tuning Imagen [36] on text-guided image inpainting. Imagen Editor's edits are faithful to the text prompts, which is accomplished by using object detectors to propose inpainting masks during training. In addition, Imagen Editor captures fine details in the input image by conditioning the cascaded pipeline on the original high resolution image. To improve qualitative and quantitative evaluation, we introduce EditBench, a systematic benchmark for text-guided image inpainting. EditBench evaluates inpainting edits on natural and generated images exploring objects, attributes, and scenes. Through extensive human evaluation on EditBench, we find that object-masking during training leads to across-the-board improvements in text-image alignment – such that Imagen Editor is preferred over DALL-E 2 [31] and Stable Diffusion [33] – and, as a cohort, these models are better at object-rendering than text-rendering, and handle material/color/size attributes better than count/shape attributes.
NICO++: Towards Better Benchmarking for Domain Generalization

Despite the remarkable performance that modern deep neural networks have achieved on independent and identically distributed (I.I.D.) data, they can crash under distribution shifts. Most current evaluation methods for do-main generalization (DG) adopt the leave-one-out strategy as a compromise on the limited number of domains. We propose a large-scale benchmark with extensive labeled domains named <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$NICO^{++}$</tex> along with more rational evaluation methods for comprehensively evaluating DC algorithms. To evaluate DG datasets, we propose two metrics to quantify covariate shift and concept shift, respectively. Two novel generalization bounds from the perspective of data construction are proposed to prove that limited concept shift and significant covariate shift favor the evaluation capability for generalization. Through extensive experiments, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$NlCO^{++}$</tex> shows its superior evaluation capability compared with current DG datasets and its contribution in alleviating unfairness caused by the leak of oracle knowledge in model selection. The data and code for the benchmark based on <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$NICO^{++}$</tex> are available at https://github.com/xxgege/NICO-plus.
Omni Aggregation Networks for Lightweight Image Super-Resolution

While lightweight ViT framework has made tremendous progress in image super-resolution, its uni-dimensional self-attention modeling, as well as homogeneous aggregation scheme, limit its effective receptive field (ERF) to include more comprehensive interactions from both spatial and channel dimensions. To tackle these drawbacks, this work proposes two enhanced components under a new Omni-SR architecture. First, an Omni Self-Attention (OSA) block is proposed based on dense interaction principle, which can simultaneously model pixel-interaction from both spatial and channel dimensions, mining the potential correlations across omni-axis (i.e., spatial and channel). Coupling with mainstream window partitioning strategies, OSA can achieve superior performance with compelling computational budgets. Second, a multi-scale interaction scheme is proposed to mitigate sub-optimal ERF (i.e., premature saturation) in shallow models, which facilitates local propagation and meso-/global-scale interactions, rendering an omni-scale aggregation building block. Extensive experiments demonstrate that Omni-SR achieves recordhigh performance on lightweight super-resolution benchmarks (e.g., 26.95dB@Urban100 x4 with only 792K parameters). Our code is available at https://github.com/Francis0625/Omni-SR.
Generating Holistic 3D Human Motion from Speech

This work addresses the problem of generating 3D holistic body motions from human speech. Given a speech recording, we synthesize sequences of 3D body poses, hand gestures, and facial expressions that are realistic and diverse. To achieve this, we first build a high-quality dataset of 3D holistic body meshes with synchronous speech. We then define a novel speech-to-motion generation framework in which the face, body, and hands are modeled separately. The separated modeling stems from the fact that face articulation strongly correlates with human speech, while body poses and hand gestures are less correlated. Specifically, we employ an autoencoder for face motions, and a compositional vector-quantized variational autoencoder (VQ- VAE) for the body and hand motions. The compositional VQ-VAE is key to generating diverse results. Additionally, we propose a cross-conditional autoregressive model that generates body poses and hand gestures, leading to coherent and realistic motions. Extensive experiments and user studies demonstrate that our proposed approach achieves state-of-the-art performance both qualitatively and quantitatively. Our dataset and code are released for research purposes at https://talkshow.is.tue.mpg.de/.
CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not

In this paper, we leverage CLIP for zero-shot sketch based image retrieval (ZS-SBIR). We are largely inspired by recent advances on foundation models and the unparalleled generalisation ability they seem to offer, but for the first time tailor it to benefit the sketch community. We put forward novel designs on how best to achieve this synergy, for both the category setting and the fine-grained setting ('all”}. At the very core of our solution is a prompt learning setup. First we show just via factoring in sketch-specific prompts, we already have a category-level ZS-SBIR system that over-shoots all prior arts, by a large margin (24.8%) - a great testimony on studying the CLIP and ZS-SBIR synergy. Moving onto the fine-grained setup is however trickier, and re-quires a deeper dive into this synergy. For that, we come up with two specific designs to tackle the fine-grained matching nature of the problem: (i) an additional regularisation loss to ensure the relative separation between sketches and photos is uniform across categories, which is not the case for the gold standard standalone triplet loss, and (ii) a clever patch shuffling technique to help establishing instance-level structural correspondences between sketch-photo pairs. With these designs, we again observe signifi-cant performance gains in the region of 26.9% over previ-ous state-of-the-art. The take-home message, if any, is the proposed CLIP and prompt learning paradigm carries great promise in tackling other sketch-related tasks (not limited to ZS-SBIR) where data scarcity remains a great challenge. Project page: https://aneeshan95.github.ioISketchLVM/
Scaling Language-Image Pre-Training via Masking

We present Fast Language-Image Pre-training (FLIP), a simple and more efficient method for training CLIP [52]. Our method randomly masks out and removes a large portion of image patches during training. Masking allows us to learn from more image-text pairs given the same wall-clock time and contrast more samples per iteration with similar memory footprint. It leads to a favorable trade-off between accuracy and training time. In our experiments on 400 million image-text pairs, FLIP improves both accuracy and speed over the no-masking baseline. On a large diversity of downstream tasks, FLIP dominantly outperforms the CLIP counterparts trained on the same data. Facilitated by the speedup, we explore the scaling behavior of increasing the model size, data size, or training length, and report encouraging results and comparisons. We hope that our work will foster future research on scaling vision-language learning.
Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective

We aim at advancing blind image quality assessment (BIQA), which predicts the human perception of image quality without any reference information. We develop a general and automated multitask learning scheme for BIQA to exploit auxiliary knowledge from other tasks, in a way that the model parameter sharing and the loss weighting are determined automatically. Specifically, we first describe all candidate label combinations (from multiple tasks) using a textual template, and compute the joint probability from the cosine similarities of the visual-textual embeddings. Predictions of each task can be inferred from the joint distribution, and optimized by carefully designed loss functions. Through comprehensive experiments on learning three tasks - BIQA, scene classification, and distortion type identification, we verify that the proposed BIQA method 1) benefits from the scene classification and distortion type identification tasks and outperforms the state-of-the-art on multiple IQA datasets, 2) is more robust in the group maximum differentiation competition, and 3) realigns the quality annotations from different IQA datasets more effectively. The source code is available at https://github.com/zwx8981/LIQE.
RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors

Existing dehazing approaches struggle to process real-world hazy images owing to the lack of paired real data and robust priors. In this work, we present a new paradigm for real image dehazing from the perspectives of synthesizing more realistic hazy data and introducing more robust priors into the network. Specifically, (1) instead of adopting the de facto physical scattering model, we rethink the degradation of real hazy images and propose a phenomenological pipeline considering diverse degradation types. (2) We propose a Real Image Dehazing network via high-quality Codebook Priors (RIDCP). Firstly, a VQGAN is pre-trained on a large-scale high-quality dataset to obtain the discrete codebook, encapsulating high-quality priors (HQPs). After replacing the negative effects brought by haze with HQPs, the decoder equipped with a novel normalized feature alignment module can effectively utilize high-quality features and produce clean results. However, although our degradation pipeline drastically mitigates the domain gap between synthetic and real data, it is still intractable to avoid it, which challenges HQPs matching in the wild. Thus, we recalculate the distance when matching the features to the HQPs by a controllable matching operation, which facilitates finding better counterparts. We provide a recommendation to control the matching based on an explainable solution. Users can also flexibly adjust the enhancement degree as per their preference. Extensive experiments verify the effectiveness of our data synthesis pipeline and the superior performance of RIDCP in real image dehazing. Code and data are released at https://rqwu.github.io/projects/RIDCP.
Universal Qudit Gate Synthesis for Transmons

Gate-based quantum computers typically encode and process information in two-dimensional units called qubits. Using d-dimensional qudits instead may offer intrinsic advantages, including more efficient circuit synthesis, problem-tailored encodings and embedded error correction. In this work, we design a superconducting qudit-based quantum processor wherein the logical space of transmon qubits is extended to higher-excited levels. We propose a universal gate set featuring a two-qudit cross-resonance entangling gate, for which we predict fidelities beyond 99% in the d=4 case of ququarts with realistic experimental parameters. Furthermore, we present a decomposition routine that compiles general qudit unitaries into these elementary gates, requiring fewer entangling gates than qubit alternatives. As proof-of-concept applications, we numerically demonstrate the synthesis of SU(16) gates for noisy quantum hardware and an embedded error-correction sequence that encodes a qubit memory in a transmon ququart to protect against pure dephasing noise. We conclude that universal qudit control—a valuable extension to the operational toolbox of superconducting quantum information processing—is within reach of current transmon-based architectures and has applications to near-term and long-term hardware.5 MoreReceived 20 December 2022Revised 2 June 2023Accepted 12 July 2023DOI:https://doi.org/10.1103/PRXQuantum.4.030327Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum computationQuantum error correctionQuantum information processingQuantum InformationCondensed Matter, Materials & Applied Physics
The impacts of Artificial Intelligence (AI) on business and its regulatory challenges

Artificial intelligence (AI) is a type of intelligence system in which machines learn and analyze data in order to make decisions similar to humans. This article discusses the rapidly growing impact of Artificial Intelligence (AI) on the business world, and the regulatory challenges that have arisen as a result. The rise of AI has brought about both opportunities and risks, with the potential to revolutionize various industries and improve processes and decision-making. However, the use of AI also raises concerns regarding ethics, privacy, and accountability, and highlights the need for clear and effective regulation. The article explores the current state of AI regulation and the challenges that governments and businesses face in balancing the benefits and risks of this technology. It also provides insights into how AI regulation may evolve in the future and the steps that organizations can take to stay ahead of the regulatory curve. This paper provides a comprehensive sketch of the legal and regulatory issues, challenges, and effects that are connected to AI and AI-related functions.
Depth-Induced Gap-Reducing Network for RGB-D Salient Object Detection: An Interaction, Guidance and Refinement Approach

Depth provides complementary information for salient object detection (SOD). However, the performance of RGB-D SOD methods is usually hindered by low quality depth map, semantic gap cross-modality and intrinsic gap between multi-level features. Although recent RGB-D SOD methods have been embedded into depth quality assessment, these methods do not consider the inconsistency of the depth format across datasets. In this paper, we propose an interpretable and effective mechanism called interference degree (ID) to assess depth quality and reweight the contribution of single-modality features without extra annotation. Then, a cross-modality interaction block (CMIB) is designed to reduce the semantic gap between RGB and depth features with the help of ID mechanism, and a mutually guided cross-level fusion (MGCF) module is designed to reduce the intrinsic gap among multi-level features. Finally, a refinement branch is proposed to enhance the salient regions and suppress the non-salient regions of fused features. Extensive experiments on six benchmark datasets show that the proposed depth-induced gap-reducing network (DIGR-Net) outperforms 20 recent state-of-the-art methods.
Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation

The remarkable achievements of Large Language Models (LLMs) have led to the emergence of a novel recommendation paradigm — Recommendation via LLM (RecLLM). Nevertheless, it is important to note that LLMs may contain social prejudices, and therefore, the fairness of recommendations made by RecLLM requires further investigation. To avoid the potential risks of RecLLM, it is imperative to evaluate the fairness of RecLLM with respect to various sensitive attributes on the user side. Due to the differences between the RecLLM paradigm and the traditional recommendation paradigm, it is problematic to directly use the fairness benchmark of traditional recommendation. To address the dilemma, we propose a novel benchmark called Fairness of Recommendation via LLM (FaiRLLM). This benchmark comprises carefully crafted metrics and a dataset that accounts for eight sensitive attributes1 in two recommendation scenarios: music and movies. By utilizing our FaiRLLM benchmark, we conducted an evaluation of ChatGPT and discovered that it still exhibits unfairness to some sensitive attributes when generating recommendations. Our code and dataset can be found at https://github.com/jizhi-zhang/FaiRLLM.
Harnessing the Potential of the Metaverse and Artificial Intelligence for the Internet of City Things: Cost-Effective XReality and Synergistic AIoT Technologies

The Metaverse represents an always-on 3D network of virtual spaces, designed to facilitate social interaction, learning, collaboration, and a wide range of activities. This emerging computing platform originates from the dynamic convergence of Extended Reality (XR), Artificial Intelligence of Things (AIoT), and platform-mediated everyday life experiences in smart cities. However, the research community faces a pressing challenge in addressing the limitations posed by the resource constraints associated with XR-enabled IoT applications within the Internet of City Things (IoCT). Additionally, there is a limited understanding of the synergies between XR and AIoT technologies in the Metaverse and their implications for IoT applications within this framework. Therefore, this study provides a detailed overview of the literature on the potential applications, opportunities, and challenges pertaining to the deployment of XR technologies in IoT applications within the broader framework of IoCT. The primary focus is on navigating the challenges pertaining to the IoT applications powered by VR and AR as key components of MR in the Metaverse. This study also explores the emerging computing paradigm of AIoT and its synergistic interplay with XR technologies in the Metaverse and in relation to future IoT applications in the realm of IoCT. This study’s contributions encompass a comprehensive literature overview of XR technologies in IoT and IoCT, providing a valuable resource for researchers and practitioners. It identifies challenges and resource constraints, identifying areas that require further investigation. It fosters interdisciplinary insights into XR, IoT, AIoT, smart cities, and IoCT, bridging the gap between them. Lastly, it offers innovation pathways for effective XR deployment in future IoT/AIoT applications within IoCT. These contributions collectively advance our understanding of synergistic opportunities and complementary strengths of cutting-edge technologies for advancing the emerging paradigms of urban development.
Artificial intelligence in diagnostic pathology

Digital pathology (DP) is being increasingly employed in cancer diagnostics, providing additional tools for faster, higher-quality, accurate diagnosis. The practice of diagnostic pathology has gone through a staggering transformation wherein new tools such as digital imaging, advanced artificial intelligence (AI) algorithms, and computer-aided diagnostic techniques are being used for assisting, augmenting and empowering the computational histopathology and AI-enabled diagnostics. This is paving the way for advancement in precision medicine in cancer. Automated whole slide imaging (WSI) scanners are now rendering diagnostic quality, high-resolution images of entire glass slides and combining these images with innovative digital pathology tools is making it possible to integrate imaging into all aspects of pathology reporting including anatomical, clinical, and molecular pathology. The recent approvals of WSI scanners for primary diagnosis by the FDA as well as the approval of prostate AI algorithm has paved the way for starting to incorporate this exciting technology for use in primary diagnosis. AI tools can provide a unique platform for innovations and advances in anatomical and clinical pathology workflows. In this review, we describe the milestones and landmark trials in the use of AI in clinical pathology with emphasis on future directions.