Reviewerly/exHarmony huggingface dataset:
ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories

Diffusion models have recently exhibited remarkable abilities to synthesize striking image samples since the introduction of denoising diffusion probabilistic models (DDPMs). Their key idea is to disrupt images into noise through a fixed forward process and learn its reverse process to generate samples from noise in a denoising way. For conditional DDPMs, most existing practices relate conditions only to the reverse process and fit it to the reversal of unconditional forward process. We find this will limit the condition modeling and generation in a small time window. In this paper, we propose a novel and flexible conditional diffusion model by introducing conditions into the forward process. We utilize extra latent space to allocate an exclusive diffusion trajectory for each condition based on some shifting rules, which will disperse condition modeling to all timesteps and improve the learning capacity of model. We formulate our method, which we call ShiftDDPMs, and provide a unified point of view on existing related methods. Extensive qualitative and quantitative experiments on image synthesis demonstrate the feasibility and effectiveness of ShiftDDPMs.
A Safe Genetic Algorithm Approach for Energy Efficient Federated Learning in Wireless Communication Networks

Federated Learning (FL) has emerged as a decentralized technique, where contrary to traditional centralized approaches, devices perform a model training in a collaborative manner, while preserving data privacy. Despite the existing efforts made in FL, its environmental impact is still under investigation, since several critical challenges regarding its applicability to wireless networks have been identified. Towards mitigating the carbon footprint of FL, the current work proposes a Genetic Algorithm (GA) approach, targeting the minimization of both the overall energy consumption of an FL process and any unnecessary resource utilization, by orchestrating the computational and communication resources of the involved devices, while guaranteeing a certain FL model performance target. A penalty function is introduced in the offline phase of the GA that penalizes the strategies that violate the constraints of the environment, ensuring a safe GA process. Evaluation results show the effectiveness of the proposed scheme compared to two state-of-the-art baseline solutions, achieving a decrease of up to 83% in the total energy consumption.
Impact of virtual reality immersion on customer experience: moderating effect of cross-sensory compensation and social interaction

Purpose In order to encourage customers to try experience virtual tourism, researchers and practitioners pay more attention on how to improve customers' perception of authenticity. Design/methodology/approach Using the theory of cross-sensory compensation, through 4 experiments, this study examines the impact of social interaction and cross-sensory strategies on the relationship between virtual reality (VR) and improving customers' perception of authenticity and customer experience. Findings Through experimental research, this paper finds that (1) VR immersion has a significant positive impact on perception of authenticity, which in turn acts on customers' virtual tourism experience. (2) In addition, social interaction strengthens the relationship between VR immersion and customers' perception of authenticity. (3) The cross-sensory compensation has a positive effect on VR immersion to enhance customers' perception of authenticity. Originality/value The research conclusion provides a direction for further discussion on how to improve customers' perception of authenticity and provides theoretical guidance and reference for the virtual tourism industry.
PeopleNet: A Novel People Counting Framework for Head-Mounted Moving Camera Videos

Traditional crowd counting (optical flow or feature matching) techniques have been upgraded to deep learning (DL) models due to their lack of automatic feature extraction and low-precision outcomes.Most of these models were tested on surveillance scene crowd datasets captured by stationary shooting equipment.It is very challenging to perform people counting from the videos shot with a head-mounted moving camera; this is mainly due to mixing the temporal information of the moving crowd with the induced camera motion.This study proposed a transfer learning-based PeopleNet model to tackle this significant problem.For this, we have made some significant changes to the standard VGG16 model, by disabling top convolutional blocks and replacing its standard fully connected layers with some new fully connected and dense layers.The strong transfer learning capability of the VGG16 network yields in-depth insights of the PeopleNet into the good quality of density maps resulting in highly accurate crowd estimation.The performance of the proposed model has been tested over a self-generated image database prepared from moving camera video clips, as there is no public and benchmark dataset for this work.The proposed framework has given promising results on various crowd categories such as dense, sparse, average, etc.To ensure versatility, we have done self and cross-evaluation on various crowd counting models and datasets, which proves the importance of the PeopleNet model in adverse defense of society.
Cross-Domain Multi-Prototypes with Contradictory Structure Learning for Semi-Supervised Domain Adaptation Segmentation of Remote Sensing Images

Recently, unsupervised domain adaptation (UDA) segmentation of remote sensing images (RSIs) has attracted a lot of attention. However, the performance of such methods still lags far behind that of their supervised counterparts. To this end, this paper focuses on a more practical yet under-investigated problem, semi-supervised domain adaptation (SSDA) segmentation of RSIs, to effectively improve the segmentation results of targeted RSIs with a few labeled samples. First, differently from the existing single-prototype mode, a novel cross-domain multi-prototype constraint is proposed, to deal with large inter-domain discrepancies and intra-domain variations. Specifically, each class is represented as a set of prototypes, so that multiple sets of prototypes corresponding to different classes can better model complex inter-class differences, while different prototypes within the same class can better describe the rich intra-class relations. Meanwhile, the multi-prototypes are calculated and updated jointly using source and target samples, which can effectively promote the utilization and fusion of the feature information in different domains. Second, a contradictory structure learning mechanism is designed to further improve the domain alignment, with an enveloping form. Third, self-supervised learning is adopted, to increase the number of target samples involved in prototype updating and domain adaptation training. Extensive experiments verified the effectiveness of the proposed method for two aspects: (1) Compared with the existing SSDA methods, the proposed method could effectively improve the segmentation performance by at least 7.38%, 4.80%, and 2.33% on the Vaihingen, Potsdam, and Urban datasets, respectively; (2) with only five labeled target samples available, the proposed method could significantly narrow the gap with its supervised counterparts, which was reduced to at least 4.04%, 6.04%, and 2.41% for the three RSIs.
Case Study of Ceramic Firing Profile for Terracotta-Based Glaze

Terracotta clay is a natural resource that has the potential as a raw material in the ceramic field. This study investigates the effects of color glaze using terracotta clay powder as an alternative substance for ceramic colorants in glaze formulation. Terracotta clay powder is proportionately added to glaze formulations to determine the potential for color added to the glaze at temperatures ranging from 1140 °C to 1200 °C. The content in terracotta clay is bounded with potash feldspar, silica, zinc oxide, and calcium carbonate. Results show the apparent color of the surface on the ceramic glaze samples with different effects.
Towards Improved Illicit Node Detection with Positive-Unlabelled Learning

Detecting illicit nodes on blockchain networks is a valuable task for strengthening future regulation. Recent machine learning-based methods proposed to tackle the tasks are using some blockchain transaction datasets with a small portion of samples labeled positive and the rest unlabelled (PU). Albeit the assumption that a random sample of unlabeled nodes are normal nodes is used in some works, we discuss that the label mechanism assumption for the hidden positive labels, and its effect on the evaluation metrics is worth considering. We further explore that PU classifiers dealing with potential hidden positive labels can have improved performance compared to regular machine learning models. We test the PU classifiers with a list of graph representation learning methods for obtaining different feature distributions for the same data to have more reliable results.
Information Controls and Internet Shutdowns in African Elections: The Politics of Electoral Integrity and Abuses of Power

Internet shutdowns in Africa are becoming increasingly widespread, particularly when governments face competitive or contentious elections. They have also come to symbolise a widening fracture between competing conceptions of the global Internet and its regulation. Governments in Africa are justifying shutdowns as able address misinformation and disinformation, protect the election process, and ensure national security. International organisations, NGOs, and social networking platforms condemn these as an inadmissible form of censorship and information control, an abuse by political actors seeking to silence critics or manipulate elections. This article offers an alternative reading on internet shutdowns by placing them in the historical context of the wide range of information controls around elections, many of which are widely regarded as being acceptable and legitimate mechanisms to support competitive elections. By offering this context, we can ask what is new about shutdowns and whether they can ever be regarded as a proportionate response to real concerns of social media and election manipulation. We conclude by highlighting the inequalities of online content moderation as an often-overlooked factor in driving the use of shutdowns, and the failure of social media companies to effectively address misinformation and disinformation in Africa, particularly around elections.
DAOs: Governance in the Blockchain Era

Blockchain technology promises to revolutionize not only the way we transact among peers but also the way we organize to create socio-economic value. Decentralized autonomous organizations (DAOs) are governed and owned by the community whose members follow a set of blockchain-embedded governance rules that define and control participation. In this chapter, we will clarify what governance decentralization and automation mean, examining DAOs’ distinguishing characteristics. We will also discuss the problems that DAOs solve (e.g., lack of extrinsic incentives, censorship, mismanagement and lack of transparency and accountability), as well as the problems they face (e.g., lack of participation, rigidity, voting misbehaviours and legal status).
A Clustering-guided Contrastive Fusion for Multi-view Representation Learning

Multi-view representation learning aims to extract comprehensive information from multiple sources. It has achieved significant success in applications such as video understanding and 3D rendering. However, how to improve the robustness and generalization of multi-view representations from unsupervised and incomplete scenarios remains an open question in this field. In this study, we discovered a positive correlation between the semantic distance of multi-view representations and the tolerance for data corruption. Moreover, we found that the information ratio of consistency and complementarity significantly impacts the performance of discriminative and generative tasks related to multi-view representations. Based on these observations, we propose an end-to-end CLustering-guided cOntrastiVE fusioN (CLOVEN) method, which enhances the robustness and generalization of multi-view representations simultaneously. To balance consistency and complementarity, we design an asymmetric contrastive fusion module. The module first combines all view-specific representations into a comprehensive representation through a scaling fusion layer. Then, the information of the comprehensive representation and view-specific representations is aligned via contrastive learning loss function, resulting in a view-common representation that includes both consistent and complementary information. We prevent the module from learning suboptimal solutions by not allowing information alignment between view-specific representations. We design a clustering-guided module that encourages the aggregation of semantically similar views. This action reduces the semantic distance of the view-common representation. We quantitatively and qualitatively evaluate CLOVEN on five datasets, demonstrating its superiority over 13 other competitive multi-view learning methods in terms of clustering and classification performance. In the data-corrupted scenario, our proposed method resists noise interference better than competitors. Additionally, the visualization demonstrates that CLOVEN succeeds in preserving the intrinsic structure of view-specific representations and improves the compactness of view-common representations. Our code can be found at https://github.com/guanzhou-ke/cloven.
Continual Training of Language Models for Few-Shot Learning

Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications.Adapting or posttraining an LM using an unlabeled domain corpus can produce even better performance for end-tasks in the domain.This paper proposes the problem of continually extending an LM by incrementally post-train the LM with a sequence of unlabeled domain corpora to expand its knowledge without forgetting its previous skills.The goal is to improve the few-shot end-task learning in these domains.The resulting system is called CPT (Continual Post-Training), which to our knowledge, is the first continual post-training system.Experimental results verify its effectiveness.
Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution

We propose the first character-level white-box adversarial attack method against transformer models.The intuition of our method comes from the observation that words are split into subtokens before being fed into the transformer models and the substitution between two close subtokens has a similar effect to the character modification.Our method mainly contains three steps.First, a gradient-based method is adopted to find the most vulnerable words in the sentence.Then we split the selected words into subtokens to replace the origin tokenization result from the transformer tokenizer.Finally, we utilize an adversarial loss to guide the substitution of attachable subtokens in which the Gumbel-softmax trick is introduced to ensure gradient propagation.Meanwhile, we introduce the visual and length constraint in the optimization process to achieve minimum character modifications.Extensive experiments on both sentence-level and token-level tasks demonstrate that our method could outperform the previous attack methods in terms of success rate and edit distance.Furthermore, human evaluation verifies our adversarial examples could preserve their origin labels.
Task Compass: Scaling Multi-task Pre-training with Task Prefix

Leveraging task-aware annotated data as supervised signals to assist with self-supervised learning on large-scale unlabeled data has become a new trend in pre-training language models.Existing studies show that multitask learning with large-scale supervised tasks suffers from negative effects across tasks.To tackle the challenge, we propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks.We conduct extensive experiments on 40 datasets, which show that our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships.The task relationships reflected by the prefixes align transfer learning performance between tasks.They also suggest directions for data augmentation with complementary tasks, which help our model achieve human-parity results on commonsense reasoning leaderboards.
Long Text and Multi-Table Summarization: Dataset and Method

Automatic document summarization aims to produce a concise summary covering the input document's salient information.Within a report document, the salient information can be scattered in the textual and non-textual content.However, existing document summarization datasets and methods usually focus on the text and filter out the non-textual content.Missing tabular data can limit produced summaries' informativeness, especially when summaries require covering quantitative descriptions of critical metrics in tables.Existing datasets and methods cannot meet the requirements of summarizing long text and multiple tables in each report.To deal with the scarcity of available data, we propose FINDSum, the first largescale dataset for long text and multi-table summarization.Built on 21,125 annual reports from 3,794 companies, it has two subsets for summarizing each company's results of operations and liquidity.To summarize the long text and dozens of tables in each report, we present three types of summarization methods.Besides, we propose a set of evaluation metrics to assess the usage of numerical information in produced summaries.Dataset analyses and experimental results indicate the importance of jointly considering input textual and tabular data when summarizing report documents.
Multi Datasource LTV User Representation (MDLUR)

In this paper, we propose a novel user representation methodology called Multi Datasource LTV User Representation (MDLUR). Our model aims to establish a universal user embedding for downstream tasks, specifically lifetime value (LTV) prediction on specific days after installation. MDLUR uses a combination of various data sources, including user information, portrait, and behavior data from the first n days after installation of the social casino game "Club Vegas Slots" developed by Bagelcode. This model overcomes the limitation of conventional approaches that struggle with effectively utilizing various data sources or accurately capturing interactions in sparse datasets. MDLUR adopts unique model architectures tailored to each data source. Coupled with robust dimensionality reduction techniques, this model succeeds in the effective integration of insights from various data sources. Comprehensive experiments on real-world industrial data demonstrate the superiority of the proposed methods compared to SOTA baselines including Two-Stage XGBoost, WhalesDector, MSDMT, and BST. Not only did it outperform these models, but it has also been efficiently deployed and tested in a live environment using MLOps demonstrating its maintainability. The representation may potentially be applied to a wide range of downstream tasks, including conversion, churn, and retention prediction, as well as user segmentation and item recommendation.
What the DAAM: Interpreting Stable Diffusion Using Cross Attention

Diffusion models are a milestone in text-toimage generation, but they remain poorly understood, lacking interpretability analyses.In this paper, we perform a text-image attribution analysis on Stable Diffusion, a recently opensourced model.To produce attribution maps, we upscale and aggregate cross-attention maps in the denoising module, naming our method DAAM.We validate it by testing its segmentation ability on nouns, as well as its generalized attribution quality on all parts of speech, rated by humans.On two generated datasets, we attain a competitive 58.8-64.8mIoU on noun segmentation and fair to good mean opinion scores (3.4-4.2) on generalized attribution.Then, we apply DAAM to study the role of syntax in the pixel space across head-dependent heat map interaction patterns for ten common dependency relations.We show that, for some relations, the head map consistently subsumes the dependent, while the opposite is true for others.Finally, we study several semantic phenomena, focusing on feature entanglement; we find that the presence of cohyponyms worsens generation quality by 9%, and descriptive adjectives attend too broadly.We are the first to interpret large diffusion models from a visuolinguistic perspective, which enables future research.Our code is at https://github.com/castorini/daam.
CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoning

Commonsense reasoning, aiming at endowing machines with a human-like ability to make situational presumptions, is extremely challenging to generalize.For someone who barely knows about meditation, while is knowledgeable about singing, he can still infer that meditation makes people relaxed from the existing knowledge that singing makes people relaxed by first conceptualizing singing as a relaxing event and then instantiating that event to meditation.This process, known as conceptual induction and deduction, is fundamental to commonsense reasoning while lacking both labeled data and methodologies to enhance commonsense modeling.To fill such a research gap, we propose CAT (Contextualized ConceptuAlization and InsTantiation), a semi-supervised learning framework that integrates event conceptualization and instantiation to conceptualize commonsense knowledge bases at scale.Extensive experiments show that our framework achieves state-of-the-art performances on two conceptualization tasks, and the acquired abstract commonsense knowledge can significantly improve commonsense inference modeling.Our code, data, and fine-tuned models are publicly available at https://github.com/HKUST-KnowComp/CAT.
Shrinking Embeddings for Hyper-Relational Knowledge Graphs

Link prediction on knowledge graphs (KGs) has been extensively studied on binary relational KGs, wherein each fact is represented by a triple.A significant amount of important knowledge, however, is represented by hyperrelational facts where each fact is composed of a primal triple and a set of qualifiers comprising a key-value pair that allows for expressing more complicated semantics.Although some recent works have proposed to embed hyper-relational KGs, these methods fail to capture essential inference patterns of hyperrelational facts such as qualifier monotonicity, qualifier implication, and qualifier mutual exclusion, limiting their generalization capability.To unlock this, we present ShrinkE, a geometric hyper-relational KG embedding method aiming to explicitly model these patterns.ShrinkE models the primal triple as a spatial-functional transformation from the head into a relation-specific box.Each qualifier "shrinks" the box to narrow down the possible answer set and, thus, realizes qualifier monotonicity.The spatial relationships between the qualifier boxes allow for modeling core inference patterns of qualifiers such as implication and mutual exclusion.Experimental results demonstrate ShrinkE's superiority on three benchmarks of hyper-relational KGs.