DOI: 10.5281/zenodo.17369249
title: HarmonyNet

description: HarmonyNet: Knowledge Graph-Driven Semantic Reasoning for Music History and Theatrical Plays

Overview

HarmonyNet is an advanced knowledge graph-driven modeling and semantic reasoning framework designed to analyze and uncover the associations between music history events and theatrical plays.

Unlike traditional approaches that struggle with temporal, thematic, and cultural complexity, HarmonyNet integrates:



A Knowledge Graph-Enhanced Model

A Semantic Resonance Strategy

A Multimodal Encoder Architecture


This enables accurate, interpretable, and scalable analysis of how historical musical contexts influence theatrical performances—and vice versa.

The framework contributes to digital humanities by bridging historical data and artistic expression, supporting cultural analytics, education, and interdisciplinary research.



✨ Features

Knowledge Graph-Enhanced Modeling



Represents entities (e.g., music events, composers, plays, playwrights) as graph nodes.

Captures relationships such as inspired by, performed in, and written by.

Supports temporal modeling with event timestamps and logical inference rules.

Enables complex querying and reasoning over cultural networks.


HarmonyNet Architecture



Employs a multimodal encoder and graph convolutional network to integrate and propagate features.

Graphical Propagation Layer enhances information flow between nodes.

Semantic Reasoning Module applies inference rules to deduce hidden connections between events and plays.📎 See Fig. 1–2 in the paper for detailed architecture diagrams.


Semantic Resonance Strategy



Uses SpectFormer-based encoder and attention blocks for contextual temporal alignment.

Maps historical events to play structures (acts, scenes).

Integrates frequency-domain channel attention for robust semantic alignment.

Supports interactive reasoning for real-time human-in-the-loop interpretation.




📊 Datasets




Dataset
Description
Purpose




Music History Events Dataset
Historical records of musical milestones (genres, instruments, composers)
Temporal and cultural analysis


Theatrical Plays Metadata Collection
Play titles, playwrights, venues, production data
Theatrical event modeling


Historical Music-Theater Correlation Dataset
Musical-theatrical intersections such as operas and musical plays
Modeling event-play relationships


Semantic Music and Play Association Dataset
Annotated links between music and plays with thematic tags
Semantic reasoning and evaluation






 




🚀 Usage



Knowledge graph embeddings

Predicted relationships between events and plays

Semantic alignment and reasoning logs

Visualization of temporal-cultural associations




🧪 Applications



Digital Humanities Research — cultural analytics and historical interpretation

Education — interactive tools for exploring music-theater history

Knowledge Graph Reasoning — automated inference of artistic and historical patterns

Content Recommendation — linking events and plays for cultural archives and platforms




🧩 Model Components




Component
Description




Knowledge Graph
Temporal, thematic, cultural representation of entities


Graphical Propagation Layer
Feature propagation across nodes


Multimodal Encoder
Integrates multiple feature types


Semantic Resonance Strategy
Enhances contextual reasoning


Interactive Reasoning Framework
Real-time interpretability and refinement






📈 Performance




Dataset
Accuracy
Recall
F1 Score
AUC




Music History Events
89.12
88.47
87.71
88.13


Theatrical Plays Metadata
91.34
90.78
90.02
90.45


Historical Music-Theater Correlation
89.53
88.94
88.36
88.65


Semantic Music & Play Association
91.25
90.72
90.18
90.45




📎 See Table 1–2 for detailed SOTA comparisons. HarmonyNet outperforms ResNet, ViT, I3D, BLIP, DenseNet, and MobileNet in accuracy and efficiency.



🧭 Future Work



Automatic enrichment of data sources to improve knowledge graph coverage.

More advanced NLP techniques to enhance interpretability.

Integration of domain-specific ontologies for richer cultural semantics.

Real-time interactive applications for education and cultural heritage projects.




📜 License

This project is licensed under the MIT License.



🙏 Acknowledgments

This work was supported by the 2024 Doctoral Research Startup Fund Project of Liaodong University (Project No. 2024BS029).Author: Cui Shiling.This research bridges knowledge graph modeling, semantic reasoning, and cultural analysis, providing new tools to understand the dynamic relationship between music history and theatrical plays.