Yumeng Hou*, Fadel Mamar Seydou & David Cian, Laboratory for Experimental Museology (eM+), EPFL Methodology This work proposes a visual search framework for embodied knowledge that leverages semi- supervised learning to compute motion similarity among the less well-annotated videos. First, a high-dimensional feature model is constructed to encode body movements combining aspects of physical postures, kinematic parameters and spatial-temporal transitions into motion time series. It then models similarity distance between vary-in- length videos using respective methods of dynamic time warping (DTW), radial basis function (RBF) and finally a model averaging approach on top. As validation, we implement a query-by-example search engine in Python and conduct top-k search tasks to evaluate its retrieval effectiveness measured by F1-score and run-time efficiency measured by computational time. Acknowledgments The Re-making of Confucian Rites is a major re-enactment project, as a collaboration between Professor Peng Lin at the Centre of Ritual Studies at Tsinghua University(Beijing), Mr. Tsong-Zung Chang and the Jia Li Hall in Hong Kong, Professor Jeffrey Shaw, City University of Hong Kong, and Professor Sarah Kenderdine, Laboratory of Experimental Museology (eM+) at EPFL. Conclusion This work proposes a motion similarity search framework that performs query-by-example for the videos from the Re-making of Confucian Rites. Experimental results conclude an effective retrieval performance, especially the model averaging approach with a confidence at ~75% for top-3 search tasks. A distance tree of video clusters with associated video contents illustrates its robust clustering quality via obvious inner-cluster similarity and inter- cluster difference with regard to human movements. Furthermore, by prototyping an interactive evaluation and annotation tool, we unveil the potential of applying such approach to searching, annotating and representing embodied knowledge in many other cultural datasets lacking established metadata schema or notation systems. References [1] Tsuchida, Shuhei, Satoru Fukayama, and Masataka Goto. "Query-by-dancing: a dance music retrieval system based on body-motion similarity." International Conference on Multimedia Modeling. Springer, Cham, 2019. [2] Cao, Zhe, et al. "OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields." IEEE transactions on pattern analysis and machine intelligence 43.1 (2019): 172-186. [3] Lin Peng. 2020. Why is re-enactment critical in the research of Confucian Rites. [4] Kenderdine, Sarah, and Jeffrey Shaw. "Archives in Motion." Museum and Archive on the Move: Changing Cultural Institutions in the Digital Era, edited by Oliver Grau Wendy Coones and Viola Rühse (2017): 211-233. For more information, visit EPFL-eM+, or contact yumeng.hou@epfl.ch Introduction Like most intangible and indigenous cultures, Confucian li is a cosmological philosophy featuring “practicality”, hereby the transmission of which is through enacting, performing rather than preserving. By computing the videos of re- performing li-rites in a machine-readable way, we instantiate a search engine that performs query-by-example in less annotated videos, also underscore motion as meaning to explore the embodied aspects of such “living etiquettes”. A Distance Tree of Video Clusters The Similarity-Based Embodied Search Framework A Clustering Evaluation Tool Integrating the Framework Experimental Evaluations of Retrieval Effectiveness with Different Methods