A one-shot domain-independent robust multimedia clustering methodology based on hybrid multimodal fusion
The existence of multiple modalities poses a challenge to the design of multimedia data clustering systems, as the unsupervised nature of the problem makes it very difficult to determine a priori whether a single modality should dominate the clustering process, or if modalities should be combined somehow. In order to fight against these indeterminacieswhich come on top of those referring to the selection of the optimal clustering algorithm and data representation for the problem at hand, this work introduces robust multimedia clustering, a one-shot methodology for domain independent multimedia data clustering based on hybrid multimodal fusion. By means of experimentation, we firstly justify the motivation of the proposed methodology by proving the relevance of multimedia clustering indeterminacies. Subsequently, a specific multimedia clustering system based on the requirements of the methodology is implemented and evaluated on three multimedia clustering applicationsmusic genres, photographic topics and audio-visual objects classificationas a proof of concept, analyzing the quality of the obtained partitions and the time complexity of the proposal. The experimental results reveal that the implemented system, which includes a self-refining consensus clustering procedure for attaining high levels of robustness, allows to obtain, in a fully unsupervised manner, better quality partitions than 93% of the clusterers available in our experiments, being even able to improve the quality of the best ones and outperforming state-of-the-art alternatives.
robust multimedia clustering hybrid multimodal fusion clustering indeterminacies self-refining consensus cluster ensembles