Reviewerly/exHarmony huggingface dataset:
On the Impact of Interactive eXtended Reality: Challenges and Opportunities for Multimedia Research

Extended Reality (XR) has been hailed as the new frontier of media, ushering new possibilities for societal areas such as communications, training, entertainment, gaming, and cultural heritage. However, despite the remarkable technical advances, current XR applications are in their majority local and individual experiences. In fact, three great barriers stand between current technology and remote immersive interactive life-like experiences, namely content realism, by means of Artificial Intelligence (AI) techniques, motion-to-photon latency, and accurate human-centric driven experiences able to map real and virtual worlds seamlessly. Overcoming these barriers will require novel solutions at all elements of the end-to-end transmission chain. In this panel, together with the leading experts of the SIGMM community, we will explore the challenges and opportunities to unlock the next generation of interactive XR applications and services.
Incomplete Multi-View Clustering with Regularized Hierarchical Graph

In this article, we propose a novel and effective incomplete multi-view clustering (IMVC) framework, referred to as incomplete multi-view clustering with regularized hierarchical graph (IMVC_RHG). Different from the existing graph learning-based IMVC methods, IMVC_RHG introduces a novel heterogeneous-graph learning and embedding strategy, which adopts the high-order structures between four tuples for each view, rather than a simple paired-sample intrinsic structure. Besides this, with the aid of the learned heterogeneous graphs, a between-view preserving strategy is designed to recover the incomplete graph for each view. Finally, a consensus representation for each sample is gained with a co-regularization term for final clustering. As a result of integrating these three learning strategies, IMVC_RHG can be flexibly applied to different types of IMVC tasks. Comparing with the other state-of-the-art methods, the proposed IMVC_RHG can achieve the best performances on real-world incomplete multi-view databases.
Toward Human Perception-Centric Video Thumbnail Generation

Video thumbnail plays an essential role in summarizing video content into a compact and concise image for users to browse efficiently. However, automatically generating attractive and informative video thumbnails remains an open problem due to the difficulty of formulating human aesthetic perception and the scarcity of paired training data. This work proposes a novel Human Perception-Centric Video Thumbnail Generation (HPCVTG) to address these challenges. Specifically, our framework first generates a set of thumbnails using a principle-based system, which conforms to established aesthetic and human perception principles, such as visual balance in the layout and avoiding overlapping elements. Then rather than designing from scratch, we ask human annotators to evaluate some of these thumbnails and select their preferred ones. A Transformer-based Variational Auto-Encoder (VAE) model is firstly pre-trained with Model-Agnostic Meta-Learning (MAML) and then fine-tuned on these human-selected thumbnails. The exploration of combining the MAML pre-training paradigm with human feedback in training can reduce human involvement and make the training process more efficient. Extensive experimental results show that our HPCVTG framework outperforms existing methods in objective and subjective evaluations, highlighting its potential to improve the user experience when browsing videos and inspire future research in human perception-centric content generation tasks. The code and dataset will be released via https://github.com/yangtao2019yt/HPCVTG.
VTLayout: A Multi-Modal Approach for Video Text Layout

The rapid explosion of video distribution is accompanied by a massive amount of video text, which encompasses rich information about the video content. While previous research has primarily focused on text extraction from videos like text detection, tracking, recognition and end to end spotting, the layout of video text has received limited attention. As different text categories convey distinct meanings, video text layout is critical for video understanding tasks such as video summarization and shooting environment comprehension. To bridge the gap between video OCR and understanding, we explore the study of video text layout in this work. We first optimize the layout annotation of the BOVText, a bilingual, open-world video text dataset, by expanding text categories and defining five clear categories: scene, subtitle, title, logo, and other. Additionally, we rectify the original unreasonable layout annotation based on these definitions. We also propose a Video-level Text Layout model (VTLayout) to address the layout problem, which fuses textual, visual, and spatial-temporal embedding of video text trajectories. To the best of our knowledge, this is the first method to tackle text layout on video level. Our method outperforms image-level layout methods across all text categories and exhibits faster inference speed. This study underscores the significance of video text layout in video understanding and offers an effective solution to this challenge. Our annotation is available at https://github.com/TencentARC/VTLayout.
MMSports '23: 6th International Workshop on Multimedia Content Analysis in Sports

The sixth ACM International Workshop on Multimedia Content Analysis in Sports (ACM MMSports'23) is part of the ACM International Conference on Multimedia 2023 (ACM Multimedia 2023). The goal of this workshop is to bring together researchers and practitioners from academia and industry to address challenges and report progress in mining, analyzing, understanding, and visualizing multimedia/multimodal data in sports, sports broadcasts, sports games and sports medicine. The combination of sports and modern technology offers a novel and intriguing field of research with promising approaches for visual broadcast augmentation and understanding, for statistical analysis and evaluation, and for sensor fusion during workouts as well as competitions. There is a lack of research communities focusing on the fusion of multiple modalities. We are helping to close this research gap with this workshop series on multimedia content analysis in sports.
SimHMR: A Simple Query-based Framework for Parameterized Human Mesh Reconstruction

Human Mesh Reconstruction (HMR) aims to recover 3D human poses and shapes from a single image. Existing parameterized HMR approaches follow the "representation-to-reasoning'' paradigm to predict human body and pose parameters. This paradigm typically involves intermediate representation and complex pipeline, where potential side effects may occur that could hinder performance. In contrast, query-based non-parameterized methods directly output 3D joints and mesh vertices, but they rely on excessive queries for prediction, leading to low efficiency and robustness. In this work, we propose a simple query-based framework, dubbed SimHMR, for parameterized human mesh reconstruction. This framework streamlines the prediction process by using a few parameterized queries, which effectively removes the need for hand-crafted intermediate representation and reasoning pipeline. Different from query-based non-parameterized HMR that uses excessive coordinate queries, SimHMR only requires a few semantic queries, which physically correspond to pose, shape, and camera. The use of semantic queries significantly improves the efficiency and robustness in extreme scenarios, e.g., occlusions. Without bells and whistles, øurs achieves state-of-the-art performance on 3DPW and Human3.6M benchmarks, and surpasses existing methods on challenging 3DPW-OCC. Code available at https://github.com/inso-13/SimHMR github.com/inso-13/SimHMR
ProTegO: Protect Text Content against OCR Extraction Attack

Online documents greatly improve the efficiency of information interaction but also cause potential security hazards, such as the ability to copy and reuse text content without authorization readily. To address copyright concerns, recent works have proposed converting reproducible text content into non-reproducible formats, making digital text content observable but not duplicable. However, as the Optical Character Recognition (OCR) technology develops, adversaries can still take screenshots of the target text region and use OCR to extract the text content. None of the existing methods can be well adapted to this kind of OCR extraction attack. In this paper, we propose "ProTegO'', a novel text content protection method against the OCR extraction attack, which generates adversarial underpaintings that do not affect human reading but can interfere with OCR after taking screenshots. Specifically, we design a text-style universal adversarial underpaintings generation framework, which can mislead both text recognition models and commercial OCR services. For invisibility, we take full advantage of the fusion property of human eyes and create complementary underpaintings to display alternatively on the screen. Experimental results demonstrate that ProTegO is a one-size-fits-all method that can ensure good visual quality while simultaneously achieving a high protection success rate on text recognition models with different architectures, outperforming the state-of-the-art methods. Furthermore, we validate the feasibility of ProTegO on a wide range of popular commercial OCR services, including Microsoft, Tencent, Alibaba, Huawei, Baidu, Apple, and Xiaomi. Codes will be available at https://github.com/Ruby-He/ProTegO.
MVFlow: Deep Optical Flow Estimation of Compressed Videos with Motion Vector Prior

In recent years, many deep learning-based methods have been proposed to tackle the problem of optical flow estimation and achieved promising results. However, they hardly consider that most videos are compressed and thus ignore the pre-computed information in compressed video streams. Motion vectors, one of the compression information, record the motion of the video frames. They can be directly extracted from the compression code stream without computational cost and serve as a solid prior for optical flow estimation. Therefore, we propose an optical flow model, MVFlow, which uses motion vectors to improve the speed and accuracy of optical flow estimation for compressed videos. In detail, MVFlow includes a key Motion-Vector Converting Module, which ensures that the motion vectors can be transformed into the same domain of optical flow and then be utilized fully by the flow estimation module. Meanwhile, we construct four optical flow datasets for compressed videos containing frames and motion vectors in pairs. The experimental results demonstrate the superiority of our proposed MVFlow, which can reduce the AEPE by 1.09 compared to existing models or save 52% time to achieve similar accuracy to existing models.
Ada3Diff: Defending against 3D Adversarial Point Clouds via Adaptive Diffusion

Deep 3D point cloud models are sensitive to adversarial attacks, which poses threats to safety-critical applications such as autonomous driving. Robust training and defend-by-denoising are typical strategies for defending adversarial perturbations. However, they either induce massive computational overhead or rely heavily upon specified priors, limiting generalized robustness against attacks of all kinds. To remedy it, this paper introduces a novel distortion-aware defense framework that can rebuild the pristine data distribution with a tailored intensity estimator and a diffusion model. To perform distortion-aware forward diffusion, we design a distortion estimation algorithm that is obtained by summing the distance of each point to the best-fitting plane of its local neighboring points, which is based on the observation of the local spatial properties of the adversarial point cloud. By iterative diffusion and reverse denoising, the perturbed point cloud under various distortions can be restored back to a clean distribution. This approach enables effective defense against adaptive attacks with varying noise budgets, enhancing the robustness of existing 3D deep recognition models.
The Impact of Eastern philosophy and Thought on American Poets

Many American poets of the 19th and 20th centuries were influenced by Eastern philosophy. Poets embraced oriental thought &amp; spirituality. These themes are reflected in their poetry through symbolism, references and imagery. These poets find peace, solace and shelter in eastern philosophy. They searched immensely for the solution to their problems but found materialism and brokenness. T.S.Eliot, Walt Whitman, Ralph Waldo Emerson and Henry David Thoreau expressed their views which resonate with eastern philosophy and spirituality. T.S. Eliot found peace in the Upanishad and the teachings of the Bhagvad Gita. One should practice the triad Da Da Da as proposed by T.S. Eliot in his Magnum opus “The Waste Land”. Walt Whitman was not a preacher of the Hindu religion but his poetry echoes oriental philosophy and ideas like non-dualism, transcendentalism, self-reliance interconnectedness, universal soul and rejection of materialism. Whitman integrated oriental and Western thought. These writers weave above mentioned themes in their verses. Henry David Thoreau wants to lead the life of a recluse through his essay Walden. Walden is the product of Henry David Thoreau's life experiments. He chooses life in the woods over materialism and wants other Americans to follow the same path. Gandhi was greatly influenced by Walden. In Walden, Thoreau advocates a life of simplicity and self-reliance. Walden is made into many chapters, each part describing one or the other aspects of simple and common life. Thoreau’s ideas on life are divided into different chapters such as economy, life, reading, solitude, the village, the ponds and higher laws. Ralph Waldo Emerson was an American poet and essayist and was one of the members of a group of New England idealists known as transcendentalists. He magically weaved God, nature and individuals into one whole. Emerson says that prayer is the disease of the soul as a creed is the disease of the intellect. He proposes that one should not pray to beg for worldly pleasures. He also puts forth that one should not travel as travel is not the food of the soul.
SECONDARY SCHOOL STUDENTS’ GRAPHIC COMPETENCE FORMATION WHILE CREATING SKETCHES OF GRAPHIC MATERIAL IN CONDITIONS OF DIGITIZATION

The study is devoted to the analysis of the process of secondary school students’ graphic competence formation in the context of modern transformations caused by the process of education digitization. The article examines the importance of graphic competence formation in creating sketches of graphic material in the context of the growing role of digital technologies in the educational process. The article analyzes scientific research aimed at understanding graphic competence and its role in the modern educational process.&#x0D; The author has analyzed various approaches to defining the category «competence» and revealed the main components of graphic competence. It is shown that this competence includes not only the technical skills of creating graphic materials but also creativity, the ability to analyze and visualize information, as well as the ability to work with various tools and resources that are opened up thanks to the digital transformation of education.&#x0D; The influence of digitalization on students’ creative potential and their ability to translate ideas into graphic material has been studied. A comprehensive approach to the educational process that takes into account traditional methods and modern information technologies is proposed. The advantages associated with the effectiveness of pedagogical approaches and methods aimed at graphic competence formation are highlighted. In particular, the teacher’s role in this competence formation has been investigated and the importance of an individual approach to each student has been emphasized.&#x0D; Examples of pedagogical practices aimed at creating conditions for the development of creative thinking and secondary school students’ graphic competence formation are considered. The author has considered the possibilities of using various software tools and online resources to increase the level of students’ graphic competence.&#x0D; Based on the analysis, recommendations aimed at further improving students’ graphic competence in the conditions of digitization of education are proposed. These recommendations are aimed at increasing the quality level of education and development of students so that they can successfully form graphic competence in the modern information society.&#x0D; The article highlights the significant topic of secondary school students’ graphic competence formation in the conditions of the digital transformation of education. It offers specific practical recommendations and examples for usage in pedagogical practice and promotes the further development of this key educational field. The study emphasizes the need to acquire competencies that allow the effective usage of graphic tools in the modern information society. The author emphasizes the importance of introducing digital tools into the educational process aimed at increasing the level of students’ graphic literacy and developing their creative competence.
DEVELOPMENT OF A SYSTEM FOR DIAGNOSING PESTS AND DISEASES IN CORN PLANTS BASED ON THE INTENSITY OF DAMAGE USING THE CERTAINTY FACTOR METHOD

Corn plants have the potential to be attacked by pests or diseases that can appear at any time such as leaf blight, leaf rust and downy mildew. Obstacles facing farmers are sometimes the lack of information about the symptoms found so that sometimes the diagnosis is not correct and the treatment is wrong, resulting in reduced corn production. Along with technological advances, a technology has also been produced that can adopt the process of human thinking, namely artificial intelligence technology or AI or Artificial Intelligence. including an expert system, which is a computer-based system that adopts the knowledge and experience of an expert, and reasoning techniques to solve a problem, similar to how an expert in a particular field solves the problem. The development that will be made is "Development of a Diagnostic System for Pests and Diseases in Corn Plants Based on Damage Intensity Using the Certainty Factor Method".
CogniWatch: Advanced Scene Analysis for Threat Detection and Public Safety

Ensuring the safety of life and property through the deployment of high-quality CCTV cameras has become indispensable in our modern world. Manual monitoring of every moment is not feasible, and the unpredictable nature of human behavior makes distinguishing between suspicious and normal activities a formidable challenge. In this research, we introduce a novel approach utilizing Convolutional Neural Networks (CNN) to discern between suspicious and routine activities within an environment. Our proposed system is designed to automatically alert relevant authorities upon detecting potentially suspicious behavior. The effectiveness of any suspicious activity detection system hinges on several critical factors, including the quality of training data, the architecture of the Machine Learning model, and the operational environment. To maintain the system's accuracy and keep it adaptive to new and evolving threats, continuous monitoring, regular updates, and ongoing improvement are imperative. Our work underscores the importance of robust data sources and the careful design of CNN-based models to ensure the system's reliability in real-world applications. This research not only addresses the pressing need for automated surveillance but also emphasizes the significance of staying current and vigilant in the ever- changing landscape of safety and security. KeyWords: CNN, Object Detection, Anomaly, Threat, Hyperparameter Tunning.
The reform of English translation teaching mode in the context of “internet plus” and the alleviation of students’ learning fear and escape symptoms

Background In recent years, research on the influencing factors of students’ English learning has found that students’ psychological factors are the core factor affecting their learning effectiveness. The innovation of English translation teaching mode in the context of “Internet plus” is of great significance to promote the improvement of English translation teaching quality, which can enhance students’ learning enthusiasm, effectively alleviate students’ learning fear and escape symptoms. Subjects and Methods We selected 120 students from a certain university who have phobia and avoidance symptoms towards English learning as the research subjects. They were divided into an experimental group and a control group. During the research period, the experimental group used the established English translation teaching mode for teaching, while the control group used the traditional teaching mode. The mental health status of two groups of students is analyzed after the three-month experimental period. Results The study used Excel 2010 and SPSS 23.0 to organize and analyze the data. The results showed that there was no statistically significant difference between the experimental group and the control group in 12 dimensions such as somatization and anxiety ( P &gt;0.05). After intervention, the scores of all dimensions in the experimental group were lower than those in the control group, and the differences were statistically significant ( P &lt;0.05). Conclusions From this, it can be seen that after the innovative reform of the English translation teaching model, the experimental group’s CCSMHS scores in all dimensions have significantly decreased. This innovative approach can effectively improve students’ mental health status.
Psychology-based analysis of cognitive barriers of corporate financial risk assessors and countermeasures

Background In corporate financial risk assessment, the cognitive ability of personnel plays an important role in accurately assessing and predicting risks. Therefore, it is of great significance to study the cognitive impairment of corporate financial risk assessment personnel and their countermeasures by combining psychological analysis. Subjects and Methods This study used psychological scales and cognitive paradigms for research. The study selected 50 staff members from a financial enterprise as the research subjects and randomly divided them into an experimental group and a control group, each consisting of 25 people. Data analysis was conducted using the Simplified Mental Intelligence Scale and the Code-recognition episodic memory paradigm, and the data was processed using SPSS22.0. Results By simplifying the analysis of MMSE sub-items, the two items with the strongest discriminative ability for MCI were identified, namely MMSE12 and MMSE19. The average cognitive speed score of the evaluators in the experimental group was 0.56 seconds, while the average cognitive speed score of the evaluators in the control group was 0.42 seconds. The t-test results showed a significant difference in cognitive speed between the two groups ( P &lt;0.01). Conclusions BABRI-mini MMSE and BABRI-EMT are suitable for rapid screening of cognitive impairment for corporate financial risk assessors. MMSE12 and MMSE19 can be used as valid indicators for assessing cognitive impairment, and the application of the situational memory scale can further improve discriminative ability. This suggests that psychology-based analysis can help corporate financial risk assessors detect and respond to cognitive impairment promptly and improve the accuracy and reliability of assessment. Acknowledgement The Hunan Provincial Department of Science and Technology, Hunan International Economics University (No. 2014GK3147); Provincial soft Science Key Project of Science and Technology Department of Hunan Province (No. 2013ZK2024).