Reviewerly/exHarmony huggingface dataset:
Graph-based Dynamic Interactive Knowledge Tracing

Knowledge Tracing (KT) is a research field that aims to trace the students’ knowledge states based on their historical learning. Much research has explored the value of relations among concepts and proposed to introduce knowledge structure into KT tasks. However, these studies suffer from two major shortcomings: 1) they only consider the dynamic state of students or the stationary property of concepts, but ignore the dynamic state of concepts and questions. 2) they do not make full use of the rich structural information of the relations between questions and concepts. In this paper, we propose a Graph-based Dynamic Interactive Knowledge Tracing (DGKT) to address these limitations. Specifically, we design two embeddings for each student, question, and concept: static embedding and dynamic embedding, which respectively represent stable attributes and time-varying properties. Additionally, DGKT utilizes graph self-supervised learning to enrich the stationary embeddings of questions and concepts. Finally, extensive experiments on three real-world datasets demonstrate that the proposed DGKT outperforms other baseline models.
Sparse Shape Encoding for Topologically Improved Instance Segmentation

Neurophysiological studies suggest that neurons in the intermediate visual area V4 of the primate cortex encode a sparse representation of object shape. While there are metabolic arguments for such sparse representations, there are also potential advantages for inference. Here we explore whether sparse shape encoding can yield benefits for object instance segmentation. Specifically, we encode 2D object shape using a Distance Transform Map (DTM) and learn a sparse basis for this representation. To make use of this encoding, we design and train an instance segmentation head to estimate the sparse coefficients representing the shape of each object, and then recover the estimated shape from the zero-crossing level set of the corresponding DTM. Our novel SparseShape encoding approach produces fewer topological errors than the state of the art, yields competitive mask AP on the COCO benchmark and exhibits superior generalization performance on the Cityscapes instance segmentation task. These results suggest that the sparse shape encoding observed in primate cortex has computational advantages that can benefit computer vision instance segmentation systems.
Fast Fine-Tuning Using Curriculum Domain Adaptation

Current deep neural networks (DNNs) have achieved remarkable accuracy in various downstream tasks. However, their training and fine-tuning are challenging due to several factors, such as limited computational resources, extended training and fine-tuning times, and over-fitting due to small datasets. To address these challenges, we propose a three-stage fast fine-tuning method that efficiently trains DNNs for edge devices. Our method combines curriculum learning and domain adaptation techniques to accelerate training while achieving comparable performance. First, we develop a data curriculum approach, which ranks the dataset according to difficulty and split it into the source domain (containing easy data) and the target domain (containing difficult data). Second, we adapt the pretrained model from the source domain to the target domain using an unsupervised domain adaptation (UDA) method called Deep CORAL. Finally, we continue training the adapted model on the source domain with fewer epochs. Our method achieves high accuracy quickly on various modern neural network architectures and datasets such as CIFAR-10, CIFAR-100, and CINIC-10.
Proof-of-Concept Study to Evaluate the Impact of Spatial Audio on Social Presence and User Behavior in Multi-Modal VR Communication

This paper presents a proof-of-concept study conducted to analyze the effect of simple diotic vs. spatial, position-dynamic binaural synthesis on social presence in VR, in comparison with face-to-face communication in the real world, for a sample two-party scenario. A conversational task with shared visual reference was realized. The collected data includes questionnaires for direct assessment, tracking data, and audio and video recordings of the individual participants' sessions for indirect evaluation. While tendencies for improvements with binaural over diotic presentation can be observed, no significant difference in social presence was found for the considered scenario. The gestural analysis revealed that participants used the same amount and type of gestures in face-to-face as in VR, highlighting the importance of non-verbal behavior in communication. As part of the research, an end-to-end framework for conducting communication studies and analysis has been developed.
Proactivity in the TV Context: Understanding the Relevance and Characteristics of Proactive Behaviours in Voice Assistants

In the context of Television (TV), intelligent voice assistants are still mainly based on reactive behaviours, only responding to users' requests. However, the addition of proactivity in TV assistants could have the potential to reduce the user-assistant interaction effort and bring an additional "human" layer, leading to a more organic adoption of these systems. Within this framework, this paper aims to contribute to the understanding of the relevance of a set of proactive scenarios for TV and get clues on how to design these behaviours to be efficient and avoid intrusion feelings. To this end, focus groups were conducted to discuss the conceptualised scenarios. The results showed that most of the idealized scenarios were relevant for the participants and that they will be positive if some characteristics can be pre-configured regarding the type of communication and intrusion of proactive suggestions.
Zero-shot virtual product placement in videos

Virtual Product Placement (VPP) is an advertising technique that digitally places branded objects into movie or TV show scenes. Despite being a billion-dollar industry, current ad rendering techniques are time-consuming, costly, and executed manually with the help of visual effects (VFX) artists. In this paper, we present a fully automated and generalized framework for placing 2D ads in any linear TV cooking show captured using a single-view camera with minimal camera movements. The framework detects empty spaces, understands the kitchen scene, handles occlusion, renders ambient lighting, and tracks ads. Our framework without requiring access to full video or production camera configuration reduces the time and cost associated with manual post-production ad rendering techniques, enabling brands to reach consumers seamlessly while preserving the continuity of their viewing experience.
Class Instance Balanced Learning for Long-Tailed Classification

The long-tailed image classification task remains important in the development of deep neural networks as it explicitly deals with large imbalances in the class frequencies of the training data. While uncommon in engineered datasets, this imbalance is almost always present in real-world data. Previous approaches have shown that combining cross-entropy and contrastive learning can improve performance on the long-tailed task, but they do not explore the tradeoff between head and tail classes. We propose a novel class instance balanced loss (CIBL), which reweights the relative contributions of a cross-entropy and a contrastive loss as a function of the frequency of class instances in the training batch. This balancing favours the contrastive loss for more common classes, leading to a learned classifier with a more balanced performance across all class frequencies. Furthermore, increasing the relative weight on the contrastive head shifts performance from common (head) to rare (tail) classes, allowing the user to skew the performance towards these classes if desired. We also show that changing the linear classifier head with a cosine classifier yields a network that can be trained to similar performance in substantially fewer epochs. We obtain competitive results on both CIFAR-100-LT and ImageNet-LT.
Tap or Swipe? Effects of Interaction Gestures for Retrieval of Match Statistics via Second Screen on Watching Soccer on TV

Accessing match statistics through second screen while watching soccer matches on TV has grown into a popular practice. Although early works have shown how gestures on touch screens performed under distracting environments, little is known regarding how specific gestures (swiping and tapping) to retrieve information on second screen affect the viewing experience of soccer games on TV. For this, a mixed-method user study, which included prototype tests of watching short clips of a soccer match, questionnaires and short interviews, was conducted with 28 participants. The results revealed that the number of people who preferred tapping was more than the number of people who favored swiping under two different second screen activity time scenarios i.e. On-Play or Off-Play. However, neither swiping nor tapping yield better performance of recalling verbatim match stats and exact comparisons in both On-Play and Off-Play. Participant evaluations in On-Play and interviews give us clues regarding such difference.
Latency-Oriented Elastic Memory Management at Task-Granularity for Stateful Streaming Processing

In a streaming application, an operator is usually instantiated into multiple tasks for parallel processing. Tasks across operators have various memory demands due to different processing logic (e.g., stateful vs. stateless tasks). The memory demands of tasks from the same operator could also vary and fluctuate due to workload variability. Improper memory provision will cause some tasks to have relatively high latency, or even unbound latency that can eventually lead to system instability. We found that the task with the maximum latency of an operator has a significant and even decisive impact on the end-to-end latency.In this paper, we present our task-level memory manager. Based on our quantitative modeling of memory and task-level latency, the manager can adaptively allocate optimal memory size to each task for minimizing the end-to-end latency. We integrate our memory management on Apache Flink. The experiments show that our memory management could significantly reduce end-to-end latency for various applications at different scales and configurations, compared to the Flink native setting.
DAGC: Data-Aware Adaptive Gradient Compression

Gradient compression algorithms are widely used to alleviate the communication bottleneck in distributed ML. However, existing gradient compression algorithms suffer from accuracy degradation in Non-IID scenarios, because a uniform compression scheme is used to compress gradients at workers with different data distributions and volumes, since workers with larger volumes of data are forced to adapt to the same aggressive compression ratios as others. Assigning different compression ratios to workers with different data distributions and volumes is thus a promising solution. In this study, we first derive a function from capturing the correlation between the number of training iterations for a model to converge to the same accuracy, and the compression ratios at different workers; This function particularly shows that workers with larger data volumes should be assigned with higher compression ratios <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> to guarantee better accuracy. Then, we formulate the assignment of compression ratios to the workers as an n-variables chi-square nonlinear optimization problem under fixed and limited total communication constrain. We propose an adaptive gradient compression strategy called DAGC, which assigns each worker a different compression ratio according to their data volumes. Our experiments confirm that DAGC can achieve better performance facing highly imbalanced data volume distribution and restricted communication.
A Reinforcement Learning Approach for Minimizing Job Completion Time in Clustered Federated Learning

Federated Learning (FL) enables potentially a large number of clients to collaboratively train a global model with the coordination of a central cloud server without exposing client raw data. However, the FL model convergence performance, often measured by the job completion time, is hindered by two critical factors: non independent and identically distributed (non-IID) data across clients and the straggler effect. In this work, we propose a clustered FL framework, MCFL, to minimize the job completion time by mitigating the influence of non-IID data and the straggler effect while guaranteeing the FL model convergence performance. MCFL builds upon a two-stage operation: i) a clustering algorithm constructs clusters, each containing clients with similar computing and communications capabilities to combat the straggler effect within a cluster; ii) a deep reinforcement learning (DRL) algorithm based on soft actor-critic with discrete actions intelligently selects a subset of clients from each cluster to mitigate the impact of non-IID data, and derives the number of intra-cluster aggregation iterations for each cluster to reduce the straggler effect among clusters. Extensive testbed experiments are conducted under various configurations to verify the efficacy of MCFL. The results show that MCFL can reduce the job completion time by up to 70% compared with three state-of-the-art FL frameworks.
InnovART2 : À l'écoute des chantiers

À l’écoute des chantiers is an immersive self-guided walking tour, a sound and visual exploration of the industrial heritage of Nantes’ Parc des Chantiers - with ten sound capsules (storytelling, interviews with former shipyard workers, and sound design) and two site-specific large-scale augmented reality projections of the transborder ferry bridge (dismantled in 1958) and the construction of the Brissac car ferry (1955). Accessible along Nantes’ popular green-line tourist path through signage on the ground, this itinerary describes, explains, and puts into perspective a landscape familiar to the people of Nantes, newcomers, as well as tourists, by giving keys to understanding a former industrial site undegoing rapid transformation and in the process of disappearing. The itinerary is available on the Nantes Patrimonia website, downloadable as a smartphone application or activated via QR codes present in the park. Part of Campus France's InnovART2 research project, À l’écoute des chantiers was part of the Voyage à Nantes 2022 and an ongoing part of the Parcours des Écoles, and questions the limits of smart tourism applications in an era of increasing digital privacy concerns that reduce geo-location capabilities.
DeepFT: Fault-Tolerant Edge Computing using a Self-Supervised Deep Surrogate Model

The emergence of latency-critical AI applications has been supported by the evolution of the edge computing paradigm. However, edge solutions are typically resource-constrained, posing reliability challenges due to heightened contention for compute capacities and faulty application behavior in the presence of overload conditions. Although a large amount of generated log data can be mined for fault prediction, labeling this data for training is a manual process and thus a limiting factor for automation. Due to this, many companies resort to unsupervised fault-tolerance models. Yet, failure models of this kind can incur a loss of accuracy when they need to adapt to non-stationary workloads and diverse host characteristics. Thus, we propose a novel modeling approach, DeepFT, to proactively avoid system overloads and their adverse effects by optimizing the task scheduling decisions. DeepFT uses a deep-surrogate model to accurately predict and diagnose faults in the system and co-simulation based self-supervised learning to dynamically adapt the model in volatile settings. Experimentation on an edge cluster shows that DeepFT can outperform state-of-the-art methods in fault-detection and QoS metrics. Specifically, DeepFT gives the highest F1 scores for fault-detection, reducing service deadline violations by up to 37% while also improving response time by up to 9%.
IOT Cloud Teaching Platform Design

The Internet of Things is the intersection and integration of various professional technologies. In order to better understand and learn the related technologies of the Internet of Things, this paper takes the two-way flow of data as the starting point to design a cloud teaching platform for the Internet of Things, from how to obtain data and upload it to the human-machine The interactive control data is issued, and the existing teaching content is integrated with the experimental links. The edge computing, IOT gateway, data storage, WEB front-end development and other links mentioned in the platform are drawn from the applied electronic technology of the college and the application technology of the Internet of Things. , network engineering technology and software technology and other related courses, and the realization process of platform functions, can enable students to deeply understand the underlying design of the Internet of Things and the knowledge structure required to realize the two-way flow of data.
Oblivion: Poisoning Federated Learning by Inducing Catastrophic Forgetting

Federated learning is exposed to model poisoning attacks as compromised clients may submit malicious model updates to pollute the global model. To defend against such attacks, robust aggregation rules are designed for the centralized server to winnow out outlier updates, and to significantly reduce the effectiveness of existing poisoning attacks. In this paper, we develop an advanced model poisoning attack against defensive aggregation rules. In particular, we exploit the catastrophic forgetting phenomenon during the process of continual learning to destroy the memory of the global model. Our proposed framework, called Oblivion, features two special components. The first component prioritizes the weights that have the most influence on the model accuracy for poisoning, which induces a more significant degradation on the global model than equally perturbing all weights. The second component smooths malicious model updates based on the number of selected compromised clients in the current round, adjusting the degree of poisoning to suit the dynamics of each training round. We implement a fully-functional prototype of Oblivion in PLATO, a real-world scalable federated learning framework. Our extensive experiments over three datasets demonstrate that Oblivion can boost the attack performance of model poisoning attacks against unknown defensive aggregation rules.
COIN: Cost-Efficient Traffic Engineering with Various Pricing Schemes in Clouds

The rapid growth of cloud services has brought a significant increase in inter-datacenter traffic. To transfer data among geographically distributed datacenters, cloud providers need to purchase bandwidth from ISPs. The data transferring cost has become one of the major expenses for cloud providers. Therefore, it is essential for a cloud provider to carefully allocate inter-datacenter traffic among the ISPs' links to minimize the costs. Exiting solutions mainly focus on the situations where all links adopt the same pricing scheme. However, in practice, ISPs usually provide multiple pricing schemes for their links due to market competition, which makes the existing solutions nonoptimal. Thus, a new traffic engineering approach that considers various pricing schemes is needed. This paper presents COIN, a new framework for cost-efficient traffic engineering with various pricing schemes. We propose a partition rounding traffic engineering algorithm based on linear independence analysis. The approximation factors and time complexity are formally analyzed. We further conduct large-scale simulations with real- world topologies and datasets. Extensive simulation results show that COIN can save the data transferring cost by up to 54.54% compared with the state-of-the-art solutions.
OPA: One-Predict-All For Efficient Deployment

Deep neural network (DNN) is the de facto standard for running a variety of computer vision applications over mobile and embedded systems. Prior to deployment, a DNN is specialized by training to fit the target use scenario (depending on computing power and visual data input). To handle its costly training and meet diverse deployment needs, a "Train Once, Deploy Everywhere" paradigm has been recently proposed by training one super-network and selecting one out of many sub-networks (part of the super-network) for the target scenario; This empowers efficient DNN deployment at low training cost (training once). However, the existing studies tackle some deployment factors like computing power and source data but largely overlook the impact of their runtime dynamics (say, time-varying visual contents and GPU/CPU workloads). In this work, we propose OPA to cover all these deployment factors, particularly those along with runtime dynamics in visual data contents and computing resources. To quickly and accurately learn which sub-network runs "best" in the dynamic deployment scenario, we devise a "One-Predict-All" approach with no need to run all the candidate sub-networks. Instead, we first develop a shallow sub-network to test the water and then use its test results to predict the performance of all other deeper sub-networks. We have implemented and evaluated OPA. Compared to the state-of-the-art, OPA has achieved up to 26% higher Top-1 accuracy for a given latency requirement.
From Ember to Blaze: Swift Interactive Video Adaptation via Meta-Reinforcement Learning

Maximizing quality of experience (QoE) for interactive video streaming has been a long-standing challenge, as its delay-sensitive nature makes it more vulnerable to bandwidth fluctuations. While reinforcement learning (RL) has demonstrated great potential, existing works are either limited by fixed models or require enormous data/time for online adaptation, which struggle to fit time-varying and diverse network states. Driven by these practical concerns, we perform large-scale measurements on WeChat for Business’s interactive video service to study real-world network fluctuations. Surprisingly, our analysis shows that, compared to time-varying network metrics, network sequences exhibit noticeable short-term continuity, sufficient for few-shot learning requirement. We thus propose Fiammetta, the first meta-RL-based bitrate adaptation algorithm for interactive video streaming. Building on the short-term continuity, Fiammetta accumulates learning experiences through offline meta-training and enables fast online adaptation to changing network states through few gradient updates. Moreover, Fiammetta innovatively incorporates a probing mechanism for real-time monitoring of network states, and proposes an adaptive meta-testing mechanism for seamless adaptation. We implement Fiammetta on a testbed whose end-to-end network follows the real-world WeChat for Business traces. The results show that Fiammetta outperforms prior algorithms significantly, improving video bitrate by 3.6%-16.2% without increasing stalling rate.