Reviewerly/exHarmony huggingface dataset:
CrispSearch

Advances in deep learning have enabled accurate language-based search and retrieval, e.g., over user photos, in the cloud. Many users prefer to store their photos in the home due to privacy concerns. As such, a need arises for models that can perform cross-modal search on resource-limited devices. State-of-the-art cross-modal retrieval models achieve high accuracy through learning entangled representations that enable fine-grained similarity calculation between a language query and an image, but at the expense of having a prohibitively high retrieval latency. Alternatively, there is a new class of methods that exhibits good performance with low latency, but requires a lot more computational resources, and an order of magnitude more training data (i.e. large web-scraped datasets consisting of millions of image-caption pairs) making them infeasible to use in a commercial context. From a pragmatic perspective, none of the existing methods are suitable for developing commercial applications for low-latency cross-modal retrieval on low-resource devices. We propose CrispSearch, a cascaded approach that greatly reduces the retrieval latency with minimal loss in ranking accuracy for on-device language-based image retrieval. The idea behind our approach is to combine a light-weight and runtime-efficient coarse model with a fine re-ranking stage. Given a language query, the coarse model effectively filters out many of the irrelevant image candidates. After this filtering, only a handful of strong candidates will be selected and sent to a fine model for re-ranking. Extensive experimental results with two SOTA models for the fine re-ranking stage, on two standard benchmark datasets (namely, MSCOCO and Flickr30k) show that CrispSearch results in a speedup of up to 38 times over the SOTA fine methods with negligible performance degradation. Moreover, our method does not require millions of training instances, making it a pragmatic solution to on-device search and retrieval.
AI-derived quality of experience prediction based on physiological signals for immersive multimedia experiences

This paper contains the research proposal of Sowmya Vijayakumar that was presented at the MMSys 2022 doctorial symposium. Multimedia applications can now be found across many application domains including but not limited to entertainment, communication, health, business, and education. It is becoming more and more important to understand the factors that influence user perceptual quality, and hence monitoring user quality of experience (QoE) for improving multimedia interaction and services is essential. In this PhD work, we propose advanced machine learning techniques to predict QoE from physiological signals for immersive multimedia experiences. The aim of this doctoral study is to investigate the utility of physiological responses for QoE assessment for different multimedia technologies. Here, the research questions and solutions proposed to address this challenge are presented. A multimodal QoE prediction model is being developed that integrates several physiological measurements to improve QoE prediction performance.
CGD

With advances in network capabilities, the gaming industry is increasingly turning towards offering "gaming on demand" solutions, with cloud gaming services such as Sony PlayStation Now, Google Stadia, and NVIDIA GeForce NOW expanding their market offerings. Similar to adaptive video streaming services, cloud gaming services typically adapt the quality of game streams (e.g., bitrate, resolution, frame rate) in accordance with current network conditions. To select the most appropriate video encoding parameters given certain conditions, it is important to understand their impact on Quality of Experience (QoE). On the other hand, network operators are interested in understanding the relationships between parameters measurable in the network and cloud gaming QoE, to be able to invoke QoE-aware network management mechanisms. To encourage developments in these areas, comprehensive datasets are crucial, including both network and application layer data. This paper presents CGD, a dataset consisting of 600 game streaming sessions corresponding to 10 games of different genres being played and streamed using the following encoding parameters: bitrate (5, 10, 20 Mbps), resolution (720p, 1080p), and frame rate (30, 60 fps). For every combination repeated five times for each game, the dataset includes: 1) gameplay video recordings, 2) network traffic traces, 3) user input logs (mouse and keyboard), and 4) streaming performance logs.
Design, development and evaluation of adaptative and interactive solutions for high-quality viewport-aware VR360 video processing and delivery

This paper contains the research proposal of Miguel Fernández Dasí that was presented at the MMSys 2022 doctoral symposium.1 The production and consumption of multimedia content is continuously increasing, and this particularly affects to immersive formats, like VR360 video. Even though significant advances have been witnessed with regard to the processing, delivery and consumption of interactive VR360 video, key challenges and research questions still need to be addressed to efficiently provide multi-camera and multi-user VR360 video services over distributed and heterogeneous environments. This research work aims at providing novel and efficient contributions to overcome existing limitations in this topic. First, it will develop an end-to-end modular VR360 video platform, including the measurement of Quality of Service (QoS) and activity metrics, to be used as a research testbed. Second, it will provide web-compliant viewport-aware video processing and delivery strategies to dynamically concentrate the video resolution on the users' viewport, with a single stream and decoding process through the web browser. Third, it will propose innovative encoding, signaling and synchronization solutions to enable an effective support for multi-camera VR360 services, with personalized, fast and in-sync switching features. Fourth, it will explore how to effectively provide social viewing scenarios between remote users while watching the same or related VR360 videos. The work also plans to contribute with low-latency delivery pipelines as well as with tools and algorithms to assess and model the Quality of Experience (QoE).
Towards low latency live streaming

Over-the-Top (OTT) service providers need faster, cheaper, and Digital Rights Management (DRM)-capable video streaming solutions. Recently, HTTP Adaptive Streaming (HAS) has become the dominant video delivery technology over the Internet. In HAS, videos are split into short intervals called segments, and each segment is encoded at various qualities/bitrates (i.e., representations) to adapt to the available bandwidth. Utilizing different HAS-based technologies with various segment formats imposes extra cost, complexity, and latency to the video delivery system. Enabling a unified format for transmitting and storing segments at Content Delivery Network (CDN) servers can alleviate the aforementioned issues. To this end, MPEG Common Media Application Format (CMAF) is presented as a standard format for cost-effective and low latency streaming. However, CMAF has not been adopted by video streaming providers yet and it is incompatible with most legacy end-user players. This paper reveals some useful steps for achieving low latency live video streaming that can be implemented for non-DRM sensitive contents before jumping to CMAF technology. We first design and instantiate our testbed in a real OTT provider environment, and then investigate the impact of changing format, segment duration, and Digital Video Recording (DVR) window length on a real live event. The results illustrate that replacing the transport stream (.ts) format with fragmented MP4 (.fMP4) and shortening segments' duration reduces live latency significantly.
Enabling scalable emulation of differentiated services in mininet

Evolving Internet applications, such as immersive multimedia and Industry 4, exhibit stringent delay, loss, and rate requirements. Realizing these requirements would be difficult without advanced dynamic traffic management solutions that leverage state-of-the-art technologies, such as Software-Defined Networking (SDN). Mininet represents a common choice for evaluating SDN solutions in a single machine. However, Mininet lacks the ability to emulate links that have multiple queues to enable differentiated service for different traffic streams. Additionally, performing a scalable emulation in Mininet would not be possible without light-weight application emulators. In this paper, we introduce two tools, namely: QLink and SPEED. QLink extends Mininet API to enable emulating links with multiple queues to differentiate between different traffic streams. SPEED represents a light-weight web traffic emulation tool that enables scalable HTTP traffic simulation in Mininet. Our performance evaluation shows that SPEED enables scalable emulation of HTTP traffic in Mininet. Additionally, we demo the benefits of using QLink to isolate three different applications (voice, web, and video) in a network bottleneck for numerous users.
Exploring Perceptions of Online Feedback in Teaching EFL Speaking and Writing Skills during the COVID-19 Pandemic

The purpose of this study was to analyze perceptions of how online feedback was provided in English as a Foreign Language (EFL) classes, particularly to enhance speaking and writing skills, during the COVID-19 pandemic. The participants were 102 teachers, 40 managers, 80 parents and 120 students from junior high school, senior high school, and higher education levels from 40 public and private educational institutions in southern Ecuador. The sample selection was based on purposeful sampling (voluntary participation). The instruments included questionnaires from students, teachers, parents and educational authorities. In addition, an interview was used to ascertain students’ views about the feedback received in speaking and writing activities. The instruments were designed to collect information about perceptions regarding online feedback in EFL remote teaching of speaking and writing skills. The questionnaires were designed to collect information about feedback for productive skills in EFL remote teaching. The collected data were tabulated and analyzed using descriptive statistics. The results indicate that online feedback is perceived as an aspect that seems to have a better quality in private institutions, especially in activities related to EFL speaking and writing skills. Another important finding was that technological tools in the EFL classroom are believed to be efficient since teachers in public and private institutions use a variety of technological resources for teaching speaking and writing skills. The fundamental aspects of feedback were time and teacher training.
A balanced butterfly optimization algorithm for numerical optimization and feature selection

Butterfly optimization algorithm (BOA) is a relatively novel optimization technique for solving function optimization as well as real-world applications. However, the paramount challenge in BOA is that it is prone to stagnation in local optima. The purpose of this study is to balance the exploration and exploitation abilities of BOA when two new strategies are introduced. The dynamic inertia weight based on Logistic model as the first strategy is introduced to modify the position updating equation. Another strategy is the opposition-based learning. A new variant of BOA called BBOA based on these two strategies is proposed. Ten widely used benchmark test functions and 30 complex benchmarks from CEC 2014 are selected to verify the effectiveness of BBOA. The used benchmark problems are composed of unimodal, multimodal, rotated, shifted, hybrid and composite functions. The experimental results and analysis show that the proposed BBOA has better exploration ability than the conventional BOA when solving different characteristics functions. Finally, BBOA is applied to solve three real-world engineering applications and sixteen feature selection problems. The results demonstrate that BBOA can outperform other competitors in terms of the accuracy of solution on constrained engineering design and feature selection problems.
Research on Evaluation Method of Digital Project Cloud Model Considering Weight Sensitivity

Digitization is a driving force for social development and corporate innovation. Digital projects have become an indispensable part of the sustainable development of enterprises. However, due to the imperfect decision-making system of digital projects and the lack of experience of traditional enterprises’ digital projects, the decision-making of digital projects is an unavoidable challenge in the digital transformation of enterprises. For the digital project decision of the STATE GRID Corporation of China, this paper conducts a sensitivity analysis of digital project evaluation index weights based on cloud model theory, on top of historical successful project experience to support digital project decision-making. Firstly, this paper establishes a comprehensive evaluation index system for digitalization projects from five aspects: economic efficiency, interconnection, intelligent management, value release, and development innovation. The coefficient of variation method is used for index screening, and the weight intervals are formed by four subjective and objective assignment methods. Then, the LSOM model is established to generate the weight values in the interval, and, finally, the sensitivity of digital project comprehensive evaluation indexes is analyzed based on the cloud model to select the most robust index weights for project evaluation and choose the optimal project. The feasibility of the proposed method is verified by arithmetic examples.
Requirements and challenges for hybrid intelligence: A case-study in education

The potential for Artificial Intelligence is widely proclaimed. Yet, in everyday educational settings the use of this technology is limited. Particularly, if we consider smart systems that actually interact with learners in a knowledgeable way and as such support the learning process. It illustrates the fact that teaching professionally is a complex challenge that is beyond the capabilities of current autonomous robots. On the other hand, dedicated forms of Artificial Intelligence can be very good at certain things. For example, computers are excellent chess players and automated route planners easily outperform humans. To deploy this potential, experts argue for a hybrid approach in which humans and smart systems collaboratively accomplish goals. How to realize this for education? What does it entail in practice? In this contribution, we investigate the idea of a hybrid approach in secondary education. As a case-study, we focus on learners acquiring systems thinking skills and our recently for this purpose developed pedagogical approach. Particularly, we discuss the kind of Artificial Intelligence that is needed in this situation, as well as which tasks the software can perform well and which tasks are better, or necessarily, left with the teacher.
Machine Learning Experimental Multipartite Entanglement Structure

Abstract With the rapid growth of controllable qubits in recent years, experimental multipartite entangled states can be created with high fidelity in various moderate‐ and large‐scale physical systems. However, the characterization of multipartite entanglement structure remains a formidable challenge, as traditionally it requires exponential number of local measurements to realize the identification. Machine learning is demonstrated to be an efficient tool to detect the underlying entanglement structure for ideal states, but it has non‐negligible underperformance when tackling imperfect experimental data in reality. Here, a modified classifier based on feed‐forward neural network to predict experimental entanglement structure in terms of entanglement intactness and depth is proposed. By preprocessing the input data, the classifier maintains efficiency and reliability against experimental noises, with the accuracy being enhanced from 69.7% to 91.2% for 6‐qubit entangled states in spin systems. This method is anticipated to shed light on future studies of entanglement structure, in particular when the number of controlled qubits reaches explosive growth in practice.
Knowledge Distillation for Lightweight 2D Single-Person Pose Estimation

The current state-of-the-art single-person pose estimation methods require heavily parameterized models for accurate predictions. A promising technique to achieve accurate yet lightweight pose estimation is knowledge distillation. However, existing pose knowledge distillation methods rely on the most common large basic building blocks and a complex multi-branch architecture. In this study, we propose a Single-branch Lightweight Knowledge Distillation method to increase pose distillation efficiency for 2D Single-person pose estimation, termed SLKD2S. First, we design a novel single-branch pose knowledge distillation framework, which is composed of connected lightweight pose estimation stages. Second, we utilize a special pose distillation loss based on the joint confidence map. Finally, we only keep the initial stage and the first refinement stage to achieve a good performance. Extensive experiments on two standard benchmark datasets show the superiority of the proposed SLKD2S in terms of cost and accuracy, and the average detection accuracies are increased by 1.43% and 2.74% compared with the top-performing pose distillation method, respectively.
Exploration and Practice of College English Multi-Blended Teaching Model Based on OBE Concept

In view of the problems existing in the teaching mode of combining online and offline English courses, such as the lack of interaction and communication, the lack of real situations, and the inability to meet the diverse needs of students.This paper takes College English IV as an example, constructs a new type of multi-online and offline blended teaching model based on the OBE concept, and explores and analyzes the application mode and implementation effect of the multi-mixed teaching model in English teaching from the design of the teaching process and assessment method.Teaching practice shows that the diversified mixed teaching model stimulates the students' learning initiative, improves the learning participation and learning investment, increases the frequency of language output, and strengthens the learning effect.
Imitative Collaboration: A mirror-neuron inspired mixed reality collaboration method with remote hands and local replicas

Mixed reality can overlay and display 3D digital content in the real world, convey abstract concepts to users, and promote the understanding of complex tasks. However, the abstract graphics overlaid on the physical space may cause a certain cognitive load for local users and reduce the efficiency of collaboration. To improve the efficiency of remote collaboration, we conducted an elicitation study on assembly tasks, explored the user needs for collaboration, and defined the design goals of our remote collaboration method. Inspired by the mirror-neuron mechanism, we present an imitative collaboration method that allows local users to imitate the interaction behavior of remote users to complete tasks. We also propose a series of interaction methods for remote users to select, copy, and interact with the local point clouds to facilitate the expression of collaboration intentions. Finally, the results of a user study evaluating our imitative collaboration method on assembly tasks are reported, confirming that our method improves collaboration efficiency while reducing the cognitive load of local users.
A motion-appearance-aware network for object change detection

Object change detection (OCD), which aims to segment moving objects from an input frame, has attracted growing attention. Most OCD algorithms rely on scene diversity or ignore the interframe spatiotemporal structural dependence, which limits their applicability. In this paper, we propose a motion-appearance-aware network (MAAN) for learning robust feature representations. Specifically, a module for mining information at multiple time scales, which can adaptively adjust information elements, is designed to refine the motion feature. Meanwhile, salient object knowledge is obtained with the help of the extracted appearance features. To enhance the semantic consistency and trim redundant connections, we construct a fusion module called multi-view feature evolution, which effectively fuses motion and appearance information by global communication and local guidance, respectively. Moreover, we develop two strategies for obtaining uniform and consistent moving objects during information propagation. One is to feed the predicted mask of the previous frame into the decoder, and the other is to match different levels of motion cues at multiple time scales to the decoder. Finally, extensive experiments on four public datasets (i.e., LASIESTA, CDnet2014, INO, and AICD) indicate that the proposed approach outperforms other methods.
Multilevel security for medical image using heterogeneous chaotic map and deoxyribonucleic acid sequence operations

Summary In telemedicine and e‐health system, the medical images are transmitted through telecommunication. In telecommunication, medical images are transmitted through open source networks. The hackers can tamper or add noise to medical images. Diagnosing the correct disease from the altered medical images is highly impossible. Hence indulge with security, confidentiality and integrity for medical images. In this work, heterogeneous hyper chaotic maps, deoxyribonucleic acid (DNA) cryptography and SHA‐512 is proposed in a new multilevel cryptosystem. The multilevel permutation is performed using heterogeneous chaotic map and zigzag transform. The multilevel diffusion is performed using DNA multiplication and DNA ADD operation. To verify the integrity the SHA‐512 is employed. All DNA coding, and DNA decoding rules are employed dynamically to generate the encrypted medical image. The introduced multilevel cryptosystem provide better security analysis for medical image transmission. The proposed method is implemented in MATLAB software. The performance of proposed approach is associated with existing approaches in terms of evaluating parameters like NPCR (number of pixel changing rate), UACI (unified average changed intensity), correlation coefficient and time taken in seconds. Hence it proves the performance and efficiency of the proposed method are high as compared to the existing methods. The proposed approach also improves the security level and offers integrity and confidentiality, and provide better medical image for encryption with less time complexity.