Reviewerly/exHarmony huggingface dataset:
SpecWands: An Efficient Priority-Based Scheduler Against Speculation Contention Attacks

Transient Execution Attacks (TEAs) have gradually become a major security threat to modern high-performance processors. They exploit the vulnerability of speculative execution to illegally access private data, and transmit them through timing-based covert channels. While new vulnerabilities are discovered continuously, the covert channels can be categorised to two types: 1) Persistent Type, in which covert channels are based on the layout changes of buffering, e.g. through caches or TLBs; 2) Volatile Type, in which covert channels are based on the contention of sharing resources, e.g. through execution units or issuing ports. The defenses against the persistent-type covert channels have been well addressed, while those for the volatile-type are still rather inadequate. Existing mitigation schemes for the volatile type such as Speculative Compression and Time-Division-Multiplexing will introduce significant overhead due to the need to stall the pipeline or to disallow resource sharing. In this paper, we look into such attacks and defenses with a new perspective, and propose a scheduling-based mitigation scheme, called SpecWands. It consists of three priority-based scheduling policies to prevent an attacker from transmitting the secret in different contention situations. SpecWands not only can defend against both inter-thread and intra-thread based attacks, but also can keep most of the performance benefit from speculative execution and resource-sharing. We evaluate its runtime overhead on SPEC 2017 benchmarks and realistic programs. The experimental results show that SpecWands has a significant performance advantage over the other two representative schemes.
REX-SC: Range-Extended Stochastic Computing Accumulation for Neural Network Acceleration

Deep learning has grown in capability and size in recent years, prompting research on alternative computing methods to cope with the increased compute cost. Stochastic computing (SC) promises higher compute efficiency with its compact compute units, but accuracy issues have prevented wide adoption, and accuracy-improving techniques have sacrificed runtime or training performance. In this work, we propose REX-SC -Range-Extended Stochastic Computing Accumulation to deal with the accuracy issues of stochastic computing. By modifying the functionality of OR-based SC accumulation, we increase SC computation accuracy without sacrificing the performance benefits. Our approach achieves a 2X reduction in stream length for the same accuracy compared to SC with OR-based accumulation and a up to 3.6X improvement in energy compared to SC with binary addition. With proper modeling, our approach improves training performance for SC-based neural networks and makes training SC models practical for large datasets like ImageNet.
Media over QUIC: Initial Testing, Findings and Results

With its advantages over TCP, QUIC created a new field for developing media-aware low-latency delivery solutions. The problem space is being examined by the new Media over QUIC (moq) working group in the IETF. In this paper, we study one of the initial proposals in detail, do a gap analysis and create an open-source testbed by introducing new essential features.
Semi-automatic mulsemedia authoring analysis from the user's perspective

Mulsemedia (Multiple Sensorial Media) authoring is a complex task that requires the author to scan the media content to identify the moments to activate sensory effects. A novel proposal is to integrate content recognition algorithms into authoring tools to alleviate the authoring effort. Such algorithms could potentially replace the work of the human author when analyzing audiovisual content, by performing automatic extraction of sensory effects. Besides that, the semi-automatic method proposes to maintain the author subjectivity, allowing the author to define which sensory effects should be automatically extracted. This paper presents an evaluation of the proposed semi-automatic authoring considering the point of view of users. Experiments were done with the STEVE 2.0 mulsemedia authoring tool. Our work uses the GQM (Goal Question Metric) methodology, a questionnaire for collecting users' feedback, and analyzes the results. We conclude that users believe that the semi-automatic authoring is a positive addition to the authoring method.
Urban Land Cover Classification from Sentinel-2 Images with Quantum-Classical Network

Exploiting deep learning techniques to automatically analyze multi-spectral remote sensing imagery plays an essential role in urban land cover and land use classification. However, the computation power required to analyze large earth observation data with complex machine learning models for this task becomes an intractable bottleneck. Leveraging quantum computing might tackle this challenge. In this paper, we present two hybrid quantum-classical deep learning frameworks. They both exploit quantum computing to extract features from multi-spectral images efficiently and classical computing for final classification. The effectiveness of our models is verified with the LCZ42 dataset through the TensorFlow Quantum platform.
Dual-Stream Multimodal Learning for Topic-Adaptive Video Highlight Detection

This paper targets at topic-adaptive video highlight detection, aiming to identify the moments in a video described by arbitrary text inputs. The fundamental challenge is the availability of annotated training data. It is costly to further scale up the number of topic-level categories which requires manually identifying and labeling corresponding highlights. To overcome this challenge, our method provides a new perspective on highlight detection by attaching importance to the semantic information of topic text rather than simply classifying whether a snippet is a highlight.Specifically, we decompose a topic into a set of key concepts and utilize the remarkable ability of visual-language pre-trained models to learn knowledge from both videos and semantic language. With the merits of this reformulation, the highlight detection task can be modeled as a snippet-text matching problem within a dual-stream multimodal learning framework, which strengthens the video representation with semantic language supervision and enables our model to accomplish open-set topic-adaptive highlight detection without any further labeled data. Our empirical evaluation shows the effectiveness of our method on several publicly available datasets, where the proposed method outperforms competitive baselines and achieves a novel state-of-the-art for topic-adaptive highlight detection. Further, when transferring our pre-trained model to the open-set video highlight detection task, our method outperforms prior supervised work by a substantial margin.
A stochastic process of software fault detection and correction for business operations

Automatic software fault detection and repair is made possible by autonomic software recovery. By incorporating this function, the software will run more efficiently and aggressively while requiring much less time and resources for maintenance. The focus of this article is on a suggested automated approach to Software Fault Detection and Recovery. The Software Fault Detection and Recovery (SFDR) procedure identifies when a software component has been damaged due to a fault and then restores the damaged component so that the software can continue functioning normally. During the design process, the SFDR is examined and created independently from the intended program. The proposed technique was implemented into an application that demonstrates the SFDR's performance and effectiveness to guarantee its practicality in real-world scenarios. The results of this experiment were encouraging. Results from experiments and comparisons to prior works show that the proposed methodology is successful.
Cross-View Sample-Enriched Graph Contrastive Learning Network for Personalized Micro-video Recommendation

Micro-video recommendation has attracted extensive research attention with the increasing popularity of micro-video sharing platforms. Recently, graph contrastive learning (GCL) is adopted for enhancing the performance of graph neural network based micro-video recommendation. However, these GCL methods may suffer from the following problems: (1) they fail to fully exploit the potential of contrastive learning for ignoring or misjudging highly similar samples, and (2) the complementary recommendation effects between graph structure information and multi-modal feature information are not effectively utilized. In this paper, we propose a novel Cross-View Sample-Enriched Graph Contrastive Learning Network (CSGCL) for micro-video recommendation. Specifically, we build a collaborative learning view and a semantic learning view to learn node representations. For the collaborative learning view, we leverage similar nodes at the structure level to construct an effective collaborative contrastive objective. For the semantic learning view, we derive the k-nearest neighbor graph generated from multi-modal features as the semantic graphs and build a semantic contrastive objective for learning high-quality micro-video representations. Finally, a cross-view contrastive objective is designed to consider the mutually complementary recommendation effects by maximizing the agreement between the two above views. Extensive experiments on three real-world datasets demonstrate that the proposed model outperforms the baselines.
Multi-granularity Separation Network for Text-Based Person Retrieval with Bidirectional Refinement Regularization

Text-based person retrieval is one of the fundamental tasks in the field of computer vision, which aims to retrieve the most relevant pedestrian image from all the candidates according to textual descriptions. Such a cross-modal retrieval task could be challenging since it requires one to properly select distinguishing clues and perform cross-modal alignments. To achieve cross-modal alignments, most previous works focus on different inter-modal constraints while overlooking the influence of intra-modal noise, yielding sub-optimal retrieved results in certain cases. To this end, we propose a novel framework termed Multi-granularity Separation Network with Bidirectional Refinement Regularization (MSN-BRR) to tackle the problem. The framework consists of two components: (1) Multi-granularity Separation Network, which extracts the multi-grained discriminative textual and visual representations at local and global semantic levels. (2) Bidirectional Refinement Regularization, which alleviates the influence of intra-modal noise and facilitates the proper alignments between the visual and textual representations. Extensive experiments on two widely used benchmarks, i.e., CUHK-PEDES and ICFG-PEDES show that our MSN-BRR method outperforms current state-of-the-art methods.
Explicit Knowledge Integration for Knowledge-Aware Visual Question Answering about Named Entities

Recent years have shown unprecedented growth of interest in Vision-Language related tasks, with the need to address the inherent challenges of integrating linguistic and visual information to solve real-world applications. Such a typical task is Visual Question Answering (VQA), which aims to answer questions about visual content. The limitations of the VQA task in terms of question redundancy and poor linguistic variability encouraged researchers to propose Knowledge-aware Visual Question Answering tasks as a natural extension of VQA. In this paper, we tackle the KVQAE (Knowledge-based Visual Question Answering about named Entities) task, which proposes to answer questions about named entities defined in a knowledge base and grounded in visual content. In particular, besides the textual and visual information, we propose to leverage the structural information extracted from syntactic dependency trees and external knowledge graphs to help answer questions about a large spectrum of entities of various types. Thus, by combining contextual and graph-based representations using Graph Convolutional Networks (GCNs), we are able to learn meaningful embeddings for Information Retrieval tasks. Experiments on the ViQuAE public dataset show how our approach improves the state-of-the-art baselines while demonstrating the interest of injecting external knowledge to enhance multimodal information retrieval.
Towards Shape-regularized Learning for Mitigating Texture Bias in CNNs

CNNs have emerged as powerful techniques for object recognition. However, the test performance of CNNs is contingent on the similarity to training distribution. Existing methods focus on data augmentation to address out-of-domain generalization. In contrast, we enforce a shape bias by encouraging our model to learn features that correlate with those learned from the shape of the object. We show that explicit shape cues enable CNNs to learn features that are robust to unseen image manipulations i.e. novel textures with the same semantic content. Our models are validated on Toys4K dataset which consists of 4179 3D objects and image pairs. To quantify texture bias, we synthesize dataset variants called Style (style-transfer with GANs), CueConflict (conflicting texture & semantics), and Scrambled datasets (obfuscating semantics by scrambling pixel blocks). Our experiments show that the benefits of using shape is not subject to specific shape representations like point clouds, rather the same benefits can be obtained from a simpler representation such as the distance transform.
Video Retrieval for Everyday Scenes With Common Objects

We propose a video retrieval system for everyday scenes with common objects. Our system exploits the predictions made by deep neural networks for image understanding tasks using natural language processing (NLP). It aims to capture the relationships between objects in a video scene as well as the ordering of the matching scenes. For each video in the database, it identifies and generates a sequence of key scene images. For each such scene, it generates most probable captions using state-of-the-art models for image captioning. The captions are parsed and represented by tree structures using NLP techniques. These are then stored and indexed in a database system. When a user poses a query video, a sequence of key scenes are generated. For each scene, its caption is generated using deep learning and parsed into its corresponding tree structure. After that, optimized tree-pattern queries are constructed and executed on the database to retrieve a set of candidate videos. Finally, these candidate videos are ranked using a combination of longest common subsequence of scene matches and tree-edit distance between parse trees. We evaluated the performance of our system using the MSR-VTT dataset, which contained everyday scenes. We observed that our system achieved higher mean average precision (mAP) compared to two recent techniques, namely, CSQ and DnS.
CMMT: Cross-Modal Meta-Transformer for Video-Text Retrieval

Video-text retrieval has drawn great attention due to the prosperity of online video contents. Most existing methods extract the video embeddings by densely sampling abundant (generally dozens of) video clips, which acquires tremendous computational cost. To reduce the resource consumption, recent works propose to sparsely sample fewer clips from each raw video with a narrow time span. However, they still struggle to learn a reliable video representation with such locally sampled video clips, especially when testing on cross-dataset setting. In this work, to overcome this problem, we sparsely and globally (with wide time span) sample a handful of video clips from each raw video, which can be regarded as different samples of a pseudo video class (i.e., each raw video denotes a pseudo video class). From such viewpoint, we propose a novel Cross-Modal Meta-Transformer (CMMT) model that can be trained in a meta-learning paradigm. Concretely, in each training step, we conduct a cross-modal fine-grained classification task where the text queries are classified with pseudo video class prototypes (each has aggregated all sampled video clips per pseudo video class). Since each classification task is defined with different/new videos (by simulating the evaluation setting), this task-based meta-learning process enables our model to generalize well on new tasks and thus learn generalizable video/text representations. To further enhance the generalizability of our model, we induce a token-aware adaptive Transformer module to dynamically update our model (prototypes) for each individual text query. Extensive experiments on three benchmarks show that our model achieves new state-of-the-art results in cross-dataset video-text retrieval, demonstrating that it has more generalizability in video-text retrieval. Importantly, we find that our new meta-learning paradigm indeed brings improvements under both cross-dataset and in-dataset retrieval settings.
Blockchain for citizens' participation in urban planning: The case of the city of Berlin. A value sensitive design approach

The aim of the paper is to investigate how a blockchain-based platform for citizens' participation in urban planning should be developed adopting a Value Sensitive Design approach (VSD). VSD can be considered as an overarching methodological framework for new technologies development which takes into account moral and human values of all the direct and indirect stakeholders which might be impacted by the technology. By considering the perspectives of the different involved stakeholders, the paper specifically addresses the call for more research about blockchain-based services which adopts a human-centred approach instead of technocentric. The paper is based on the empirical analysis of a DApp called BBBlockchain which was specifically created with the purpose to improve citizens participation in urban planning decisions in two major residential development projects in Berlin, Germany. BBBlockchain is a proof of concept which is currently running on its own in-house test network but it can run on every EVM compatible blockchain. From the analysis of the information layer of participation, it can be argued that the most important values include transparency, inclusiveness and confidentiality.
SEPE Dataset: 8K Video Sequences and Images for Analysis and Development

This paper provides an overview of our open (Software Engineering Practice and Education) SEPE 8K dataset which is made of 40 different 8K (8192 x 4320) video sequences and 40 variant 8K (8192 x 5464) images. The video sequences were captured at a framerate of 29.97 frames per second (FPS) and had been encoded into videos using AVC/H.264, HEVC/H.265, and AV1 codecs at resolutions from 8K to 480p. The images, video sequences, encoded videos, and various other statistics related to the media that make the dataset are stored online, published, and maintained on the repo on GitHub for non-commercial use. In this paper, the dataset components are described and analyzed using various methods. The proposed dataset is - as far as we know - the first to publish true 8K natural sequences; thus, it is important for the next level of applications dealing with multimedia such as video quality assessment, super-resolution, video coding, video compression, and many more.
Hate Speech Detection using Deep Learning and Text Analysis

Hate speech is any form of speech, gesture, written or physical expression that threatens a person or a group based on their race, ethnicity, religion, gender, sexual orientation, nationality, disability, or any other characteristic that is protected by law. Hate speech can take many forms, ranging from verbal harassment to physical violence. Hate speech detection has become an important task in NLP due to the growing frequency of hate speech on online forums and social media. The proposed research work aims to improve hate speech detection by doing modification in standard i.e., Modified bi-LS TM model vs RCNN. The study examines how well the modified model performs on tasks involving the classification of hate speech when compared to a conventional LS TM model. The improved bi-LS TM model is intended to capture the context and relationships more accurately between the words in hate speech utterances.The study uses a publicly accessible dataset of tweets containing hate speech and tweets without any hate speech. The proposed model is trained and tested with the help of various performance metrics such as F1-score, accuracy and precision, recall. The research outcomes show that the proposed model outperforms the standard IS TM model in detecting hate speech.
Design and Implementation of Dynamic Interactive E-Book System

Under the background of flourishing computer technology, books gradually become electronic. Electronic books bring great convenience to people’s learning, but at present their advantages are more reflected in the digitization of content than in the computability and interactivity. In order to further explore the value of electronic books, this paper studies the design and implementation of a dynamic interactive e-book system. This system can present knowledge in a dynamic way, and has obvious practical significance in promoting learners’ deep understanding of knowledge. This paper first introduces the relationship between the modules in the system, then takes the geometry subsystem as an example to discuss its architecture and technical implementation methods, and finally gives the demonstration of this subsystem.
Blockchain Integration with Artificial Intelligence and Internet of Things Technologies

Technologies are essential for human life because they improve efficiency, productivity, and convenience in various aspects of daily life. This paper explores the integration of Blockchain, Artificial intelligence (AI), and Internet of Things (IoT) technologies and their potential impact on various industries. It highlights the benefits of combining these technologies, including increased security, transparency, and automation. The paper also discusses the challenges associated with integrating these technologies and the potential solutions to address them. The study concludes by providing insight into the future of this integration and its potential to revolutionize various industries including finance, healthcare, and transportation. the limitations and issues of existing models before the integration of blockchain, artificial intelligence, and IoT technologies included a lack of transparency, data privacy and security concerns, and a lack of automation.
Multifarious Face Attendance System using Machine Learning and Deep Learning

The process for recording attendance depends on a variety of variables including the quantity of participants, locations, events and time intervals where attendance must be collected. Recording and collecting attendance for an event or program is very much essential to keep track of the intended audience. For example, in schools and colleges roll call can be used to take attendance for the presenteeism in the classroom. Similarly, you can use this process for programs or events with a smaller number of participants. If participants have more attendance maintaining system is tedious process, if it is done manually. So, there is a need for an effective attendance system so that it can monitor the presents in all the ways. So here a multifarious face attendance system is implemented which involves machine learning and deep learning for face detection using MT-CNN and VGGFace2 large scale face recognition dataset model. The execution of this attendance system can be demonstrated for the educational institution. So, the problem of fraudulent attendance and proxies can be resolved by utilizing this technique. The main task of this system is to recognize multiple faces using VGGFace2 and map attendance based on it. So that thereby allowing us to analyze the attendees of the program or function. Following this, the discovered faces can be compared by cross-referencing with the student face database. The attendance and records of attendees can be effectively maintained with the help of this effective technique.