Reviewerly/exHarmony huggingface dataset:
Content-Based Video Big Data Retrieval with Extensive Features and Deep Learning

In the era of digital media, the rapidly increasing volume and complexity of multimedia data cause many problems in storing, processing, and querying information in a reasonable time. Feature extraction and processing time play an extremely important role in large-scale video retrieval systems and currently receive much attention from researchers. We, therefore, propose an efficient approach to feature extraction on big video datasets using deep learning techniques. It focuses on the main features, including subtitles, speeches, and objects in video frames, by using a combination of three techniques: optical character recognition (OCR), automatic speech recognition (ASR), and object identification with deep learning techniques. We provide three network models developed from networks of Faster R-CNN ResNet, Faster R-CNN Inception ResNet V2, and Single Shot Detector MobileNet V2. The approach is implemented in Spark, the next-generation parallel and distributed computing environment, which reduces the time and space costs of the feature extraction process. Experimental results show that our proposal achieves an accuracy of 96% and a processing time reduction of 50%. This demonstrates the feasibility of the approach for content-based video retrieval systems in a big data context.
Cross-view panorama image synthesis with progressive attention GANs

Despite the significant progress of conditional image generation, it remains difficult to synthesize a ground-view panorama image from a top-view aerial image. Among the core challenges are the vast differences in image appearance and resolution between aerial images and panorama images, and the limited aside information available for top-to-ground viewpoint transformation. To address these challenges, we propose a new Progressive Attention Generative Adversarial Network (PAGAN) with two novel components: a multistage progressive generation framework and a cross-stage attention module. In the first stage, an aerial image is fed into a U-Net-like network to generate one local region of the panorama image and its corresponding segmentation map. Then, the synthetic panorama image region is extended and refined through the following generation stages with our proposed cross-stage attention module that passes semantic information forward stage-by-stage. In each of the successive generation stages, the synthetic panorama image and segmentation map are separately fed into an image discriminator and a segmentation discriminator to compute both later real and fake, as well as feature alignment score maps for discrimination. The model is trained with a novel orientation-aware data augmentation strategy based on the geometric relation between aerial and panorama images. Extensive experimental results on two cross-view datasets show that PAGAN generates high-quality panorama images with more convincing details than state-of-the-art methods.
AR-assisted children book for smart teaching and learning of Turkish alphabets

Augmented reality (AR), virtual reality (VR), and remote-controlled devices are driving the need for a better 5G infrastructure to support faster data transmission. This paper emphasizes that mobile AR is a viable and widespread solution that can easily scale to millions of end-users and educators since it is lightweight, low-cost, and cross-platform. Low-efficiency smart devices and lengthy latency for real-time interactions via regular mobile networks are major barriers to using AR in education. The good news is that the upcoming 5G cellular networks can mitigate some of these issues via network slicing, device-to-device communication, and mobile edge computing. In this paper, we rely on technology to solve some of these problems. The proposed software monitors Image Targets on a printed book and renders 3D objects and alphabet models. In addition, the application considers phonetics. The sound (Phonetic) and 3D representation of the letter are played as soon as an image target is detected. The Turkish alphabet 3D models were created in Adobe Photoshop using Unity3D and the Vuforia SDK. The proposed application teaches Turkish alphabets and phonetics by using 3D object models, 3D letters, and 3D phrases including those letters and sounds.
Adaptive Multi-View and Temporal Fusing Transformer for 3D Human Pose Estimation

This article proposes a unified framework dubbed Multi-view and Temporal Fusing Transformer (MTF-Transformer) to adaptively handle varying view numbers and video length without camera calibration in 3D Human Pose Estimation (HPE). It consists of Feature Extractor, Multi-view Fusing Transformer (MFT), and Temporal Fusing Transformer (TFT). Feature Extractor estimates 2D pose from each image and fuses the prediction according to the confidence. It provides pose-focused feature embedding and makes subsequent modules computationally lightweight. MFT fuses the features of a varying number of views with a novel Relative-Attention block. It adaptively measures the implicit relative relationship between each pair of views and reconstructs more informative features. TFT aggregates the features of the whole sequence and predicts 3D pose via a transformer. It adaptively deals with the video of arbitrary length and fully unitizes the temporal information. The migration of transformers enables our model to learn spatial geometry better and preserve robustness for varying application scenarios. We report quantitative and qualitative results on the Human3.6M, TotalCapture, and KTH Multiview Football II. Compared with state-of-the-art methods with camera parameters, MTF-Transformer obtains competitive results and generalizes well to dynamic capture with an arbitrary number of unseen views.
Object Detection and Recognition in Real Time Using Deep Learning for Visually Impaired People

This paper aims at combining object detection at real time and recognition with suitable deep learning methods in order to detect and recognize objects position as well as the names of multiple objects detected by the camera using an object detector algorithm. This is to aid the visually impaired user without the help of any other person. The image and video processing algorithms were designed to take real-time inputs from the camera, Deep Neural Networks were used to predict the objects and uses Google’s famous Text-To-Speech (GTTS) API module for the anticipated voice output precisely detecting and recognizing the category or class of objects and locations contained. Our best result shows that the system recognizes 91 categories of outdoor objects and produces the output in speech i.e. in an audio format even when a reduced amount of spectral information from the data is available.
Effectiveness of Using Augmented Reality for Training in the Medical Professions: Meta-analysis

Augmented reality (AR) is an interactive technology that uses persuasive digital data and real-world surroundings to expand the user's reality, wherein objects are produced by various computer applications. It constitutes a novel advancement in medical care, education, and training.The aim of this work was to assess how effective AR is in training medical students when compared to other educational methods in terms of skills, knowledge, confidence, performance time, and satisfaction.We performed a meta-analysis on the effectiveness of AR in medical training that was constructed by using the Cochrane methodology. A web-based literature search was performed by using the Cochrane Library, Web of Science, PubMed, and Embase databases to find studies that recorded the effect of AR in medical training up to April 2021. The quality of the selected studies was assessed by following the Cochrane criteria for risk of bias evaluations.In total, 13 studies with a total of 654 participants were included in the meta-analysis. The findings showed that using AR in training can improve participants' performance time (I2=99.9%; P<.001), confidence (I2=97.7%; P=.02), and satisfaction (I2=99.8%; P=.006) more than what occurs under control conditions. Further, AR did not have any effect on the participants' knowledge (I2=99.4%; P=.90) and skills (I2=97.5%; P=.10). The meta-regression plot shows that there has been an increase in the number of articles discussing AR over the years and that there is no publication bias in the studies used for the meta-analysis.The findings of this work suggest that AR can effectively improve performance time, satisfaction, and confidence in medical training but is not very effective in areas such as knowledge and skill. Therefore, more AR technologies should be implemented in the field of medical training and education. However, to confirm these findings, more meticulous research with more participants is needed.
A Survey of Blockchain Technology: Architecture, Applied Domains, Platforms, and Security Threats

Blockchain technology is at the peak of hype and contemporary research area across the world. It is a distributed ledger that keeps records of transactions with verifiable and immutable structures and continuously grows with the new block of transactions. Blockchain provides better transparency, enhanced security and privacy, and true traceability over the traditional approaches. Due to its advanced and secure features, it is being used in various fields such as trade finance, digital transactions, the Internet of Things (IoT), the healthcare industry, and energy sector. Blockchain technology has a great impact in all its applied fields with the prominent features of privacy and reliability of data and transactions. This survey intends to present the architecture of blockchain, its potential applications, and practices in different domains other than cryptocurrency along with the platform of blockchain. This paper will explore some potential benefits of blockchain and some future directions and open challenges that are expected to come in future research.
Revisiting Two-tower Models for Unbiased Learning to Rank

Two-tower architecture is commonly used in real-world systems for Unbiased Learning to Rank (ULTR), where a Deep Neural Network (DNN) tower models unbiased relevance predictions, while another tower models observation biases inherent in the training data like user clicks. This two-tower architecture introduces inductive biases to allow more efficient use of limited observational logs and better generalization during deployment than single-tower architecture that may learn spurious correlations between relevance predictions and biases. However, despite their popularity, it is largely neglected in the literature that existing two-tower models assume that the joint distribution of relevance prediction and observation probabilities are completely factorizable. In this work, we revisit two-tower models for ULTR. We rigorously show that the factorization assumption can be too strong for real-world user behaviors, and existing methods may easily fail under slightly milder assumptions. We then propose several novel ideas that consider a wider spectrum of user behaviors while still under the two-tower framework to maintain simplicity and generalizability. Our concerns of existing two-tower models and the effectiveness of our proposed methods are validated on both controlled synthetic and large-scale real-world datasets.
A grounded theory of coordination in remote-first and hybrid software teams

While the long-term effects of the COVID-19 pandemic on software professionals and organizations are difficult to predict, it seems likely that working from home, remote-first teams, distributed teams, and hybrid (part-remote/part-office) teams will be more common. It is therefore important to investigate the challenges that software teams and organizations face with new remote and hybrid work. Consequently, this paper reports a year-long, participant-observation, constructivist grounded theory study investigating the impact of working from home on software development. This study resulted in a theory of software team coordination. Briefly, shifting from in-office to at-home work fundamentally altered coordination within software teams. While group cohesion and more effective communication appear protective, coordination is undermined by distrust, parenting and communication bricolage. Poor coordination leads to numerous problems including misunderstandings, help requests, lower job satisfaction among team members, and more ill-defined tasks. These problems, in turn, reduce overall project success and prompt professionals to alter their software development processes (in this case, from Scrum to Kanban). Our findings suggest that software organizations with many remote employees can improve performance by encouraging greater engagement within teams and supporting employees with family and childcare responsibilities.
Using reinforcement learning for load testing of video games

Different from what happens for most types of software systems, testing video games has largely remained a manual activity performed by human testers. This is mostly due to the continuous and intelligent user interaction video games require. Recently, reinforcement learning (RL) has been exploited to partially automate functional testing. RL enables training smart agents that can even achieve super-human performance in playing games, thus being suitable to explore them looking for bugs. We investigate the possibility of using RL for load testing video games. Indeed, the goal of game testing is not only to identify functional bugs, but also to examine the game's performance, such as its ability to avoid lags and keep a minimum number of frames per second (FPS) when high-demanding 3D scenes are shown on screen. We define a methodology employing RL to train an agent able to play the game as a human while also trying to identify areas of the game resulting in a drop of FPS. We demonstrate the feasibility of our approach on three games. Two of them are used as proof-of-concept, by injecting artificial performance bugs. The third one is an open-source 3D game that we load test using the trained agent showing its potential to identify areas of the game resulting in lower FPS.
Curriculum Contrastive Context Denoising for Few-shot Conversational Dense Retrieval

Conversational search is a crucial and promising branch in information retrieval. In this paper, we reveal that not all historical conversational turns are necessary for understanding the intent of the current query. The redundant noisy turns in the context largely hinder the improvement of search performance. However, enhancing the context denoising ability for conversational search is quite challenging due to data scarcity and the steep difficulty for simultaneously learning conversational query encoding and context denoising. To address these issues, in this paper, we present a novel Curriculum cOntrastive conTExt Denoising framework, COTED, towards few-shot conversational dense retrieval. Under a curriculum training order, we progressively endow the model with the capability of context denoising via contrastive learning between noised samples and denoised samples generated by a new conversation data augmentation strategy. Three curriculums tailored to conversational search are exploited in our framework. Extensive experiments on two few-shot conversational search datasets, i.e., CAsT-19 and CAsT-20, validate the effectiveness and superiority of our method compared with the state-of-the-art baselines.
DH-HGCN: Dual Homogeneity Hypergraph Convolutional Network for Multiple Social Recommendations

Social relations are often used as auxiliary information to improve recommendations. In the real-world, social relations among users are complex and diverse. However, most existing recommendation methods assume only single social relation (i.e., exploit pairwise relations to mine user preferences), ignoring the impact of multifaceted social relations on user preferences (i.e., high order complexity of user relations). Moreover, an observing fact is that similar items always have similar attractiveness when exposed to users, indicating a potential connection among the static attributes of items. Here, we advocate modeling the dual homogeneity from social relations and item connections by hypergraph convolution networks, named DH-HGCN, to obtain high-order correlations among users and items. Specifically, we use sentiment analysis to extract comment relation and use the k-means clustering to construct item-item correlations, and we then optimize those heterogeneous graphs in a unified framework. Extensive experiments on two real-world datasets demonstrate the effectiveness of our model.
Multi-Faceted Global Item Relation Learning for Session-Based Recommendation

As an emerging paradigm, session-based recommendation is aimed at recommending the next item based on a set of anonymous sessions. Effectively representing a session that is normally a short interaction sequence renders a major technical challenge. In view of the limitations of pioneering studies that explore collaborative information from other sessions, in this paper we propose a new direction to enhance session representations by learning multi-faceted session-independent global item relations. In particular, we identify three types of advantageous global item relations, including negative relations that have not been studied before, and propose different graph construction methods to capture such relations. We then devise a novel multi-faceted global item relation (MGIR) model to encode different relations using different aggregation layers and generate enhanced session representations by fusing positive and negative relations. Our solution is flexible to accommodate new item relations and can easily integrate existing session representation learning methods to generate better representations from global relation enhanced session information. Extensive experiments on three benchmark datasets demonstrate the superiority of our model over a large number of state-of-the-art methods. Specifically, we show that learning negative relations is critical for session-based recommendation.
FIRA

Commit messages summarize code changes of each commit in natural language, which help developers understand code changes without digging into detailed implementations and play an essential role in comprehending software evolution. To alleviate human efforts in writing commit messages, researchers have proposed various automated techniques to generate commit messages, including template-based, information retrieval-based, and learning-based techniques. Although promising, previous techniques have limited effectiveness due to their coarse-grained code change representations.
A case study of using Alexa for out-of-class, self-directed Japanese language learning

Intelligent personal assistants, otherwise known as virtual assistants, are artificial intelligence-powered conversational agents that utilize machine learning to understand and appropriately respond to user requests. In recent years, research interest on the use of this emerging technology for foreign language learning has increased. Several studies have examined the use of intelligent personal assistants for English as a foreign language learning. However, no study has studied intelligent personal assistants in the context of a non-English foreign language. Thus, this case study addressed this gap in the research by investigating the out-of-class, self-directed use of the intelligent personal assistant, Alexa, among six Japanese as a second language learners. The primary aim of the study was to better understand the experiences and perceptions of these learners when interacting with Alexa in the target language. According to an analysis of the participants’ usage history, it was found that the intelligent personal assistant could understand over 80% of the learners’ commands. When faced with a communication difficulty, the learners most often resorted to an abandonment of the previously misunderstood command, with rephrase and repeat strategies being used far less frequently. Concerning the learners’ opinions towards Alexa for autonomous foreign language learning, views were mostly positive. Namely, they believed interaction with Alexa was fun and supported their Japanese development. These results demonstrate that intelligent personal assistants have the potential to be a useful technology for self-directed foreign language learning for languages besides English.
High Accuracy Real-Time Insulator String Defect Detection Method Based on Improved YOLOv5

Insulator string is a special insulation component which plays an important role in overhead transmission lines. However, working outdoors for a long time, insulators often have defects because of various environmental and weather conditions, which affect the normal operation of transmission lines and even cause huge economic losses. Therefore, insulator defect recognition is a crucial issue. Traditional insulator defect identification relies on manual work, which is time-consuming and inefficient. Therefore, the use of artificial intelligence to detect the defect location and recognize its class has become a key research field. By improving the classical YOLOv5 (you only look once) model, this article proposes a new method to enable high accuracy and real-time detection. Our method has three advantages: 1) Efficient-IoU (EIoU) replaces intersection over union (IoU) to calculate the loss of box regression, which overcomes that the detection is sensitive to various scale insulators in aerial images. 2) Since YOLOv5 itself detects some natural scenes in the real world, some anchors setting by default are not suitable for defect detection, this article introduces Assumption-free K-MC2 (AFK-MC2) algorithm into YOLOv5 to modify the K-means algorithm to improve accuracy and speed. 3) The cluster non-maximum suppression (Cluster-NMS) algorithm is introduced to avoid missing detection of insulators because of mutual occlusion in images and improve the computation speed at the same time. The experiments’ results show that this model can improve detection accuracy compared with YOLOv5 and realize real-time detection.