Reviewerly/exHarmony huggingface dataset:
JDSR-GAN: Constructing an Efficient Joint Learning Network for Masked Face Super-Resolution

With the growing importance of preventing the COVID-19 virus, face images obtained in most video surveillance scenarios are low resolution with mask simultaneously. However, most of the previous face super-resolution solutions can not handle both tasks in one model. In this work, we treat the mask occlusion as image noise and construct a joint and collaborative learning network, called JDSR-GAN, for the masked face super-resolution task. Given a low-quality face image with the mask as input, the role of the generator composed of a denoising module and super-resolution module is to acquire a high-quality high-resolution face image. The discriminator utilizes some carefully designed loss functions to ensure the quality of the recovered face images. Moreover, we incorporate the identity information and attention mechanism into our network for feasible correlated feature expression and informative feature learning. By jointly performing denoising and face super-resolution, the two tasks can complement each other and attain promising performance. Extensive qualitative and quantitative results show the superiority of our proposed JDSR-GAN over some comparable methods which perform the previous two tasks separately.
Short text topic modelling using local and global word-context semantic correlation

Nowadays, people use short text to portray their opinions on platforms of social media such as Twitter, Facebook, and YouTube, as well as on e-commerce websites such as Amazon and Flipkart to share their commercial purchasing experiences. Every day, billions of short texts are created worldwide in tweets, tags, keywords, search queries etc. However, this short text possesses inadequate contextual information, which can be ambiguous, sparse, noisy, remains a major challenge. State-of-the-art strategies of topic modeling such as Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis are not suitable as it contains a limited number of words in a single document. This work proposes a new model named G_SeaNMF (Gensim_SeaNMF) to improve the word-context semantic relationship by using local and global word embedding techniques. Word embeddings learned from a large corpus provide general semantic and syntactic information about words; it can guide topic modeling for short text collections as supporting information for sparse co-occurrence patterns. In the proposed model, SeaNMF (Semantics-assisted Non-negative Matrix Factorization) is incorporated with word2vec model of Gensim library to strengthen the word's semantic relationship. In this article, a short text topic modeling techniques based on DMM (Dirichlet Multinomial Mixture), self-aggregation and global word co-occurrence were explored. These are evaluated using different measures to gauge cluster coherence on real-world datasets such as Search Snippet, Biomedicine, Pascal Flickr, Tweet and TagMyNews. Empirical evaluation shows that a combination of local and global word embedding provides more appropriate words under each topic with improved outcomes.
2D/3D Deep Registration Along Trajectories With Spatiotemporal Context: Application to Prostate Biopsy Navigation

The accuracy of biopsy targeting is a major issue for prostate cancer diagnosis and therapy. However, navigation to biopsy targets remains challenging due to the limitations of transrectal ultrasound (TRUS) guidance added to prostate motion issues. This article describes a rigid 2D/3D deep registration method, which provides a continuous tracking of the biopsy location w.r.t the prostate for enhanced navigation.A spatiotemporal registration network (SpT-Net) is proposed to localize the live 2D US image relatively to a previously aquired US reference volume. The temporal context relies on prior trajectory information based on previous registration results and probe tracking. Different forms of spatial context were compared through inputs (local, partial or global) or using an additional spatial penalty term. The proposed 3D CNN architecture with all combinations of spatial and temporal context was evaluated in an ablation study. For providing a realistic clinical validation, a cumulative error was computed through series of registrations along trajectories, simulating a complete clinical navigation procedure. We also proposed two dataset generation processes with increasing levels of registration complexity and clinical realism.The experiments show that a model using local spatial information combined with temporal information performs better than more complex spatiotemporal combination.The best proposed model demonstrates robust real-time 2D/3D US cumulated registration performance on trajectories. Those results respect clinical requirements, application feasibility, and they outperform similar state-of-the-art methods.Our approach seems promising for clinical prostate biopsy navigation assistance or other US image-guided procedure.
Human Behavior Identification Based on Graphology Using Artificial Neural Network

Handwriting reflects a person's true nature, phobias, emotional outbursts, honesty, defenses and many more characteristics.Analysis of handwriting, also known as graphology, is a science that uses the strokes and patterns disclosed by handwriting to identify, evaluate, and analyze personality.It is the study of the patterns and physical characteristics of handwriting to identify the author, indicate the author's psychological state while writing, or analyze personality traits.Traditionally, professionals also called graphologists predict the behavior of the writer by analyzing their handwriting, but this procedure is tedious and expensive.Therefore, this paper focuses on developing an application for personality identification that can predict behavioral characteristics directly using a computer without any human involvement.Most of the existing applications use English as the primary language to identify the personality trait of the writer however, our approach uses Devanagari scripts for prediction, thereby eliminating the language barrier.Our proposed method uses a machine learning approach to predict personality by analyzing Devanagari samples using Artificial Neural Network.We have created our own Devanagari word dataset.There are almost 4000 images which belong to 5 classes namely Introvert, Extrovert, Optimistic, Pessimistic and Stable mind-set.The testing accuracy achieved by the proposed method is 94.75%.
Adopting distributed pair programming as an effective team learning activity: a systematic review

As online learning has become an inevitable trend in the post-peak era of the COVID-19 pandemic, distributed pair programming (DPP) is gaining momentum in both education and industry. DDP serves as a collaborative programming approach and also benefits the development of computational thinking, a fundamental skill in today's world. This study conducted a systematic review of studies on DPP published after 2010 to understand the themes and factors that impact the team effectiveness of DPP and thus inform future research and practices on how to better leverage this approach for teaching and learning. The results showed that individual characteristics attracted major investigations in the selected 23 studies, including prior programming experience, actual skill, perceived skill, gender, personality, time management, confidence, and self-esteem, with pair compatibility identified as a critical team design factor that significantly affects programmers' satisfaction. Although the feel-good factor in the team process was investigated, no significant impact was found. Under the team environment theme, we compared different opinions on the orientation (e.g., scripted roles) and the use of technology (e.g., integrated development environment tools). Future research should investigate how task structure influences team effectiveness of DPP and relates to computational thinking education. Additionally, because most studies were conducted in higher education contexts, more research in primary and secondary educational contexts is also needed.
Networks of random lasers: current perspective and future challenges [Invited]

Artificial neural networks are widely used in many different applications because of their ability to deal with a range of complex problems generally involving massive data sets. These networks are made up of nodes, connections, and nonlinear response connections, which are typically implemented as software code running on ordinary electronic computers. In such systems, electrons, with their advantages and drawbacks, are in charge of storing, processing, and transmitting information. Signal processing in the optical domain can provide ultrafast, parallel operation, nonlinear dynamics, and high energy efficiency, making photonics a suitable technology for the realization of neuroinspired computing platforms. This advantage stimulated the development of photonics neural networks based on single and multiple lasers with classical optical cavities. Recently, networks made of random lasers emerged as a novel concept that uses randomly placed scattering elements to create nonlinearity and complexity in photonics neural networks. In this review paper, we present the general framework for networks of coupled lasers, discuss recent advances in networks of random lasers, and outline future directions in this area. We also examine the challenges and limitations of using random lasers in photonic networks, as well as potential solutions. By harnessing the properties of random lasers, such as their unique spectral characteristics in pulsed emission mode and their robustness against noise, networks of interacting random lasers can explore new and exciting possibilities for photonics technology that could find applications in a variety of fields, including image recognition and encryption.
Ergo, SMIRK is safe: a safety case for a machine learning component in a pedestrian automatic emergency brake system

Abstract Integration of machine learning (ML) components in critical applications introduces novel challenges for software certification and verification. New safety standards and technical guidelines are under development to support the safety of ML-based systems, e.g., ISO 21448 SOTIF for the automotive domain and the Assurance of Machine Learning for use in Autonomous Systems (AMLAS) framework. SOTIF and AMLAS provide high-level guidance but the details must be chiseled out for each specific case. We initiated a research project with the goal to demonstrate a complete safety case for an ML component in an open automotive system. This paper reports results from an industry-academia collaboration on safety assurance of SMIRK, an ML-based pedestrian automatic emergency braking demonstrator running in an industry-grade simulator. We demonstrate an application of AMLAS on SMIRK for a minimalistic operational design domain, i.e., we share a complete safety case for its integrated ML-based component. Finally, we report lessons learned and provide both SMIRK and the safety case under an open-source license for the research community to reuse.
Phase Unwrapping Error Correction Based on Multiple Linear Regression Analysis

Fringe projection profilometry (FPP) is prone to phase unwrapping error (PUE) due to phase noise and measurement conditions. Most of the existing PUE-correction methods detect and correct PUE on a pixel-by-pixel or partitioned block basis and do not make full use of the correlation of all information in the unwrapped phase map. In this study, a new method for detecting and correcting PUE is proposed. First, according to the low rank of the unwrapped phase map, multiple linear regression analysis is used to obtain the regression plane of the unwrapped phase, and thick PUE positions are marked on the basis of the tolerance set according to the regression plane. Then, an improved median filter is used to mark random PUE positions and finally correct marked PUE. Experimental results show that the proposed method is effective and robust. In addition, this method is progressive in the treatment of highly abrupt or discontinuous regions.
Siamese PointNet: 3D Head Pose Estimation with Local Feature Descriptor

Head pose estimation is an important part of the field of face analysis technology. It can be applied to driver attention monitoring, passenger monitoring, effective information screening, etc. However, illumination changes and partial occlusion interfere with the task, and due to the non-stationary characteristic of the head pose change process, normal regression networks are unable to achieve very accurate results on large-scale synthetic training data. To address the above problems, a Siamese network based on 3D point clouds was proposed, which adopts a share weight network with similar pose samples to constrain the regression process of the pose’s angles; meanwhile, a local feature descriptor was introduced to describe the local geometric features of the objects. In order to verify the performance of our method, we conducted experiments on two public datasets: the Biwi Kinect Head Pose dataset and Pandora. The results show that compared with the latest methods, our standard deviation was reduced by 0.4, and the mean error was reduced by 0.1; meanwhile, our network also maintained a good real-time performance.
An Enhanced Deep Learning Approach for Breast Cancer Detection in Histopathology Images

Breast cancer is defined as abnormal cellular proliferation in the breast. The most common kind of cancer that affects the breast and causes mortality in women is invasive ductal carcinoma (IDC). As a result, early diagnosis and prognosis have become critical to maximize survival and minimize mortality. Mammograms, computerized tomography (CT) scans, and ultrasounds are among the breast cancer tests available. On the other hand, histopathology evaluation with a biopsy is regarded as one of the most trustworthy techniques for determining if suspicious lesions are malignant. This paper proposes an enhanced approach for classifying breast tumors using a proposed CNN model that we named (CancerNet). We evaluate the proposed model on a benchmark dataset containing 277,524 patches. Compared to several types of CNN-based models, our proposed model has achieved accuracy, Area Under Curve (AUC), precision, recall, and F1-score of 86%, 92%, 81%, 84%, and 83%, respectively, outperforming the previous work on the same benchmark.
Static Hand Gesture Recognition Using Capacitive Sensing and Machine Learning

Automated hand gesture recognition is a key enabler of Human-to-Machine Interfaces (HMIs) and smart living. This paper reports the development and testing of a static hand gesture recognition system using capacitive sensing. Our system consists of a 6×18 array of capacitive sensors that captured five gestures—Palm, Fist, Middle, OK, and Index—of five participants to create a dataset of gesture images. The dataset was used to train Decision Tree, Naïve Bayes, Multi-Layer Perceptron (MLP) neural network, and Convolutional Neural Network (CNN) classifiers. Each classifier was trained five times; each time, the classifier was trained using four different participants’ gestures and tested with one different participant’s gestures. The MLP classifier performed the best, achieving an average accuracy of 96.87% and an average F1 score of 92.16%. This demonstrates that the proposed system can accurately recognize hand gestures and that capacitive sensing is a viable method for implementing a non-contact, static hand gesture recognition system.
COVID-19 now and then: Reflections on mobile communication and the pandemic

The COVID-19 pandemic may soon be coming to its end, but COVID-19 still kills thousands of people every single day (at time of writing). Even if COVID-19 now represents less of a health risk, and less disruption to our personal lives, we know this won’t be the last pandemic. Preparing for the next pandemic includes understanding the past and planning for the future. It includes rethinking “normal” ways of interacting with others, our technologies, and the spaces in which we live. In this introduction, we show how the pandemic has challenged the role of mobile communication in our everyday lives, making us rethink the very meaning of mobile communication—from simply communicating while on the move, to a networked resource that supports emotional and personal connections. During the pandemic, mobile communication practices and the development of new mobile technologies, such as contact-tracing apps and mobile mapping, was strongly tied to the infrastructural politics that took place through government and private companies’ interventions. In addition, mobile technologies became a primary source of support for those who became immobile, or were forced to move. However, mobile communication is not only enabled by end devices; it happens at the intersection of both end devices and the infrastructures that enable them to work. The articles in this special issue reflect some of these themes, and address how the pandemic has shaped and rearranged our mobile communication, sociability, and networked urban mobility practices around the world. Although each article engages with the challenges of the pandemic in its unique and original way, in this introduction we highlight some overlapping topics and methodologies that run across multiple articles, namely historical perspectives on the pandemic, urban and transnational networked mobilities, the use of mobile apps and interfaces for community and self-care, pandemic context in the Global South, and networks and infrastructures.
Hybrid MoS<sub>2</sub>/PbS Quantum Dots Toward In-Sensor Reservoir Computing

Neuromorphic visual systems are potential to overcome the von Neumann bottleneck, emulating the human retina to implement in situ vision preprocessing. Specifically, in-sensor reservoir computing (RC) is capable of processing spatio-temporal vision information with high efficiency. However, traditional photodetectors are incapable of the fading memory ability, which is the core of the RC systems for temporal information processing. Here, we report a hybrid MoS2/PbS quantum dots based optoelectronic device with short-term memory properties for RC demonstration. The photogating effect induced dynamic temporal photoresponse behavior can be effectively regulated by the incident optical intensity and time. As a proof of concept, image classifying with a high accuracy of 95.8% and time series prediction are demonstrated, paving a way for intelligent edge computing.
Changes to Captions: An Attentive Network for Remote Sensing Change Captioning

In recent years, advanced research has focused on the direct learning and analysis of remote-sensing images using natural language processing (NLP) techniques. The ability to accurately describe changes occurring in multi-temporal remote sensing images is becoming increasingly important for geospatial understanding and land planning. Unlike natural image change captioning tasks, remote sensing change captioning aims to capture the most significant changes, irrespective of various influential factors such as illumination, seasonal effects, and complex land covers. In this study, we highlight the significance of accurately describing changes in remote sensing images and present a comparison of the change captioning task for natural and synthetic images and remote sensing images. To address the challenge of generating accurate captions, we propose an attentive changes-to-captions network, called Chg2Cap for short, for bi-temporal remote sensing images. The network comprises three main components: 1) a Siamese CNN-based feature extractor to collect high-level representations for each image pair; 2) an attentive encoder that includes a hierarchical self-attention block to locate change-related features and a residual block to generate the image embedding; and 3) a transformer-based caption generator to decode the relationship between the image embedding and the word embedding into a description. The proposed Chg2Cap network is evaluated on two representative remote sensing datasets, and a comprehensive experimental analysis is provided. The code and pre-trained models will be available online at https://github.com/ShizhenChang/Chg2Cap.
Progressive multi-level distillation learning for pruning network

Abstract Although the classification method based on the deep neural network has achieved excellent results in classification tasks, it is difficult to apply to real-time scenarios because of high memory footprints and prohibitive inference times. Compared to unstructured pruning, structured pruning techniques can reduce the computation cost of the model runtime more effectively, but inevitably reduces the precision of the model. Traditional methods use fine tuning to restore model damage performance. However, there is still a large gap between the pruned model and the original one. In this paper, we use progressive multi-level distillation learning to compensate for the loss caused by pruning. Pre-pruning and post-pruning networks serve as the teacher and student networks. The proposed approach utilizes the complementary properties of structured pruning and knowledge distillation, which allows the pruned network to learn the intermediate and output representations of the teacher network, thus reducing the influence of the model subject to pruning. Experiments demonstrate that our approach performs better on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets with different pruning rates. For instance, GoogLeNet can achieve near lossless pruning on the CIFAR-10 dataset with 60% pruning. Moreover, this paper also proves that using the proposed distillation learning method during the pruning process achieves more significant performance gains than after completing the pruning.
Prompting Is Programming: A Query Language for Large Language Models

Large language models have demonstrated outstanding performance on a wide range of tasks such as question answering and code generation. On a high level, given an input, a language model can be used to automatically complete the sequence in a statistically-likely way. Based on this, users prompt these models with language instructions or examples, to implement a variety of downstream tasks. Advanced prompting methods can even imply interaction between the language model, a user, and external tools such as calculators. However, to obtain state-of-the-art performance or adapt language models for specific tasks, complex task- and model-specific programs have to be implemented, which may still require ad-hoc interaction. Based on this, we present the novel idea of Language Model Programming (LMP). LMP generalizes language model prompting from pure text prompts to an intuitive combination of text prompting and scripting. Additionally, LMP allows constraints to be specified over the language model output. This enables easy adaption to many tasks while abstracting language model internals and providing high-level semantics. To enable LMP, we implement LMQL (short for Language Model Query Language), which leverages the constraints and control flow from an LMP prompt to generate an efficient inference procedure that minimizes the number of expensive calls to the underlying language model. We show that LMQL can capture a wide range of state-of-the-art prompting methods in an intuitive way, especially facilitating interactive flows that are challenging to implement with existing high-level APIs. Our evaluation shows that we retain or increase the accuracy on several downstream tasks, while also significantly reducing the required amount of computation or cost in the case of pay-to-use APIs (26-85% cost savings).