Reviewerly/exHarmony huggingface dataset:
Photographic Lighting Design with Photographer-in-the-Loop Bayesian Optimization

It is important for photographers to have the best possible lighting configuration at the time of shooting; otherwise, they need post-processing on images, which may cause artifacts and deterioration. Thus, photographers often struggle to find the best possible lighting configuration by manipulating lighting devices, including light sources and modifiers, in a trial-and-error manner. In this paper, we propose a novel computational framework to support photographers. This framework assumes that every lighting device is programmable; that is, its adjustable parameters (e.g., orientation, intensity, and color temperature) can be set using a program. Using our framework, photographers do not need to learn how the parameter values affect the resulting lighting, and even do not need to determine the strategy of the trial-and-error process; instead, photographers need only concentrate on evaluating which lighting configuration is more desirable among options suggested by the system. The framework is enabled by our novel photographer-in-the-loop Bayesian optimization, which is sample-efficient (i.e., the number of required evaluation steps is small) and which can also be guided by providing a rough painting of the desired lighting configuration if any. We demonstrate how the framework works in both simulated virtual environments and a physical environment, suggesting that it could find pleasing lighting configurations quickly in around 10 iterations. Our user study suggests that the framework enables the photographer to concentrate on the look of captured images rather than the parameters, compared with the traditional manual lighting workflow.
Record Once, Post Everywhere: Automatic Shortening of Audio Stories for Social Media

Following the prevalence of short-form video, short-form voice content has emerged on social media platforms like Twitter and Facebook. A challenge that creators face is hard constraints on the content length. If the initial recording is not short enough, they need to re-record or edit their content. Both are time-consuming, and the latter, if supported, can have a learning curve. Moreover, creators need to manually create multiple versions to publish content on platforms with different length constraints. To simplify this process, we present ROPE1 (Record Once, Post Everywhere). Creators can record voice content once, and our system will automatically shorten it to all length limits by removing parts of the recording for each target. We formulate this as a combinatorial optimization problem and propose a novel algorithm that automatically selects optimal sentence combinations from the original content to comply with each length constraint. Creators can customize the algorithmically shortened content by specifying sentences to include or exclude. Our system can also use the user-specified constraints to recompute and provides a new version. We conducted a user study comparing ROPE with a sentence-based manual editing baseline. The results show that ROPE can generate high-quality edits, alleviating the cognitive loads of creators for shortening content. While our system and user study address short-form voice content specifically, we believe that the same concept can also be applied to other media such as video with narration and dialog.
Synthesis-Assisted Video Prototyping From a Document

Video productions commonly start with a script, especially for talking head videos that feature a speaker narrating to the camera. When the source materials come from a written document – such as a web tutorial, it takes iterations to refine content from a text article to a spoken dialogue, while considering visual compositions in each scene. We propose Doc2Video, a video prototyping approach that converts a document to interactive scripting with a preview of synthetic talking head videos. Our pipeline decomposes a source document into a series of scenes, each automatically creating a synthesized video of a virtual instructor. Designed for a specific domain – programming cookbooks, we apply visual elements from the source document, such as a keyword, a code snippet or a screenshot, in suitable layouts. Users edit narration sentences, break or combine sections, and modify visuals to prototype a video in our Editing UI. We evaluated our pipeline with public programming cookbooks. Feedback from professional creators shows that our method provided a reasonable starting point to engage them in interactive scripting for a narrated instructional video.
SemanticOn: Specifying Content-Based Semantic Conditions for Web Automation Programs

Data scientists, researchers, and clerks often create web automation programs to perform repetitive yet essential tasks, such as data scraping and data entry. However, existing web automation systems lack mechanisms for defining conditional behaviors where the system can intelligently filter candidate content based on semantic filters (e.g., extract texts based on key ideas or images based on entity relationships). We introduce SemanticOn, a system that enables users to specify, refine, and incorporate visual and textual semantic conditions in web automation programs via two methods: natural language description via prompts or information highlighting. Users can coordinate with SemanticOn to refine the conditions as the program continuously executes or reclaim manual control to repair errors. In a user study, participants completed a series of conditional web automation tasks. They reported that SemanticOn helped them effectively express and refine their semantic intent by utilizing visual and textual conditions.
Expert Goggles: Detecting and Annotating Visualizations using a Machine Learning Classifier

Data visualizations are an increasingly common way to communicate information online. However, not everyone has the data literacy skills necessary to interpret complex visualizations effectively. We present Expert Goggles, a Chrome browser extension that provides just-in-time guidance to help non-experts interpret the visualizations they encounter on the web. Expert Goggles uses a machine learning classifier to automatically determine the visualization type and uses this context to deliver relevant learning materials. We discuss how this approach to automatically detect the context and provide just-in-time support might transform everyday experiences into informal learning opportunities.
Removal of noise on mammogram breast images using filtering methods

Summary Detecting Breast Cancer (BC) at an early stage can reduce the rate of mortality. A mammogram is a preliminary examination of breast cancer with an optimistic and accurate method, for the same test will be used to know the behavior of the tumor in all stages of cancer. Unpolished Images need to process for a clear image to analyze, and enhanced images can use to view the tissue, muscle, and tumor. Image processing methods are not only used to remove the noisy pixels but also help to extract the Region of Interest (RoI) from the image. The size and structure of the tumor will be extracted using statistical analysis. This study was experimented on mammogram images by analyzing the types of noise on the image and filtering methods applied for a specific class of noise data. The different filtering approaches like Adaptive, Gaussian, Threshold, Mean, Mean‐Median, Bilateral, Wiener, and Tri‐State Filters were used. Mammogram images are grayscale images. The objective of this research experiment is noise data will reduce the detection factors of the tumor with True−Ve and False+Ve results. Fault results will impact the accuracy to identify the RoI. Image Preprocessing will help to improve the quality of the images and help in the treatment plan and expert advice in the decision of the disease stage. Results : The Dataset used images from CBIS‐DDSM (The Curated Breast Imaging Subset of DDSM‐[Digital Database for Screening Mammography]). The obtained Peak‐signal‐to‐Noise Ratio (PSNR) achieved is 80% minimum.
A novel MobileNet with selective depth multiplier to compromise complexity and accuracy

Abstract In the last few years, convolutional neural networks (CNNs) have demonstrated good performance while solving various computer vision problems. However, since CNNs exhibit high computational complexity, signal processing is performed on the server side. To reduce the computational complexity of CNNs for edge computing, a lightweight algorithm, such as a MobileNet, is proposed. Although MobileNet is lighter than other CNN models, it commonly achieves lower classification accuracy. Hence, to find a balance between complexity and accuracy, additional hyperparameters for adjusting the size of the model have recently been proposed. However, significantly increasing the number of parameters makes models dense and unsuitable for devices with limited computational resources. In this study, we propose a novel MobileNet architecture, in which the number of parameters is adaptively increased according to the importance of feature maps. We show that our proposed network achieves better classification accuracy with fewer parameters than the conventional MobileNet.
Fate of Heavy Metals in the Surface Water-Dump Rock System of the Mine Lupikko I (Karelia): Field Observations and Geochemical Modeling

Abandoned mines are sources of potentially toxic chemical elements, although the development of these objects was completed. The Lupikko I mine area (Karelia Republic, Russia) is an excellent example of such technogenic objects. It is one of the largest mines in the Pitkäranta area, which was abandoned more than one hundred years ago. The dump rocks here are characterized by significant mineral diversity. Disseminated ore mineralization of the study area contains heavy metals, which enter the natural waters due to the oxidative dissolution of sulfides. Dump rocks and water from the Lupikko I mine area were collected to research the behavior of toxic elements. The samples were analyzed using ICP-MS, ICP-AES, potentiometric titration, ionic chromatography, X-ray microanalysis, X-ray fluorescence, and SEM to obtain information about the geochemical environment. According to new data, the content of Fe, Zn, Cu, Pb, Cd, and Ni in the natural waters of the mine significantly exceeds the geochemical background. For a more detailed study of the behavior of heavy metals, equilibrium-kinetic modeling, which considers the dissolution rate of ore minerals and the accumulation of toxic elements over time, was applied. A comparison of modeling data and field observations agreed. It was also found that for accurate modeling of Fe behavior, it is necessary to consider the organic matter content. Despite some model limitations, such retrospective assessments allow us to approve the applicability of this method for forecasting estimates.
A Generalized Laser Simulator Algorithm for Mobile Robot Path Planning with Obstacle Avoidance

This paper aims to develop a new mobile robot path planning algorithm, called generalized laser simulator (GLS), for navigating autonomously mobile robots in the presence of static and dynamic obstacles. This algorithm enables a mobile robot to identify a feasible path while finding the target and avoiding obstacles while moving in complex regions. An optimal path between the start and target point is found by forming a wave of points in all directions towards the target position considering target minimum and border maximum distance principles. The algorithm will select the minimum path from the candidate points to target while avoiding obstacles. The obstacle borders are regarded as the environment's borders for static obstacle avoidance. However, once dynamic obstacles appear in front of the GLS waves, the system detects them as new dynamic obstacle borders. Several experiments were carried out to validate the effectiveness and practicality of the GLS algorithm, including path-planning experiments in the presence of obstacles in a complex dynamic environment. The findings indicate that the robot could successfully find the correct path while avoiding obstacles. The proposed method is compared to other popular methods in terms of speed and path length in both real and simulated environments. According to the results, the GLS algorithm outperformed the original laser simulator (LS) method in path and success rate. With application of the all-direction border scan, it outperforms the A-star (A*) and PRM algorithms and provides safer and shorter paths. Furthermore, the path planning approach was validated for local planning in simulation and real-world tests, in which the proposed method produced the best path compared to the original LS algorithm.
Design of UVC Surface Disinfection Robot with Coverage Path Planning Using Map-Based Approach At-The-Edge

In response to the issue of virus contamination in the cold-chain warehouse or hospital environment under the influence of the COVID-19, we propose the design work of a disinfection robot based on the UVC radiation mechanism using the low-computational path optimization at-the-edge. To build a surface disinfection robot with less computing power to generate a collision-free path with shorter total distance in studies, a 2D map is used as a graph-based approach to automatically generate a closed-loop disinfection path to cover all the accessible surfaces. The discrete disinfection points from the map are extracted with effective disinfection distances and sorted by a nearest-neighbor (NN) search over historical trajectory data and improved A * algorithm to obtain an efficient coverage path to all accessible boundaries of the entire area. The purpose of improved A * algorithm with NN is not to find the optimal path solution but to optimize one with reasonable computing power. The proposed algorithm enhances the path-finding efficiency by a dynamically weighted heuristic function and reduces the path turning angles, which improves the path smoothness significantly requiring less computing power. The Gazebo simulation is conducted, and the prototype disinfection robot has been built and tested in a real lab environment. Compared with the classic A * algorithm, the improved A * algorithm with NN has improved the path-finding efficiency and reduced the path length while covering the same area. Both the simulation and experimental results show that this approach can provide the design to balance the tradeoffs among the path-finding efficiency, smoothness, disinfection coverage, and computation resources.
Image Large Rotation and Scale Estimation Using the Gabor Filter

This paper proposes a novel image large rotation and scale estimation method based on the Gabor filter and pulse-coupled neural network (PCNN). First, the Gabor features of the template image and its rotated one are extracted by performing the Gabor filter. Second, we present a modified PCNN model to measure the similarity between the Gabor features of the image and its rotated one. Finally, the rotation angle is calculated by searching the global minimum of the correlation coefficients. Besides rotation estimation, we also propose a scale estimation method based on the max-projection strategy. The Gabor feature image is projected along the estimated rotation angle, and the scale is calculated by searching the peak of this projection result. Moreover, experiments illustrate that the proposed method has high accuracy on rotation and scale estimation and is robust to noise. Compared with the state-of-the-art methods, the proposed approach has a more stable performance.
Applications and Challenges of Federated Learning Paradigm in the Big Data Era with Special Emphasis on COVID-19

Federated learning (FL) is one of the leading paradigms of modern times with higher privacy guarantees than any other digital solution. Since its inception in 2016, FL has been rigorously investigated from multiple perspectives. Some of these perspectives are extensions of FL’s applications in different sectors, communication overheads, statistical heterogeneity problems, client dropout issues, the legitimacy of FL system results, privacy preservation, etc. Recently, FL is being increasingly used in the medical domain for multiple purposes, and many successful applications exist that are serving mankind in various ways. In this work, we describe the novel applications and challenges of the FL paradigm with special emphasis on the COVID-19 pandemic. We describe the synergies of FL with other emerging technologies to accomplish multiple services to fight the COVID-19 pandemic. We analyze the recent open-source development of FL which can help in designing scalable and reliable FL models. Lastly, we suggest valuable recommendations to enhance the technical persuasiveness of the FL paradigm. To the best of the authors’ knowledge, this is the first work that highlights the efficacy of FL in the era of COVID-19. The analysis enclosed in this article can pave the way for understanding the technical efficacy of FL in medical field, specifically COVID-19.
Self-Supervised Reinforcement Learning with dual-reward for knowledge-aware recommendation

To improve the recommendation accuracy and offer explanations for recommendations, Reinforcement Learning (RL) has been applied to path reasoning over knowledge graphs. However, in recommendation tasks, most existing RL methods learn the path-finding policy using only a short-term or single reward, leading to a local optimum and losing some potential paths. To address these issues, we propose a Self-Supervised Reinforcement Learning (SSRL) framework combined with dual-reward for knowledge-aware recommendation reasoning over knowledge graphs. Then, we improve Actor–Critic algorithm by using a dual-reward driven strategy, which combines short-term reward with long-term incremental evaluation. The improved algorithm helps the policy guide path reasoning in an overall situation. In addition, to find the most potential paths, in the improved Actor–Critic algorithm, a loss constraint of each sample is used as a reinforced signal to update the gradients. With some improvements against baselines, experimental results demonstrate the effectiveness of our framework.
SAM-X: sorting algorithm for musculoskeletal x-ray radiography

To develop a two-phased deep learning sorting algorithm for post-X-ray image acquisition in order to facilitate large musculoskeletal image datasets according to their anatomical entity.In total, 42,608 unstructured and pseudonymized radiographs were retrieved from the PACS of a musculoskeletal tumor center. In phase 1, imaging data were sorted into 1000 clusters by a self-supervised model. A human-in-the-loop radiologist assigned weak, semantic labels to all clusters and clusters with the same label were merged. Three hundred thirty-two non-musculoskeletal clusters were discarded. In phase 2, the initial model was modified by "injecting" the identified labels into the self-supervised model to train a classifier. To provide statistical significance, data split and cross-validation were applied. The hold-out test set consisted of 50% external data. To gain insight into the model's predictions, Grad-CAMs were calculated.The self-supervised clustering resulted in a high normalized mutual information of 0.930. The expert radiologist identified 28 musculoskeletal clusters. The modified model achieved a classification accuracy of 96.2% and 96.6% for validation and hold-out test data for predicting the top class, respectively. When considering the top two predicted class labels, an accuracy of 99.7% and 99.6% was accomplished. Grad-CAMs as well as final cluster results underlined the robustness of the proposed method by showing that it focused on similar image regions a human would have considered for categorizing images.For efficient dataset building, we propose an accurate deep learning sorting algorithm for classifying radiographs according to their anatomical entity in the assessment of musculoskeletal diseases.• Classification of large radiograph datasets according to their anatomical entity. • Paramount importance of structuring vast amounts of retrospective data for modern deep learning applications. • Optimization of the radiological workflow and increase in efficiency as well as decrease of time-consuming tasks for radiologists through deep learning.
An Effective ELECTRA-Based Pipeline for Sentiment Analysis of Tourist Attraction Reviews

In the era of information explosion, it is difficult for people to decide on a tourist destination quickly. Online travel review texts provide valuable references and suggestions to assist in decision making. However, tourist attraction reviews are primarily informal and noisy. Most works in this field focus on shallow machine learning models or non-pretrained deep learning models. These approaches struggle to generate satisfactory classification results. To solve this issue, the paper proposes a pipeline model. In the first step of this paper, we preprocess tourist attraction reviews by performing stopword removal, special character removal, redundancy deletion and negation substitution to reduce noise. Then, we propose an ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately) classifier for sentiment analysis of tourist attraction review. Finally, we compare our pipeline model with several representative deep text classification models. Extensive experiments have demonstrated the effectiveness of our approach to sentiment analysis of tourist attraction reviews. We not only provide one high-quality dataset for tourist attraction reviews, but our work can also expand and promote the development of sentiment analysis in other domains.
FPGA Implementation of Compact Hardware Accelerators for Ring-Binary-LWE-based Post-quantum Cryptography

Post-quantum cryptography (PQC) has recently drawn substantial attention from various communities owing to the proven vulnerability of existing public-key cryptosystems against the attacks launched from well-established quantum computers. The Ring-Binary-Learning-with-Errors (RBLWE), a variant of Ring-LWE, has been proposed to build PQC for lightweight applications. As more Field-Programmable Gate Array (FPGA) devices are being deployed in lightweight applications like Internet-of-Things (IoT) devices, it would be interesting if the RBLWE-based PQC can be implemented on the FPGA with ultra-low complexity and flexible processing. However, thus far, limited information is available for such implementations. In this article, we propose novel RBLWE-based PQC accelerators on the FPGA with ultra-low implementation complexity and flexible timing. We first present the process of deriving the key operation of the RBLWE-based scheme into the proposed algorithmic operation. The corresponding hardware accelerator is then efficiently mapped from the proposed algorithm with the help of algorithm-to-architecture implementation techniques and extended to obtain higher-throughput designs. The final complexity analysis and implementation results (on a variety of FPGAs) show that the proposed accelerators have significantly smaller area-time complexities than the state-of-the-art designs. Overall, the proposed accelerators feature low implementation complexity and flexible processing, making them desirable for emerging FPGA-based lightweight applications.