Reviewerly/exHarmony huggingface dataset:
Intelligent Services Development Technology Using the IACPaaS Cloud Platform

The paper presents a technology for the development of intelligent multi-agent services. This technology is put to reduce the labor-intensiveness of their creation and maintenance. Its key aspect is an independent development of knowledge and databases, a user interface and a problem solver (as a set of agents), and their assembly into an intelligent service. The principal specifics of the technology are: two-level ontological approach to the formation of knowledge and databases with a clear separation between the ontology and information formed on its basis; the ontology-based specification of not only the domain terminology and structure, but also of the rules for knowledge and data formation, control of their integrity and completeness; division of all software components into procedural and declarative parts; unified semantic representation of the latter (including declarative parts of software components), etc. The proposed approaches are implemented on the IACPaaS cloud platform where expert-oriented formation of each service component with an appropriate tool is provided. An example of service development which demonstrates the technology usage is given.
A Blockchain-based Multi-layer Decentralized Framework for Robust Federated Learning

With the expansion of the Internet of Things (IoT) development and application, federated learning has gained higher popularity in industrial researching fields. However, the security issues in federated learning have become hot-spots in the research area, such as privacy-preserving and poisoning attacks. This paper proposes a robust blockchained multi-layer decentralized federated learning (RBML-DFL) framework to ensure the federated learning's robustness. Firstly, by adopting the three-layered framework, the blockchain connects the federated learning components to secure the privacy and data safety of federated learning. Secondly, the proposed framework provides resilience on poisoning attacks to the central model compared to typical federated learning frameworks. Lastly, the decentralized structure associated with the blockchain tracing back mechanism can prevent the central server failure or mal-function compared to centralized federated learning. We evaluate and compare the proposed framework with other state-of-the-art federated learning frameworks on the accuracy, latency, and system robustness under poisoning attacks. The results show that the proposed RBML-DFL framework outperforms state-of-the-art baseline frameworks on all three metrics: accuracy, latency, and the robustness of the federated learning.
Watermark Vaccine: Adversarial Attacks to Prevent Watermark Removal

As a common security tool, visible watermarking has been widely applied to protect copyrights of digital images. However, recent works have shown that visible watermarks can be removed by DNNs without damaging their host images. Such watermark-removal techniques pose a great threat to the ownership of images. Inspired by the vulnerability of DNNs on adversarial perturbations, we propose a novel defence mechanism by adversarial machine learning for good. From the perspective of the adversary, blind watermark-removal networks can be posed as our target models; then we actually optimize an imperceptible adversarial perturbation on the host images to proactively attack against watermark-removal networks, dubbed Watermark Vaccine. Specifically, two types of vaccines are proposed. Disrupting Watermark Vaccine (DWV) induces to ruin the host image along with watermark after passing through watermark-removal networks. In contrast, Inerasable Watermark Vaccine (IWV) works in another fashion of trying to keep the watermark not removed and still noticeable. Extensive experiments demonstrate the effectiveness of our DWV/IWV in preventing watermark removal, especially on various watermark removal networks. The Code is released in https://github.com/thinwayliu/Watermark-Vaccine .
Sign Language Recognition Using the Reuse of Estimate Results by Each Epoch

Various researchers have proposed systems with high recognition rates for sign language recognition. One of the methods for sign language video recognition is to perform preprocessing, such as optical flow and posture estimation from images, followed by machine learning processing using models such as 3DCNN. Recognition rates tend to be particularly high for models that use information from posture estimation. In addition, many studies have recently used multiple recognition methods and fused their results to increase recognition rates further. In those studies, a Skeleton Aware Multi-modal SLR (SAM-SLR) model used the Turkish sign language dataset AUTSL to achieve very high recognition rates. The next version, SAM-SLR-v2, proposed a skeleton-aware multi-model framework that supports three sign language datasets and uses a global ensemble model for isolated signs. In particular, one of the three datasets, Turkish Sign Language, contains both RGB and Depth videos, and SAM-SLR-v2 achieved high recognition rates of 98.00% and 98.10%, respectively. We wanted to pursue the possibility of achieving even higher recognition rates. Therefore, in this study, based on this SAM-SLR, we first propose a method to increase the recognition rate by reusing the previous training models and estimate results created at each epoch when creating training models for each recognition method using the posture estimation Joint and Bone features. Next, we report the recognition rate of the Multi-stream, Modal-free Late-fusion Ensemble using the proposed method with Joint and Bone. We demonstrate that our proposed method is easy to implement and achieves improved performance because it reuses the estimate results already obtained. Our method achieved a recognition rate of 96.10% and 96.18% for Joint and Bone, respectively, and the results were reflected in the Modal-free Latefusion Ensemble, which achieved a recognition rate of 98.05% when the optimal parameters were given manually in RGB videos.
ANALYSIS OF MEDIA LITERACY AMONG JOURNALISM STUDENTS

In modern society, media literacy issues are particularly relevant due to the spread of fake information and unverified news. The analysis of the level of media literacy as the competence of future journalists seems to be a promising area of scientific and practical research, since journalists are both authors and consumers of information. As the main methods of studying the level of media literacy, we use a pedagogical experiment and a questionnaire survey. We conducted a mass questionnaire survey to determine the level of media literacy of 1st and 4th year students enrolled in the direction of 42.03.02 “Journalism” at Northern (Arctic) Federal University named after M. V. Lomonosov. As a result, the authors came to the conclusion that having a special education in journalism and initial experience in the media sphere does not guarantee achieving a high level.
Decentralizing Electronic Medical Records on the Blockchain Using Smart Contracts

The blockchain technology is considered as the most secure and decentralized way to manage your payments with cryptocurrencies.Furthermore, with the advent of NFT's which are a way to make owning digital art a possibility we can see how blockchain has multiple purposes.Blockchain gained traction because of its attributes such as security, immutability, expandability and decentralism.We can see blockchain being used more and more for non-financial applications.On the topic of electronic medical records which need to be secure and be accessed globally, it is still being stored locally or on the cloud which is confined to an organization or a group of organizations.We can observe that healthcare is a field that can benefit massively from blockchain technology.It can make electronic medical records be securely accessed in case of emergencies using smart contracts making the system effectively be used hand in hand with conventional database systems.Smart contracts can be used to manage the accessibility of electronic medical data by third party organizations globally without the data being tampered, manipulated or misused.The system can be used with the current database as it is interoperable.With the successful implementation of this system, it will give access to patients, health care organizations and other third parties the ability to view and manage the electronic medical data.
Privacy-Preserving Federated Learning for Detecting Fraudulent Financial Transactions in Japanese Banks

To tackle financial crimes including fraudulent financial transactions (FFTs), money laundering, illegal money transfers, and bank transfer scams, several attempts have been considered to employ artificial intelligence (AI)-based FFT detection systems, particularly, deep learning-based ones. However, to the best of our knowledge, no federated learning systems using real transaction data among financial institutions have been implemented so far. This is because there are several issues to be addressed as follows: (1) it is difficult to prepare sufficient amount of transaction data for training by one financial institution (e.g., a local bank), and a small amount of dataset does not promise high accuracy for detection, (2) each transaction data contains personal information, and thus it is restricted by Act on the Protection of Personal Information in Japan to provide the transaction data to a third party. In this paper, we introduce out demonstration experimental results of privacy-preserving federated learning with five banks in Japan: the Chiba Bank, Ltd., MUFG Bank, Ltd., the Chugoku Bank, Ltd., Sumitomo Mitsui Trust Bank, Ltd., and the Iyo Bank, Ltd. As the underlying cryptographic tool, we proposed a privacy-preserving federated learning protocol which we call Deepprotect, for detecting fraudulent financial transactions. Briefly, Deepprotect allows parties to execute the stochastic gradient descent algorithm using a set of techniques for the privacy-preserving deep learning algorithms (IEEE TIFS 2018, 2019). In the demonstration experiments, we built machine learning models for detecting two types of financial frauds — detecting fraudulent transactions in customers/victims' accounts and detecting criminals' bank accounts. We show that our federated learning system detected FFTs that could not be detected by the model built using the dataset from a single bank and detected criminals' bank accounts before the bank actually froze them.
Coevolution of education and business: adaptive interaction

The study focuses on innovative means of interaction between educational and business entities. These means are aimed at drawing a parallel between established educational objectives and practical experience. Business education has to transform to ensure coevolution of the business needs and capabilities of the business education, and this challenge is discussed in the article. It explains why it is urgent to introduce new ways of interaction and involvement of the students in managerial and economic processes. An expanded typology of modern business education models has been described in this study. Their practical implementation would prove fruitful in the long run. In particular, it would allow the educational institution to evolve along with the business environment and constantly adapt to new challenges. Businesses would be able to improve and replenish labour resources, and graduates of business schools would be more competitive on the market.
Distributed Mission Planning of Complex Tasks for Heterogeneous Multi-Robot Systems

In this paper, we propose a distributed multistage optimization method for planning complex missions for heterogeneous multi-robot systems. This class of problems involves tasks that can be executed in different ways and are associated with cross-schedule dependencies that constrain the schedules of the different robots in the system. The proposed approach involves a multi-objective heuristic search of the mission, represented as a hierarchical tree that defines the mission goal. This procedure outputs several favorable ways to fulfill the mission, which directly feed into the next stage of the method. We utilize a distributed metaheuristic based on evolutionary computation to allocate tasks and generate schedules for the set of chosen decompositions. The solution is evaluated in a simulation setup of an automated greenhouse use case, where we demonstrate the system’s ability to adapt the planning strategy depending on the available robots and the given optimization criteria.
A region-adaptive non-local denoising algorithm for low-dose computed tomography images

&lt;abstract&gt; &lt;p&gt;Low-dose computed tomography (LDCT) can effectively reduce radiation exposure in patients. However, with such dose reductions, large increases in speckled noise and streak artifacts occur, resulting in seriously degraded reconstructed images. The non-local means (NLM) method has shown potential for improving the quality of LDCT images. In the NLM method, similar blocks are obtained using fixed directions over a fixed range. However, the denoising performance of this method is limited. In this paper, a region-adaptive NLM method is proposed for LDCT image denoising. In the proposed method, pixels are classified into different regions according to the edge information of the image. Based on the classification results, the adaptive searching window, block size and filter smoothing parameter could be modified in different regions. Furthermore, the candidate pixels in the searching window could be filtered based on the classification results. In addition, the filter parameter could be adjusted adaptively based on intuitionistic fuzzy divergence (IFD). The experimental results showed that the proposed method performed better in LDCT image denoising than several of the related denoising methods in terms of numerical results and visual quality.&lt;/p&gt; &lt;/abstract&gt;
COO: Comic Onomatopoeia Dataset for Recognizing Arbitrary or Truncated Texts

Recognizing irregular texts has been a challenging topic in text recognition. To encourage research on this topic, we provide a novel comic onomatopoeia dataset (COO), which consists of onomatopoeia texts in Japanese comics. COO has many arbitrary texts, such as extremely curved, partially shrunk texts, or arbitrarily placed texts. Furthermore, some texts are separated into several parts. Each part is a truncated text and is not meaningful by itself. These parts should be linked to represent the intended meaning. Thus, we propose a novel task that predicts the link between truncated texts. We conduct three tasks to detect the onomatopoeia region and capture its intended meaning: text detection, text recognition, and link prediction. Through extensive experiments, we analyze the characteristics of the COO. Our data and code are available at https://github.com/ku21fan/COO-Comic-Onomatopoeia .
Self-supervised Social Relation Representation for Human Group Detection

Human group detection, which splits crowd of people into groups, is an important step for video-based human social activity analysis. The core of human group detection is the human social relation representation and division. In this paper, we propose a new two-stage multi-head framework for human group detection. In the first stage, we propose a human behavior simulator head to learn the social relation feature embedding, which is self-supervised trained by leveraging the socially grounded multi-person behavior relationship. In the second stage, based on the social relation embedding, we develop a self-attention inspired network for human group detection. Remarkable performance on two state-of-the-art large-scale benchmarks, i.e., PANDA and JRDB-Group, verifies the effectiveness of the proposed framework. Benefiting from the self-supervised social relation embedding, our method can provide promising results with very few (labeled) training data. We have released the source code to the public.
Multimodal Feature Extraction for Memes Sentiment Classification

In this study, we propose feature extraction for multimodal meme classification using Deep Learning approaches. A meme is usually a photo or video with text shared by the young generation on social media platforms that expresses a culturally relevant idea. Since they are an efficient way to express emotions and feelings, a good classifier that can classify the sentiment behind the meme is important. To make the learning process more efficient, reduce the likelihood of overfitting, and improve the generalizability of the model, one needs a good approach for joint feature extraction from all modalities. In this work, we proposed to use different multimodal neural network approaches for multimodal feature extraction and use the extracted features to train a classifier to identify the sentiment in a meme.
SiRi: A Simple Selective Retraining Mechanism for Transformer-Based Visual Grounding

In this paper, we investigate how to achieve better visual grounding with modern vision-language transformers, and propose a simple yet powerful Selective Retraining (SiRi) mechanism for this challenging task. Particularly, SiRi conveys a significant principle to the research of visual grounding, i.e., a better initialized vision-language encoder would help the model converge to a better local minimum, advancing the performance accordingly. In specific, we continually update the parameters of the encoder as the training goes on, while periodically re-initialize rest of the parameters to compel the model to be better optimized based on an enhanced encoder. SiRi can significantly outperform previous approaches on three popular benchmarks. Specifically, our method achieves 83.04% Top1 accuracy on RefCOCO+ testA, outperforming the state-of-the-art approaches (training from scratch) by more than 10.21%. Additionally, we reveal that SiRi performs surprisingly superior even with limited training data. We also extend it to transformer-based visual grounding models and other vision-language tasks to verify the validity. Code is available at https://github.com/qumengxue/siri-vg.git .
Trends of Quantum Computing Applications to Computer Vision

As the development in quantum computation is on the rise, its potential application in various research areas has been investigated, including to artificial intelligence and machine learning (AI/ML). Computer vision, the fastest growing field in the AI/ML field, has also seen potential applications of quantum computing to solve computer vision tasks. In this paper, we discuss the current approach of utilizing quantum computing to computer vision. In particular, we firstly elaborate on classical approaches to solving several computer vision tasks, then briefly describe the background of different paradigms of quantum computers, i.e., the adiabatic quantum computer (quantum annealer), and the gate model, also known as the universal quantum computer. Subsequently, we summarize the problem formulation and quantum computing-based solutions to computer vision tasks that appeared in the major AI/ML conferences in the past years. Additionally, the strengths and drawbacks of each case are also discussed, providing beneficial insights into the current state of quantum computing usage for computer vision.
Efficient Fine-Tuning of BERT Models on the Edge

Resource-constrained devices are increasingly the deployment targets of machine learning applications. Static models, however, do not always suffice for dynamic environments. On-device training of models allows for quick adaptability to new scenarios. With the increasing size of deep neural networks, as noted with the likes of BERT and other natural language processing models, comes increased resource requirements, namely memory, computation, energy, and time. Furthermore, training is far more resource intensive than inference. Resource-constrained on-device learning is thus doubly difficult, especially with large BERT-like models. By reducing the memory usage of fine-tuning, pre-trained BERT models can become efficient enough to fine-tune on resource-constrained devices. We propose Freeze And ReconFigure (FAR), a memory-efficient training regime for BERT-like models that reduces the memory usage of activation maps during fine-tuning by avoiding unnecessary parameter updates. FAR reduces fine-tuning time on the DistilBERT model and CoLA dataset by 30 %, and time spent on memory operations by 47%. More broadly, reductions in metric performance on the GLUE and SQuAD datasets are around 1% on average.
A Federated Learning Based Privacy-Preserving Data Sharing Scheme for Internet of Vehicles

The data analysis in the process of vehicle collaboration for the Internet of Vehicles (IoV) environment improves the driving experience and service quality. However, the privacy issue is becoming one of the problems of obstructing the development of data sharing among vehicles. To overcome the disadvantage, in this work, we propose a privacy-preserving data sharing scheme based on federated learning by the collaboration of participants, which can resist gradient leakage, poisoning attacks, etc. Firstly, the gradient data is encrypted by random masking to protect the privacy of training data. Then, the Pearson correlation coefficient is utilized to distinguish the correctness of the model parameters uploaded from the vehicle at uplink. Finally, the proposed scheme can verify the correctness of the global model distributed from AS at downlink using the Lagrange interpolation The experimental results show that the proposed privacy-preserving data sharing scheme provides higher learning accuracy by eliminating malicious gradients.
SemAug: Semantically Meaningful Image Augmentations for Object Detection Through Language Grounding

Data augmentation is an essential technique in improving the generalization of deep neural networks. The majority of existing image-domain augmentations either rely on geometric and structural transformations, or apply different kinds of photometric distortions. In this paper, we propose an effective technique for image augmentation by injecting contextually meaningful knowledge into the scenes. Our method of semantically meaningful image augmentation for object detection via language grounding, SemAug, starts by calculating semantically appropriate new objects that can be placed into relevant locations in the image (the what and where problems). Then it embeds these objects into their relevant target locations, thereby promoting diversity of object instance distribution. Our method allows for introducing new object instances and categories that may not even exist in the training set. Furthermore, it does not require the additional overhead of training a context network, so it can be easily added to existing architectures. Our comprehensive set of evaluations showed that the proposed method is very effective in improving the generalization, while the overhead is negligible. In particular, for a wide range of model architectures, our method achieved 2–4% and 1–2% mAP improvements for the task of object detection on the Pascal VOC and COCO datasets, respectively. Code is available as supplementary.