Reviewerly/exHarmony huggingface dataset:
SKILL: Structured Knowledge Infusion for Large Language Models

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks.However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from text.In this work, we propose a method to infuse structured knowledge into LLMs, by directly training T5 models on factual triples of knowledge graphs (KGs).We show that models pre-trained on Wikidata KG with our method outperform the T5 baselines on FreebaseQA and WikiHop, as well as the Wikidata-answerable subset of Triv-iaQA and NaturalQuestions.The models pretrained on factual triples compare competitively with the ones on natural language sentences that contain the same knowledge.Trained on a smaller size KG, WikiMovies, we saw 3× improvement of exact match score on MetaQA task compared to T5 baseline.The proposed method has an advantage that no alignment between the knowledge graph and text corpus is required in curating training data.This makes our method particularly useful when working with industry-scale knowledge graphs.
Evaluating time series encoding techniques for Predictive Maintenance

Predictive Maintenance has become an important component in modern industrial scenarios, as a way to minimize down-times and fault rate for different equipment. In this sense, while machine learning and deep learning approaches are promising due to their accurate predictive abilities, their data-heavy requirements make them significantly limited in real world applications. Since one of the main issues to overcome is lack of consistent training data, recent work has explored the possibility of adapting well-known deep-learning models for image recognition, by exploiting techniques to encode time series as images. In this paper, we propose a framework for evaluating some of the best known time series encoding techniques, together with Convolutional Neural Network-based image classifiers applied to predictive maintenance tasks. We conduct an extensive empirical evaluation of these approaches for the failure prediction task on two real-world datasets (PAKDD2020 Alibaba AI OPS Competition and NASA bearings), also comparing their performances with respect to the state-of-the-art approaches. We further discuss advantages and limitation of the exploited models when coupled with proper data augmentation techniques.
An end-to-end framework for information extraction from Italian resumes

Nowadays, recruitment processes are increasingly being automated by intelligent systems which provide best candidates for companies' open positions, and vice versa. However, extracting information from the unstructured documents involved in these processes (e.g. resumes, jobs' descriptions) still represents an open challenge because of their high heterogeneity (in the form and style) and the lack of pre-defined standards between different companies and/or countries. In this paper, we address the resume information extraction problem, focusing on documents within the Italian Labor Market. Specifically, we propose an effective and efficient end-to-end framework capable of providing a complete candidate overview including his personal information, skills and work experiences. Specifically, after having extracted the raw data from the resume documents, the system segments them into semantically consistent parts using linguistics patterns. Each segment is further processed with a NER algorithm, based on pre-trained language models, to extract relevant information which an HR specialist could consult in order to assess the suitability of a candidate for a job offer. We collected (and labeled) a new Italian resume dataset and our results prove the effectiveness of the proposed method, especially considering the great advantages our segmentation strategy brings to the NER performance with respect to standard line-based segmentation approaches. In addition, our system achieves promising performance when combined with modern NLP models.
Sports Field Registration via Keypoints-aware Label Condition

We propose a novel deep learning framework for sports field registration. The typical algorithmic flow for sports field registration involves extracting field-specific features (e.g., corners, lines, etc.) from field image and estimating the homography matrix between a 2D field template and the field image using the extracted features. Unlike previous methods that strive to extract sparse field features from field images with uniform appearance, we tackle the problem differently. First, we use a grid of uniformly distributed keypoints as our field-specific features to increase the likelihood of having sufficient field features under various camera poses. Then we formulate the keypoints detection problem as an instance segmentation with dynamic filter learning. In our model, the convolution filters are generated dynamically, conditioned on the field image and associated keypoint identity, thus improving the robustness of prediction results. To extensively evaluate our method, we introduce a new soccer dataset, called TS-WorldCup, with detailed field markings on 3812 time-sequence images from 43 videos of Soccer World Cup 2014 and 2018. The experimental results demonstrate that our method outperforms state-of-the-arts on the TS-WorldCup dataset in both quantitative and qualitative evaluations. Both the code and dataset are available online <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Multi-Camera Vehicle Tracking System for AI City Challenge 2022

Multi-Target Multi-Camera tracking is a fundamental task for intelligent traffic systems. The track 1 of AI City Challenge 2022 aims at the city-scale multi-camera vehicle tracking task. In this paper we propose an accurate vehicle tracking system composed of 4 parts, including: (1) State-of-the-art detection and re-identification models for vehicle detection and feature extraction. (2) Single camera tracking, where we introduce augmented tracks prediction and multi-level association method on top of tracking-by-detection paradigm.(3) Zone-based singe-camera track-let merging strategy. (4) Multi-camera spatial-temporal matching and clustering strategy. The proposed system achieves promising results and ranks the second place in Track 1 of the AI City Challenge 2022 with a IDF1 score of 0.8437.
Mapping Climate Parameters over the Territory of Botswana Using GMT and Gridded Surface Data from TerraClimate

This articles presents a new series of maps showing the climate and environmental variability of Botswana. Situated in southern Africa, Botswana has an arid to semi-arid climate, which significantly varies in its different regions: Kalahari Desert, Makgadikgadi Pan and Okavango Delta. While desert regions are prone to droughts and periods of extreme heat during the summer months, other regions experience heavy downpours, as well as episodic and unpredictable rains that affect agricultural activities. Such climatic variations affect social and economic aspects of life in Botswana. This study aimed to visualise the non-linear correlations between the topography and climate setting at the country’s scale. Variables included T °C min, T °C max, precipitation, soil moisture, evapotranspiration (PET and AET), downward surface shortwave radiation, vapour pressure and vapour pressure deficit (VPD), wind speed and Palmer Drought Severity Index (PDSI). The dataset was taken from the TerraClimate source and GEBCO for topographic mapping. The mapping approach included the use of Generic Mapping Tools (GMT), a console-based scripting toolset, which enables the use of a scripting method of automated mapping. Several GMT modules were used to derive a set of climate parameters for Botswana. The data were supplemented with the adjusted cartographic elements and inspected by the Geospatial Data Abstraction Library (GDAL). The PDSI in Botswana in 2018 shows stepwise variation with seven areas of drought: (1) −3.7 to −2.2. (extreme); (2) −2.2 to −0.8 (strong, southern Kalahari); (3) −0.8 to 0.7 (significant, central Kalahari; (4) 0.7 to 2.1 (moderate); (5) 2.1 to 3.5 (lesser); (6) 3.5 to 4.9 (low); (7) 4.9 to 6.4 (least). The VPD has a general trend towards the south-western region (Kalahari Desert, up to 3.3), while it is lower in the north-eastern region of Botswana (up to 1.4). Other values vary respectively, as demonstrated in the presented 12 maps of climate and environmental inventory in Botswana.
Adversarial attacks and defenses in deep learning for image recognition: A survey

In recent years, researches on adversarial attacks and defense mechanisms have obtained much attention. It’s observed that adversarial examples crafted with small malicious perturbations would mislead the deep neural network (DNN) model to output wrong prediction results. These small perturbations are imperceptible to humans. The existence of adversarial examples poses great threat to the robustness of DNN-based models. It is necessary to study the principles behind it and develop their countermeasures. This paper surveys and summarizes the recent advances in attack and defense methods extensively and in detail, analyzes and compares the pros and cons of various attack and defense schemes. Finally we discuss the main challenges and future research directions in this field.
Two-stream transformer network for sensor-based human activity recognition

Human Activity Recognition (HAR) based on wearable devices has always been a hot topic in health applications, human-object interaction, and smart homes. Despite significant improvements achieved by convolutional neural networks, long short-term memory networks, transformer networks and various hybrid models, there are still two fundamental issues. First, the spatial–temporal dependencies of sensor signals are difficult to be effectively modeled. Second, in multimodal environments, sensors placed on various body positions contribute distinctly to the classification results. In this work, we propose a self-attention based Two-stream Transformer Network (TTN). In view of the former issue, we use two streams, named temporal stream and spatial stream, respectively, to extract the readings-over-time and time-over-readings features from sensor signals. These features extracted from two streams are complementary since the time-over-readings features are able to express additional information which cannot be captured from sensor signals directly. To deal with the latter issue, we assign attention weights to each sensor axis in the spatial channel based on their classification scores. It makes sense that different axis-readings with distinct recognition contributions caused by data heterogeneity be treated unequally. Extensive experiments on four available benchmark datasets (PAMAP2, Opportunity, USC–HAD, and Skoda) reveal that our proposed model is better suited for multimodal HAR than previous state-of-the-art methods.
Students' perspectives of digital transformation of higher education in Bosnia and Herzegovina

Abstract Since the COVID‐19 pandemic prompted numerous global changes, educational institutions in Bosnia and Herzegovina were forced to adapt their educational approaches, with schools and universities implementing alternative teaching and learning practices. Thus, this study aimed to investigate students' perspectives on the digital transformation of higher education in Bosnia and Herzegovina, including their preferences for hybrid, online, or face‐to‐face teaching models, their internet habits and readiness for e‐learning, their attitudes and satisfaction with online learning, and their computer anxiety. The research sample consisted of 330 students studying in a variety of fields and academic years at both public and private universities. The data was gathered via an online questionnaire. A multinomial logistic regression analysis revealed a significant effect of students' field of study, year of study, and university status on their preferences for hybrid, online, and face‐to‐face classes. However, the MANOVA test revealed that the aforementioned factors have an insignificant effect on students' internet habits and readiness for e‐learning, attitudes and satisfaction with online learning, and computer anxiety, all of which have an insignificant impact on student's academic achievement as determined by standard multiple regression analyses. Additionally, multivariate analysis revealed that students' preferences for hybrid, online, and face‐to‐face classes have a significant influence on their internet habits and readiness for e‐learning, attitudes, and satisfaction with online learning, and computer anxiety. The findings of this study may aid in a better understanding of tertiary education's digital transformation and the improvement of educational policy, curricula, and instructional and learning strategies.
Deep Learning for Clothing Style Recognition Using YOLOv5

With the rapid development of artificial intelligence, much more attention has been paid to deep learning. However, as the complexity of learning algorithms increases, the needs of computation power of hardware facilities become more crucial. Instead of the focus being on computing devices like GPU computers, a lightweight learning algorithm could be the answer for this problem. Cross-domain applications of deep learning have attracted great interest amongst researchers in academia and industries. For beginners who do not have enough support with software and hardware, an open-source development environment is very helpful. In this paper, a relatively lightweight algorithm YOLOv5s is addressed, and the Google Colab is used for model training and testing. Based on the developed environment, many state-of-art learning algorithms can be studied for performance comparisons. To highlight the benefits of one-stage object detection algorithms, the recognition of clothing styles is investigated. The image samples are selected from datasets of fashion clothes and the web crawling of online stores. The image data are categorized into five groups: plaid; plain; block; horizontal; and vertical. Average precison, mean average precison, recall, F1-score, model size, and frame per second are the metrics used for performance validations. From the experimental outcomes, it shows that YOLOv5s is better than other learning algorithms in the recognition accuracy and detection speed.
Protecting Fiber-Optic Quantum Key Distribution Sources against Light-Injection Attacks

A well-protected and characterized source in a quantum key distribution system is needed for its security. Unfortunately, the source is vulnerable to light-injection attacks, such as Trojan-horse, laser-seeding, and laser-damage attacks, in which an eavesdropper actively injects bright light to hack the source unit. The hacking laser could be a high-power one that can modify properties of components via the laser-damage attack and also further help the Trojan-horse and other light-injection attacks. Here we propose a countermeasure against the light-injection attacks, consisting of an additional sacrificial component placed at the exit of the source. This component should either withstand high-power incoming light while attenuating it to a safe level that cannot modify the rest of the source, or get destroyed into a permanent high-attenuation state that breaks up the line. We demonstrate experimentally that off-the-shelf fiber-optic isolators and circulators have these desired properties, at least under attack by a continuous-wave high-power laser.3 MoreReceived 3 February 2022Revised 9 July 2022Accepted 29 August 2022DOI:https://doi.org/10.1103/PRXQuantum.3.040307Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum communicationQuantum cryptographyQuantum Information
Artificial Intelligence in Breast Cancer Screening and Diagnosis

Cancer is a disease that continues to plague our modern society. Among all types of cancer, breast cancer is now the most common type of cancer occurring in women worldwide. Various factors, including genetics, lifestyle, and the environment, have contributed to the rise in the prevalence of breast cancer among women of all socioeconomic strata. Therefore, proper screening for early diagnosis and treatment becomes a major factor when fighting the disease. Artificial intelligence (AI) continues to revolutionize various spheres of our lives with its numerous applications. Using AI in the existing screening process makes obtaining results even easier and more convenient. Faster, more accurate results are some of the benefits of AI methods in breast cancer screening. Nonetheless, there are many challenges in the process of the integration of AI that needs to be addressed systematically. The following is a review of the application of AI in breast cancer screening.
Contextual counters and multimodal Deep Learning for activity-level traffic classification of mobile communication apps during COVID-19 pandemic

The COVID-19 pandemic has reshaped Internet traffic due to the huge modifications imposed to lifestyle of people resorting more and more to collaboration and communication apps to accomplish daily tasks. Accordingly, these dramatic changes call for novel traffic management solutions to adequately countermeasure such unexpected and massive changes in traffic characteristics. In this paper, we focus on communication and collaboration apps whose traffic experienced a sudden growth during the last two years. Specifically, we consider nine apps whose traffic we collect, reliably label, and publicly release as a new dataset (MIRAGE-COVID-CCMA-2022) to the scientific community. First, we investigate the capability of state-of-art single-modal and multimodal Deep Learning-based classifiers in telling the specific app, the activity performed by the user, or both. While we highlight that state-of-art solutions reports a more-than-satisfactory performance in addressing app classification (96%-98% F-measure), evident shortcomings stem out when tackling activity classification (56%-65% F-measure) when using approaches that leverage the transport-layer payload and/or per-packet information attainable from the initial part of the biflows. In line with these limitations, we design a novel set of inputs (namely Context Inputs) providing clues about the nature of a biflow by observing the biflows coexisting simultaneously. Based on these considerations, we propose Mimetic-All a novel early traffic classification multimodal solution that leverages Context Inputs as an additional modality, achieving ≥82% F-measure in activity classification. Also, capitalizing the multimodal nature of Mimetic-All, we evaluate different combinations of the inputs. Interestingly, experimental results witness that Mimetic-ConSeq-a variant that uses the Context Inputs but does not rely on payload information (thus gaining greater robustness to more opaque encryption sub-layers possibly going to be adopted in the future)-experiences only ≈1% F-measure drop in performance w.r.t. Mimetic-All and results in a shorter training time.
Face-mask-aware Facial Expression Recognition based on Face Parsing and Vision Transformer

As wearing face masks is becoming an embedded practice due to the COVID-19 pandemic, facial expression recognition (FER) that takes face masks into account is now a problem that needs to be solved. In this paper, we propose a face parsing and vision Transformer-based method to improve the accuracy of face-mask-aware FER. First, in order to improve the precision of distinguishing the unobstructed facial region as well as those parts of the face covered by a mask, we re-train a face-mask-aware face parsing model, based on the existing face parsing dataset automatically relabeled with a face mask and pixel label. Second, we propose a vision Transformer with a cross attention mechanism-based FER classifier, capable of taking both occluded and non-occluded facial regions into account and reweigh these two parts automatically to get the best facial expression recognition performance. The proposed method outperforms existing state-of-the-art face-mask-aware FER methods, as well as other occlusion-aware FER methods, on two datasets that contain three kinds of emotions (M-LFW-FER and M-KDDI-FER datasets) and two datasets that contain seven kinds of emotions (M-FER-2013 and M-CK+ datasets).
The Risk-Based Approach of the European Union’s Proposed Artificial Intelligence Regulation: Some Comments from a Tort Law Perspective

Abstract How can tort law contribute to a better understanding of the risk-based approach in the European Union’s (EU) Artificial Intelligence Act proposal and evolving liability regime? In a new legal area of intense development, it is pivotal to make the best use possible of existing regulation and legal knowledge. The main objective of this article is thus to investigate the relationship between traditional tort law principles, with a focus on risk assessments, and the developing legislation on artificial intelligence (AI) in the EU. The article offers a critical analysis and evaluation from a tort law perspective of the risk-based approach in the proposed AI Act and the European Parliament resolution on a civil liability regime for AI, with comparisons also to the proposal for a revised and AI-adapted product liability directive and the recently proposed directive on civil liability for AI. The discussion leads to the illumination of both challenges and possibilities in the interplay between AI, tort law and the concept of risk, displaying the large potential of tort law as a tool for handling rising AI issues.
Flow-Guided Transformer for Video Inpainting

We propose a flow-guided transformer, which innovatively leverage the motion discrepancy exposed by optical flows to instruct the attention retrieval in transformer for high fidelity video inpainting. More specially, we design a novel flow completion network to complete the corrupted flows by exploiting the relevant flow features in a local temporal window. With the completed flows, we propagate the content across video frames, and adopt the flow-guided transformer to synthesize the rest corrupted regions. We decouple transformers along temporal and spatial dimension, so that we can easily integrate the locally relevant completed flows to instruct spatial attention only. Furthermore, we design a flow-reweight module to precisely control the impact of completed flows on each spatial transformer. For the sake of efficiency, we introduce window partition strategy to both spatial and temporal transformers. Especially in spatial transformer, we design a dual perspective spatial MHSA, which integrates the global tokens to the window-based attention. Extensive experiments demonstrate the effectiveness of the proposed method qualitatively and quantitatively. Codes are available at https://github.com/hitachinsk/FGT .