Reviewerly/exHarmony huggingface dataset:
Discovering Human Interactions With Novel Objects via Zero-Shot Learning

We aim to detect human interactions with novel objects through zero-shot learning. Different from previous works, we allow unseen object categories by using its semantic word embedding. To do so, we design a human-object region proposal network specifically for the human-object interaction detection task. The core idea is to leverage human visual clues to localize objects which are interacting with humans. We show that our proposed model can outperform existing methods on detecting interacting objects, and generalize well to novel objects. To recognize objects from unseen categories, we devise a zero-shot classification module upon the classifier of seen categories. It utilizes the classifier logits for seen categories to estimate a vector in the semantic space, and then performs nearest search to find the closest unseen category. We validate our method on V-COCO and HICO-DET datasets, and obtain superior results on detecting human interactions with both seen and unseen objects.
Data journals: incentivizing data access and documentation within the scholarly communication system

Data journals provide strong incentives for data creators to verify, document and disseminate their data. They also bring data access and documentation into the mainstream of scholarly communication, rewarding data creators through existing mechanisms of peer-reviewed publication and citation tracking. These same advantages are not generally associated with data repositories, or with conventional journals’ data-sharing mandates. This article describes the unique advantages of data journals. It also examines the data journal landscape, presenting the characteristics of 13 data journals in the fields of biology, environmental science, chemistry, medicine and health sciences. These journals vary considerably in size, scope, publisher characteristics, length of data reports, data hosting policies, time from submission to first decision, article processing charges, bibliographic index coverage and citation impact. They are similar, however, in their peer review criteria, their open access license terms and the characteristics of their editorial boards.
Spatiotemporal‐based sentiment analysis on tweets for risk assessment of event using deep learning approach

Summary Social media plays a vital role in analyzing the actual emotions of people after and during a disaster. Sentiment analysis is a method to detect a pattern from the emotions and feedback of the user. The main objective of the proposed work is to perform sentiment analysis on the tweets on a specific disaster context for a particular location at different intervals of time. LSTM network with word embedding algorithm is used to derive keywords based on the history of tweets and the context of the tweets. The proposed algorithm risk assessment sentiment analysis (RASA) uses the keywords generated from the network to classify the tweets and sentiment score for each location is identified. The model is validated with various state‐of‐art algorithms, namely, support vector machine, Naive‐Bayes, maximum entropy, logistic regression, random forest, XGBoost, stochastic gradient descent, and convolution neural networks in 2‐fold scenario: one for binary class and the other multiclass with three target classes. The results infer that the proposed RASA performs better in a binary class scenario with an increase of 1% when compared with XGBoost and 30% in multiclass scenario on an average when compared with all the other techniques. The model helps the government to take preventive measures to manage the posteffect of the disaster event in a location.
Macaw: An Extensible Conversational Information Seeking Platform

Conversational information seeking (CIS) has been recognized as a major emerging research area in information retrieval. Such research will require data and tools, to allow the implementation and study of conversational systems. This paper introduces Macaw, an open-source framework with a modular architecture for CIS research. Macaw supports multi-turn, multi-modal, and mixed-initiative interactions, and enables research for tasks such as document retrieval, question answering, recommendation, and structured data exploration. It has a modular design to encourage the study of new CIS algorithms, which can be evaluated in batch mode. It can also integrate with a user interface, which allows user studies and data collection in an interactive mode, where the back end can be fully algorithmic or a wizard of oz setup. Macaw is distributed under the MIT License.
Transmission dynamics: Data sharing in the <scp>COVID</scp>‐19 era

The current coronavirus disease 2019 (COVID-19) pandemic underscores the need for building and sustaining public health data infrastructure to support a rapid local, regional, national, and international response. Despite a historical context of public health crises, data sharing agreements and transactional standards do not uniformly exist between institutions which hamper a foundational infrastructure to meet data sharing and integration needs for the advancement of public health.There is a growing need to apply population health knowledge with technological solutions to data transfer, integration, and reasoning, to improve health in a broader learning health system ecosystem. To achieve this, data must be combined from healthcare provider organizations, public health departments, and other settings. Public health entities are in a unique position to consume these data, however, most do not yet have the infrastructure required to integrate data sources and apply computable knowledge to combat this pandemic.Herein, we describe lessons learned and a framework to address these needs, which focus on: (a) identifying and filling technology "gaps"; (b) pursuing collaborative design of data sharing requirements and transmission mechanisms; (c) facilitating cross-domain discussions involving legal and research compliance; and (d) establishing or participating in multi-institutional convening or coordinating activities.While by no means a comprehensive evaluation of such issues, we envision that many of our experiences are universal. We hope those elucidated can serve as the catalyst for a robust community-wide dialogue on what steps can and should be taken to ensure that our regional and national health care systems can truly learn, in a rapid manner, so as to respond to this and future emergent public health crises.
What Machines See Is Not What They Get: Fooling Scene Text Recognition Models With Adversarial Text Images

The research on scene text recognition (STR) has made remarkable progress in recent years with the development of deep neural networks (DNNs). Recent studies on adversarial attack have verified that a DNN model designed for non-sequential tasks (e.g., classification, segmentation and retrieval) can be easily fooled by adversarial examples. Actually, STR is an application highly related to security issues. However, there are few studies considering the safety and reliability of STR models that make sequential prediction. In this paper, we make the first attempt in attacking the state-of-the-art DNN-based STR models. Specifically, we propose a novel and efficient optimization-based method that can be naturally integrated to different sequential prediction schemes, i.e., connectionist temporal classification (CTC) and attention mechanism. We apply our proposed method to five state-of-the-art STR models with both targeted and untargeted attack modes, the comprehensive results on 7 real-world datasets and 2 synthetic datasets consistently show the vulnerability of these STR models with a significant performance drop. Finally, we also test our attack method on a real-world STR engine of Baidu OCR, which demonstrates the practical potentials of our method.
Revealing the Character of Nodes in a Blockchain With Supervised Learning

The term blockchain has its roots in cryptocurrencies. However, its applications are now more widespread, and in many areas, this technology has become the foundation of the distributed ledger. The blockchain protocol assumes that all the participants of the system are both contributors and safeguards of this ledger, since the lack of a trusted third party requires other security precautions in order to maintain the consistency of transactions. In this work, we investigate whether for the participants of a blockchain-based system that does not require revealing the character explicitly, it can be discovered by other means. In order to verify this, we built and publicly released a dataset of nearly 9,000 addresses of nodes in the most popular cryptocurrency - Bitcoin, and then labelled them. These labels represent the character the nodes have in the network, e.g. miners or exchanges. We then developed a set of features that quantify the behaviour of nodes in the network and used supervised machine learning algorithms to find out whether the character of nodes can be revealed based on these features. Our results demonstrate, due to the F-score reaching over 95% in the best-performing algorithms, that it is hard to hide the role the node has in a blockchain-based network. These results indicate that to build trustworthy blockchain-based systems that fully comply with original blockchain assumptions, specific countermeasures are needed in order to preserve the desired level of anonymity.
A Novel False Alarm Suppression Method for CNN-Based SAR Ship Detector

Synthetic aperture radar (SAR) ship detection is an important part of remote sensing applications. With the development of computer vision, SAR ship detection methods based on convolutional neural network (CNN) can directly perform end-to-end detection of near-shore ship targets. However, CNN-based methods are prone to generate false targets on land areas, especially when using a rotatable bounding box (RBox) for detection. Therefore, how to reduce the false alarm rate becomes a key direction in research for SAR ship detection. In this letter, the problem of negative sample intraclass imbalance in the training stage of CNN-based detection methods is pointed out for the first time, which is considered to be an important reason for the excessive false alarm rate in the land area. Then, a method is proposed to reduce the false targets generated in the land area by CNN-based detection methods. First, an RBox-based model is proposed as the basic architecture for detection. Then, a new loss function is adopted to guide the model to balance the loss contribution of different negative samples during the training stage. The experimental results prove that the proposed method can effectively reduce the false alarm rate of the model and boost the performance of CNN-based detection methods.
RankMI: A Mutual Information Maximizing Ranking Loss

We introduce an information-theoretic loss function, RankMI, and an associated training algorithm for deep representation learning for image retrieval. Our proposed framework consists of alternating updates to a network that estimates the divergence between distance distributions of matching and non-matching pairs of learned embeddings, and an embedding network that maximizes this estimate via sampled negatives. In addition, under this information-theoretic lens we draw connections between RankMI and commonly-used ranking losses, e.g., triplet loss. We extensively evaluate RankMI on several standard image retrieval datasets, namely, CUB-200-2011, CARS-196, and Stanford Online Products. Our method achieves competitive results or significant improvements over previous reported results on all datasets.
How Furiously Can Colorless Green Ideas Sleep? Sentence Acceptability in Context

We study the influence of context on sentence acceptability. First we compare the acceptability ratings of sentences judged in isolation, with a relevant context, and with an irrelevant context. Our results show that context induces a cognitive load for humans, which compresses the distribution of ratings. Moreover, in relevant contexts we observe a discourse coherence effect that uniformly raises acceptability. Next, we test unidirectional and bidirectional language models in their ability to predict acceptability ratings. The bidirectional models show very promising results, with the best model achieving a new state-of-the-art for unsupervised acceptability prediction. The two sets of experiments provide insights into the cognitive aspects of sentence processing and central issues in the computational modeling of text and discourse.
In object detection deep learning methods, YOLO shows supremum to Mask R-CNN

Abstract Deep learning concept and algorithm play a pivotal role in solving various complicated problems such as playing games, forecasting economic future values, detecting objects in images. It could break through the bottle neck in conventional methods of neural networks and artificial intelligence. This paper will compare two influential deep learning algorithms in image processing and object detection, that is, Mask R-CNN and YOLO. Today, detection tasks become more complex when they come to numerous variations in the humans’ perceived appearance, formation, attire, reasoning and the dynamic nature of their behaviour. It is also a challenging task to understand subtle details in their surroundings. For instance, radiance conditions, background clutter and partial or full occlusion. When a machine tries to interact with human or try to take pictures, it becomes hard for them to magnify the details of a human surrounding. In this study we have focused to detect humans effectively. The main objective of the present work is to compare the performance of YOLO and Mask R-CNN, which unveils the inability of Mask R-CNN in detecting tiny human figures among other prominent human images, and illustrate YOLO was successful in detecting most of the human figures in an image with higher accuracy. Therefore, the paper evaluates and differentiates the performance of YOLO from the deep learning method Mask R-CNN in two points, (1) detection ability and (2) computation time. Since, the machine learning algorithms are mostly data specific, the authors believe that the presented results might vary with the varying nature of the data under observation. In another way, the presented data might be seen as a counter example of unveiling the detection inaccuracy of the Mask R-CNN.
Amazon SageMaker Autopilot

We present Amazon SageMaker Autopilot: a fully managed system that provides an automatic machine learning solution. Given a tabular dataset and the target column name, Autopilot identifies the problem type, analyzes the data and produces a diverse set of complete ML pipelines, which are tuned to generate a leaderboard of candidate models that the customer can choose from. The diversity allows users to balance between different needs such as model accuracy vs. latency. By exposing not only the final models but the way they are trained, meaning the pipelines, we allow to customize the generated training pipeline, thus catering the need of users with different levels of expertise. This trait is crucial for users and is the main novelty of Autopilot; it provides a solution that on one hand is not fully black-box and can be further worked on, while on the other hand is not a do it yourself solution, requiring expertise in all aspects of machine learning. This paper describes the different components in the eco-system of Autopilot, emphasizing the infrastructure choices that allow scalability, high quality models, editable ML pipelines, consumption of artifacts of offline meta-learning, and a convenient integration with the entire SageMaker system allowing these trained models to be used in a production setting.
Effect of Organic Matter Content on the Spectral Signature of Iron Oxides across the VIS–NIR Spectral Region in Artificial Mixtures: An Example from a Red Soil from Israel

The investigation of iron oxides in soil using spectral reflectance is very common. Their spectral signal is significant across the visible–near infrared (VIS–NIR) spectral range (400–1000 nm). However, this range overlaps with other soil chromophores, such as those for water and soil organic matter (SOM). This study aimed to investigate the effect of different SOM species on red soil from Israel, which is rich in hematite iron oxide, under air-dried conditions. We constructed datasets of artificially mixed soil and organic matter (OM) with different percentages of added compost from two sources (referred to as A2 and A5). Eighty subsamples of mixed soil–OM were prepared for each of the OM (compost) types. To investigate the effect of OM on the strong iron-oxide absorbance at 880 nm, we generated two indices: CRDC, the absorbance spectral depth change at 880 nm after continuous removal, and NRIR, the normalized red index ratio using 880 and 780 nm wavelengths. The different OM types influenced the soil reflectance differently. At low %SOM, up to 1.5%, the OM types behaved more similarly, but as the OM content increased, their effect on the iron-oxide signal was greater, enhancing the significant differences between the two OM sources. Moreover, as the SOM content increased, the iron-oxide signal decreased until it was completely masked out from the reflectance spectrum. The masking point was observed at different SOM contents: 4% for A5 and 8% for A2. A mechanism that explains the indirect chromophore activity of SOM in the visible region, which is related to the iron-oxide spectral features, was provided. We also compared the use of synthetic linear-mixing practices (soil–OM) to the authentic mixed samples. The synthetic mixture could not imitate the authentic soil reflectance status, especially across the overlapping spectral position of the iron oxides and OM, and hence may hinder real conditions.
Location-based Sentiment Analyses and Visualization of Twitter Election Data

In this article, we perform sentiment analyses of Twitter location data. We use two case studies: US presidential elections of 2016 and UK general elections of 2017. For US elections, we plot state-wise user sentiment towards Hillary Clinton and Donald Trump. For UK elections, we download two disparate datasets, using keywords and location coordinates, looking for similar tendencies in sentiment towards political candidates and parties. The overall objective of the two case studies is to evaluate similarity between sentiment of location-based tweets and on-ground public opinion reflected in election results. We discover Twitter location sentiment does indeed corroborate with the election result in both cases. We also discover similar tendencies in Twitter sentiment towards political candidates and parties regardless of the methodology adopted for data collection.
AI Surveillance during Pandemics: Ethical Implementation Imperatives

Abstract Artificial intelligence surveillance can be used to diagnose individual cases, track the spread of Covid‐19, and help provide care. The use of AI for surveillance purposes (such as detecting new Covid‐19 cases and gathering data from healthy and ill individuals) in a pandemic raises multiple concerns ranging from privacy to discrimination to access to care. Luckily, there exist several frameworks that can help guide stakeholders, especially physicians but also AI developers and public health officials, as they navigate these treacherous shoals. While these frameworks were not explicitly designed for AI surveillance during a pandemic, they can be adapted to help address concerns regarding privacy, human rights, and due process and equality. In a time where the rapid implementation of all tools available is critical to ending a pandemic, physicians, public health officials, and technology companies should understand the criteria for the ethical implementation of AI surveillance .
Hierarchical Multi-Blockchain Architecture for Scalable Internet of Things Environment

Many researchers challenge the possibility of using blockchain and smart contracts to disrupt the Internet of Things (IoT) architecture because of their security and decentralization guarantees. However, the state-of-the-art blockchain architecture is not scalable enough to satisfy the requirements of massive data traffics in the IoT environment. The main reason for this issue is one needs to choose the consensus trade-off between either coping with a high throughput or a high number of nodes. Consequently, this issue prevents the applicability of blockchain for IoT use cases. In this paper, we propose a scalable two-tiered hierarchical blockchain architecture for IoT. The first tier is a Core Engine, which is based on a Practical Byzantine Fault Tolerance (PBFT) consensus to cope with a high throughput, that supervises the underlying subordinate engines (sub-engines) as its second tier. This second tier comprises of the Payment, Compute, and Storage Engine, respectively. We can deploy multiple instances of these sub-engines as many as we need and as local as possible near to the IoT domains, where IoT devices reside, to cope with a high number of nodes. Furthermore, to further extend the scalability of the proposed architecture, we also provide additional scalability features on the Core Engine such as request aggregation, request prioritization, as well as sub-engine parallelism. We implement all of our engines and expose them to IoT applications through the Engine APIs. With these APIs, developers can build and run IoT applications in our architecture. Our evaluation results show that our proposed features on the Core Engine can indeed enhance the overall performance of our architecture. Moreover, based on our proof-of-concept IoT car rental application, we also show that the interoperability between sub-engines through the Core Engine is possible, even when the particular sub-engine is under sub-engine parallelism.
On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms

We study the convergence of a class of gradient-based Model-Agnostic Meta-Learning (MAML) methods and characterize their overall complexity as well as their best achievable accuracy in terms of gradient norm for nonconvex loss functions. We start with the MAML method and its first-order approximation (FO-MAML) and highlight the challenges that emerge in their analysis. By overcoming these challenges not only we provide the first theoretical guarantees for MAML and FO-MAML in nonconvex settings, but also we answer some of the unanswered questions for the implementation of these algorithms including how to choose their learning rate and the batch size for both tasks and datasets corresponding to tasks. In particular, we show that MAML can find an $\epsilon$-first-order stationary point ($\epsilon$-FOSP) for any positive $\epsilon$ after at most $\mathcal{O}(1/\epsilon^2)$ iterations at the expense of requiring second-order information. We also show that FO-MAML which ignores the second-order information required in the update of MAML cannot achieve any small desired level of accuracy, i.e., FO-MAML cannot find an $\epsilon$-FOSP for any $\epsilon>0$. We further propose a new variant of the MAML algorithm called Hessian-free MAML which preserves all theoretical guarantees of MAML, without requiring access to second-order information.
Improved Differential-Linear Attacks with Applications to ARX Ciphers

We present several improvements to the framework of differential-linear attacks with a special focus on ARX ciphers. As a demonstration of their impact, we apply them to Chaskey and ChaCha and we are able to significantly improve upon the best attacks published so far.