Reviewerly/exHarmony huggingface dataset:
Multimodal graph neural network for video procedural captioning

Video procedural captioning aims to generate detailed descriptive captions for all steps in a long instructional video. The peculiarity of this problem is the procedural dependency between the events to generate consistent captions among the video. However, existing video (dense) captioning methods only consider intra-event or sequential inter-event context and are hard to model the non-sequential context dependency between events. In this paper, inspired by the recent success of graph neural networks in capturing the relations for structured data, we propose a novel Multimodal Graph Neural Network (MGNN) for dense video procedural captioning in capturing the procedural structure between events. Specifically, we construct temporal sequential graph and semantic non-sequential graph for a multimodal heterogeneous graph. Moreover, we adopt the graph neural network to enhance the visual and text features, and fuse both features for further caption generation. Extensive experiments demonstrate the proposed MGNN is effective in generating coherent captions on both the Youcook2 and Activitynet Captions benchmark.
Face Image Retrieval with Attribute Manipulation

Current face image retrieval solutions are limited, since they treat different facial attributes the same and cannot incorporate user’s preference for a subset of attributes in their search criteria. This paper introduces a new face image retrieval framework, where the input face query is augmented by both an adjustment vector that specifies the desired modifications to the facial attributes, and a preference vector that assigns different levels of importance to different attributes. For example, a user can ask for retrieving images similar to a query image, but with a different hair color, and no preference for absence/presence of eyeglasses in the results. To achieve this, we propose to disentangle the semantics, corresponding to various attributes, by learning a set of sparse and orthogonal basis vectors in the latent space of StyleGAN. Such basis vectors are then employed to decompose the dissimilarity between face images in terms of dissimilarity between their attributes, assign preference to the attributes, and adjust the attributes in the query. Enforcing sparsity on the basis vectors helps us to disentangle the latent space and adjust each attribute independently from other attributes, while enforcing orthogonality facilitates preference assignment and the dissimilarity decomposition. The effectiveness of our approach is illustrated by achieving state-of-the-art results for the face image retrieval task.
Ontology-based knowledge representation and semantic topic modeling for intelligent trademark legal precedent research

An intelligent methodology and its prototype system are developed for automatically discovering legal precedents using semantic analysis. The concept of the trademark legal precedent recommendation was originated from our TE2020 conference paper. The approach is to identify matching cases related to given seed case with respect to their legal case brief attributes using advanced text mining techniques. In the paper, dynamic topic modeling is further developed to analyze the dataset over three time-sequential cohorts to identify trademark law topics varied over time. Further, the prototype system was demonstrated and verified using real trademark case analysis with satisfactory results.
Adversarial Deep Learning for Indoor Localization With Channel State Information Tensors

Fingerprinting-based indoor localization has been a research focus for GPS denied areas. The development of neural networks has greatly promoted its application in indoor localization systems. However, recent studies showed that the machine learning models, including state-of-the-art neural networks, are vulnerable to adversarial examples, and thus, neural network-based indoor localization systems are also under the threat of adversarial attacks. To investigate the effect of adversarial attacks on indoor localization systems and to make such systems resilient to adversarial attacks, we propose AdvLoc, an adversarial deep learning for indoor localization system. With the proposed AdvLoc system, the effect of adversarial attacks on indoor localization is studied under six types of adversarial attack methods in both black-box attack and white-box attack scenarios. Furthermore, adversarial training is utilized in offline training of the proposed AdvLoc system, which is effective against first-order adversarial attacks. The proposed AdvLoc system is implemented with commodity WiFi devices and evaluated with extensive experiments in two representative indoor environments. The experimental results verify the robustness of the proposed system against first-order adversarial attacks in representative indoor environments.
Precarious Data: Crack, Opioids, and Enacting a Social Justice Ethic in Data Visualization Practice

<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Background:</b> The linguistic framing strategies used in media reporting on illegal drugs have been extensively documented, but less attention has been directed toward visuals, particularly data visualizations. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Literature review:</b> Positioning illegal drug use as a criminal justice problem or a public health issue are types of frameworks that use specific rhetorical strategies. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Research questions:</b> 1. What are the rhetorical strategies used in data visualizations published during the crack and opioid drug epidemics, respectively? 2. Do these strategies advance dominant media narratives that crack addiction should be criminalized but opioid addiction should be treated like a public health issue? And if so, how is this accomplished? <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methodology:</b> Drawing from the media studies approach previously employed in a study in technical and professional communication (TPC) on information design trends, I apply the concept of “scripto-visual” rhetoric to select data visualizations published by mainstream news media during both drug epidemics. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</b> I argue these graphics escalated the perceived threat during both drug epidemics but different scripto-visual rhetorical strategies were used. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusions:</b> Attending to ethical considerations in the creation of data visualizations has long been important in TPC, while scholarship has integrated social justice as a core component of the discipline. In the last section of this article, I bring these themes together by arguing that a social justice ethic is needed in data design work. I then propose a critical heuristic constructed from Jones et al.’s positionality, privilege, and power framework that can be used analytically or as an inventional tool to tease out the ways particular scripto-visual rhetorical decisions may be promoting inequities.
Label-Only Membership Inference Attacks and Defenses in Semantic Segmentation Models

Recent research has discovered that deep learning models are vulnerable to membership inference attacks, which can reveal whether a sample is in the training dataset of the victim model or not. Most membership inference attacks rely on confidence scores from the victim model for the attack purpose. However, a few studies indicate that prediction labels of the victim model's output are sufficient for launching successful attacks. Besides the well-studied classification models, segmentation models are also vulnerable to this type of attack. In this article, for the first time, we propose the label-only membership inference attacks against semantic segmentation models. With a well-designed framework of the attacks, we can achieve a considerably higher successful attacking rate compared to previous work. In addition, we have discussed several possible defense mechanisms to counter such a threat.
Word Sense Disambiguation using Cooperative Game Theory and Fuzzy Hindi WordNet based on ConceptNet

Natural Language is fuzzy in nature. The fuzziness of Hindi language was captured in the Fuzzy Hindi WordNet (FHWN) . FHWN assigned membership values to fuzzy relationships by consulting experts from various domains. However, these membership values need to be corrected. In the proposed work, we compute the membership values of fuzzy semantic relations using ConceptNet. Later, we perform WSD of Hindi text using cooperative game theoretic approach. We used the Shapley Value centrality measure where we predict which coalition of players (word senses) proves to be the most beneficial. We tested and compared our algorithm with the existing state-of-the-art approaches of Hindi on three datasets and results are better on all the three datasets. One more notable aspect is that the results are quite stable even if the fuzzy membership values of fuzzy graphs changes.
Digital IIR Filter Design Using a Differential Evolution Algorithm with Polar Coordinates

A differential evolution algorithm is presented that uses a combination of rectangular and polar coordinates. The algorithm was developed using thirteen commonly used numerical optimization test functions, including a spherical, three hyper-ellipsoid, the sum of different powers, Rastrigin's, Schwefel's, Griewank's, Rosenbrock's valley, Styblinski- Tang, Ackley's Path, Price- Rosenbrock, and Eggholder's functions. The algorithm was then applied to design a digital infinite impulse response (IIR) filter.
Cross Channel Scripting and Code Injection Attacks on Web and Cloud-Based Applications: A Comprehensive Review

Cross channel scripting (XCS) is a common web application vulnerability, which is a variant of a cross-site scripting (XSS) attack. An XCS attack vector can be injected through network protocol and smart devices that have web interfaces such as routers, photo frames, and cameras. In this attack scenario, the network devices allow the web administrator to carry out various functions related to accessing the web content from the server. After the injection of malicious code into web interfaces, XCS attack vectors can be exploited in the client browser. In addition, scripted content can be injected into the networked devices through various protocols, such as network file system, file transfer protocol (FTP), and simple mail transfer protocol. In this paper, various computational techniques deployed at the client and server sides for XCS detection and mitigation are analyzed. Various web application scanners have been discussed along with specific features. Various computational tools and approaches with their respective characteristics are also discussed. Finally, shortcomings and future directions related to the existing computational techniques for XCS are presented.
A Comparative Analysis of Optimizers in Recurrent Neural Networks for Text Classification

The performance of any deep learning model depends heavily on the choice of optimizers and their corresponding hyper-parameters. For any given problem researchers struggle to select the best possible optimizer from a myriad of optimizers proposed in existing literature. Currently the process of optimizer selection in practice is anecdotal at best whereby practitioners either randomly select an optimizer or rely on best practices or online recommendations not grounded on empirical evidence base. In our paper, we delve deep into this problem of picking the right optimizer for text based datasets and linguistic classification problems, by bench-marking ten optimizers on three different RNN models (Bi-GRU, Bi-LSTM and BRNN) on three spam email based benchmark datasets. We analyse the performance of models employing these optimizers using train accuracy, train loss, validation accuracy, validation loss, test accuracy, test loss and RO-AUC score as metrics. The results show that Adaptive Optimization methods (RMSprop, Adam, Adam weight decay and Nadam) with default hyper-parameters outperform other optimizers in all three datasets and RNN model variations.
What Can Knowledge Bring to Machine Learning?—A Survey of Low-shot Learning for Structured Data

Supervised machine learning has several drawbacks that make it difficult to use in many situations. Drawbacks include heavy reliance on massive training data, limited generalizability, and poor expressiveness of high-level semantics. Low-shot Learning attempts to address these drawbacks. Low-shot learning allows the model to obtain good predictive power with very little or no training data, where structured knowledge plays a key role as a high-level semantic representation of human. This article will review the fundamental factors of low-shot learning technologies, with a focus on the operation of structured knowledge under different low-shot conditions. We also introduce other techniques relevant to low-shot learning. Finally, we point out the limitations of low-shot learning, the prospects and gaps of industrial applications, and future research directions.
Freedom of speech: A relational defence

Much of the recent literature on freedom of speech has focused on the arguments for and against the regulation of certain kinds of speech. Discussions of hate speech and offensive speech, for example, abound in this literature, as do debates concerning the permissibility of pornography. Less attention has been paid, however, at least recently, to the normative foundations of freedom of speech where three classic justifications still prevail, based on the values of truth, autonomy and democracy. In this paper we argue, first, that none of these justifications meet all four intuitive desiderata for an adequate theory of free speech. We go on to sketch an original relational view of free speech, one which grounds its value in the recognition that speakers grant each other when engaging in speech practices, and its limits in the republican ideal of non-dominated co-exercisable liberty. We briefly illustrate the relational approach’s implications for the debate on hate speech regulation and for the response to fake news.
Eliminating the high false‐positive rate in defect prediction through <scp>BayesNet</scp> with adjustable weight

Abstract In defect prediction, a high false‐positive rate (FPR) caused by class imbalance not only increases the workload of testing and development but also consumes unnecessary costs. Many defect models against class imbalance have been proposed to improve the accuracy of defect prediction, but their ability to reduce FPR is unclear. To solve these problems, we first proposed a BayesNet with adjustable weights, called WBN, to reduce the FPR in software defect prediction, which is an algorithm independent of data preprocessing techniques. The mechanism of our WBN is to change the sampling probability of the misclassified instances when training the defect model, making the BayesNet model focus more on false alarm instances. And then, we investigate the FPR of five mainstream defect models for solving class imbalance and select them as comparison models to test the validity of our methods. The experimental result on eight open‐source projects shows that a) our WBN, in in‐version defect prediction (IVDP) and cross‐version defect prediction (CVDP), effectively reduces FPR with means of 0.384 and 0.322, respectively; b) compared with improved subclass discriminant analysis (ISDA) that is the lowest FPR in all control models, our WBN not only reduced the FPR but maintained recall whose mean value was 0.797, whereas ISDA did not, with an average recall of only 0.397; c) our WBN, in CVDP, not only reduces FPR, but also has significant superiority over five control defect models and baseline. Besides, we also found that the class imbalance difference between the test set and the training set has an impact on CVDP performance, recommending that practitioners choose the best dataset for CVDP from the defect data of the historical version through special technology.
Neural Network Based Data Estimation for Unique Word OFDM

Model-based methods have been employed for data estimation for several decades. Due to the incredible success of data-driven machine learning methods, efforts have recently been made to utilize neural networks (NNs) for data estimation in general multiple-input multiple-output (MIMO) communication systems. In this paper, NN-based data estimation is conducted for a communication system employing the unique word orthogonal frequency division multiplexing (UW-OFDM) signaling scheme. In particular, we utilize the so-called DetNet, an NN that has been proposed for data estimation in a general MIMO system. However, to achieve satisfying results for data estimation in a UW-OFDM system an appropriate pre-processing of the input data of DetNet has to be introduced. We investigate its bit error ratio performance in indoor frequency selective environments, we conduct a brief complexity analysis, and we highlight its partially peculiar estimation characteristics.
FusionSum: Abstractive summarization with sentence fusion and cooperative reinforcement learning

When summarizing an article, humans are habituated to fuse multiple related sentences to make the summary more concise and coherent. But most of the previous work focuses on the grammaticality of the fusion process and neglects the mechanism behind which sentences should be fused together. And there also lacks an effective training method for bridging the modules in the model to approach a global optimization. In this paper we propose FusionSum, a novel framework that imitates the behaviors of humans in summarization by explicitly modeling the sentence grouping and fusion process. It consists of an entity-aware sentence grouping module to identify salient sentences and combine them into groups, after which a unified sentence fusion module rewrites each group into a summary sentence. Furthermore, we also study the collaboration problem between these two modules and propose cooperative reinforcement learning method, which plays a maximax two-player game to approach global optimization. Automatic evaluation shows that our model significantly outperforms the strong baselines on three prevalent corpora. Human evaluation further demonstrates the summaries generated by our model are more concise and coherent.
Study of Blockchain Application in the Logistics Industry

In this article, a study of blockchain technology application in the logistics industry is presented.The main problems in the logistics area are delayed in delivery, loss of documentation, unknown source of products, errors, among others, which can be minimized and even avoided by implementing this technology.This blockchain technology initially emerged along with Bitcoin and stands out due to its qualities related to privacy, security, and authenticity.Thus, the objective of this work, exploratory and explanatory, was to evaluate the level of knowledge about the tool, the difficulties in the logistics sector and the advantages in supply chain management.This was done through the application of forms and a questionnaire aimed at determining the level of knowledge about blockchain of professionals in the logistics area and the adoption in the organizations where they work.This work aims to conduct a bibliographic study on the benefits of applying blockchain technology, proposing improvements in the information and contracting network in the logistics industry and a quality tracking system.A survey was conducted with a sample of 40 respondents.The results obtained revealed that only 42.5% of respondents had due knowledge concerning this technology, because it is recent in the logistics area.87.5% agree that blockchain technology would solve some problems in the logistics area, however only 17.5% of the companies use the tool.About the training for the new digital era addressed in the survey, only 27.5% described that their companies invest in technology.
Detecting Steganography in JPEG Images Recompressed With the Same Quantization Matrix

JPEG steganalysis aims to detect stego JPEG images. For some robust steganography methods, in order to enhance stego images’ robustness of resisting JPEG recompression from lossy channel such SNS or photo sharing websites, steganographer may intentionally recompress cover image several times with quantization matrix of targeted channel, which thereby make it possible to transmit stego data in such channel for better disguise. In addition, there are huge number of cover JPEG images may be recompressed for various reasons, such as processing by some tools. Thus a better steganalysis method for such images is needed. In this paper, we investigate the steganalysis method for images recompressed with the same quantization matrix, namely, discriminate recompressed JPEG cover images and its stego images. We present some observed phenomenon on recompressed JPEG images, and design methods to enhance the sensitivity of feature based and deep model based steganalysis methods for this task. To verify their effectiveness with different acquisition of recompression prior-knowledge, we conduct experiments in various settings including conventional setting and mixing samples of different recompressing times in training. Their results demonstrate that the proposed method can notably improve detection accuracy on recompressed JPEG images.
LIANet: Layer Interactive Attention Network for RGB-D Salient Object Detection

RGB-D salient object detection (SOD) usually describes two modes’ classification or regression problem, namely RGB and depth. The existing RGB-D SOD methods use depth hints to increase the detection performance, meanwhile they focus on the quality of little depth maps. In practical application, the interference of various problems in the acquisition process affects the depth map quality, which dramatically reduces the detection effect. In this paper, to minimize interference in depth mapping and emphasize prominent objects in RGB images, we put forward a layered interactive attention network (LIANet). The whole network model adopts the idea of double branch structure to integrate RGB information and Depth information. The network has three parts: feature coding, layered fusion mechanism, and feature decoding. In the feature coding stage, a simple attention module (SAM) is added, which defines the energy function considering the weights of channel and space dimensions. This module enables the network to learn more discriminating neurons without adding parameters. By refining these neurons, the high-level semantic features of images can be fully mined. The layered interactive fusion module (LIFM) is the most critical part of this paper. This module effectively enhances the cross-modal interaction between RGB features and depth features. The RGB-depth-RGB modulation feedback mechanism successfully eliminates interference in the depth map and accurately highlights the features of salient objects. In addition, we also used mixed losses to optimize further and train our model. Finally, a mass of experiments on six standard datasets demonstrated the importance of the method, and a timely detection speed reaches 30 fps on every dataset.