Reviewerly/exHarmony huggingface dataset:
WangchanBERTa: Pretraining transformer-based Thai Language Models

Transformer-based language models, more specifically BERT-based architectures have achieved state-of-the-art performance in many downstream tasks. However, for a relatively low-resource language such as Thai, the choices of models are limited to training a BERT-based model based on a much smaller dataset or finetuning multi-lingual models, both of which yield suboptimal downstream performance. Moreover, large-scale multi-lingual pretraining does not take into account language-specific features for Thai. To overcome these limitations, we pretrain a language model based on RoBERTa-base architecture on a large, deduplicated, cleaned training set (78GB in total size), curated from diverse domains of social media posts, news articles and other publicly available datasets. We apply text processing rules that are specific to Thai most importantly preserving spaces, which are important chunk and sentence boundaries in Thai before subword tokenization. We also experiment with word-level, syllable-level and SentencePiece tokenization with a smaller dataset to explore the effects on tokenization on downstream performance. Our model wangchanberta-base-att-spm-uncased trained on the 78.5GB dataset outperforms strong baselines (NBSVM, CRF and ULMFit) and multi-lingual models (XLMR and mBERT) on both sequence classification and token classification tasks in human-annotated, mono-lingual contexts.
Fast Convergence of DETR with Spatially Modulated Co-Attention

The recently proposed Detection Transformer (DETR) model successfully applies Transformer to objects detection and achieves comparable performance with two-stage object detection frameworks, such as Faster-RCNN. However, DETR suffers from its slow convergence. Training DETR from scratch needs 500 epochs to achieve a high accuracy. To accelerate its convergence, we propose a simple yet effective scheme for improving the DETR framework, namely Spatially Modulated Co-Attention (SMCA) mechanism. The core idea of SMCA is to conduct location-aware co-attention in DETR by constraining co-attention responses to be high near initially estimated bounding box locations. Our proposed SMCA increases DETR's convergence speed by replacing the original co-attention mechanism in the decoder while keeping other operations in DETR unchanged. Furthermore, by integrating multi-head and scale-selection attention designs into SMCA, our fully-fledged SMCA can achieve better performance compared to DETR with a dilated convolution-based backbone (45.6 mAP at 108 epochs vs. 43.3 mAP at 500 epochs). We perform extensive ablation studies on COCO dataset to validate SMCA. Code is released at https://github.com/gaopengcuhk/SMCA-DETR .
Surveillance Capitalism as a Form of Biopower in Historical Perspective

Information has become the most important commodity of markets since the Oil Crisis of 1973. This period is also called as New Economy in which industry lost its previous value while services using the new information technologies are put in the core of economic activities. The rise of information technologies has brought forth new ways to generate profits. IT companies have directed their marketing interests to generate profits in behavioral forecasting models through which future behaviors of individuals can be predicted by means of machine intelligence. Accordingly, since the capital accumulation of the new stage is based on the surveillance of individuals, Shoshana Zuboff called this new stage as “surveillance capitalism”. This shift in the market interests can be understood within its historical context in which the management strategies were already being shifted from a sociological bureaucracy to new Taylorist scientific management based on continual surveillance. Thus, surveillance capitalism can be interpreted as a new “technology of the self” aiming at subordination of individuals through psychological manipulations within neoliberal governmentality. This article aims to put forth the historical link between the emergence of surveillance capitalism and the use of individualized management strategies as a new form of capitalistic biopower.
Penetration Testing in Accordance with NIST SP 800-115 Standard

Relevance. Security issues of information systems in critical infrastructure objects become important now. However, current tasks of information security audit of critical infrastructure objects are mainly limited to checking them for compliance with requirements of standards and documents. With this approach to the audit, security of these objects from real attacks by hackers remains unclear. Therefore, objects are subjected to a testing procedure, namely, penetration testing, in order to objectively verify their security. For example, there are instructions of the Bank of Russia to carry out such testing when the information security of banking systems are checked. However, there is no formal national standard for conducting penetration testing in Russia. This is the deterrent factor to testing critical infrastructure objects. The goal of the paper is to analysis of the American testing standard – NIST SP 800-115 to estimate the possibility of its used for development of the Russian national penetration testing standard. Research methods. Methods of analysis and decomposition from the theory of system analysis are used in the paper to achieve the research goal. Results. In-depth analysis of the NIST SP 800-115 standard is provided in the paper. The following are considered: types of information security assessment measures; stages of information security assessment; methods of analysis and testing which used in the assessment of information security; types and sequence of penetration testing; tested vulnerabilities; recommended tools for analysis and testing, are presented in NIST SP 800-11. Conclusions about the strengths and weaknesses of the NIST SP 800-115 standard are made. Recommendations about as NIST SP 800-115 is used in the development of the national Russian standard of penetration testing are presented.
Few-Shot Question Answering by Pretraining Span Selection

In several question answering benchmarks, pretrained models have reached human parity through fine-tuning on an order of 100,000 annotated questions and answers. We explore the more realistic few-shot setting, where only a few hundred training examples are available, and observe that standard models perform poorly, highlighting the discrepancy between current pretraining objectives and question answering. We propose a new pretraining scheme tailored for question answering: recurring span selection. Given a passage with multiple sets of recurring spans, we mask in each set all recurring spans but one, and ask the model to select the correct span in the passage for each masked span. Masked spans are replaced with a special token, viewed as a question representation, that is later used during fine-tuning to select the answer span. The resulting model obtains surprisingly good results on multiple benchmarks (e.g., 72.7 F1 on SQuAD with only 128 training examples), while maintaining competitive performance in the high-resource setting.
A Fusion NLP Model for the Inference of Standardized Thyroid Nodule Malignancy Scores from Radiology Report Text.

Radiology reports are a rich resource for advancing deep learning applications for medical images, facilitating the generation of large-scale annotated image databases. Although the ambiguity and subtlety of natural language poses a significant challenge to information extraction from radiology reports. Thyroid Imaging Reporting and Data Systems (TI-RADS) has been proposed as a system to standardize ultrasound imaging reports for thyroid cancer screening and diagnosis, through the implementation of structured templates and a standardized thyroid nodule malignancy risk scoring system; however there remains significant variation in radiologist practice when it comes to diagnostic thyroid ultrasound interpretation and reporting. In this work, we propose a computerized approach using a contextual embedding and fusion strategy for the large-scale inference of TI-RADS final assessment categories from narrative ultrasound (US) reports. The proposed model has achieved high accuracy on an internal data set, and high performance scores on an external validation dataset.
Development of Professional Skills of Bachelors of Musical Art in the Context of Dual Education

The relevance of the study is conditioned by the need to investigate the interaction of the following aspects: studying the curriculum in a higher educational institution by listening to lecture courses, preparing for the session and passing it, implementing and further developing existing professional skills directly in the work. The purpose of this research paper is to reveal the ways of developing and improving professional skills of bachelors of musical art in the context of dual education, combining educational (theoretical) and labour (practical) activities at the same time. The research materials are data from monitoring the quality of education acquired in conditions of duality, and information on the principles of combining these aspects at the same time. Methods of studying the problem – collecting and systematising sources devoted to it, deducing the categories that make up the core of the analysed issue, and drawing up a unified picture of the development of professional skills of students of musical educational institutions in the conditions of dual education. As for the study results, they contain information about the internal content and nature of such a phenomenon as duality in the acquisition of a profession. The prospects of this phenomenon, its uniqueness, significance, and opportunities provided to future specialists are also outlined. The results of the study can be used by teachers to improve the professional training of music students, and by researchers and students to find ways to improve the interaction between music education institutions and organisations that offer jobs
WhatsApp and political messaging at the periphery: Insights from northern Ghana

With relatively cheap smartphones available across the country; timelimited data bundles and schemes such as Facebook's Free Basics (which provide unlimited or free access to various social media platforms); 1 'cultural values of sociality, interconnectedness, interdependence and conviviality'; 2 and a widespread popular interest in current affairs, 3 it is unsurprising that an increasing number of Ghanaians are active social media users. 4 This includes the country's political aspirants and party activists who -in an extremely competitive electoral system that sees regular transfers of power between the National Democratic Congress (NDC) and New Patriotic Party (NPP) -are always looking for a competitive edge.In 2016, the NPP seemed to gain this edge, at least in part, through their more effective use of social media.As the incumbent president and NDC flagbearer, John Mahama, lamented in the wake of his defeat:Social media was coming into its own … and the NPP used it very effectively to really create the perception -exaggerated -of our administration that the economy was in a big mess. 5 as one NPP official claimed, 'we used [social media] extensively [in 2016] and it contributed about 40 per cent to our victory.' 6 This common
Building a Vietnamese Dataset for Natural Language Inference Models

Natural language inference models are essential resources for many natural language understanding applications. These models are possibly built by training or fine-tuning using deep neural network architectures for state-of-the-art results. That means high-quality annotated datasets are essential for building state-of-the-art models. Therefore, we propose a method to build a Vietnamese dataset for training Vietnamese inference models which work on native Vietnamese texts. Our approach aims at two issues: removing cue marks and ensuring the writing style of Vietnamese texts. If a dataset contains cue marks, the trained models will identify the relationship between a premise and a hypothesis without semantic computation. For evaluation, we fine-tuned a BERT model, viNLI, on our dataset and compared it to a BERT model, viXNLI, which was fine-tuned on XNLI dataset. The viNLI model has an accuracy of 94.79%, while the viXNLI model has an accuracy of 64.04% when testing on our Vietnamese test set. In addition, we also conducted an answer selection experiment with these two models in which the P@1 of viNLI and of viXNLI are 0.4949 and 0.4044, respectively. That means our method can be used to build a high-quality Vietnamese natural language inference dataset.
Air Writing Recognition Using Mediapipe and Opencv

Have you ever wished you could draw something just by waving your finger in the air? This can be accomplished in this day and age using a variety of technologies. This paper describes an application that uses the Python modules MediaPipe, Numpy, and OpenCV to write/draw geometrical shapes in real time over air in front of a camera. The proposed method accomplishes two goals: first, it recognizes the hand and the fingers from video frames, and then it monitors hand movements to create diverse designs. Furthermore, this technology allows for genuine Human-System interaction without the use of any form of character input device. The application is created in Python and tested in a variety of circumstances with variable illumination. This Software-based solution is relevantly basic, quick, and straightforward to use.
Haematological Studies and Micronucleus Assay of Straw-Coloured Fruit Bats (Eidolon helvum).

The straw-coloured fruit bats (Eidolon helvum) are the most widely distributed megachiropteran species in Africa. Studies have shown that they migrate, and are likely exposed to environmental pollutants across population. This study was designed to investigate genotoxicity via the bone marrow micronucleus assay and haematological alterations of Eidolon helvum in the tropics. Healthy straw-coloured fruit bats (Eidolon helvum; n=20) were captured from two geographical regions, Ogun and Gombe States in Nigeria and were grouped based on sex and age. Blood samples were collected for haematology and osmotic fragility, and bone marrow samples for genotoxicity studies. Results showed no significant differences in erythrocytes and leucocytes values across age and sex. The erythrocytes osmotic fragility was higher in juvenile than in adults at 0 and 0.1%NaCl, while it was higher in adult males than in adult females at 0 and 0.3% NaCl. The erythrocytes and leucocytes parameters in straw colored fruit bats were within the reference values seen in literature except the higher monocyte counts suggesting chronic inflammation. There were increased levels of micronucleated polychromatic erythrocytes and normochromatic erythrocytes in the straw-coloured fruit bats indicating genotoxicity and cytotoxicity, respectively. The present study provided baseline research data on the haematology and micronucleus profile of the straw-coloured fruit bats in Nigeria. This is perhaps the first study on haematology and micronucleus assay of in straw-colored fruit bats in the tropics.
Temporally-Transferable Perturbations: Efficient, One-Shot Adversarial Attacks for Online Visual Object Trackers

In recent years, the trackers based on Siamese networks have emerged as highly effective and efficient for visual object tracking (VOT). While these methods were shown to be vulnerable to adversarial attacks, as most deep networks for visual recognition tasks, the existing attacks for VOT trackers all require perturbing the search region of every input frame to be effective, which comes at a non-negligible cost, considering that VOT is a real-time task. In this paper, we propose a framework to generate a single temporally transferable adversarial perturbation from the object template image only. This perturbation can then be added to every search image, which comes at virtually no cost, and still, successfully fool the tracker. Our experiments evidence that our approach outperforms the state-of-the-art attacks on the standard VOT benchmarks in the untargeted scenario. Furthermore, we show that our formalism naturally extends to targeted attacks that force the tracker to follow any given trajectory by precomputing diverse directional perturbations.
Smartajweed Automatic Recognition of Arabic Quranic Recitation Rules

Tajweed is a set of rules to read the Quran in a correct Pronunciation of the letters with all its Qualities, while Reciting the Quran. which means you have to give every letter in the Quran its due of characteristics and apply it to this particular letter in this specific situation while reading, which may differ in other times. These characteristics include melodic rules, like where to stop and for how long, when to merge two letters in pronunciation or when to stretch some, or even when to put more strength on some letters over other. Most of the papers focus mainly on the main recitation rules and the pronunciation but not (Ahkam AL Tajweed) which give different rhythm and different melody to the pronunciation with every different rule of (Tajweed). Which is also considered very important and essential in Reading the Quran as it can give different meanings to the words. In this paper we discuss in detail full system for automatic recognition of Quran Recitation Rules (Tajweed) by using support vector machine and threshold scoring system
Soft-IntroVAE: Analyzing and Improving the Introspective Variational Autoencoder

The recently introduced introspective variational autoencoder (IntroVAE) exhibits outstanding image generations, and allows for amortized inference using an image encoder. The main idea in IntroVAE is to train a VAE adversarially, using the VAE encoder to discriminate between generated and real data samples. However, the original IntroVAE loss function relied on a particular hinge-loss formulation that is very hard to stabilize in practice, and its theoretical convergence analysis ignored important terms in the loss. In this work, we take a step towards better understanding of the IntroVAE model, its practical implementation, and its applications. We propose the Soft-IntroVAE, a modified IntroVAE that replaces the hinge-loss terms with a smooth exponential loss on generated samples. This change significantly improves training stability, and also enables theoretical analysis of the complete algorithm. Interestingly, we show that the IntroVAE converges to a distribution that minimizes a sum of KL distance from the data distribution and an entropy term. We discuss the implications of this result, and demonstrate that it induces competitive image generation and reconstruction. Finally, we describe two applications of Soft-IntroVAE to unsupervised image translation and out-of-distribution detection, and demonstrate compelling results. Code and additional information is available on the project website -- https://taldatech.github.io/soft-intro-vae-web
ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language

Transformers have been shown to emulate logical deduction over natural language theories (logical rules expressed in natural language), reliably assigning true/false labels to candidate implications. However, their ability to generate implications of a theory has not yet been demonstrated, and methods for reconstructing proofs of answers are imperfect. In this work we show that a generative model, called ProofWriter, can reliably generate both implications of a theory and the natural language proof(s) that support them. In particular, iterating a 1-step implication generator results in proofs that are highly reliable, and represent actual model decisions (rather than post-hoc rationalizations). On the RuleTaker dataset, the accuracy of ProofWriter's proofs exceed previous methods by +9% absolute, and in a way that generalizes to proof depths unseen in training and on out-of-domain problems. We also show that generative techniques can perform a type of abduction with high precision: Given a theory and an unprovable conclusion, identify a missing fact that allows the conclusion to be proved, along with a proof. These results significantly improve the viability of neural methods for systematically reasoning over natural language.
Private-Shared Disentangled Multimodal VAE for Learning of Hybrid Latent Representations

Multi-modal generative models represent an important family of deep models, whose goal is to facilitate representation learning on data with multiple views or modalities. However, current deep multi-modal models focus on the inference of shared representations, while neglecting the important private aspects of data within individual modalities. In this paper, we introduce a disentangled multi-modal variational autoencoder (DMVAE) that utilizes disentangled VAE strategy to separate the private and shared latent spaces of multiple modalities. We specifically consider the instance where the latent factor may be of both continuous and discrete nature, leading to the family of general hybrid DMVAE models. We demonstrate the utility of DMVAE on a semi-supervised learning task, where one of the modalities contains partial data labels, both relevant and irrelevant to the other modality. Our experiments on several benchmarks indicate the importance of the private-shared disentanglement as well as the hybrid latent representation.
Influencer Videos: Unboxing the Mystique

Influencer marketing has become a very popular tool to reach customers. Despite the rapid growth in influencer videos, there has been little research on the effectiveness of their constituent features in explaining video engagement. We study YouTube influencers and analyze their unstructured video data across text, audio and images using an "interpretable deep learning" framework that accomplishes both goals of prediction and interpretation. Our prediction-based approach analyzes unstructured data and finds that "what is said" in words (text) is more influential than "how it is said" in imagery (images) or acoustics (audio). Our novel interpretation-based approach is implemented after completion of model prediction by analyzing the same source of unstructured data to measure importance attributed to the video features. We eliminate several spurious relationships in two steps, identifying a subset of relationships which are confirmed using theory. We uncover novel findings that establish distinct associations for measures of shallow and deep engagement based on the dual-system framework of human thinking. Our approach is validated using simulated data, and we discuss the learnings from our findings for influencers and brands.
Limitations of Deep Neural Networks: a discussion of G. Marcus' critical appraisal of deep learning

Deep neural networks have triggered a revolution in artificial intelligence, having been applied with great results in medical imaging, semi-autonomous vehicles, ecommerce, genetics research, speech recognition, particle physics, experimental art, economic forecasting, environmental science, industrial manufacturing, and a wide variety of applications in nearly every field. This sudden success, though, may have intoxicated the research community and blinded them to the potential pitfalls of assigning deep learning a higher status than warranted. Also, research directed at alleviating the weaknesses of deep learning may seem less attractive to scientists and engineers, who focus on the low-hanging fruit of finding more and more applications for deep learning models, thus letting short-term benefits hamper long-term scientific progress. Gary Marcus wrote a paper entitled Deep Learning: A Critical Appraisal, and here we discuss Marcus' core ideas, as well as attempt a general assessment of the subject. This study examines some of the limitations of deep neural networks, with the intention of pointing towards potential paths for future research, and of clearing up some metaphysical misconceptions, held by numerous researchers, that may misdirect them.