Reviewerly/exHarmony huggingface dataset:
VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding

Pre-trained language models have achieved promising performance on general benchmarks, but underperform when migrated to a specific domain.Recent works perform pre-training from scratch or continual pre-training on domain corpora.However, in many specific domains, the limited corpus can hardly support obtaining precise representations.To address this issue, we propose a novel Transformer-based language model named VarMAE for domainadaptive language understanding.Under the masked autoencoding objective, we design a context uncertainty learning module to encode the token's context into a smooth latent distribution.The module can produce diverse and well-formed contextual representations.Experiments on science-and finance-domain NLU tasks demonstrate that VarMAE can be efficiently adapted to new domains with limited resources.
Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization

Efficient k-nearest neighbor search is a fundamental task, foundational for many problems in NLP.When the similarity is measured by dot-product between dual-encoder vectors or ℓ 2 -distance, there already exist many scalable and efficient search methods.But not so when similarity is measured by more accurate and expensive black-box neural similarity models, such as cross-encoders, which jointly encode the query and candidate neighbor.The cross-encoders' high computational cost typically limits their use to reranking candidates retrieved by a cheaper model, such as dual encoder or TF-IDF.However, the accuracy of such a two-stage approach is upper-bounded by the recall of the initial candidate set, and potentially requires additional training to align the auxiliary retrieval model with the cross-encoder model.In this paper, we present an approach that avoids the use of a dual-encoder for retrieval, relying solely on the cross-encoder.Retrieval is made efficient with CUR decomposition, a matrix decomposition approach that approximates all pairwise cross-encoder distances from a small subset of rows and columns of the distance matrix.Indexing items using our approach is computationally cheaper than training an auxiliary dual-encoder model through distillation.Empirically, for k > 10, our approach provides test-time recall-vs-computational cost trade-offs superior to the current widely-used methods that re-rank items retrieved using a dual-encoder or TF-IDF.
Discourse Context Predictability Effects in Hindi Word Order

We test the hypothesis that discourse predictability influences Hindi syntactic choice.While prior work has shown that a number of factors (e.g., information status, dependency length, and syntactic surprisal) influence Hindi word order preferences, the role of discourse predictability is underexplored in the literature.Inspired by prior work on syntactic priming, we investigate how the words and syntactic structures in a sentence influence the word order of the following sentences.Specifically, we extract sentences from the Hindi-Urdu Treebank corpus (HUTB), permute the preverbal constituents of those sentences, and build a classifier to predict which sentences actually occurred in the corpus against artificially generated distractors.The classifier uses a number of discourse-based features and cognitive features to make its predictions, including dependency length, surprisal, and information status.We find that information status and LSTM-based discourse predictability influence word order choices, especially for non-canonical objectfronted orders.We conclude by situating our results within the broader syntactic priming literature.
Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction

Neural language models encode rich knowledge about entities and their relationships which can be extracted from their representations using probing.Common properties of nouns (e.g., red strawberries, small ant) are, however, more challenging to extract compared to other types of knowledge because they are rarely explicitly stated in texts.We hypothesize this to mainly be the case for perceptual properties which are obvious to the participants in the communication.We propose to extract these properties from images and use them in an ensemble model, in order to complement the information that is extracted from language models.We consider perceptual properties to be more concrete than abstract properties (e.g., interesting, flawless).We propose to use the adjectives' concreteness score as a lever to calibrate the contribution of each source (text vs. images).We evaluate our ensemble model in a ranking task where the actual properties of a noun need to be ranked higher than other non-relevant properties.Our results show that the proposed combination of text and images greatly improves noun property prediction compared to powerful text-based language models.
Late Fusion with Triplet Margin Objective for Multimodal Ideology Prediction and Analysis

Prior work on ideology prediction has largely focused on single modalities, i.e., text or images.In this work, we introduce the task of multimodal ideology prediction, where a model predicts binary or five-point scale ideological leanings, given a text-image pair with political content.We first collect five new large-scale datasets with English documents and images along with their ideological leanings, covering news articles from a wide range of mainstream media in US and social media posts from Reddit and Twitter.We conduct in-depth analyses on news articles and reveal differences in image content and usage across the political spectrum.Furthermore, we perform extensive experiments and ablation studies, demonstrating the effectiveness of targeted pretraining objectives on different model components.Our bestperforming model, a late-fusion architecture pretrained with a triplet objective over multimodal content, outperforms the state-of-the-art text-only model by almost 4% and a strong multimodal baseline with no pretraining by over 3%.
I2C-Huelva at SemEval-2023 Task 10: Ensembling Transformers Models for the Detection of Online Sexism

This work details our approach for addressing Tasks A and B of the Semeval 2023 Task 10: Explainable Detection of Online Sexism (EDOS). For Task A a simple ensemble based of majority vote system was presented. To build our proposal, first a review of transformers was carried out and the 3 best performing models were selected to be part of the ensemble. Next, for these models, the best hyperpameters were searched using a reduced data set. Finally, we trained these models using more data. During the development phase, our ensemble system achieved an f1-score of 0.8403. For task B, we developed a model based on the deBERTa transformer, utilizing the hyperparameters identified for task A. During the development phase, our proposed model attained an f1-score of 0.6467. Overall, our methodology demonstrates an effective approach to the tasks, leveraging advanced machine learning techniques and hyperparameters searches to achieve high performance in detecting and classifying instances of sexism in online text.
CDNet: Cross-Domain Description and Detection for 2D - 3D Learning Local Features

By integrating descriptors with keypoint detectors, we can unlock the possibility of achieving comprehensive image matching and retrieval, which presents significant implications for various modern applications in computer vision and image processing. Despite the numerous proposals for learning-based feature detection and keypoint description in 2D or 3D, the matching between 2D images and 3D point clouds has not received thorough investigation. In this work, we propose a two-branch fully convolutional network framework that maps 2D images and 3D point clouds into a latent space for feature description and feature point detection. Our model leverages two parallel branches, one for extracting features from 2D images and the other from 3D point clouds, while facilitating information exchange through weight sharing. This approach enables us to fully exploit the correlations between 2D images and 3D point clouds, enhancing the expressive power of features and achieving more accurate and robust performance in image matching and retrieval tasks. Additionally, we have designed a novel loss function to enhance descriptor performance and enable more accurate keypoint detection. Finally, we extensively evaluate our model on the SceneNN and 3DMatch datasets, demonstrating its strong performance in accurate and efficient 2D-3D image matching. Our findings have significant implications for various applications, including augmented reality, autonomous navigation, and 3D reconstruction.
Cheater’s Bowl: Human vs. Computer Search Strategies for Open-Domain QA

For humans and computers, the first step in answering an open-domain question is retrieving a set of relevant documents from a large corpus.However, the strategies that computers use fundamentally differ from those of humans.To better understand these differences, we design a gamified interface for data collection-Cheater's Bowl-where a human answers complex questions with access to both traditional and modern search tools.We collect a dataset of human search sessions, analyze human search strategies, and compare them to state-of-the-art multi-hop QA models.Humans query logically, apply dynamic search chains, and use world knowledge to boost searching.We demonstrate how human queries can improve the accuracy of existing systems and propose improving the future design of QA models.
Deep Learning and Imbalance Handling on Movie Review Sentiment Analysis

Before watching a movie, people usually read reviews written by movie critics or regular audiences to gain insights about the movie’s quality and discover recommended films. However, analyzing movie reviews can be challenging due to several reasons. Firstly, popular movies can receive hundreds of reviews, each comprising several paragraphs, making it time-consuming and effort-intensive to read them all. Secondly, different reviews may express varying opinions about the movie, making it difficult to draw definitive conclusions. To address these challenges, sentiment analysis using CNN and LSTM models, known for their effectiveness in classifying text in various datasets, was performed on the movie reviews. Additionally, techniques such as TF-IDF, Word2Vec, and data balancing with SMOTEN were applied to enhance the analysis. The CNN achieved an impressive sentiment analysis accuracy of 98.56%, while the LSTM achieved a close 98.53%. Moreover, both classifiers performed well in terms of the F1-score, with CNN obtaining 77.87% and LSTM achieving 78.92%. These results demonstrate the effectiveness of the sentiment analysis approach in extracting valuable insights from movie reviews and helping people make informed decisions about which movies to watch.
White Blood Cell Detection Using Yolov8 Integration with DETR to Improve Accuracy

One of the body's most crucial blood cell kinds is the white blood cell. White blood cells, called leukocytes, are crucial for the body's defence mechanism and against hazardous foreign substances, tumour cells, and infectious bacteria. This paper suggests a computer-based automated system for detecting white blood cells using the YOLOV8 transformer and white blood cell analysis in digital images of blood cells. The Generate process uses Yolov8. In Generate, this will produce image processing in the form of annotation results on each type of white blood cell and dataset with COCO format. The DETR Model training conducted in this study is to increase the accuracy value of the white output of the blood cell picture formation. Test results using recall, precision, f1 score and object detection values. In the lymphocyte and basophil datasets, the number of white blood cell images used is only 10 images. Following the results of training from yolov8 using Roboflow, the results were increased relatively high, with an average increase of 0.68 in all five images of white blood cells. This test also gets an average improvement in detection results from Yolo to DETR, getting a fairly significant result of 68%, which is because YOLO cannot handle undetected objects (which are not in the training dataset; furthermore, DETR can handle multiple objects in a single image. Typically, detecting traditional objects such as YOLO requires repeatedly multiple object detection with a fixed batch size
Community-based Dynamic Graph Learning for Popularity Prediction

Popularity prediction, which aims to forecast how many users would like to interact with a target item or online content in the future, can help online shopping or social media platforms to identify popular items or digital contents. Many efforts have been made to study how the multi-faceted factors, such as item features, user preferences, and social influence, affect user-item interactions, but little work has focused on the evolutionary dynamics of these factors for individuals or groups. In that light, this paper develops a community-based dynamic graph learning method for popularity prediction. First, a dynamic graph learning framework is proposed to maintain a dynamic representation for each item or user entity and update the representations according to the newly observed user-item interactions. Second, a community detection module is designed to capture the evolving community structures and identify the most influential nodes. More importantly, our framework leverages a community-level message passing during the learning process to balance local and global information propagation. Finally, we predict the popularity of the target item or online content based on the learned representations. Our experimental results based on three real-world datasets demonstrate that the proposed method achieves better performance than the baselines. Our method could not only model the changes in a user's preferences, but also capture how the communities evolve over time.
Trend Forecasting of the Top 3 Indonesian Bank Stocks Using the ARIMA Method

The number of investors in Indonesia increases annually. This is due to the growing popularity of investing, particularly stock investment. There are currently three largest equities in the banking industry, namely BBCA, BBRI, and BMRI. Stock prices fluctuate and form multiple patterns of price movements; therefore, investors must be able to recognize the patterns and trends of securities on the capital market in order to plan long-term investments, maximize potential profits, and reduce the risk of investment losses. In addition to knowing the trajectory of the stock market's trend, investors rely heavily on forecasting. Forecasting is necessary so that investors can anticipate future prices. The Autoregressive Integrated Moving Average (ARIMA) method is a frequently used method for forecasting time series data. In general, ARIMA is represented by the formula ARIMA (p, d, q), where p represents the Autoregressive (AR) order, d represents the difference, and q represents the Moving Average (MA) order. The trend of BBCA, BBRI, and BMRI stock data was effectively predicted using the ARIMA method. The results of this study are presented as diagrams of actual and forecasted data for the next 12 periods, as well as predictions of the optimal purchase price points for stocks. The ARIMA model of each stock also generates a low MAPE error value, with MAPE values of 4% for BBCA, 5% for BBRI, and 7% for BMRI. The MAPE value derived by each model is incorporated into the MAPE value with a high degree of precision, as it falls below 10%.
Generative Perturbation Analysis for Probabilistic Black-Box Anomaly Attribution

We address the task of probabilistic anomaly attribution in the black-box regression setting, where the goal is to compute the probability distribution of the attribution score of each input variable, given an observed anomaly. The training dataset is assumed to be unavailable. This task differs from the standard XAI (explainable AI) scenario, since we wish to explain the anomalous deviation from a black-box prediction rather than the black-box model itself.
MetricPrompt: Prompting Model as a Relevance Metric for Few-shot Text Classification

Prompting methods have shown impressive performance in a variety of text mining tasks and applications, especially few-shot ones. Despite the promising prospects, the performance of prompting model largely depends on the design of prompt template and verbalizer. In this work, we propose MetricPrompt, which eases verbalizer design difficulty by reformulating few-shot text classification task into text pair relevance estimation task. MetricPrompt adopts prompting model as the relevance metric, further bridging the gap between Pre-trained Language Model's (PLM) pre-training objective and text classification task, making possible PLM's smooth adaption. Taking a training sample and a query one simultaneously, MetricPrompt captures cross-sample relevance information for accurate relevance estimation. We conduct experiments on three widely used text classification datasets across four few-shot settings. Results show that MetricPrompt outperforms manual verbalizer and other automatic verbalizer design methods across all few-shot settings, achieving new state-of-the-art (SOTA) performance.
Workshop Resensi Buku: Menguatkan Kemampuan Menulis Mahasiswa Dan Staf Perpustakaan UIN Gusdur

Community service activities are carried out to strengthen writing skills, especially writing reviews. Service activities are carried out by way of workshops which are delivered with presentation techniques explaining the material. Presentation of the material is done using the lecture method, discussion-question-answer, and assignments. The results of the workshop activities: the activity participants have a good understanding of the material presented and have basic skills related to review writing efforts. This activity also has a social impact based on the good response given by the activity participants as well as the results of reviews that have been well assessed.
Supplemental Material: Rheological heterogeneities at the roots of the seismogenic zone

&lt;p&gt;SM1 (Coulomb stress change); SM2 (basement geometries); SM3 (clustering methods); SM4 (cross-correlation analysis); SM5 (clusters); SM6 (experimental apparatus and procedures); SM7 (basement gouges); and SM8 (seismicity cutoff evolution). &lt;/p&gt;
Toward a Dynamic Annotation Scheme

This chapter works toward the specification of a dynamic annotation scheme, called dSpace. It extends the scope of ISO-Space to the domains of space and time over motions by being amalgamated with ISO-TimeML. In dSpace, various types of temporal relations interact with spatial relations. The temporal dimension characterizes various types of paths and motions anchored to each location on the paths; dSpace also generalizes the notion of paths by classifying them into four types: static, dynamic, projected, and oriented, while introducing a relational link, called pathLink, over paths with various relation types such as meet and deviate.
Temporal Ontology

Viewing ontology as a science of things, this chapter treats times as real objects in the world. Such a view of ontology of times, called temporal ontology, conforms to Neo-Davidsonian semantics and to the type-theoretic semantics, which treats time points as one of the basic types that include individual objects, events, and spatial points. It is thus designed to provide a sound basis for the development of a semantics for the annotation and interpretation of event-based temporal information in language. In this chapter, I first introduce OWL-Time ontology which classifies temporal entities into instances and intervals. I then introduce an interval temporal calculus with 13 base relations over time points and intervals. I also discuss how eventualities are temporalized to be treated as denoting time intervals. Eventualities are then temporally related to times. To apply the notions of time points and intervals to the interpretation of tenses and aspects of language, especially the progessive aspect and the present perfective aspect, I define the notion of neighborhood and apply it to the definition of the present perfect as denoting the neighborhood of the present moment.
Data Segmentation

Data can be segmented into minimal units. Such a process is called base segmentation. In this chapter, I discuss three types of base segmentation of language data, depending on its three media types: phoneme segmentation, image segmentation, and text segmentation. They can be grouped into larger units. Base segmented text, for instance, undergo tokenization, annotated segmentation such as word segmentation, and chunking with POS-tagging. The semantic annotation of language data, whether written, spoken, or visualized, requires the target data to be segmented and preferably annotated with appropriate morpho-syntactic information.
What Does Semantics Do for Annotation?

The task of semantics for annotation is two-fold. First, semantics validates the construction of a syntax for the generation of well-formed annotation structures. Second, semantics provides a formalism for interpreting those annotation structures that are generated by the syntax. In this chapter, my main concern is to present a general view of what kind of semantics is needed to interpret annotation structures and to lay a general ground for constructing an interpretation scheme for temporal and spatial annotation. This semantics follows the ordinary steps of doing model-theoretic formal semantics such as Montague semantics. It goes through an intermediate step of representing semantic content or denotations in logical forms and then interprets them with respect to a model with truth definitions.
Extending the Range of Temporal Annotation

In Chapter 9, I introduce eXTimeML, an extended variant of ISO-TimeML, with three extensions. (i) Temporal measure expressions are annotated as part of generalized measure: e.g., 30 hours. (ii) Quantified temporal expressions are annotated as part of generalized quantification: e.g., every day. (iii) Adjectives and adverbs are annotated as modifiers of nouns and verbs, respectively: e.g., daily, never. Temporal measures and temporal quantifiers are treated as part of generalized measures and quantifiers. I then illustrate how the representation language of ABS applies to each of these extensions in eXTimeML by deriving appropriate (logical) semantic forms from the well-formed annotation structures of temporal measures, quantifiers, and modifiers. Semantic forms are then interpreted with respect to admissible models, constrained by the formal definitions of logical predicates such as twice or three thousand.