Reviewerly/exHarmony huggingface dataset:
Neural Network Compression using Binarization and Few Full-Precision Weights

Quantization and pruning are two effective Deep Neural Networks model compression methods. In this paper, we propose Automatic Prune Binarization (APB), a novel compression technique combining quantization with pruning. APB enhances the representational capability of binary networks using a few full-precision weights. Our technique jointly maximizes the accuracy of the network while minimizing its memory impact by deciding whether each weight should be binarized or kept in full precision. We show how to efficiently perform a forward pass through layers compressed using APB by decomposing it into a binary and a sparse-dense matrix multiplication. Moreover, we design two novel efficient algorithms for extremely quantized matrix multiplication on CPU, leveraging highly efficient bitwise operations. The proposed algorithms are 6.9x and 1.5x faster than available state-of-the-art solutions. We extensively evaluate APB on two widely adopted model compression datasets, namely CIFAR10 and ImageNet. APB delivers better accuracy/memory trade-off compared to state-of-the-art methods based on i) quantization, ii) pruning, and iii) combination of pruning and quantization. APB outperforms quantization in the accuracy/efficiency trade-off, being up to 2x faster than the 2-bit quantized model with no loss in accuracy.
DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning

Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations and is essential for the development of communicative social agents. In this paper, we introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding. Compared with previous works that treat different figurative expressions (e.g. metaphor, sarcasm) as individual tasks, DiPlomat provides a cohesive framework towards general pragmatic understanding. Our dataset is created through the utilization of Amazon Mechanical Turk ( AMT ), resulting in a total of 4, 177 multi-turn dialogues. In conjunction with the dataset, we propose two tasks, Pragmatic Identification and Reasoning (PIR) and Conversational Question Answering (CQA). Experimental results with state-of-the-art (SOTA) neural architectures reveal several significant findings: 1) large language models ( LLMs) exhibit poor performance in tackling this subjective domain; 2) comprehensive comprehension of context emerges as a critical factor for establishing benign human-machine interactions; 3) current models defect in the application of pragmatic reasoning. As a result, we call on more attention to improve the ability of context understanding, reasoning, and implied meaning modeling.
Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration

Although instruction-tuned large language models (LLMs) have exhibited remarkable capabilities across various NLP tasks, their effectiveness on other data modalities beyond text has not been fully studied. In this work, we propose Macaw-LLM, a novel multi-modal LLM that seamlessly integrates visual, audio, and textual information. Macaw-LLM consists of three main components: a modality module for encoding multi-modal data, a cognitive module for harnessing pretrained LLMs, and an alignment module for harmonizing diverse representations. Our novel alignment module seamlessly bridges multi-modal features to textual features, simplifying the adaptation process from the modality modules to the cognitive module. In addition, we construct a large-scale multi-modal instruction dataset in terms of multi-turn dialogue, including 69K image instances and 50K video instances. We have made our data, code and model publicly available, which we hope can pave the way for future research in multi-modal LLMs and expand the capabilities of LLMs to handle diverse data modalities and address complex real-world scenarios.
Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection

Modern machine learning models deployed in the wild can encounter both covariate and semantic shifts, giving rise to the problems of out-of-distribution (OOD) generalization and OOD detection respectively. While both problems have received significant research attention lately, they have been pursued independently. This may not be surprising, since the two tasks have seemingly conflicting goals. This paper provides a new unified approach that is capable of simultaneously generalizing to covariate shifts while robustly detecting semantic shifts. We propose a margin-based learning framework that exploits freely available unlabeled data in the wild that captures the environmental test-time OOD distributions under both covariate and semantic shifts. We show both empirically and theoretically that the proposed margin constraint is the key to achieving both OOD generalization and detection. Extensive experiments show the superiority of our framework, outperforming competitive baselines that specialize in either OOD generalization or OOD detection. Code is publicly available at https://github.com/deeplearning-wisc/scone.
Neural World Models for Computer Vision

Humans navigate in their environment by learning a mental model of the world through passive observation and active interaction. Their world model allows them to anticipate what might happen next and act accordingly with respect to an underlying objective. Such world models hold strong promises for planning in complex environments like in autonomous driving. A human driver, or a self-driving system, perceives their surroundings with their eyes or their cameras. They infer an internal representation of the world which should: (i) have spatial memory (e.g. occlusions), (ii) fill partially observable or noisy inputs (e.g. when blinded by sunlight), and (iii) be able to reason about unobservable events probabilistically (e.g. predict different possible futures). They are embodied intelligent agents that can predict, plan, and act in the physical world through their world model. In this thesis we present a general framework to train a world model and a policy, parameterised by deep neural networks, from camera observations and expert demonstrations. We leverage important computer vision concepts such as geometry, semantics, and motion to scale world models to complex urban driving scenes. First, we propose a model that predicts important quantities in computer vision: depth, semantic segmentation, and optical flow. We then use 3D geometry as an inductive bias to operate in the bird's-eye view space. We present for the first time a model that can predict probabilistic future trajectories of dynamic agents in bird's-eye view from 360{\deg} surround monocular cameras only. Finally, we demonstrate the benefits of learning a world model in closed-loop driving. Our model can jointly predict static scene, dynamic scene, and ego-behaviour in an urban driving environment.
High-Resolution Convolutional Neural Networks on Homomorphically Encrypted Data via Sharding Ciphertexts

Recently, Deep Convolutional Neural Networks (DCNNs) including the ResNet-20 architecture have been privately evaluated on encrypted, low-resolution data with the Residue-Number-System Cheon-Kim-Kim-Song (RNS-CKKS) homomorphic encryption scheme. We extend methods for evaluating DCNNs on images with larger dimensions and many channels, beyond what can be stored in single ciphertexts. Additionally, we simplify and improve the efficiency of the recently introduced multiplexed image format, demonstrating that homomorphic evaluation can work with standard, row-major matrix packing and results in encrypted inference time speedups by $4.6-6.5\times$. We also show how existing DCNN models can be regularized during the training process to further improve efficiency and accuracy. These techniques are applied to homomorphically evaluate a DCNN with high accuracy on the high-resolution ImageNet dataset, achieving $80.2\%$ top-1 accuracy. We also achieve an accuracy of homomorphically evaluated CNNs on the CIFAR-10 dataset of $98.3\%$.
DaMuEL: A Large Multilingual Dataset for Entity Linking

We present DaMuEL, a large Multilingual Dataset for Entity Linking containing data in 53 languages. DaMuEL consists of two components: a knowledge base that contains language-agnostic information about entities, including their claims from Wikidata and named entity types (PER, ORG, LOC, EVENT, BRAND, WORK_OF_ART, MANUFACTURED); and Wikipedia texts with entity mentions linked to the knowledge base, along with language-specific text from Wikidata such as labels, aliases, and descriptions, stored separately for each language. The Wikidata QID is used as a persistent, language-agnostic identifier, enabling the combination of the knowledge base with language-specific texts and information for each entity. Wikipedia documents deliberately annotate only a single mention for every entity present; we further automatically detect all mentions of named entities linked from each document. The dataset contains 27.9M named entities in the knowledge base and 12.3G tokens from Wikipedia texts. The dataset is published under the CC BY-SA license at https://hdl.handle.net/11234/1-5047.
DreamHuman: Animatable 3D Avatars from Text

We present DreamHuman, a method to generate realistic animatable 3D human avatar models solely from textual descriptions. Recent text-to-3D methods have made considerable strides in generation, but are still lacking in important aspects. Control and often spatial resolution remain limited, existing methods produce fixed rather than animated 3D human models, and anthropometric consistency for complex structures like people remains a challenge. DreamHuman connects large text-to-image synthesis models, neural radiance fields, and statistical human body models in a novel modeling and optimization framework. This makes it possible to generate dynamic 3D human avatars with high-quality textures and learned, instance-specific, surface deformations. We demonstrate that our method is capable to generate a wide variety of animatable, realistic 3D human models from text. Our 3D models have diverse appearance, clothing, skin tones and body shapes, and significantly outperform both generic text-to-3D approaches and previous text-based 3D avatar generators in visual fidelity. For more results and animations please check our website at https://dream-human.github.io.
Quantum JPEG

The JPEG algorithm compresses a digital image by filtering its high spatial-frequency components. Similarly, we introduce a quantum algorithm that uses the quantum Fourier transform to discard the high spatial-frequency qubits of an image, downsampling it to a lower resolution. This allows one to capture, compress, and send images even with limited quantum resources for storage and communication. We show under which conditions this protocol is advantageous with respect to its classical counterpart.
UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video

We show how to build a model that allows realistic, free-viewpoint renderings of a scene under novel lighting conditions from video. Our method -- UrbanIR: Urban Scene Inverse Rendering -- computes an inverse graphics representation from the video. UrbanIR jointly infers shape, albedo, visibility, and sun and sky illumination from a single video of unbounded outdoor scenes with unknown lighting. UrbanIR uses videos from cameras mounted on cars (in contrast to many views of the same points in typical NeRF-style estimation). As a result, standard methods produce poor geometry estimates (for example, roofs), and there are numerous ''floaters''. Errors in inverse graphics inference can result in strong rendering artifacts. UrbanIR uses novel losses to control these and other sources of error. UrbanIR uses a novel loss to make very good estimates of shadow volumes in the original scene. The resulting representations facilitate controllable editing, delivering photorealistic free-viewpoint renderings of relit scenes and inserted objects. Qualitative evaluation demonstrates strong improvements over the state-of-the-art.
Span-Selective Linear Attention Transformers for Effective and Robust Schema-Guided Dialogue State Tracking

In schema-guided dialogue state tracking models estimate the current state of a conversation using natural language descriptions of the service schema for generalization to unseen services. Prior generative approaches which decode slot values sequentially do not generalize well to variations in schema, while discriminative approaches separately encode history and schema and fail to account for inter-slot and intent-slot dependencies. We introduce SPLAT, a novel architecture which achieves better generalization and efficiency than prior approaches by constraining outputs to a limited prediction space. At the same time, our model allows for rich attention among descriptions and history while keeping computation costs constrained by incorporating linear-time attention. We demonstrate the effectiveness of our model on the Schema-Guided Dialogue (SGD) and MultiWOZ datasets. Our approach significantly improves upon existing models achieving 85.3 JGA on the SGD dataset. Further, we show increased robustness on the SGD-X benchmark: our model outperforms the more than 30$\times$ larger D3ST-XXL model by 5.0 points.
Generative Proxemics: A Prior for 3D Social Interaction from Images

Social interaction is a fundamental aspect of human behavior and communication. The way individuals position themselves in relation to others, also known as proxemics, conveys social cues and affects the dynamics of social interaction. Reconstructing such interaction from images presents challenges because of mutual occlusion and the limited availability of large training datasets. To address this, we present a novel approach that learns a prior over the 3D proxemics two people in close social interaction and demonstrate its use for single-view 3D reconstruction. We start by creating 3D training data of interacting people using image datasets with contact annotations. We then model the proxemics using a novel denoising diffusion model called BUDDI that learns the joint distribution over the poses of two people in close social interaction. Sampling from our generative proxemics model produces realistic 3D human interactions, which we validate through a perceptual study. We use BUDDI in reconstructing two people in close proximity from a single image without any contact annotation via an optimization approach that uses the diffusion model as a prior. Our approach recovers accurate and plausible 3D social interactions from noisy initial estimates, outperforming state-of-the-art methods. Our code, data, and model are availableat our project website at: muelea.github.io/buddi.
PaReprop: Fast Parallelized Reversible Backpropagation

The growing size of datasets and deep learning models has made faster and memory-efficient training crucial. Reversible transformers have recently been introduced as an exciting new method for extremely memory-efficient training, but they come with an additional computation overhead of activation re-computation in the backpropagation phase. We present PaReprop, a fast Parallelized Reversible Backpropagation algorithm that parallelizes the additional activation re-computation overhead in reversible training with the gradient computation itself in backpropagation phase. We demonstrate the effectiveness of the proposed PaReprop algorithm through extensive benchmarking across model families (ViT, MViT, Swin and RoBERTa), data modalities (Vision & NLP), model sizes (from small to giant), and training batch sizes. Our empirical results show that PaReprop achieves up to 20% higher training throughput than vanilla reversible training, largely mitigating the theoretical overhead of 25% lower throughput from activation recomputation in reversible training. Project page: https://tylerzhu.com/pareprop.
METACOGNITIVE APPROACH IN STUDENT EDUCATION IN THE CONTEXT OF EDUCATIONAL INNOVATIONS

The article is devoted to the study of metacognitive learning. The advantages of using metacognitive learning methods in the educational process are given. Different approaches and models used to describe the purpose of cognitive learning are illustrated. A framework for determining different personality types of students, according to available metacognitive skills, is described. Tips are described that contribute to the development of metacognitive skills, which, in our opinion, form the foundation of metacognitive learning. The result of this exploration is the assumption that the use of metacognitive skills used in metacognitive learning helps students to become aware of their strengths and weaknesses. The key element is realizing the limits of your knowledge or abilities and then figuring out how to extend or extend that knowledge possibility. Those who know their strengths and weaknesses in these areas are more likely to "actively monitor their learning strategies and resources and assess their readiness for specific tasks and performance." Which, in turn, will contribute to the successful educational activities of students.
Write Around Strategy Upshot Upon Students’ Writing Skill Competence

The aim of study was to find out the effect of Write Around Strategy towards students’ writing skill in descriptive text. This study applied quantitative method with experimental study type. The technique of collecting the data of the study was using Pre-test and Post-test. The data analysis technique used median, mode and mean to find the effect of student`s writing skill in descriptive text. Based on the explanation above, the researcher concluded that Write Around Strategy was positive effect towards students` writing skill in descriptive text. Based on the data analysis, the researcher found that the result of t- test was 5.92 and the level of 0.05 (9,5), then t-table 200. Therefore, write around strategy were accepted. This meant that the null hypothesis (Ho) which said, “Write Around Strategy has no positive effect towards students’ writing skill in descriptive text.” was rejected, while the alternative hypothesis (Ha) which said, “Write Around Strategy Write Around Strategy has positive effect towards students` writing skill in descriptive text” was accepted.
Optimal ElGamal Encryption with Hybrid Deep-Learning-Based Classification on Secure Internet of Things Environment

The Internet of Things (IoT) is a kind of advanced information technology that has grabbed the attention of society. Stimulators and sensors were generally known as smart devices in this ecosystem. In parallel, IoT security provides new challenges. Internet connection and the possibility of communication with smart gadgets cause gadgets to indulge in human life. Thus, safety is essential in devising IoT. IoT contains three notable features: intelligent processing, overall perception, and reliable transmission. Due to the IoT span, the security of transmitting data becomes a crucial factor for system security. This study designs a slime mold optimization with ElGamal Encryption with a Hybrid Deep-Learning-Based Classification (SMOEGE-HDL) model in an IoT environment. The proposed SMOEGE-HDL model mainly encompasses two major processes, namely data encryption and data classification. At the initial stage, the SMOEGE technique is applied to encrypt the data in an IoT environment. For optimal key generation in the EGE technique, the SMO algorithm has been utilized. Next, in the later stage, the HDL model is utilized to carry out the classification process. In order to boost the classification performance of the HDL model, the Nadam optimizer is utilized in this study. The experimental validation of the SMOEGE-HDL approach is performed, and the outcomes are inspected under distinct aspects. The proposed approach offers the following scores: 98.50% for specificity, 98.75% for precision, 98.30% for recall, 98.50% for accuracy, and 98.25% for F1-score. This comparative study demonstrated the enhanced performance of the SMOEGE-HDL technique compared to existing techniques.
Development of Android-Based Interactive Multimedia On Thematic Learning

This research and development were conducted at SD Madrasah Ibtidaiyah Muhammadiyah using the ADDIE development model. The model consists of analysis, design, development, implementation, and evaluation stages. The result of this research is an interactive multimedia product based on the Android platform for thematic learning, specifically focusing on the subject of national historical events during the colonial era, intended for elementary school fifth-grade students with a focus on the Social Studies subject.The interactive multimedia based on Android was developed using Unity and Photoshop applications, incorporating text, images, and sound components. This multimedia can be downloaded and installed on Android-based smartphones. The product testing results showed that the validation by media experts reached 85.30%, while the validation by subject matter experts reached 95.45%. The average of these two validations falls into the category of highly valid. The effectiveness test through evaluation questions yielded a result of 96.24%, which falls into the category of highly effective and achieved a high level of completion. The practicality test obtained a percentage of 96.7% from teachers and 96.30% from students, which falls into the category of highly practical. The attractiveness test yielded a percentage of 96.72% from students, also falling into the category of highly attractive.Therefore, based on the above presentation, it can be concluded that the interactive multimedia based on Android for thematic learning on the topic of national historical events during the colonial era is suitable for use in elementary school fifth-grade students. This product meets the criteria of being valid, effective, practical, and attractive.
Analisis Praktikalitas Media Pembelajaran Interaktif Berbasis Swishmax Dalam Materi Hukum Newton Dan Penerapannya Kelas VIII SMP

In the era of technology and information that continues to develop, the use of interactive learning media is an important focus in education. However, in-depth research is still needed to test the effectiveness of SwishMax-based interactive learning media in improving student learning outcomes. This study aims to analyze the effectiveness of SwishMax-based interactive learning media on Newton's law material and its application in class VIII SMP. The method used in this research is 4D (Define, Design, Develop, Disseminate) in the development of learning media. This research trial involved students and teachers to collect data on the practicality of the developed learning media. The results showed that SwishMax-based interactive learning media was very practical for students to use in learning with a practicality percentage of 89.37%. In addition, this learning media was also found to be very practical when used in other classes. In conclusion, interactive learning media based on SwishMax on Newton's Laws and Their Applications in class VIII of junior high school meets the criteria of being very practical and effective.