OpenRaiser/Intern-Atlas huggingface dataset:
AIM: Accelerating Arbitrary-precision Integer Multiplication on Heterogeneous Reconfigurable Computing Platform Versal ACAP
Venue: arXiv cs.AR 2023
Year: 2023

Arbitrary-precision integer multiplication is the core kernel of many applications in simulation, cryptography, etc. Existing acceleration of arbitrary-precision integer multiplication includes CPUs, GPUs, FPGAs, and ASICs. Among these accelerators, FPGAs are promised to provide both good energy efficiency and flexibility. Surprisingly, in our implementations, FPGA has the lowest energy efficiency, i.e., 0.29x of the CPU and 0.17x of the GPU with the same generation fabrication. Therefore, key questions arise: Where do the energy efficiency gains of CPUs and GPUs come from? Can reconfigurable computing do better? If can, how to achieve that? We identify that the biggest energy efficiency gains of the CPUs and GPUs come from the dedicated vector units. FPGA uses DSPs and lookup tables to compose the needed computation, which incurs overhead when compared to using vector units directly. New reconfigurable computing, e.g., 'FPGA+vector units' is a novel and feasible solution to improve energy efficiency. In this paper, we propose to map arbitrary-precision integer multiplication onto such a heterogeneous platform, i.e., AMD/Xilinx Versal ACAP architecture. Designing on Versal ACAP incurs several challenges and we propose AIM: Arbitrary-precision Integer Multiplication on Versal ACAP to automate and optimize the design. AIM framework includes design space exploration and AIM automatic code generation to facilitate the system design and verification. We deploy the AIM framework on three different applications, including large integer multiplication (LIM), RSA, and Mandelbrot, on the AMD/Xilinx Versal ACAP VCK190 evaluation board. Our experimental results show that AIM achieves up to 12.6x, and 2.1x energy efficiency gains over the Intel Xeon Ice Lake 6346 CPU, and NVidia A5000 GPU respectively, which brings reconfigurable computing the most energy-efficient platform among CPUs and GPUs.
See to Touch: Learning Tactile Dexterity through Visual Incentives
Venue: arXiv cs.RO 2023
Year: 2023

Equipping multi-fingered robots with tactile sensing is crucial for achieving the precise, contact-rich, and dexterous manipulation that humans excel at. However, relying solely on tactile sensing fails to provide adequate cues for reasoning about objects' spatial configurations, limiting the ability to correct errors and adapt to changing situations. In this paper, we present Tactile Adaptation from Visual Incentives (TAVI), a new framework that enhances tactile-based dexterity by optimizing dexterous policies using vision-based rewards. First, we use a contrastive-based objective to learn visual representations. Next, we construct a reward function using these visual representations through optimal-transport based matching on one human demonstration. Finally, we use online reinforcement learning on our robot to optimize tactile-based policies that maximize the visual reward. On six challenging tasks, such as peg pick-and-place, unstacking bowls, and flipping slender objects, TAVI achieves a success rate of 73% using our four-fingered Allegro robot hand. The increase in performance is 108% higher than policies using tactile and vision-based rewards and 135% higher than policies without tactile observational input. Robot videos are best viewed on our project website: https://see-to-touch.github.io/.
LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent
Venue: arXiv cs.CV 2023
Year: 2023

3D visual grounding is a critical skill for household robots, enabling them to navigate, manipulate objects, and answer questions based on their environment. While existing approaches often rely on extensive labeled data or exhibit limitations in handling complex language queries, we propose LLM-Grounder, a novel zero-shot, open-vocabulary, Large Language Model (LLM)-based 3D visual grounding pipeline. LLM-Grounder utilizes an LLM to decompose complex natural language queries into semantic constituents and employs a visual grounding tool, such as OpenScene or LERF, to identify objects in a 3D scene. The LLM then evaluates the spatial and commonsense relations among the proposed objects to make a final grounding decision. Our method does not require any labeled training data and can generalize to novel 3D scenes and arbitrary text queries. We evaluate LLM-Grounder on the ScanRefer benchmark and demonstrate state-of-the-art zero-shot grounding accuracy. Our findings indicate that LLMs significantly improve the grounding capability, especially for complex language queries, making LLM-Grounder an effective approach for 3D vision-language tasks in robotics. Videos and interactive demos can be found on the project website https://chat-with-nerf.github.io/ .
Onchain Sports Betting using UBET Automated Market Maker
Venue: arXiv cs.CY 2023
Year: 2023

The paper underscores how decentralization in sports betting addresses the drawbacks of traditional centralized platforms, ensuring transparency, security, and lower fees. Non-custodial solutions empower bettors with ownership of funds, bypassing geographical restrictions. Decentralized platforms enhance security, privacy, and democratic decision-making. However, decentralized sports betting necessitates automated market makers (AMMs) for efficient liquidity provision. Existing AMMs like Uniswap lack alignment with fair odds, creating risks for liquidity providers. To mitigate this, the paper introduces UBET AMM (UAMM), utilizing smart contracts and algorithms to price sports odds fairly. It establishes an on-chain betting framework, detailing market creation, UAMM application, collateral liquidity pools, and experiments that exhibit positive outcomes. UAMM enhances decentralized sports betting by ensuring liquidity, decentralized pricing, and global accessibility, promoting trustless and efficient betting.
The E.U.'s Artificial Intelligence Act: An Ordoliberal Assessment
Venue: arXiv cs.CY 2023
Year: 2023

In light of the rise of generative AI and recent debates about the socio-political implications of large-language models and chatbots, this article investigates the E.U.'s Artificial Intelligence Act (AIA), the world's first major attempt by a government body to address and mitigate the potentially negative impacts of AI technologies. The article critically analyzes the AIA from a distinct economic ethics perspective, i.e., ordoliberalism 2.0 - a perspective currently lacking in the academic literature. It evaluates, in particular, the AIA's ordoliberal strengths and weaknesses and proposes reform measures that could be taken to strengthen the AIA.
Integrating Competency-Based Education in Interactive Learning Systems
Venue: arXiv cs.CY 2023
Year: 2023

Artemis is an interactive learning system that organizes courses, hosts lecture content and interactive exercises, conducts exams, and creates automatic assessments with individual feedback. Research shows that students have unique capabilities, previous experiences, and expectations. However, the course content on current learning systems, including Artemis, is not tailored to a student's competencies. The main goal of this paper is to describe how to make Artemis capable of competency-based education and provide individual course content based on the unique characteristics of every student. We show how instructors can define relations between competencies to create a competency relation graph, how Artemis measures and visualizes the student's progress toward mastering a competency, and how the progress can generate a personalized learning path for students that recommends relevant learning resources. Finally, we present the results of a user study regarding the usability of the newly designed competency visualization and give an outlook on possible improvements and future visions.
Antagonising explanation and revealing bias directly through sequencing and multimodal inference
Venue: arXiv cs.HC 2023
Year: 2023

Deep generative models produce data according to a learned representation, e.g. diffusion models, through a process of approximation computing possible samples. Approximation can be understood as reconstruction and the large datasets used to train models as sets of records in which we represent the physical world with some data structure (photographs, audio recordings, manuscripts). During the process of reconstruction, e.g., image frames develop each timestep towards a textual input description. While moving forward in time, frame sets are shaped according to learned bias and their production, we argue here, can be considered as going back in time; not by inspiration on the backward diffusion process but acknowledging culture is specifically marked in the records. Futures of generative modelling, namely in film and audiovisual arts, can benefit by dealing with diffusion systems as a process to compute the future by inevitably being tied to the past, if acknowledging the records as to capture fields of view at a specific time, and to correlate with our own finite memory ideals. Models generating new data distributions can target video production as signal processors and by developing sequences through timelines we ourselves also go back to decade-old algorithmic and multi-track methodologies revealing the actual predictive failure of contemporary approaches to synthesis in moving image, both as relevant to composition and not explanatory.
An overview of research on human-centered design in the development of artificial general intelligence
Venue: arXiv cs.CY 2023
Year: 2023

Abstract: This article offers a comprehensive analysis of Artificial General Intelligence (AGI) development through a humanistic lens. Utilizing a wide array of academic and industry resources, it dissects the technological and ethical complexities inherent in AGI's evolution. Specifically, the paper underlines the societal and individual implications of AGI and argues for its alignment with human values and interests. Purpose: The study aims to explore the role of human-centered design in AGI's development and governance. Design/Methodology/Approach: Employing content analysis and literature review, the research evaluates major themes and concepts in human-centered design within AGI development. It also scrutinizes relevant academic studies, theories, and best practices. Findings: Human-centered design is imperative for ethical and sustainable AGI, emphasizing human dignity, privacy, and autonomy. Incorporating values like empathy, ethics, and social responsibility can significantly influence AGI's ethical deployment. Talent development is also critical, warranting interdisciplinary initiatives. Research Limitations/Implications: There is a need for additional empirical studies focusing on ethics, social responsibility, and talent cultivation within AGI development. Practical Implications: Implementing human-centered values in AGI development enables ethical and sustainable utilization, thus promoting human dignity, privacy, and autonomy. Moreover, a concerted effort across industry, academia, and research sectors can secure a robust talent pool, essential for AGI's stable advancement. Originality/Value: This paper contributes original research to the field by highlighting the necessity of a human-centered approach in AGI development, and discusses its practical ramifications.
Optimizing Traversing and Retrieval Speed of Large Breached Databases
Venue: arXiv cs.DB 2023
Year: 2023

Breached data refers to the unauthorized access, theft, or exposure of confidential or sensitive information. Breaches typically occur when malicious actors or unauthorized users breach secure systems or networks, resulting in compromised personally identifiable information (PII), protected or personal health information (PHI), payment card industry (PCI) information, or other sensitive data. Data breaches are often the result of malicious activities such as hacking, phishing, insider threats, malware, or physical theft. The misuse of breached data can lead to identity theft, fraud, spamming, or blackmailing. Organizations that experience data breaches may face legal and financial consequences, reputational damage, and harm to their customers or users. Breached records are commonly sold on the dark web or made available on various public forums. To counteract these malicious activities, it is possible to collect breached databases and mitigate potential harm. These databases can be quite large, reaching sizes of up to 150 GB or more. Typically, breached data is stored in the CSV (Comma Separated Value) format due to its simplicity and lightweight nature, which reduces storage requirements. Analyzing and traversing large breached databases necessitates substantial computational power. However, this research explores techniques to optimize database traversal speed without the need to rent expensive cloud machines or virtual private servers (VPS). This optimization will enable individual security researchers to analyze and process large databases on their personal computer systems while significantly reducing costs.
Shedding Light on the Ageing of Extra Virgin Olive Oil: Probing the Impact of Temperature with Fluorescence Spectroscopy and Machine Learning Techniques
Venue: arXiv cs.LG 2023
Year: 2023

This work systematically investigates the oxidation of extra virgin olive oil (EVOO) under accelerated storage conditions with UV absorption and total fluorescence spectroscopy. With the large amount of data collected, it proposes a method to monitor the oil's quality based on machine learning applied to highly-aggregated data. EVOO is a high-quality vegetable oil that has earned worldwide reputation for its numerous health benefits and excellent taste. Despite its outstanding quality, EVOO degrades over time owing to oxidation, which can affect both its health qualities and flavour. Therefore, it is highly relevant to quantify the effects of oxidation on EVOO and develop methods to assess it that can be easily implemented under field conditions, rather than in specialized laboratories. The following study demonstrates that fluorescence spectroscopy has the capability to monitor the effect of oxidation and assess the quality of EVOO, even when the data are highly aggregated. It shows that complex laboratory equipment is not necessary to exploit fluorescence spectroscopy using the proposed method and that cost-effective solutions, which can be used in-field by non-scientists, could provide an easily-accessible assessment of the quality of EVOO.
Multimodal Deep Learning for Scientific Imaging Interpretation
Venue: arXiv cs.LG 2023
Year: 2023

In the domain of scientific imaging, interpreting visual data often demands an intricate combination of human expertise and deep comprehension of the subject materials. This study presents a novel methodology to linguistically emulate and subsequently evaluate human-like interactions with Scanning Electron Microscopy (SEM) images, specifically of glass materials. Leveraging a multimodal deep learning framework, our approach distills insights from both textual and visual data harvested from peer-reviewed articles, further augmented by the capabilities of GPT-4 for refined data synthesis and evaluation. Despite inherent challenges--such as nuanced interpretations and the limited availability of specialized datasets--our model (GlassLLaVA) excels in crafting accurate interpretations, identifying key features, and detecting defects in previously unseen SEM images. Moreover, we introduce versatile evaluation metrics, suitable for an array of scientific imaging applications, which allows for benchmarking against research-grounded answers. Benefiting from the robustness of contemporary Large Language Models, our model adeptly aligns with insights from research papers. This advancement not only underscores considerable progress in bridging the gap between human and machine interpretation in scientific imaging, but also hints at expansive avenues for future research and broader application.
Mechanizing Session-Types using a Structural View: Enforcing Linearity without Linearity
Venue: arXiv cs.PL 2023
Year: 2023

Session types employ a linear type system that ensures that communication channels cannot be implicitly copied or discarded. As a result, many mechanizations of these systems require modeling channel contexts and carefully ensuring that they treat channels linearly. We demonstrate a technique that localizes linearity conditions as additional predicates embedded within type judgments, which allows us to use structural typing contexts instead of linear ones. This technique is especially relevant when leveraging (weak) higher-order abstract syntax to handle channel mobility and the intricate binding structures that arise in session-typed systems. Following this approach, we mechanize a session-typed system based on classical linear logic and its type preservation proof in the proof assistant Beluga, which uses the logical framework LF as its encoding language. We also prove adequacy for our encoding. This shows the tractability and effectiveness of our approach in modelling substructural systems such as session-typed languages.
SAVME: Efficient Safety Validation for Autonomous Systems Using Meta-Learning
Venue: arXiv cs.RO 2023
Year: 2023

Discovering potential failures of an autonomous system is important prior to deployment. Falsification-based methods are often used to assess the safety of such systems, but the cost of running many accurate simulation can be high. The validation can be accelerated by identifying critical failure scenarios for the system under test and by reducing the simulation runtime. We propose a Bayesian approach that integrates meta-learning strategies with a multi-armed bandit framework. Our method involves learning distributions over scenario parameters that are prone to triggering failures in the system under test, as well as a distribution over fidelity settings that enable fast and accurate simulations. In the spirit of meta-learning, we also assess whether the learned fidelity settings distribution facilitates faster learning of the scenario parameter distributions for new scenarios. We showcase our methodology using a cutting-edge 3D driving simulator, incorporating 16 fidelity settings for an autonomous vehicle stack that includes camera and lidar sensors. We evaluate various scenarios based on an autonomous vehicle pre-crash typology. As a result, our approach achieves a significant speedup, up to 18 times faster compared to traditional methods that solely rely on a high-fidelity simulator.
Knowledge Graph Embedding: An Overview
Venue: arXiv cs.AI 2023
Year: 2023

Many mathematical models have been leveraged to design embeddings for representing Knowledge Graph (KG) entities and relations for link prediction and many downstream tasks. These mathematically-inspired models are not only highly scalable for inference in large KGs, but also have many explainable advantages in modeling different relation patterns that can be validated through both formal proofs and empirical results. In this paper, we make a comprehensive overview of the current state of research in KG completion. In particular, we focus on two main branches of KG embedding (KGE) design: 1) distance-based methods and 2) semantic matching-based methods. We discover the connections between recently proposed models and present an underlying trend that might help researchers invent novel and more effective models. Next, we delve into CompoundE and CompoundE3D, which draw inspiration from 2D and 3D affine operations, respectively. They encompass a broad spectrum of techniques including distance-based and semantic-based methods. We will also discuss an emerging approach for KG completion which leverages pre-trained language models (PLMs) and textual descriptions of entities and relations and offer insights into the integration of KGE embedding methods with PLMs for KG completion.
License Plate Super-Resolution Using Diffusion Models
Venue: arXiv cs.CV 2023
Year: 2023

In surveillance, accurately recognizing license plates is hindered by their often low quality and small dimensions, compromising recognition precision. Despite advancements in AI-based image super-resolution, methods like Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) still fall short in enhancing license plate images. This study leverages the cutting-edge diffusion model, which has consistently outperformed other deep learning techniques in image restoration. By training this model using a curated dataset of Saudi license plates, both in low and high resolutions, we discovered the diffusion model's superior efficacy. The method achieves a 12.55\% and 37.32% improvement in Peak Signal-to-Noise Ratio (PSNR) over SwinIR and ESRGAN, respectively. Moreover, our method surpasses these techniques in terms of Structural Similarity Index (SSIM), registering a 4.89% and 17.66% improvement over SwinIR and ESRGAN, respectively. Furthermore, 92% of human evaluators preferred our images over those from other algorithms. In essence, this research presents a pioneering solution for license plate super-resolution, with tangible potential for surveillance systems.
Confidence Calibration for Systems with Cascaded Predictive Modules
Venue: arXiv cs.LG 2023
Year: 2023

Existing conformal prediction algorithms estimate prediction intervals at target confidence levels to characterize the performance of a regression model on new test samples. However, considering an autonomous system consisting of multiple modules, prediction intervals constructed for individual modules fall short of accommodating uncertainty propagation over different modules and thus cannot provide reliable predictions on system behavior. We address this limitation and present novel solutions based on conformal prediction to provide prediction intervals calibrated for a predictive system consisting of cascaded modules (e.g., an upstream feature extraction module and a downstream regression module). Our key idea is to leverage module-level validation data to characterize the system-level error distribution without direct access to end-to-end validation data. We provide theoretical justification and empirical experimental results to demonstrate the effectiveness of proposed solutions. In comparison to prediction intervals calibrated for individual modules, our solutions generate improved intervals with more accurate performance guarantees for system predictions, which are demonstrated on both synthetic systems and real-world systems performing overlap prediction for indoor navigation using the Matterport3D dataset.
Distributed CSMA/CA MAC Protocol for RIS-Assisted Networks
Venue: arXiv cs.IT 2023
Year: 2023

This paper focuses on achieving optimal multi-user channel access in distributed networks using a reconfigurable intelligent surface (RIS). The network includes wireless channels with direct links between users and RIS links connecting users to the RIS. To maximize average system throughput, an optimal channel access strategy is proposed, considering the trade-off between exploiting spatial diversity gain with RIS assistance and the overhead of channel probing. The paper proposes an optimal distributed Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) strategy with opportunistic RIS assistance, based on statistics theory of optimal sequential observation planned decision. Each source-destination pair makes decisions regarding the use of direct links and/or probing source-RIS-destination links. Channel access occurs in a distributed manner after successful channel contention. The optimality of the strategy is rigorously derived using multiple-level pure thresholds. A distributed algorithm, which achieves significantly lower online complexity at $O(1)$, is developed to implement the proposed strategy. Numerical simulations verify the theoretical results and demonstrate the superior performance compared to existing approaches.