OpenRaiser/Intern-Atlas huggingface dataset:
Precision in Building Extraction: Comparing Shallow and Deep Models using LiDAR Data
Venue: arXiv cs.CV 2023
Year: 2023

Building segmentation is essential in infrastructure development, population management, and geological observations. This article targets shallow models due to their interpretable nature to assess the presence of LiDAR data for supervised segmentation. The benchmark data used in this article are published in NORA MapAI competition for deep learning model. Shallow models are compared with deep learning models based on Intersection over Union (IoU) and Boundary Intersection over Union (BIoU). In the proposed work, boundary masks from the original mask are generated to improve the BIoU score, which relates to building shapes' borderline. The influence of LiDAR data is tested by training the model with only aerial images in task 1 and a combination of aerial and LiDAR data in task 2 and then compared. shallow models outperform deep learning models in IoU by 8% using aerial images (task 1) only and 2% in combined aerial images and LiDAR data (task 2). In contrast, deep learning models show better performance on BIoU scores. Boundary masks improve BIoU scores by 4% in both tasks. Light Gradient-Boosting Machine (LightGBM) performs better than RF and Extreme Gradient Boosting (XGBoost).
Self-Calibrating, Fully Differentiable NLOS Inverse Rendering
Venue: arXiv cs.CV 2023
Year: 2023

Existing time-resolved non-line-of-sight (NLOS) imaging methods reconstruct hidden scenes by inverting the optical paths of indirect illumination measured at visible relay surfaces. These methods are prone to reconstruction artifacts due to inversion ambiguities and capture noise, which are typically mitigated through the manual selection of filtering functions and parameters. We introduce a fully-differentiable end-to-end NLOS inverse rendering pipeline that self-calibrates the imaging parameters during the reconstruction of hidden scenes, using as input only the measured illumination while working both in the time and frequency domains. Our pipeline extracts a geometric representation of the hidden scene from NLOS volumetric intensities and estimates the time-resolved illumination at the relay wall produced by such geometric information using differentiable transient rendering. We then use gradient descent to optimize imaging parameters by minimizing the error between our simulated time-resolved illumination and the measured illumination. Our end-to-end differentiable pipeline couples diffraction-based volumetric NLOS reconstruction with path-space light transport and a simple ray marching technique to extract detailed, dense sets of surface points and normals of hidden scenes. We demonstrate the robustness of our method to consistently reconstruct geometry and albedo, even under significant noise levels.
How-to Guides for Specific Audiences: A Corpus and Initial Findings
Venue: arXiv cs.CL 2023
Year: 2023

Instructional texts for specific target groups should ideally take into account the prior knowledge and needs of the readers in order to guide them efficiently to their desired goals. However, targeting specific groups also carries the risk of reflecting disparate social norms and subtle stereotypes. In this paper, we investigate the extent to which how-to guides from one particular platform, wikiHow, differ in practice depending on the intended audience. We conduct two case studies in which we examine qualitative features of texts written for specific audiences. In a generalization study, we investigate which differences can also be systematically demonstrated using computational methods. The results of our studies show that guides from wikiHow, like other text genres, are subject to subtle biases. We aim to raise awareness of these inequalities as a first step to addressing them in future work.
OSN-MDAD: Machine Translation Dataset for Arabic Multi-Dialectal Conversations on Online Social Media
Venue: arXiv cs.CL 2023
Year: 2023

While resources for English language are fairly sufficient to understand content on social media, similar resources in Arabic are still immature. The main reason that the resources in Arabic are insufficient is that Arabic has many dialects in addition to the standard version (MSA). Arabs do not use MSA in their daily communications; rather, they use dialectal versions. Unfortunately, social users transfer this phenomenon into their use of social media platforms, which in turn has raised an urgent need for building suitable AI models for language-dependent applications. Existing machine translation (MT) systems designed for MSA fail to work well with Arabic dialects. In light of this, it is necessary to adapt to the informal nature of communication on social networks by developing MT systems that can effectively handle the various dialects of Arabic. Unlike for MSA that shows advanced progress in MT systems, little effort has been exerted to utilize Arabic dialects for MT systems. While few attempts have been made to build translation datasets for dialectal Arabic, they are domain dependent and are not OSN cultural-language friendly. In this work, we attempt to alleviate these limitations by proposing an online social network-based multidialect Arabic dataset that is crafted by contextually translating English tweets into four Arabic dialects: Gulf, Yemeni, Iraqi, and Levantine. To perform the translation, we followed our proposed guideline framework for content translation, which could be universally applicable for translation between foreign languages and local dialects. We validated the authenticity of our proposed dataset by developing neural MT models for four Arabic dialects. Our results have shown a superior performance of our NMT models trained using our dataset. We believe that our dataset can reliably serve as an Arabic multidialectal translation dataset for informal MT tasks.
Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features
Venue: arXiv cs.CV 2023
Year: 2023

The rapid development of 3D object detection systems for self-driving cars has significantly improved accuracy. However, these systems struggle to generalize across diverse driving environments, which can lead to safety-critical failures in detecting traffic participants. To address this, we propose a method that utilizes unlabeled repeated traversals of multiple locations to adapt object detectors to new driving environments. By incorporating statistics computed from repeated LiDAR scans, we guide the adaptation process effectively. Our approach enhances LiDAR-based detection models using spatial quantized historical features and introduces a lightweight regression head to leverage the statistics for feature regularization. Additionally, we leverage the statistics for a novel self-training process to stabilize the training. The framework is detector model-agnostic and experiments on real-world datasets demonstrate significant improvements, achieving up to a 20-point performance gain, especially in detecting pedestrians and distant objects. Code is available at https://github.com/zhangtravis/Hist-DA.
Information Forensics and Security: A quarter-century-long journey
Venue: arXiv cs.CR 2023
Year: 2023

Information Forensics and Security (IFS) is an active R&D area whose goal is to ensure that people use devices, data, and intellectual properties for authorized purposes and to facilitate the gathering of solid evidence to hold perpetrators accountable. For over a quarter century since the 1990s, the IFS research area has grown tremendously to address the societal needs of the digital information era. The IEEE Signal Processing Society (SPS) has emerged as an important hub and leader in this area, and the article below celebrates some landmark technical contributions. In particular, we highlight the major technological advances on some selected focus areas in the field developed in the last 25 years from the research community and present future trends.
BOMs Away! Inside the Minds of Stakeholders: A Comprehensive Study of Bills of Materials for Software Systems
Venue: arXiv cs.SE 2023
Year: 2023

Software Bills of Materials (SBOMs) have emerged as tools to facilitate the management of software dependencies, vulnerabilities, licenses, and the supply chain. While significant effort has been devoted to increasing SBOM awareness and developing SBOM formats and tools, recent studies have shown that SBOMs are still an early technology not yet adequately adopted in practice. Expanding on previous research, this paper reports a comprehensive study that investigates the current challenges stakeholders encounter when creating and using SBOMs. The study surveyed 138 practitioners belonging to five stakeholder groups (practitioners familiar with SBOMs, members of critical open source projects, AI/ML, cyber-physical systems, and legal practitioners) using differentiated questionnaires, and interviewed 8 survey respondents to gather further insights about their experience. We identified 12 major challenges facing the creation and use of SBOMs, including those related to the SBOM content, deficiencies in SBOM tools, SBOM maintenance and verification, and domain-specific challenges. We propose and discuss 4 actionable solutions to the identified challenges and present the major avenues for future research and development.
Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognition
Venue: arXiv cs.CL 2023
Year: 2023

In this study, we present synchronous bilingual Connectionist Temporal Classification (CTC), an innovative framework that leverages dual CTC to bridge the gaps of both modality and language in the speech translation (ST) task. Utilizing transcript and translation as concurrent objectives for CTC, our model bridges the gap between audio and text as well as between source and target languages. Building upon the recent advances in CTC application, we develop an enhanced variant, BiL-CTC+, that establishes new state-of-the-art performances on the MuST-C ST benchmarks under resource-constrained scenarios. Intriguingly, our method also yields significant improvements in speech recognition performance, revealing the effect of cross-lingual learning on transcription and demonstrating its broad applicability. The source code is available at https://github.com/xuchennlp/S2T.
Weakly-supervised Automated Audio Captioning via text only training
Venue: arXiv cs.SD 2023
Year: 2023

In recent years, datasets of paired audio and captions have enabled remarkable success in automatically generating descriptions for audio clips, namely Automated Audio Captioning (AAC). However, it is labor-intensive and time-consuming to collect a sufficient number of paired audio and captions. Motivated by the recent advances in Contrastive Language-Audio Pretraining (CLAP), we propose a weakly-supervised approach to train an AAC model assuming only text data and a pre-trained CLAP model, alleviating the need for paired target data. Our approach leverages the similarity between audio and text embeddings in CLAP. During training, we learn to reconstruct the text from the CLAP text embedding, and during inference, we decode using the audio embeddings. To mitigate the modality gap between the audio and text embeddings we employ strategies to bridge the gap during training and inference stages. We evaluate our proposed method on Clotho and AudioCaps datasets demonstrating its ability to achieve a relative performance of up to ~$83\%$ compared to fully supervised approaches trained with paired target data.
SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative References
Venue: arXiv cs.CL 2023
Year: 2023

Evaluation of QA systems is very challenging and expensive, with the most reliable approach being human annotations of correctness of answers for questions. Recent works (AVA, BEM) have shown that transformer LM encoder based similarity metrics transfer well for QA evaluation, but they are limited by the usage of a single correct reference answer. We propose a new evaluation metric: SQuArE (Sentence-level QUestion AnsweRing Evaluation), using multiple reference answers (combining multiple correct and incorrect references) for sentence-form QA. We evaluate SQuArE on both sentence-level extractive (Answer Selection) and generative (GenQA) QA systems, across multiple academic and industrial datasets, and show that it outperforms previous baselines and obtains the highest correlation with human annotations.
SALSA-CLRS: A Sparse and Scalable Benchmark for Algorithmic Reasoning
Venue: arXiv cs.LG 2023
Year: 2023

We introduce an extension to the CLRS algorithmic learning benchmark, prioritizing scalability and the utilization of sparse representations. Many algorithms in CLRS require global memory or information exchange, mirrored in its execution model, which constructs fully connected (not sparse) graphs based on the underlying problem. Despite CLRS's aim of assessing how effectively learned algorithms can generalize to larger instances, the existing execution model becomes a significant constraint due to its demanding memory requirements and runtime (hard to scale). However, many important algorithms do not demand a fully connected graph; these algorithms, primarily distributed in nature, align closely with the message-passing paradigm employed by Graph Neural Networks. Hence, we propose SALSA-CLRS, an extension of the current CLRS benchmark specifically with scalability and sparseness in mind. Our approach includes adapted algorithms from the original CLRS benchmark and introduces new problems from distributed and randomized algorithms. Moreover, we perform a thorough empirical evaluation of our benchmark. Code is publicly available at https://github.com/jkminder/SALSA-CLRS.
Inspire the Large Language Model by External Knowledge on BioMedical Named Entity Recognition
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) have demonstrated dominating performance in many NLP tasks, especially on generative tasks. However, they often fall short in some information extraction tasks, particularly those requiring domain-specific knowledge, such as Biomedical Named Entity Recognition (NER). In this paper, inspired by Chain-of-thought, we leverage the LLM to solve the Biomedical NER step-by-step: break down the NER task into entity span extraction and entity type determination. Additionally, for entity type determination, we inject entity knowledge to address the problem that LLM's lack of domain knowledge when predicting entity category. Experimental results show a significant improvement in our two-step BioNER approach compared to previous few-shot LLM baseline. Additionally, the incorporation of external knowledge significantly enhances entity category determination performance.
SIAK-NG User Interface Design with Design Thinking Method to Support System Integration
Venue: arXiv cs.HC 2023
Year: 2023

This research aims to improve the user interface of SIAK-NG (Next Generation Academic Information System), the academic portal website of the University of Indonesia through a design thinking approach. Despite being in existence for several decades and undergoing multiple improvements, there has been no specific effort to evaluate the quality of the user interface. In line with the revamping of SIAK-NG according to the University of Indonesia 2019-2024 master plan, this study aims to provide the necessary insights. The research focuses on redesigning the website interface of SIAK-NG to address user complaints and difficulties, particularly related to the unsatisfactory interface design. The Design Thinking approach is employed to generate solutions that meet the needs of active University of Indonesia students who are the primary users. Through methods such as storyboarding, empathy mapping, usability testing, and others, the author will design recommendations that align with user requirements.
Drone Flight Path Architecture
Venue: arXiv cs.HC 2023
Year: 2023

This project was built from a pre-existing architecture that facilitates the planning and automatic execution of drone routes in a known space through a 3D virtual reality environment. Our work consisted in extending this architecture by integrating a new web component, making use of a 3D map API, to facilitate to people who do not have access to virtual reality hardware, the possibility of planning flight routes that have as parameters the <latitude, longitude> with respect to the globe and also a component in meters that represents the height at which the drone rises in a certain point. Additionally, the configuration possibilities of a route were extended in order to take advantage of one of the components that gives more value and potential to unmanned aircrafts: the use of the camera in multiple contexts and scenarios. The extension of this solution allows the user to assign different camera tasks along the route, see in real time what the camera is capturing and, after the flight, retrieve the multimedia content that was created
A Case for AI Safety via Law
Venue: arXiv cs.CY 2023
Year: 2023

How to make artificial intelligence (AI) systems safe and aligned with human values is an open research question. Proposed solutions tend toward relying on human intervention in uncertain situations, learning human values and intentions through training or observation, providing off-switches, implementing isolation or simulation environments, or extrapolating what people would want if they had more knowledge and more time to think. Law-based approaches--such as inspired by Isaac Asimov--have not been well regarded. This paper makes a case that effective legal systems are the best way to address AI safety. Law is defined as any rules that codify prohibitions and prescriptions applicable to particular agents in specified domains/contexts and includes processes for enacting, managing, enforcing, and litigating such rules.
The Impact of Live Polling Quizzes on Student Engagement and Performance in Computer Science Lectures
Venue: arXiv cs.CY 2023
Year: 2023

Prior to the COVID-19 pandemic, the adoption of live polling and real-time feedback tools gained traction in higher education to enhance student engagement and learning outcomes. Integrating live polling activities has been shown to boost attention, participation, and understanding of course materials. However, recent changes in learning behaviours due to the pandemic necessitate a reevaluation of these active learning technologies. In this context, our study focuses on the Computer Science (CS) domain, investigating the impact of Live Polling Quizzes (LPQs) in undergraduate CS lectures. These quizzes comprise fact-based, formally defined questions with clear answers, aiming to enhance engagement, learning outcomes, and overall perceptions of the course module. A survey was conducted among 70 undergraduate CS students, attending CS modules with and without LPQs. The results revealed that while LPQs contributed to higher attendance, other factors likely influenced attendance rates more significantly. LPQs were generally viewed positively, aiding comprehension and maintaining student attention and motivation. However, careful management of LPQ frequency is crucial to prevent overuse for some students and potential reduced motivation. Clear instructions for using the polling software were also highlighted as essential.
ActiveAI: Introducing AI Literacy for Middle School Learners with Goal-based Scenario Learning
Venue: arXiv cs.CY 2023
Year: 2023

The ActiveAI project addresses key challenges in AI education for grades 7-9 students by providing an engaging AI literacy learning experience based on the AI4K12 knowledge framework. Utilizing learning science mechanisms such as goal-based scenarios, immediate feedback, project-based learning, and intelligent agents, the app incorporates a variety of learner inputs like sliders, steppers, and collectors to enhance understanding. In these courses, students work on real-world scenarios like analyzing sentiment in social media comments. This helps them learn to effectively engage with AI systems and develop their ability to evaluate AI-generated output. The Learning Engineering Process (LEP) guided the project's creation and data instrumentation, focusing on design and impact. The project is currently in the implementation stage, leveraging the intelligent tutor design principles for app development. The extended abstract presents the foundational design and development, with further evaluation and research to be conducted in the future.
Exploring IoT in Smart Cities: Practices, Challenges and Way Forward
Venue: arXiv cs.CY 2023
Year: 2023

The rise of Internet of things (IoT) technology has revolutionized urban living, offering immense potential for smart cities in which smart home, smart infrastructure, and smart industry are essential aspects that contribute to the development of intelligent urban ecosystems. The integration of smart home technology raises concerns regarding data privacy and security, while smart infrastructure implementation demands robust networking and interoperability solutions. Simultaneously, deploying IoT in industrial settings faces challenges related to scalability, standardization, and data management. This research paper offers a systematic literature review of published research in the field of IoT in smart cities including 55 relevant primary studies that have been published in reputable journals and conferences. This extensive literature review explores and evaluates various aspects of smart home, smart infrastructure, and smart industry and the challenges like security and privacy, smart sensors, interoperability and standardization. We provide a unified perspective, as we seek to enhance the efficiency and effectiveness of smart cities while overcoming security concerns. It then explores their potential for collective integration and impact on the development of smart cities. Furthermore, this study addresses the challenges associated with each component individually and explores their combined impact on enhancing urban efficiency and sustainability. Through a comprehensive analysis of security concerns, this research successfully integrates these IoT components in a unified approach, presenting a holistic framework for building smart cities of the future. Integrating smart home, smart infrastructure, and smart industry, this research highlights the significance of an integrated approach in developing smart cities.
Transitioning a Project-Based Course between Onsite and Online. An Experience Report
Venue: arXiv cs.CY 2023
Year: 2023

We present an investigation regarding the challenges faced by student teams across four consecutive iterations of a team-focused, project-based course in software engineering. The studied period includes the switch to fully online activities in the spring of 2020, and covers the return to face-to-face teaching two years later. We cover the feedback provided by over 1,500 students, collected in a free-text form on the basis of a survey. A qualitative research method was utilized to discern and examine the challenges and perceived benefits of a course that was conducted entirely online. We show that technical challenges remain a constant in project-based courses, with time management being the most affected by the move to online. Students reported that the effective use of collaborative tools eased team organization and communication while online. We conclude by providing a number of action points regarding the integration of online activities in face-to-face course unfolding related to project management, communication tools, the importance of teamwork, and of active mentor participation.