OpenRaiser/Intern-Atlas huggingface dataset:
Exploring the Effect of Multiple Natural Languages on Code Suggestion Using GitHub Copilot
Venue: arXiv cs.SE 2024
Year: 2024

GitHub Copilot is an AI-enabled tool that automates program synthesis. It has gained significant attention since its launch in 2021. Recent studies have extensively examined Copilot's capabilities in various programming tasks, as well as its security issues. However, little is known about the effect of different natural languages on code suggestion. Natural language is considered a social bias in the field of NLP, and this bias could impact the diversity of software engineering. To address this gap, we conducted an empirical study to investigate the effect of three popular natural languages (English, Japanese, and Chinese) on Copilot. We used 756 questions of varying difficulty levels from AtCoder contests for evaluation purposes. The results highlight that the capability varies across natural languages, with Chinese achieving the worst performance. Furthermore, regardless of the type of natural language, the performance decreases significantly as the difficulty of questions increases. Our work represents the initial step in comprehending the significance of natural languages in Copilot's capability and introduces promising opportunities for future endeavors.
GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting
Venue: arXiv cs.CV 2024
Year: 2024

Gaussian Splatting (GS) is a novel, state-of-the-art technique for rendering points in a 3D scene by approximating their contribution to image pixels through Gaussian distributions, warranting fast training and real-time rendering. The main drawback of GS is the absence of a well-defined approach for its conditioning due to the necessity of conditioning several hundred thousand Gaussian components. To solve this, we introduce the Gaussian Mesh Splatting (GaMeS) model, which allows modification of Gaussian components in a similar way as meshes. We parameterize each Gaussian component by the vertices of the mesh face. Furthermore, our model needs mesh initialization on input or estimated mesh during training. We also define Gaussian splats solely based on their location on the mesh, allowing for automatic adjustments in position, scale, and rotation during animation. As a result, we obtain a real-time rendering of editable GS.
Visual Gyroscope: Combination of Deep Learning Features and Direct Alignment for Panoramic Stabilization
Venue: arXiv cs.CV 2024
Year: 2024

In this article we present a visual gyroscope based on equirectangular panoramas. We propose a new pipeline where we take advantage of combining three different methods to obtain a robust and accurate estimation of the attitude of the camera. We quantitatively and qualitatively validate our method on two image sequences taken with a $360^\circ$ dual-fisheye camera mounted on different aerial vehicles.
A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation
Venue: arXiv cs.CL 2024
Year: 2024

Generating natural language text from graph-structured data is essential for conversational information seeking. Semantic triples derived from knowledge graphs can serve as a valuable source for grounding responses from conversational agents by providing a factual basis for the information they communicate. This is especially relevant in the context of large language models, which offer great potential for conversational interaction but are prone to hallucinating, omitting, or producing conflicting information. In this study, we conduct an empirical analysis of conversational large language models in generating natural language text from semantic triples. We compare four large language models of varying sizes with different prompting techniques. Through a series of benchmark experiments on the WebNLG dataset, we analyze the models' performance and identify the most common issues in the generated predictions. Our findings show that the capabilities of large language models in triple verbalization can be significantly improved through few-shot prompting, post-processing, and efficient fine-tuning techniques, particularly for smaller models that exhibit lower zero-shot performance.
Developing and Evaluating a Design Method for Positive Artificial Intelligence
Venue: arXiv cs.AI 2024
Year: 2024

As artificial intelligence (AI) continues advancing, ensuring positive societal impacts becomes critical, especially as AI systems become increasingly ubiquitous in various aspects of life. However, developing "AI for good" poses substantial challenges around aligning systems with complex human values. Presently, we lack mature methods for addressing these challenges. This article presents and evaluates the Positive AI design method aimed at addressing this gap. The method provides a human-centered process to translate wellbeing aspirations into concrete practices. First, we explain the method's four key steps: contextualizing, operationalizing, optimizing, and implementing wellbeing supported by continuous measurement for feedback cycles. We then present a multiple case study where novice designers applied the method, revealing strengths and weaknesses related to efficacy and usability. Next, an expert evaluation study assessed the quality of the resulting concepts, rating them moderately high for feasibility, desirability, and plausibility of achieving intended wellbeing benefits. Together, these studies provide preliminary validation of the method's ability to improve AI design, while surfacing areas needing refinement like developing support for complex steps. Proposed adaptations such as examples and evaluation heuristics could address weaknesses. Further research should examine sustained application over multiple projects. This human-centered approach shows promise for realizing the vision of 'AI for Wellbeing' that does not just avoid harm, but actively benefits humanity.
Code-Switched Language Identification is Harder Than You Think
Venue: arXiv cs.CL 2024
Year: 2024

Code switching (CS) is a very common phenomenon in written and spoken communication but one that is handled poorly by many natural language processing applications. Looking to the application of building CS corpora, we explore CS language identification (LID) for corpus building. We make the task more realistic by scaling it to more languages and considering models with simpler architectures for faster inference. We also reformulate the task as a sentence-level multi-label tagging problem to make it more tractable. Having defined the task, we investigate three reasonable models for this task and define metrics which better reflect desired performance. We present empirical evidence that no current approach is adequate and finally provide recommendations for future work in this area.
Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence
Venue: arXiv cs.LG 2024
Year: 2024

Based on SGD, previous works have proposed many algorithms that have improved convergence speed and generalization in stochastic optimization, such as SGDm, AdaGrad, Adam, etc. However, their convergence analysis under non-convex conditions is challenging. In this work, we propose a unified framework to address this issue. For any first-order methods, we interpret the updated direction $g_t$ as the sum of the stochastic subgradient $\nabla f_t(x_t)$ and an additional acceleration term $\frac{2|\langle v_t, \nabla f_t(x_t) \rangle|}{\|v_t\|_2^2} v_t$, thus we can discuss the convergence by analyzing $\langle v_t, \nabla f_t(x_t) \rangle$. Through our framework, we have discovered two plug-and-play acceleration methods: \textbf{Reject Accelerating} and \textbf{Random Vector Accelerating}, we theoretically demonstrate that these two methods can directly lead to an improvement in convergence rate.
K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning
Venue: arXiv cs.CL 2024
Year: 2024

Strategic reasoning is a complex yet essential capability for intelligent agents. It requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments. Unlike static reasoning tasks, success in these contexts depends on anticipating other agents' beliefs and actions while continuously adjusting strategies to achieve individual goals. LLMs and LLM agents often struggle with strategic reasoning due to the absence of a reasoning framework that enables them to dynamically infer others' perspectives and adapt to changing environments. Inspired by the Level-K framework from game theory and behavioral economics, which extends reasoning from simple reactions to structured strategic depth, we propose a novel framework: "K-Level Reasoning with Large Language Models (K-R)." This framework employs recursive mechanisms to enable LLMs to achieve varying levels of strategic depth, allowing agents to form higher order beliefs - beliefs about others' beliefs. We validate this framework through rigorous testing on four testbeds: two classical game theory problems and two social intelligence tasks. The results demonstrate the advantages of K-R in strategic reasoning. Our work presents the first recursive implementation of strategic depth in large language models (LLMs). It establishes a foundation for future research into theory of mind and strategic reasoning in LLMs.
Mapping acceptance: micro scenarios as a dual-perspective approach for assessing public opinion and individual differences in technology perception
Venue: arXiv cs.CY 2024
Year: 2024

Understanding public perception of technology is crucial to aligning research, development, and governance of technology. This article introduces micro scenarios as an integrative method to evaluate mental models and social acceptance across numerous technologies and concepts using a few single-item scales within a single comprehensive survey. This approach contrasts with traditional methods that focus on detailed assessments of as few as one scenario. The data can be interpreted in two ways: Perspective (1): Average evaluations of each participant can be seen as individual differences, providing reflexive measurements across technologies or topics. This helps in understanding how perceptions of technology relate to other personality factors. Perspective (2): Average evaluations of each technology or topic can be interpreted as technology attributions. This makes it possible to position technologies on visuo-spatial maps to simplify identification of critical issues, conduct comparative rankings based on selected criteria, and to analyze the interplay between different attributions. This dual approach enables the modeling of acceptance-relevant factors that shape public opinion. It offers a framework for researchers, technology developers, and policymakers to identify pivotal factors for acceptance at both the individual and technology levels. I illustrate this methodology with examples from my research, provide practical guidelines, and include R code to enable others to conduct similar studies. This paper aims to bridge the gap between technological advancement and societal perception, offering a tool for more informed decision-making in technology development and policy-making.
Generative AI for Education (GAIED): Advances, Opportunities, and Challenges
Venue: arXiv cs.CY 2024
Year: 2024

This survey article has grown out of the GAIED (pronounced "guide") workshop organized by the authors at the NeurIPS 2023 conference. We organized the GAIED workshop as part of a community-building effort to bring together researchers, educators, and practitioners to explore the potential of generative AI for enhancing education. This article aims to provide an overview of the workshop activities and highlight several future research directions in the area of GAIED.
Integration of LaTeX formula in computer-based test application for academic purposes
Venue: arXiv cs.CY 2024
Year: 2024

LaTeX is a free document preparation system that handles the typesetting of mathematical expressions smoothly and elegantly. It has become the standard format for creating and publishing research articles in mathematics and many scientific fields. Computer-based testing (CBT) has become widespread in recent years. Most establishments now use it to deliver assessments as an alternative to using the pen-paper method. To deliver an assessment, the examiner would first add a new exam or edit an existing exam using a CBT editor. Thus, the implementation of CBT should comprise both support for setting and administering questions. Existing CBT applications used in the academic space lacks the capacity to handle advanced formulas, programming codes, and tables, thereby resorting to converting them into images which takes a lot of time and storage space. In this paper, we discuss how we solvde this problem by integrating latex technology into our CBT applications. This enables seamless manipulation and accurate rendering of tables, programming codes, and equations to increase readability and clarity on both the setting and administering of questions platforms. Furthermore, this implementation has reduced drastically the sizes of system resources allocated to converting tables, codes, and equations to images. Those in mathematics, statistics, computer science, engineering, chemistry, etc. will find this application useful.
Nie pozwól algorytmom rządzić Twoim koszykiem: systemy rekomendacyjne w dobie Omnibusa
Venue: arXiv cs.CY 2024
Year: 2024

The Omnibus Directive is an essential part of the European Union's New Deal for Consumers. The Directive introduces new regulations in trade, including e-commerce, with the main goal being to increase transparency, fairness and consumer protection. The authors critically draw attention to a significant oversight in the Omnibus Directive, namely the lack of consideration of recommendation systems. Recommendation engines can be a source of potentially harmful practices affecting consumers, hence the need for a directive extension. The proposals presented in this article include the introduction of ethical supervision over recommendation systems to minimize the risk of negative effects of their recommendations, as well as a clear explanation of the criteria on which recommendations are made -- similar to search result rankings. -- Dyrektywa Omnibus stanowi istotną część Nowego Ładu dla Konsumentów (ang. \emph{New Deal for Consumers}) Unii Europejskiej. Dyrektywa wprowadza nowe regulacje w handlu, w tym e-commerce, których głównym celem jest zwiększenie przejrzystości, uczciwości i ochrony konsumentów. Autorzy krytycznie zwracają uwagę na istotne zaniedbanie w dyrektywie Omnibus, jakim jest brak uwzględnienia systemów rekomendacyjnych. Silniki rekomendacyjne mogą być źródłem potencjalnie szkodliwych praktyk uderzających w konsumentów, stąd niezbędne jest rozszerzenie dyrektywy. Propozycje przedstawione w niniejszym artykule obejmują wprowadzenie etycznego nadzoru nad systemami rekomendującymi, aby zminimalizować ryzyko negatywnych skutków ich rekomendacji, a także jasne wyjaśnienie kryteriów, na podstawie których dokonywane są rekomendacje -- analogicznie do rankingów wyników wyszukiwania.
MULTIVERSE: Exposing Large Language Model Alignment Problems in Diverse Worlds
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Model (LLM) alignment aims to ensure that LLM outputs match with human values. Researchers have demonstrated the severity of alignment problems with a large spectrum of jailbreak techniques that can induce LLMs to produce malicious content during conversations. Finding the corresponding jailbreaking prompts usually requires substantial human intelligence or computation resources. In this paper, we report that LLMs have different levels of alignment in various contexts. As such, by systematically constructing many contexts, called worlds, leveraging a Domain Specific Language describing possible worlds (e.g., time, location, characters, actions and languages) and the corresponding compiler, we can cost-effectively expose latent alignment issues. Given the low cost of our method, we are able to conduct a large scale study regarding LLM alignment issues in different worlds. Our results show that our method outperforms the-state-of-the-art jailbreaking techniques on both effectiveness and efficiency. In addition, our results indicate that existing LLMs are extremely vulnerable to nesting worlds and programming language worlds. They imply that existing alignment training focuses on the real-world and is lacking in various (virtual) worlds where LLMs can be exploited.
Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Data
Venue: arXiv cs.CL 2024
Year: 2024

The inherent complexity of structured longitudinal Electronic Health Records (EHR) data poses a significant challenge when integrated with Large Language Models (LLMs), which are traditionally tailored for natural language processing. Motivated by the urgent need for swift decision-making during new disease outbreaks, where traditional predictive models often fail due to a lack of historical data, this research investigates the adaptability of LLMs, like GPT-4, to EHR data. We particularly focus on their zero-shot capabilities, which enable them to make predictions in scenarios in which they haven't been explicitly trained. In response to the longitudinal, sparse, and knowledge-infused nature of EHR data, our prompting approach involves taking into account specific EHR characteristics such as units and reference ranges, and employing an in-context learning strategy that aligns with clinical contexts. Our comprehensive experiments on the MIMIC-IV and TJH datasets demonstrate that with our elaborately designed prompting framework, LLMs can improve prediction performance in key tasks such as mortality, length-of-stay, and 30-day readmission by about 35\%, surpassing ML models in few-shot settings. Our research underscores the potential of LLMs in enhancing clinical decision-making, especially in urgent healthcare situations like the outbreak of emerging diseases with no labeled data. The code is publicly available at https://github.com/yhzhu99/llm4healthcare for reproducibility.
TrICy: Trigger-guided Data-to-text Generation with Intent aware Attention-Copy
Venue: arXiv cs.CL 2024
Year: 2024

Data-to-text (D2T) generation is a crucial task in many natural language understanding (NLU) applications and forms the foundation of task-oriented dialog systems. In the context of conversational AI solutions that can work directly with local data on the user's device, architectures utilizing large pre-trained language models (PLMs) are impractical for on-device deployment due to a high memory footprint. To this end, we propose TrICy, a novel lightweight framework for an enhanced D2T task that generates text sequences based on the intent in context and may further be guided by user-provided triggers. We leverage an attention-copy mechanism to predict out-of-vocabulary (OOV) words accurately. Performance analyses on E2E NLG dataset (BLEU: 66.43%, ROUGE-L: 70.14%), WebNLG dataset (BLEU: Seen 64.08%, Unseen 52.35%), and our Custom dataset related to text messaging applications, showcase our architecture's effectiveness. Moreover, we show that by leveraging an optional trigger input, data-to-text generation quality increases significantly and achieves the new SOTA score of 69.29% BLEU for E2E NLG. Furthermore, our analyses show that TrICy achieves at least 24% and 3% improvement in BLEU and METEOR respectively over LLMs like GPT-3, ChatGPT, and Llama 2. We also demonstrate that in some scenarios, performance improvement due to triggers is observed even when they are absent in training.
Bloom-epistemic and sentiment analysis hierarchical classification in course discussion forums
Venue: arXiv cs.CY 2024
Year: 2024

Online discussion forums are widely used for active textual interaction between lecturers and students, and to see how the students have progressed in a learning process. The objective of this study is to compare appropriate machine-learning models to assess sentiments and Bloomś epistemic taxonomy based on textual comments in educational discussion forums. Our proposed method is called the hierarchical approach of Bloom-Epistemic and Sentiment Analysis (BE-Sent). The research methodology consists of three main steps. The first step is the data collection from the internal discussion forum and YouTube comments of a Web Programming channel. The next step is text preprocessing to annotate the text and clear unimportant words. Furthermore, with the text dataset that has been successfully cleaned, sentiment analysis and epistemic categorization will be done in each sentence of the text. Sentiment analysis is divided into three categories: positive, negative, and neutral. Bloomś epistemic is divided into six categories: remembering, understanding, applying, analyzing, evaluating, and creating. This research has succeeded in producing a course learning subsystem that assesses opinions based on text reviews of discussion forums according to the category of sentiment and epistemic analysis.
Business Models for Digitalization Enabled Energy Efficiency and Flexibility in Industry: A Survey with Nine Case Studies
Venue: arXiv cs.CY 2024
Year: 2024

Digitalization is challenging in heavy industrial sectors, and many pi-lot projects facing difficulties to be replicated and scaled. Case studies are strong pedagogical vehicles for learning and sharing experience & knowledge, but rarely available in the literature. Therefore, this paper conducts a survey to gather a diverse set of nine industry cases, which are subsequently subjected to analysis using the business model canvas (BMC). The cases are summarized and compared based on nine BMC components, and a Value of Business Model (VBM) evaluation index is proposed to assess the business potential of industrial digital solutions. The results show that the main partners are industry stakeholders, IT companies and academic institutes. Their key activities for digital solutions include big-data analysis, machine learning algorithms, digital twins, and internet of things developments. The value propositions of most cases are improving energy efficiency and enabling energy flexibility. Moreover, the technology readiness levels of six industrial digital solutions are under level 7, indicating that they need further validation in real-world environments. Building upon these insights, this paper proposes six recommendations for future industrial digital solution development: fostering cross-sector collaboration, prioritizing comprehensive testing and validation, extending value propositions, enhancing product adaptability, providing user-friendly platforms, and adopting transparent recommendations.