OpenRaiser/Intern-Atlas huggingface dataset:
Dynamic Interval Restrictions on Action Spaces in Deep Reinforcement Learning for Obstacle Avoidance
Venue: arXiv cs.LG 2023
Year: 2023

Deep reinforcement learning algorithms typically act on the same set of actions. However, this is not sufficient for a wide range of real-world applications where different subsets are available at each step. In this thesis, we consider the problem of interval restrictions as they occur in pathfinding with dynamic obstacles. When actions that lead to collisions are avoided, the continuous action space is split into variable parts. Recent research learns with strong assumptions on the number of intervals, is limited to convex subsets, and the available actions are learned from the observations. Therefore, we propose two approaches that are independent of the state of the environment by extending parameterized reinforcement learning and ConstraintNet to handle an arbitrary number of intervals. We demonstrate their performance in an obstacle avoidance task and compare the methods to penalties, projection, replacement, as well as discrete and continuous masking from the literature. The results suggest that discrete masking of action-values is the only effective method when constraints did not emerge during training. When restrictions are learned, the decision between projection, masking, and our ConstraintNet modification seems to depend on the task at hand. We compare the results with varying complexity and give directions for future work.
Tune As You Scale: Hyperparameter Optimization For Compute Efficient Training
Venue: arXiv cs.LG 2023
Year: 2023

Hyperparameter tuning of deep learning models can lead to order-of-magnitude performance gains for the same amount of compute. Despite this, systematic tuning is uncommon, particularly for large models, which are expensive to evaluate and tend to have many hyperparameters, necessitating difficult judgment calls about tradeoffs, budgets, and search bounds. To address these issues and propose a practical method for robustly tuning large models, we present Cost-Aware Pareto Region Bayesian Search (CARBS), a Bayesian optimization algorithm that performs local search around the performance-cost Pareto frontier. CARBS does well even in unbounded search spaces with many hyperparameters, learns scaling relationships so that it can tune models even as they are scaled up, and automates much of the "black magic" of tuning. Among our results, we effectively solve the entire ProcGen benchmark just by tuning a simple baseline (PPO, as provided in the original ProcGen paper). We also reproduce the model size vs. training tokens scaling result from the Chinchilla project (Hoffmann et al. 2022), while simultaneously discovering scaling laws for every other hyperparameter, via an easy automated process that uses significantly less compute and is applicable to any deep learning problem (not just language models).
Symbolic Regression via Control Variable Genetic Programming
Venue: arXiv cs.NE 2023
Year: 2023

Learning symbolic expressions directly from experiment data is a vital step in AI-driven scientific discovery. Nevertheless, state-of-the-art approaches are limited to learning simple expressions. Regressing expressions involving many independent variables still remain out of reach. Motivated by the control variable experiments widely utilized in science, we propose Control Variable Genetic Programming (CVGP) for symbolic regression over many independent variables. CVGP expedites symbolic expression discovery via customized experiment design, rather than learning from a fixed dataset collected a priori. CVGP starts by fitting simple expressions involving a small set of independent variables using genetic programming, under controlled experiments where other variables are held as constants. It then extends expressions learned in previous generations by adding new independent variables, using new control variable experiments in which these variables are allowed to vary. Theoretically, we show CVGP as an incremental building approach can yield an exponential reduction in the search space when learning a class of expressions. Experimentally, CVGP outperforms several baselines in learning symbolic expressions involving multiple independent variables.
Soft Soil Gait Planning and Control for Biped Robot using Deep Deterministic Policy Gradient Approach
Venue: arXiv cs.RO 2023
Year: 2023

Biped robots have plenty of benefits over wheeled, quadruped, or hexapod robots due to their ability to behave like human beings in tough and non-flat environments. Deformable terrain is another challenge for biped robots as it has to deal with sinkage and maintain stability without falling. In this study, we are proposing a Deep Deterministic Policy Gradient (DDPG) approach for motion control of a flat-foot biped robot walking on deformable terrain. We have considered a 7-link biped robot for our proposed approach. For soft soil terrain modeling, we have considered triangular Mesh to describe its geometry, where mesh parameters determine the softness of soil. All simulations have been performed on PyChrono, which can handle soft soil environments.
Can ChatGPT Enable ITS? The Case of Mixed Traffic Control via Reinforcement Learning
Venue: arXiv cs.AI 2023
Year: 2023

The surge in Reinforcement Learning (RL) applications in Intelligent Transportation Systems (ITS) has contributed to its growth as well as highlighted key challenges. However, defining objectives of RL agents in traffic control and management tasks, as well as aligning policies with these goals through an effective formulation of Markov Decision Process (MDP), can be challenging and often require domain experts in both RL and ITS. Recent advancements in Large Language Models (LLMs) such as GPT-4 highlight their broad general knowledge, reasoning capabilities, and commonsense priors across various domains. In this work, we conduct a large-scale user study involving 70 participants to investigate whether novices can leverage ChatGPT to solve complex mixed traffic control problems. Three environments are tested, including ring road, bottleneck, and intersection. We find ChatGPT has mixed results. For intersection and bottleneck, ChatGPT increases number of successful policies by 150% and 136% compared to solely beginner capabilities, with some of them even outperforming experts. However, ChatGPT does not provide consistent improvements across all scenarios.
Ergonomic-Centric Holography: Optimizing Realism,Immersion, and Comfort for Holographic Display
Venue: arXiv cs.GR 2023
Year: 2023

We introduce ergonomic-centric holography, an algorithmic framework that simultaneously optimizes for realistic incoherent defocus, unrestricted pupil movements in the eye box, and high-order diffractions for filtering-free holography. The proposed method outperforms prior algorithms on holographic display prototypes operating in unfiltered and pupil-mimicking modes, offering the potential to enhance next-generation virtual and augmented reality experiences.
The interplay of fatigue dynamics and task achievement using optimal control predictive simulation
Venue: arXiv cs.RO 2023
Year: 2023

Predictive simulation of human motion could provide insight into optimal techniques. In repetitive or long-duration tasks, these simulations must predict fatigue-induced adaptation. However, most studies minimize cost function terms related to actuator activations, assuming it minimizes fatigue. An additional modeling layer is needed to consider the previous use of muscles to reveal adaptive strategies to the decreased force production capability. Here, we propose interfacing Xia's three-compartment fatigue dynamics model with rigid-body dynamics. A stabilization invariant was added to Xia's model. We simulated the maximum repetition of dumbbell biceps curls as an optimal control problem (OCP) using direct multiple shooting. We explored three cost functions (minimizing minimum torque, fatigue, or both) and two OCP formulations (full-horizon and sliding-horizon approaches). We found that Xia's model modified with the stabilization invariant (10 or 5) was adapted to direct multiple shooting. Sliding-horizon OCPs achieved 20 to 21 repetitions. The kinematic strategy slowly deviated from a plausible dumbbell lifting task to a swinging strategy as fatigue onset increasingly compromised the ability to keep the arm vertical. In full-horizon OCPs, the latter kinematic strategy was used over the whole motion, resulting in 32 repetitions. We showed that sliding-horizon OCPs revealed a reactive strategy to fatigue when only torque was included in the cost function, whereas an anticipatory strategy was revealed when the fatigue term was included in the cost function. Overall, the proposed approach has the potential to be a valuable tool in optimizing performance and helping reduce fatigue-related injuries in a variety of fields.
ChatGPT vs. Lightweight Security: First Work Implementing the NIST Cryptographic Standard ASCON
Venue: arXiv cs.CR 2023
Year: 2023

This study, to the best of our knowledge, is the first to explore the intersection between lightweight cryptography (LWC) and advanced artificial intelligence (AI) language models. LWC, in particular the ASCON algorithm which has been selected as the LWC standard by the National Institute of Standards and Technology (NIST) in Feb. 2023, has become increasingly significant for preserving data security given the quick expansion and resource limitations of Internet of Things (IoT) devices. On the other hand, OpenAI's large language model (LLM) ChatGPT has demonstrated significant potential in producing complex, human-like text. This paper offers a novel method for implementing the NIST LWC standard, ASCON, using the GPT-4 model. Moreover, this paper details the design and functionality of ASCON, the procedures and actual Python implementation of ASCON using ChatGPT, and a discussion of the outcomes. The results contribute valuable insights into the efficient application of advanced AI language models in cryptography, particularly in constrained environments. Source code can be found at: https://github.com/DrCintas/ASCON-with-ChatGPT.
ZeroForge: Feedforward Text-to-Shape Without 3D Supervision
Venue: arXiv cs.CV 2023
Year: 2023

Current state-of-the-art methods for text-to-shape generation either require supervised training using a labeled dataset of pre-defined 3D shapes, or perform expensive inference-time optimization of implicit neural representations. In this work, we present ZeroForge, an approach for zero-shot text-to-shape generation that avoids both pitfalls. To achieve open-vocabulary shape generation, we require careful architectural adaptation of existing feed-forward approaches, as well as a combination of data-free CLIP-loss and contrastive losses to avoid mode collapse. Using these techniques, we are able to considerably expand the generative ability of existing feed-forward text-to-shape models such as CLIP-Forge. We support our method via extensive qualitative and quantitative evaluations
Subjective-objective policy making approach: Coupling of resident-values multiple regression analysis with value-indices, multi-agent-based simulation
Venue: arXiv cs.CY 2023
Year: 2023

Given the concerns around the existing subjective and objective policy evaluation approaches, this study proposes a new combined subjective-objective policy evaluation approach to choose better policy that reflects the will of citizens and is backed up by objective facts. Subjective approaches, such as the Life Satisfaction Approach and the Contingent Valuation Method, convert subjectivity into economic value, raising the question whether a higher economic value really accords with what citizens want. Objective policy evaluation approaches, such as Evidence Based Policy Making and Multi-Agent-Based Simulation, do not take subjectivity into account, making it difficult to choose from diverse and pluralistic candidate policies. The proposed approach establishes a subjective target function based on a multiple regression analysis of the results of a residents questionnaire survey, and uses MABS to calculate the objective evaluation indices for a number of candidate policies. Next, a new subjective-objective coupling target function, combining the explanatory variables of the subjective target function with objective evaluation indices, is set up, optimized to select the preferred policies from numerous candidates. To evaluate this approach, we conducted a verification of renewable energy introduction policies at Takaharu Town in Miyazaki Prefecture, Japan. The results show a good potential for using a new subjective-objective coupling target function to select policies consistent with the residents values for well-being from 20,000 policy candidates for social, ecological, and economic values obtained in MABS. Using the new approach to compare several policies enables concrete expression of the will of stakeholders with diverse values, and contributes to constructive discussions and consensus-building.
Contrastive Loss is All You Need to Recover Analogies as Parallel Lines
Venue: arXiv cs.CL 2023
Year: 2023

While static word embedding models are known to represent linguistic analogies as parallel lines in high-dimensional space, the underlying mechanism as to why they result in such geometric structures remains obscure. We find that an elementary contrastive-style method employed over distributional information performs competitively with popular word embedding models on analogy recovery tasks, while achieving dramatic speedups in training time. Further, we demonstrate that a contrastive loss is sufficient to create these parallel structures in word embeddings, and establish a precise relationship between the co-occurrence statistics and the geometric structure of the resulting word embeddings.
Protecting User Privacy in Remote Conversational Systems: A Privacy-Preserving framework based on text sanitization
Venue: arXiv cs.CR 2023
Year: 2023

Large Language Models (LLMs) are gaining increasing attention due to their exceptional performance across numerous tasks. As a result, the general public utilize them as an influential tool for boosting their productivity while natural language processing researchers endeavor to employ them in solving existing or new research problems. Unfortunately, individuals can only access such powerful AIs through APIs, which ultimately leads to the transmission of raw data to the models' providers and increases the possibility of privacy data leakage. Current privacy-preserving methods for cloud-deployed language models aim to protect privacy information in the pre-training dataset or during the model training phase. However, they do not meet the specific challenges presented by the remote access approach of new large-scale language models. This paper introduces a novel task, "User Privacy Protection for Dialogue Models," which aims to safeguard sensitive user information from any possible disclosure while conversing with chatbots. We also present an evaluation scheme for this task, which covers evaluation metrics for privacy protection, data availability, and resistance to simulation attacks. Moreover, we propose the first framework for this task, namely privacy protection through text sanitization. Before sending the input to remote large models, it filters out the sensitive information, using several rounds of text sanitization based on privacy types that users define. Upon receiving responses from the larger model, our framework automatically restores privacy to ensure that the conversation goes smoothly, without intervention from the privacy filter. Experiments based on real-world datasets demonstrate the efficacy of our privacy-preserving approach against eavesdropping from potential attackers.
Extracting Information from Twitter Screenshots
Venue: arXiv cs.IR 2023
Year: 2023

Screenshots are prevalent on social media as a common approach for information sharing. Users rarely verify before sharing a screenshot whether the post it contains is fake or real. Information sharing through fake screenshots can be highly responsible for misinformation and disinformation spread on social media. Our ultimate goal is to develop a tool that could take a screenshot of a tweet and provide a probability that the tweet is real, using resources found on the live web and in web archives. This paper provides methods for extracting the tweet text, timestamp, and Twitter handle from a screenshot of a tweet.
GraphVine: A Data Structure to Optimize Dynamic Graph Processing on GPUs
Venue: arXiv cs.DS 2023
Year: 2023

Graph processing on GPUs is gaining momentum due to the high throughputs observed compared to traditional CPUs, attributed to the vast number of processing cores on GPUs that can exploit parallelism in graph analytics. This paper discusses a graph data structure for dynamic graph processing on GPUs. Unlike static graphs, dynamic graphs mutate over their lifetime through vertex and/or edge batch updates. The proposed work aims to provide fast batch updates and graph querying without consuming too much GPU memory. Experimental results show improved initialization timings by 1968-1269024%, improved batch edge insert timings by 30-30047%, and improved batch edge delete timings by 50-25262% while consuming less memory when the batch size is large.
A Contemporary Survey on 6G Wireless Networks: Potentials, Recent Advances, Technical Challenges and Future Trends
Venue: arXiv cs.NI 2023
Year: 2023

Smart services based on Internet of everything (IoE) are prophesied to reap notable attention by both academia and industry in the future. Although fifth-generation (5G) is a promising communication technology, however it cannot fulfill complete demands of novel applications. Sixth-generation (6G) technology is envisaged to overcome limitations of 5G technology. The vision and planning of future 6G network has been started with this aim to meet the stringent requirements of mobile communication. Our aim is to explore recent advances and potential challenges to enable 6G technology in this review. We have devised a taxonomy based on computing technologies, networking technologies, communication technologies, use cases, machine learning algorithms and key enabler technologies. In this regard, we subsequently highlight potential features and key areas of 6G. Key technological breakthroughs which include quantum communication, tactile communication, holographic communication, terahertz communication, visible light communication (VLC) Internet of Bio Nano Things, which can put profound impact on wireless communication, have been elaborated at length in this review. In this review, our prime focus is to discuss potential enabling technologies which can develop seamless and sustainable network, encompassing symbiotic radio, blockchain, new communication paradigm, VLC and terahertz. In addition, we have investigated open research challenges which can hamper the performance of 6G network. Finally, we have outlined several practical considerations, 6G key projects and future directions. We envision 6G undergoing unprecedented breakthroughs to eliminate technical uncertainties and provide enlightening research directions for subsequent future studies. Although it is impossible to envisage complete details of 6G, we believe this study will pave the way for future research work.
Differentially Private Wireless Federated Learning Using Orthogonal Sequences
Venue: arXiv cs.IT 2023
Year: 2023

We propose a privacy-preserving uplink over-the-air computation (AirComp) method, termed FLORAS, for single-input single-output (SISO) wireless federated learning (FL) systems. From the perspective of communication designs, FLORAS eliminates the requirement of channel state information at the transmitters (CSIT) by leveraging the properties of orthogonal sequences. From the privacy perspective, we prove that FLORAS offers both item-level and client-level differential privacy (DP) guarantees. Moreover, by properly adjusting the system parameters, FLORAS can flexibly achieve different DP levels at no additional cost. A new FL convergence bound is derived which, combined with the privacy guarantees, allows for a smooth tradeoff between the achieved convergence rate and differential privacy levels. Experimental results demonstrate the advantages of FLORAS compared with the baseline AirComp method, and validate that the analytical results can guide the design of privacy-preserving FL with different tradeoff requirements on the model convergence and privacy levels.
A reinforcement learning strategy for p-adaptation in high order solvers
Venue: arXiv cs.LG 2023
Year: 2023

Reinforcement learning (RL) has emerged as a promising approach to automating decision processes. This paper explores the application of RL techniques to optimise the polynomial order in the computational mesh when using high-order solvers. Mesh adaptation plays a crucial role in improving the efficiency of numerical simulations by improving accuracy while reducing the cost. Here, actor-critic RL models based on Proximal Policy Optimization offer a data-driven approach for agents to learn optimal mesh modifications based on evolving conditions. The paper provides a strategy for p-adaptation in high-order solvers and includes insights into the main aspects of RL-based mesh adaptation, including the formulation of appropriate reward structures and the interaction between the RL agent and the simulation environment. We discuss the impact of RL-based mesh p-adaptation on computational efficiency and accuracy. We test the RL p-adaptation strategy on a 1D inviscid Burgers' equation to demonstrate the effectiveness of the strategy. The RL strategy reduces the computational cost and improves accuracy over uniform adaptation, while minimising human intervention.
Research on Named Entity Recognition in Improved transformer with R-Drop structure
Venue: arXiv cs.CL 2023
Year: 2023

To enhance the generalization ability of the model and improve the effectiveness of the transformer for named entity recognition tasks, the XLNet-Transformer-R model is proposed in this paper. The XLNet pre-trained model and the Transformer encoder with relative positional encodings are combined to enhance the model's ability to process long text and learn contextual information to improve robustness. To prevent overfitting, the R-Drop structure is used to improve the generalization capability and enhance the accuracy of the model in named entity recognition tasks. The model in this paper performs ablation experiments on the MSRA dataset and comparison experiments with other models on four datasets with excellent performance, demonstrating the strategic effectiveness of the XLNet-Transformer-R model.