OpenRaiser/Intern-Atlas huggingface dataset:
A Theory for Discrete-time Boolean Finite Dynamical Systems with Uncertainty
Venue: arXiv cs.CC 2022
Year: 2022

Dynamical Systems is a field that studies the collective behavior of objects that update their states according to some rules. Discrete-time Boolean Finite Dynamical System (DT-BFDS) is a subfield where the systems have some finite number of objects whose states are Boolean values, and the state updates occur in discrete time. In the subfield of DT-BFDS, researchers aim to (i) design models for capturing real-world phenomena and using the models to make predictions and (ii) develop simulation techniques for acquiring insights about the systems' behavior. Useful for both aims is understanding the system dynamics mathematically before executing the systems. Obtaining a mathematical understanding of BFDS is quite challenging, even for simple systems, because the state space of a system grows exponentially in the number of objects. Researchers have used computational complexity to circumvent the challenge. The complexity theoretic research in DT-BFDS has successfully produced complete characterizations for many dynamical problems. The DT-BFDS studies have mainly dealt with deterministic models, where the update at each time step is deterministic, so the system dynamics are completely determinable from the initial setting. However, natural systems have uncertainty. Models having uncertainty may lead to far-better understandings of nature. Although a few attempts have explored DT-BFDS with uncertainty, including stochastic initialization and tie-breaking, they have scratched only a tiny surface of models with uncertainty. The introduction of uncertainty can be through two schemes. One is the introduction of alternate update functions. The other is the introduction of alternate update schedules. 37This paper establishes a theory of models with uncertainty and proves some fundamental results.
An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we revisit and improve the convergence of policy gradient (PG), natural PG (NPG) methods, and their variance-reduced variants, under general smooth policy parametrizations. More specifically, with the Fisher information matrix of the policy being positive definite: i) we show that a state-of-the-art variance-reduced PG method, which has only been shown to converge to stationary points, converges to the globally optimal value up to some inherent function approximation error due to policy parametrization; ii) we show that NPG enjoys a lower sample complexity; iii) we propose SRVR-NPG, which incorporates variance-reduction into the NPG update. Our improvements follow from an observation that the convergence of (variance-reduced) PG and NPG methods can improve each other: the stationary convergence analysis of PG can be applied to NPG as well, and the global convergence analysis of NPG can help to establish the global convergence of (variance-reduced) PG methods. Our analysis carefully integrates the advantages of these two lines of works. Thanks to this improvement, we have also made variance-reduction for NPG possible, with both global convergence and an efficient finite-sample complexity.
A Metaheuristic Approach for Mining Gradual Patterns
Venue: arXiv cs.NE 2022
Year: 2022

Swarm intelligence is a discipline that studies the collective behavior that is produced by local interactions of a group of individuals with each other and with their environment. In Computer Science domain, numerous swarm intelligence techniques are applied to optimization problems that seek to efficiently find best solutions within a search space. Gradual pattern mining is another Computer Science field that could benefit from the efficiency of swarm based optimization techniques in the task of finding gradual patterns from a huge search space. A gradual pattern is a rule-based correlation that describes the gradual relationship among the attributes of a data set. For example, given attributes {G,H} of a data set a gradual pattern may take the form: "the less G, the more H". In this paper, we propose a numeric encoding for gradual pattern candidates that we use to define an effective search space. In addition, we present a systematic study of several meta-heuristic optimization techniques as efficient solutions to the problem of finding gradual patterns using our search space.
Automatic Evaluation of Excavator Operators using Learned Reward Functions
Venue: arXiv cs.RO 2022
Year: 2022

Training novice users to operate an excavator for learning different skills requires the presence of expert teachers. Considering the complexity of the problem, it is comparatively expensive to find skilled experts as the process is time-consuming and requires precise focus. Moreover, since humans tend to be biased, the evaluation process is noisy and will lead to high variance in the final score of different operators with similar skills. In this work, we address these issues and propose a novel strategy for the automatic evaluation of excavator operators. We take into account the internal dynamics of the excavator and the safety criterion at every time step to evaluate the performance. To further validate our approach, we use this score prediction model as a source of reward for a reinforcement learning agent to learn the task of maneuvering an excavator in a simulated environment that closely replicates the real-world dynamics. For a policy learned using these external reward prediction models, our results demonstrate safer solutions following the required dynamic constraints when compared to policy trained with task-based reward functions only, making it one step closer to real-life adoption. For future research, we release our codebase at https://github.com/pranavAL/InvRL_Auto-Evaluate and video results https://drive.google.com/file/d/1jR1otOAu8zrY8mkhUOUZW9jkBOAKK71Z/view?usp=share_link .
An Overview on Controllable Text Generation via Variational Auto-Encoders
Venue: arXiv cs.CL 2022
Year: 2022

Recent advances in neural-based generative modeling have reignited the hopes of having computer systems capable of conversing with humans and able to understand natural language. The employment of deep neural architectures has been largely explored in a multitude of context and tasks to fulfill various user needs. On one hand, producing textual content that meets specific requirements is of priority for a model to seamlessly conduct conversations with different groups of people. On the other hand, latent variable models (LVM) such as variational auto-encoders (VAEs) as one of the most popular genres of generative models are designed to characterize the distributional pattern of textual data. Thus they are inherently capable of learning the integral textual features that are worth exploring for controllable pursuits. \noindent This overview gives an introduction to existing generation schemes, problems associated with text variational auto-encoders, and a review of several applications about the controllable generation that are instantiations of these general formulations,\footnote{A detailed paper list is available at \url{https://github.com/ImKeTT/CTG-latentAEs}} as well as related datasets, metrics and discussions for future researches. Hopefully, this overview will provide an overview of living questions, popular methodologies and raw thoughts for controllable language generation under the scope of variational auto-encoder.
The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck
Venue: arXiv cs.LG 2022
Year: 2022

Machine learning (ML) is revolutionizing the world, affecting almost every field of science and industry. Recent algorithms (in particular, deep networks) are increasingly data-hungry, requiring large datasets for training. Thus, the dominant paradigm in ML today involves constructing large, task-specific datasets. However, obtaining quality datasets of such magnitude proves to be a difficult challenge. A variety of methods have been proposed to address this data bottleneck problem, but they are scattered across different areas, and it is hard for a practitioner to keep up with the latest developments. In this work, we propose a taxonomy of these methods. Our goal is twofold: (1) We wish to raise the community's awareness of the methods that already exist and encourage more efficient use of resources, and (2) we hope that such a taxonomy will contribute to our understanding of the problem, inspiring novel ideas and strategies to replace current annotation-heavy approaches.
A Simple and Efficient Lagrange Multiplier Based Mixed Finite Element for Gradient Damage
Venue: arXiv cs.CE 2022
Year: 2022

A novel finite element formulation for gradient-regularized damage models is presented which allows for the robust, efficient, and mesh-independent simulation of damage phenomena in engineering and biological materials. The paper presents a Lagrange multiplier based mixed finite element formulation for finite strains. Thereby, no numerical stabilization or penalty parameters are required. On the other hand, no additional degrees of freedom appear for the Lagrange multiplier which is achieved through a suitable FE-interpolation scheme allowing for static condensation. In contrast to competitive approaches from the literature with similar efficiency, the proposed formulation does not require cross-element information and thus, a straightforward implementation using standard element routine interfaces is enabled. Numerical tests show mesh-independent solutions, robustness of the solution procedure for states of severe damage and under cyclic loading conditions. It is demonstrated that the computing time of the gradient damage calculations exceeds the one of purely elastic computations only by an insignificant amount. Furthermore, an improved convergence behavior compared to alternative approaches is shown.
Spatiotemporal modeling of European paleoclimate using doubly sparse Gaussian processes
Venue: arXiv cs.LG 2022
Year: 2022

Paleoclimatology -- the study of past climate -- is relevant beyond climate science itself, such as in archaeology and anthropology for understanding past human dispersal. Information about the Earth's paleoclimate comes from simulations of physical and biogeochemical processes and from proxy records found in naturally occurring archives. Climate-field reconstructions (CFRs) combine these data into a statistical spatial or spatiotemporal model. To date, there exists no consensus spatiotemporal paleoclimate model that is continuous in space and time, produces predictions with uncertainty, and can include data from various sources. A Gaussian process (GP) model would have these desired properties; however, GPs scale unfavorably with data of the magnitude typical for building CFRs. We propose to build on recent advances in sparse spatiotemporal GPs that reduce the computational burden by combining variational methods based on inducing variables with the state-space formulation of GPs. We successfully employ such a doubly sparse GP to construct a probabilistic model of European paleoclimate from the Last Glacial Maximum (LGM) to the mid-Holocene (MH) that synthesizes paleoclimate simulations and fossilized pollen proxy data.
Grasping the Inconspicuous
Venue: arXiv cs.CV 2022
Year: 2022

Transparent objects are common in day-to-day life and hence find many applications that require robot grasping. Many solutions toward object grasping exist for non-transparent objects. However, due to the unique visual properties of transparent objects, standard 3D sensors produce noisy or distorted measurements. Modern approaches tackle this problem by either refining the noisy depth measurements or using some intermediate representation of the depth. Towards this, we study deep learning 6D pose estimation from RGB images only for transparent object grasping. To train and test the suitability of RGB-based object pose estimation, we construct a dataset of RGB-only images with 6D pose annotations. The experiments demonstrate the effectiveness of RGB image space for grasping transparent objects.
Desarollo de un Dron Low-Cost para Tareas Indoor
Venue: arXiv cs.RO 2022
Year: 2022

Commercial drones are not yet dimensioned to perform indoor autonomous tasks, since they use GPS for their location in the environment. When it comes to a space with physical obstacles (walls, metal, etc.) between the communication of the drone and the satellites that allow the precise location of the same, there is great difficulty in finding the satellites or it generates interference for this location. This problem can cause an unexpected action of the drone, a collision and a possible accident can occur. The work to follow presents the development of a drone capable of operating in a physical space (indoor), without the need for GPS. In this proposal, a prototype of a system for detecting the distance (lidar) that the drone is from the walls is also developed, with the aim of being able to take this information as the location of the drone.
On Inferring User Socioeconomic Status with Mobility Records
Venue: arXiv cs.LG 2022
Year: 2022

When users move in a physical space (e.g., an urban space), they would have some records called mobility records (e.g., trajectories) generated by devices such as mobile phones and GPS devices. Naturally, mobility records capture essential information of how users work, live and entertain in their daily lives, and therefore, they have been used in a wide range of tasks such as user profile inference, mobility prediction and traffic management. In this paper, we expand this line of research by investigating the problem of inferring user socioeconomic statuses (such as prices of users' living houses as a proxy of users' socioeconomic statuses) based on their mobility records, which can potentially be used in real-life applications such as the car loan business. For this task, we propose a socioeconomic-aware deep model called DeepSEI. The DeepSEI model incorporates two networks called deep network and recurrent network, which extract the features of the mobility records from three aspects, namely spatiality, temporality and activity, one at a coarse level and the other at a detailed level. We conduct extensive experiments on real mobility records data, POI data and house prices data. The results verify that the DeepSEI model achieves superior performance than existing studies. All datasets used in this paper will be made publicly available.
A Comparative Study of Machine Learning and Deep Learning Techniques for Prediction of Co2 Emission in Cars
Venue: arXiv cs.LG 2022
Year: 2022

The most recent concern of all people on Earth is the increase in the concentration of greenhouse gas in the atmosphere. The concentration of these gases has risen rapidly over the last century and if the trend continues it can cause many adverse climatic changes. There have been ways implemented to curb this by the government by limiting processes that emit a higher amount of CO2, one such greenhouse gas. However, there is mounting evidence that the CO2 numbers supplied by the government do not accurately reflect the performance of automobiles on the road. Our proposal of using artificial intelligence techniques to improve a previously rudimentary process takes a radical tack, but it fits the bill given the situation. To determine which algorithms and models produce the greatest outcomes, we compared them all and explored a novel method of ensembling them. Further, this can be used to foretell the rise in global temperature and to ground crucial policy decisions like the adoption of electric vehicles. To estimate emissions from vehicles, we used machine learning, deep learning, and ensemble learning on a massive dataset.
An FNet based Auto Encoder for Long Sequence News Story Generation
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, we design an auto encoder based off of Google's FNet Architecture in order to generate text from a subset of news stories contained in Google's C4 dataset. We discuss previous attempts and methods to generate text from autoencoders and non LLM Models. FNET poses multiple advantages to BERT based encoders in the realm of efficiency which train 80% faster on GPUs and 70% faster on TPUs. We then compare outputs of how this autencoder perfroms on different epochs. Finally, we analyze what outputs the encoder produces with different seed text.
Deep-Learning-Empowered Inverse Design for Freeform Reconfigurable Metasurfaces
Venue: arXiv cs.LG 2022
Year: 2022

The past decade has witnessed the advances of artificial intelligence with various applications in engineering. Recently, artificial neural network empowered inverse design for metasurfaces has been developed that can design on-demand meta-atoms with diverse shapes and high performance, where the design process based on artificial intelligence is fast and automatic. However, once the inverse-designed static meta-atom is fabricated, the function of the metasurface is fixed. Reconfigurable metasurfaces can realize dynamic functions, while applying artificial intelligence to design practical reconfigurable meta-atoms inversely has not been reported yet. Here, we present a deep-learning-empowered inverse design method for freeform reconfigurable metasurfaces, which can generate on-demand reconfigurable coding meta-atoms at self-defined frequency bands. To reduce the scale of dataset, a decoupling method of the reconfigurable meta-atom based on microwave network theory is proposed at first, which can convert the inverse design process for reconfigurable coding meta-atoms to the inverse design for static structures. A convolutional neural network model is trained to predict the responses of free-shaped meta-atoms, and the genetic algorithm is applied to generate the optimal structure patterns rapidly. As a demonstration of concept, several inverse-designed examples are generated with different self-defined spectrum responses in microwave band, and an inverse-designed wideband reconfigurable metasurface prototype is fabricated and measured for beam scanning applications with broad bandwidth. Our work paves the way for the fast and automatic design process of high-performance reconfigurable metasurfaces.
Air Pollution Hotspot Detection and Source Feature Analysis using Cross-domain Urban Data
Venue: arXiv cs.LG 2022
Year: 2022

Air pollution is a major global environmental health threat, in particular for people who live or work near pollution sources. Areas adjacent to pollution sources often have high ambient pollution concentrations, and those areas are commonly referred to as air pollution hotspots. Detecting and characterizing pollution hotspots are of great importance for air quality management, but are challenging due to the high spatial and temporal variability of air pollutants. In this work, we explore the use of mobile sensing data (i.e., air quality sensors installed on vehicles) to detect pollution hotspots. One major challenge with mobile sensing data is uneven sampling, i.e., data collection can vary by both space and time. To address this challenge, we propose a two-step approach to detect hotspots from mobile sensing data, which includes local spike detection and sample-weighted clustering. Essentially, this approach tackles the uneven sampling issue by weighting samples based on their spatial frequency and temporal hit rate, so as to identify robust and persistent hotspots. To contextualize the hotspots and discover potential pollution source characteristics, we explore a variety of cross-domain urban data and extract features from them. As a soft-validation of the extracted features, we build hotspot inference models for cities with and without mobile sensing data. Evaluation results using real-world mobile sensing air quality data as well as cross-domain urban data demonstrate the effectiveness of our approach in detecting and inferring pollution hotspots. Furthermore, the empirical analysis of hotspots and source features yields useful insights regarding neighborhood pollution sources.
iTUAVs: Intermittently Tethered UAVs for Future Wireless Networks
Venue: arXiv cs.NI 2022
Year: 2022

We propose the intermittently tethered unmanned aerial vehicle (iTUAV) as a tradeoff between the power availability of a tethered UAV (TUAV) and the flexibility of an untethered UAV. An iTUAV can provide cellular connectivity while being temporarily tethered to the most adequate ground anchor. Also, it can flexibly detach from one anchor, travel, then attach to another one to maintain/improve the coverage quality for mobile users. Hence, we discuss here the existing UAV-based cellular networking technologies, followed by a detailed description of the iTUAV system, its components, and mode of operation. Subsequently, we present a comparative study of the existing and proposed systems highlighting the differences in key features such as mobility and energy. To emphasize the potential of iTUAV systems, we conduct a case study, evaluate the iTUAV performance, and compare it to benchmarks. Obtained results show that with only 10 anchors in the area, the iTUAV system can serve up to 90% of the users covered by the untethered UAV swapping system. Moreover, results from a small case study prove that the iTUAV allows to balance performance/cost and can be implemented realistically. For instance, when user locations are clustered, with only 2 active iTUAVs and 4 anchors, achieved performance is superior to that of the system with 3 TUAVs, while when considering a single UAV on a 100 minutes event, a system with only 6 anchors outperforms the untethered UAV as it combines location flexibility with increased mission time.
Automatic penalty and degree continuation for parallel pre-conditioned mesh curving on virtual geometry
Venue: arXiv cs.CE 2022
Year: 2022

We present a distributed parallel mesh curving method for virtual geometry. The main application is to generate large-scale curved meshes on complex geometry suitable for analysis with unstructured high-order methods. Accordingly, we devise the technique to generate geometrically accurate meshes composed of high-quality elements. To this end, we advocate for degree continuation on a penalty-based second-order optimizer that uses global tight tolerances to converge the distortion residuals. To reduce the method memory footprint, waiting time, and energy consumption, we combine three main ingredients. First, we propose a matrix-free GMRES solver pre-conditioned with successive over-relaxation by blocks to reduce the memory footprint three times. We also propose an adaptive penalty technique, to reduce the number of non-linear iterations. Third, we propose an indicator of the required linear solver tolerance to reduce the number of linear iterations. On thousands of cores, the method curves meshes composed of millions of quartic elements featuring highly stretched elements while matching a virtual topology.
Power-law Scaling to Assist with Key Challenges in Artificial Intelligence
Venue: arXiv cs.LG 2022
Year: 2022

Power-law scaling, a central concept in critical phenomena, is found to be useful in deep learning, where optimized test errors on handwritten digit examples converge as a power-law to zero with database size. For rapid decision making with one training epoch, each example is presented only once to the trained network, the power-law exponent increased with the number of hidden layers. For the largest dataset, the obtained test error was estimated to be in the proximity of state-of-the-art algorithms for large epoch numbers. Power-law scaling assists with key challenges found in current artificial intelligence applications and facilitates an a priori dataset size estimation to achieve a desired test accuracy. It establishes a benchmark for measuring training complexity and a quantitative hierarchy of machine learning tasks and algorithms.