OpenRaiser/Intern-Atlas huggingface dataset:
META-STORM: Generalized Fully-Adaptive Variance Reduced SGD for Unbounded Functions
Venue: arXiv cs.LG 2022
Year: 2022

We study the application of variance reduction (VR) techniques to general non-convex stochastic optimization problems. In this setting, the recent work STORM [Cutkosky-Orabona '19] overcomes the drawback of having to compute gradients of "mega-batches" that earlier VR methods rely on. There, STORM utilizes recursive momentum to achieve the VR effect and is then later made fully adaptive in STORM+ [Levy et al., '21], where full-adaptivity removes the requirement for obtaining certain problem-specific parameters such as the smoothness of the objective and bounds on the variance and norm of the stochastic gradients in order to set the step size. However, STORM+ crucially relies on the assumption that the function values are bounded, excluding a large class of useful functions. In this work, we propose META-STORM, a generalized framework of STORM+ that removes this bounded function values assumption while still attaining the optimal convergence rate for non-convex optimization. META-STORM not only maintains full-adaptivity, removing the need to obtain problem specific parameters, but also improves the convergence rate's dependency on the problem parameters. Furthermore, META-STORM can utilize a large range of parameter settings that subsumes previous methods allowing for more flexibility in a wider range of settings. Finally, we demonstrate the effectiveness of META-STORM through experiments across common deep learning tasks. Our algorithm improves upon the previous work STORM+ and is competitive with widely used algorithms after the addition of per-coordinate update and exponential moving average heuristics.
Improving Understanding of Biocide Availability in Facades through Immersive Analytics
Venue: arXiv cs.HC 2022
Year: 2022

The durability of facades is heavily affected by multiple factors like microbial growth and weather conditions among others. Biocides are often used to resist these factors and protect the facades. However, the biocides get washed out due to rains and other factors like geometric structure of the facade, orientation of the building. It is therefore, important to understand how these factors affect the durability of facades, leading to a requirement of expert analysis. In this paper, we propose a technical pipeline and a set of interaction techniques to support data analysis within the immersive environment for our case study. Our technical pipeline mainly consists of three steps: 3D reconstruction, embedding sensor data and visualization and interaction techniques. We made a formative evaluation of our prototype to get insights from microbiology, biology and VR experts. The remarks from the experts and the results of the evaluation suggest that an immersive analytic system in our case study could be beneficial for both experts and non-expert users.
OpenDSU: Digital Sovereignty in PharmaLedger
Venue: arXiv cs.CR 2022
Year: 2022

Distributed ledger networks, chiefly those based on blockchain technologies, currently are heralding a next generation of computer systems that aims to suit modern users' demands. Over the recent years, several technologies for blockchains, off-chaining strategies, as well as decentralised and respectively self-sovereign identity systems have shot up so fast that standardisation of the protocols is lagging behind, severely hampering the interoperability of different approaches. Moreover, most of the currently available solutions for distributed ledgers focus on either home users or enterprise use case scenarios, failing to provide integrative solutions addressing the needs of both. Herein we introduce the OpenDSU platform that allows to interoperate generic blockchain technologies, organised - and possibly cascaded in a hierarchical fashion - in domains. To achieve this flexibility, we seamlessly integrated a set of well conceived OpenDSU components to orchestrate off-chain data with granularly resolved and cryptographically secure access levels that are nested with sovereign identities across the different domains. Employing our platform to PharmaLedger, an inter-European network for the standardisation of data handling in the pharmaceutical industry and in healthcare, we demonstrate that OpenDSU can cope with generic demands of heterogeneous use cases in both, performance and handling substantially different business policies. Importantly, whereas available solutions commonly require a pre-defined and fixed set of components, no such vendor lock-in restrictions on the blockchain technology or identity system exist in OpenDSU, making systems built on it flexibly adaptable to new standards evolving in the future.
Joint Embedding Self-Supervised Learning in the Kernel Regime
Venue: arXiv cs.LG 2022
Year: 2022

The fundamental goal of self-supervised learning (SSL) is to produce useful representations of data without access to any labels for classifying the data. Modern methods in SSL, which form representations based on known or constructed relationships between samples, have been particularly effective at this task. Here, we aim to extend this framework to incorporate algorithms based on kernel methods where embeddings are constructed by linear maps acting on the feature space of a kernel. In this kernel regime, we derive methods to find the optimal form of the output representations for contrastive and non-contrastive loss functions. This procedure produces a new representation space with an inner product denoted as the induced kernel which generally correlates points which are related by an augmentation in kernel space and de-correlates points otherwise. We analyze our kernel model on small datasets to identify common features of self-supervised learning algorithms and gain theoretical insights into their performance on downstream tasks.
Joint Convexity of Error Probability in Blocklength and Transmit Power in the Finite Blocklength Regime
Venue: arXiv cs.IT 2022
Year: 2022

To support ultra-reliable and low-latency services for mission-critical applications, transmissions are usually carried via short blocklength codes, i.e., in the so-called finite blocklength (FBL) regime. Different from the infinite blocklength regime where transmissions are assumed to be arbitrarily reliable at the Shannon's capacity, the reliability and capacity performances of an FBL transmission are impacted by the coding blocklength. The relationship among reliability, coding rate, blocklength and channel quality has recently been characterized in the literature, considering the FBL performance model. In this paper, we follow this model, and prove the joint convexity of the FBL error probability with respect to blocklength and transmit power within a region of interest, as a key enabler for designing systems to achieve globally optimal performance levels. Moreover, we apply the joint convexity to general use cases and efficiently solve the joint optimization problem in the setting with multiple users. We also extend the applicability of the proposed approach by proving that the joint convexity still holds in fading channels, as well as in relaying networks. Via simulations, we validate our analytical results and demonstrate the advantage of leveraging the joint convexity compared to other commonly-applied approaches.
Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals
Venue: arXiv cs.HC 2022
Year: 2022

Language models are increasingly attracting interest from writers. However, such models lack long-range semantic coherence, limiting their usefulness for longform creative writing. We address this limitation by applying language models hierarchically, in a system we call Dramatron. By building structural context via prompt chaining, Dramatron can generate coherent scripts and screenplays complete with title, characters, story beats, location descriptions, and dialogue. We illustrate Dramatron's usefulness as an interactive co-creative system with a user study of 15 theatre and film industry professionals. Participants co-wrote theatre scripts and screenplays with Dramatron and engaged in open-ended interviews. We report critical reflections both from our interviewees and from independent reviewers who watched stagings of the works to illustrate how both Dramatron and hierarchical text generation could be useful for human-machine co-creativity. Finally, we discuss the suitability of Dramatron for co-creativity, ethical considerations -- including plagiarism and bias -- and participatory models for the design and deployment of such tools.
DreamFusion: Text-to-3D using 2D Diffusion
Venue: arXiv cs.CV 2022
Year: 2022

Recent breakthroughs in text-to-image synthesis have been driven by diffusion models trained on billions of image-text pairs. Adapting this approach to 3D synthesis would require large-scale datasets of labeled 3D data and efficient architectures for denoising 3D data, neither of which currently exist. In this work, we circumvent these limitations by using a pretrained 2D text-to-image diffusion model to perform text-to-3D synthesis. We introduce a loss based on probability density distillation that enables the use of a 2D diffusion model as a prior for optimization of a parametric image generator. Using this loss in a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a Neural Radiance Field, or NeRF) via gradient descent such that its 2D renderings from random angles achieve a low loss. The resulting 3D model of the given text can be viewed from any angle, relit by arbitrary illumination, or composited into any 3D environment. Our approach requires no 3D training data and no modifications to the image diffusion model, demonstrating the effectiveness of pretrained image diffusion models as priors.
REST: REtrieve & Self-Train for generative action recognition
Venue: arXiv cs.CV 2022
Year: 2022

This work is on training a generative action/video recognition model whose output is a free-form action-specific caption describing the video (rather than an action class label). A generative approach has practical advantages like producing more fine-grained and human-readable output, and being naturally open-world. To this end, we propose to adapt a pre-trained generative Vision & Language (V&L) Foundation Model for video/action recognition. While recently there have been a few attempts to adapt V&L models trained with contrastive learning (e.g. CLIP) for video/action, to the best of our knowledge, we propose the very first method that sets outs to accomplish this goal for a generative model. We firstly show that direct fine-tuning of a generative model to produce action classes suffers from severe overfitting. To alleviate this, we introduce REST, a training framework consisting of two key components: an unsupervised method for adapting the generative model to action/video by means of pseudo-caption generation and Self-training, i.e. without using any action-specific labels; (b) a Retrieval approach based on CLIP for discovering a diverse set of pseudo-captions for each video to train the model. Importantly, we show that both components are necessary to obtain high accuracy. We evaluate REST on the problem of zero-shot action recognition where we show that our approach is very competitive when compared to contrastive learning-based methods. Code will be made available.
A deep learning approach to the probabilistic numerical solution of path-dependent partial differential equations
Venue: arXiv cs.LG 2022
Year: 2022

Recent work on Path-Dependent Partial Differential Equations (PPDEs) has shown that PPDE solutions can be approximated by a probabilistic representation, implemented in the literature by the estimation of conditional expectations using regression. However, a limitation of this approach is to require the selection of a basis in a function space. In this paper, we overcome this limitation by the use of deep learning methods, and we show that this setting allows for the derivation of error bounds on the approximation of conditional expectations. Numerical examples based on a two-person zero-sum game, as well as on Asian and barrier option pricing, are presented. In comparison with other deep learning approaches, our algorithm appears to be more accurate, especially in large dimensions.
Does Collaborative Editing Help Mitigate Security Vulnerabilities in Crowd-Shared IoT Code Examples?
Venue: arXiv cs.SE 2022
Year: 2022

Background: With the proliferation of crowd-sourced developer forums, software developers are increasingly sharing more coding solutions to programming problems with others in forums. The decentralized nature of knowledge sharing on sites has raised the concern of sharing security vulnerable code, which then can be reused into mission critical software systems - making those systems vulnerable in the process. Collaborative editing has been introduced in forums like Stack Overflow to improve the quality of the shared contents. Aim: In this paper, we investigate whether code editing can mitigate shared vulnerable code examples by analyzing IoT code snippets and their revisions in three Stack Exchange sites: Stack Overflow, Arduino, and Raspberry Pi. Method:We analyze the vulnerabilities present in shared IoT C/C++ code snippets, as C/C++ is one of the most widely used languages in mission-critical devices and low-powered IoT devices. We further analyse the revisions made to these code snippets, and their effects. Results: We find several vulnerabilities such as CWE 788 - Access of Memory Location After End of Buffer, in 740 code snippets . However, we find the vast majority of posts are not revised, or revisions are not made to the code snippets themselves (598 out of 740). We also find that revisions are most likely to result in no change to the number of vulnerabilities in a code snippet rather than deteriorating or improving the snippet. Conclusions: We conclude that the current collaborative editing system in the forums may be insufficient to help mitigate vulnerabilities in the shared code.
Improving Generative Flow Networks with Path Regularization
Venue: arXiv cs.LG 2022
Year: 2022

Generative Flow Networks (GFlowNets) are recently proposed models for learning stochastic policies that generate compositional objects by sequences of actions with the probability proportional to a given reward function. The central problem of GFlowNets is to improve their exploration and generalization. In this work, we propose a novel path regularization method based on optimal transport theory that places prior constraints on the underlying structure of the GFlowNets. The prior is designed to help the GFlowNets better discover the latent structure of the target distribution or enhance its ability to explore the environment in the context of active learning. The path regularization controls the flow in GFlowNets to generate more diverse and novel candidates via maximizing the optimal transport distances between two forward policies or to improve the generalization via minimizing the optimal transport distances. In addition, we derive an efficient implementation of the regularization by finding its closed form solutions in specific cases and a meaningful upper bound that can be used as an approximation to minimize the regularization term. We empirically demonstrate the advantage of our path regularization on a wide range of tasks, including synthetic hypergrid environment modeling, discrete probabilistic modeling, and biological sequence design.
Improving Molecular Pretraining with Complementary Featurizations
Venue: arXiv cs.LG 2022
Year: 2022

Molecular pretraining, which learns molecular representations over massive unlabeled data, has become a prominent paradigm to solve a variety of tasks in computational chemistry and drug discovery. Recently, prosperous progress has been made in molecular pretraining with different molecular featurizations, including 1D SMILES strings, 2D graphs, and 3D geometries. However, the role of molecular featurizations with their corresponding neural architectures in molecular pretraining remains largely unexamined. In this paper, through two case studies -- chirality classification and aromatic ring counting -- we first demonstrate that different featurization techniques convey chemical information differently. In light of this observation, we propose a simple and effective MOlecular pretraining framework with COmplementary featurizations (MOCO). MOCO comprehensively leverages multiple featurizations that complement each other and outperforms existing state-of-the-art models that solely relies on one or two featurizations on a wide range of molecular property prediction tasks.
Data Querying with Ciphertext Policy Attribute Based Encryption
Venue: arXiv cs.CR 2022
Year: 2022

Data encryption limits the power and efficiency of queries. Direct processing of encrypted data should ideally be possible to avoid the need for data decryption, processing, and re-encryption. It is vital to keep the data searchable and sortable. That is, some information is intentionally leaked. This intentional leakage technology is known as "querying over encrypted data schemes", which offer confidentiality as well as querying over encrypted data, but it is not meant to provide flexible access control. This paper suggests the use of Ciphertext Policy Attributes Based Encryption (CP-ABE) to address three security requirements, namely: confidentiality, queries over encrypted data, and flexible access control. By combining flexible access control and data confidentiality, CP-ABE can authenticate who can access data and possess the secret key. Thus, this paper identifies how much data leakage there is in order to figure out what kinds of operations are allowed when data is encrypted by CP-ABE.
Hidden in Plain Sight: Exploring Encrypted Channels in Android apps
Venue: arXiv cs.CR 2022
Year: 2022

As privacy features in Android operating system improve, privacy-invasive apps may gradually shift their focus to non-standard and covert channels for leaking private user/device information. Such leaks also remain largely undetected by state-of-the-art privacy analysis tools, which are very effective in uncovering privacy exposures via regular HTTP and HTTPS channels. In this study, we design and implement, ThirdEye, to significantly extend the visibility of current privacy analysis tools, in terms of the exposures that happen across various non-standard and covert channels, i.e., via any protocol over TCP/UDP (beyond HTTP/S), and using multi-layer custom encryption over HTTP/S and non-HTTP protocols. Besides network exposures, we also consider covert channels via storage media that also leverage custom encryption layers. Using ThirdEye, we analyzed 12,598 top-apps in various categories from Androidrank, and found that 2887/12,598 (22.92%) apps used custom encryption/decryption for network transmission and storing content in shared device storage, and 2465/2887 (85.38%) of those apps sent device information (e.g., advertising ID, list of installed apps) over the network that can fingerprint users. Besides, 299 apps transmitted insecure encrypted content over HTTP/non-HTTP protocols; 22 apps that used authentication tokens over HTTPS, happen to expose them over insecure (albeit custom encrypted) HTTP/non-HTTP channels. We found non-standard and covert channels with multiple levels of obfuscation (e.g., encrypted data over HTTPS, encryption at nested levels), and the use of vulnerable keys and cryptographic algorithms. Our findings can provide valuable insights into the evolving field of non-standard and covert channels, and help spur new countermeasures against such privacy leakage and security issues.
Batch Multivalid Conformal Prediction
Venue: arXiv cs.LG 2022
Year: 2022

We develop fast distribution-free conformal prediction algorithms for obtaining multivalid coverage on exchangeable data in the batch setting. Multivalid coverage guarantees are stronger than marginal coverage guarantees in two ways: (1) They hold even conditional on group membership -- that is, the target coverage level $1-α$ holds conditionally on membership in each of an arbitrary (potentially intersecting) group in a finite collection $\mathcal{G}$ of regions in the feature space. (2) They hold even conditional on the value of the threshold used to produce the prediction set on a given example. In fact multivalid coverage guarantees hold even when conditioning on group membership and threshold value simultaneously. We give two algorithms: both take as input an arbitrary non-conformity score and an arbitrary collection of possibly intersecting groups $\mathcal{G}$, and then can equip arbitrary black-box predictors with prediction sets. Our first algorithm (BatchGCP) is a direct extension of quantile regression, needs to solve only a single convex minimization problem, and produces an estimator which has group-conditional guarantees for each group in $\mathcal{G}$. Our second algorithm (BatchMVP) is iterative, and gives the full guarantees of multivalid conformal prediction: prediction sets that are valid conditionally both on group membership and non-conformity threshold. We evaluate the performance of both of our algorithms in an extensive set of experiments. Code to replicate all of our experiments can be found at https://github.com/ProgBelarus/BatchMultivalidConformal
Embedded System Performance Analysis for Implementing a Portable Drowsiness Detection System for Drivers
Venue: arXiv cs.CV 2022
Year: 2022

Drowsiness on the road is a widespread problem with fatal consequences; thus, a multitude of systems and techniques have been proposed. Among existing methods, Ghoddoosian et al. utilized temporal blinking patterns to detect early signs of drowsiness, but their algorithm was tested only on a powerful desktop computer, which is not practical to apply in a moving vehicle setting. In this paper, we propose an efficient platform to run Ghoddosian's algorithm, detail the performance tests we ran to determine this platform, and explain our threshold optimization logic. After considering the Jetson Nano and Beelink (Mini PC), we concluded that the Mini PC is the most efficient and practical to run our embedded system in a vehicle. To determine this, we ran communication speed tests and evaluated total processing times for inference operations. Based on our experiments, the average total processing time to run the drowsiness detection model was 94.27 ms for Jetson Nano and 22.73 ms for the Beelink (Mini PC). Considering the portability and power efficiency of each device, along with the processing time results, the Beelink (Mini PC) was determined to be most suitable. Also, we propose a threshold optimization algorithm, which determines whether the driver is drowsy or alert based on the trade-off between the sensitivity and specificity of the drowsiness detection model. Our study will serve as a crucial next step for drowsiness detection research and its application in vehicles. Through our experiment, we have determinend a favorable platform that can run drowsiness detection algorithms in real-time and can be used as a foundation to further advance drowsiness detection research. In doing so, we have bridged the gap between an existing embedded system and its actual implementation in vehicles to bring drowsiness technology a step closer to prevalent real-life implementation.