Reviewerly/exHarmony huggingface dataset:
Model Generalization: A Sharpness Aware Optimization Perspective

Sharpness-Aware Minimization (SAM) and adaptive sharpness-aware minimization (ASAM) aim to improve the model generalization. And in this project, we proposed three experiments to valid their generalization from the sharpness aware perspective. And our experiments show that sharpness aware-based optimization techniques could help to provide models with strong generalization ability. Our experiments also show that ASAM could improve the generalization performance on un-normalized data, but further research is needed to confirm this.
Visual Localization via Few-Shot Scene Region Classification

Visual (re)localization addresses the problem of estimating the 6-DoF (Degree of Freedom) camera pose of a query image captured in a known scene, which is a key building block of many computer vision and robotics applications. Recent advances in structure-based localization solve this problem by memorizing the mapping from image pixels to scene coordinates with neural networks to build 2D-3D correspondences for camera pose optimization. However, such memorization requires training by amounts of posed images in each scene, which is heavy and inefficient. On the contrary, few-shot images are usually sufficient to cover the main regions of a scene for a human operator to perform visual localization. In this paper, we propose a scene region classification approach to achieve fast and effective scene memorization with few-shot images. Our insight is leveraging a) pre-learned feature extractor, b) scene region classifier, and c) meta-learning strategy to accelerate training while mitigating overfitting. We evaluate our method on both indoor and outdoor benchmarks. The experiments validate the effectiveness of our method in the few-shot setting, and the training time is significantly reduced to only a few minutes. Code available at: \url{https://github.com/siyandong/SRC}
Synthesis of Semantic Actions in Attribute Grammars

Attribute grammars allow the association of semantic actions to the production rules in context-free grammars, providing a simple yet effective formalism to define the semantics of a language. However, drafting the semantic actions can be tricky and a large drain on developer time. In this work, we propose a synthesis methodology to automatically infer the semantic actions from a set of examples associating strings to their meanings. We also propose a new coverage metric, derivation coverage. We use it to build a sampler to effectively and automatically draw strings to drive the synthesis engine. We build our ideas into our tool, PANINI, and empirically evaluate it on twelve benchmarks, including a forward differentiation engine, an interpreter over a subset of Java bytecode, and a mini-compiler for C language to two-address code. Our results show that PANINI scales well with the number of actions to be synthesized and the size of the context-free grammar, significantly outperforming simple baselines.
Thermodynamics of deterministic finite automata operating locally and periodically

Real-world computers have operational constraints that cause nonzero entropy production (EP). In particular, almost all real-world computers are ``periodic'', iteratively undergoing the same physical process; and ``local", in that subsystems evolve whilst physically decoupled from the rest of the computer. These constraints are so universal because decomposing a complex computation into small, iterative calculations is what makes computers so powerful. We first derive the nonzero EP caused by the locality and periodicity constraints for deterministic finite automata (DFA), a foundational system of computer science theory. We then relate this minimal EP to the computational characteristics of the DFA. We thus divide the languages recognised by DFA into two classes: those that can be recognised with zero EP, and those that necessarily have non-zero EP. We also demonstrate the thermodynamic advantages of implementing a DFA with a physical process that is agnostic about the inputs that it processes.
Shuffle Instances-based Vision Transformer for Pancreatic Cancer ROSE Image Classification

The rapid on-site evaluation (ROSE) technique can signifi-cantly accelerate the diagnosis of pancreatic cancer by im-mediately analyzing the fast-stained cytopathological images. Computer-aided diagnosis (CAD) can potentially address the shortage of pathologists in ROSE. However, the cancerous patterns vary significantly between different samples, making the CAD task extremely challenging. Besides, the ROSE images have complicated perturbations regarding color distribution, brightness, and contrast due to different staining qualities and various acquisition device types. To address these challenges, we proposed a shuffle instances-based Vision Transformer (SI-ViT) approach, which can reduce the perturbations and enhance the modeling among the instances. With the regrouped bags of shuffle instances and their bag-level soft labels, the approach utilizes a regression head to make the model focus on the cells rather than various perturbations. Simultaneously, combined with a classification head, the model can effectively identify the general distributive patterns among different instances. The results demonstrate significant improvements in the classification accuracy with more accurate attention regions, indicating that the diverse patterns of ROSE images are effectively extracted, and the complicated perturbations are significantly reduced. It also suggests that the SI-ViT has excellent potential in analyzing cytopathological images. The code and experimental results are available at https://github.com/sagizty/MIL-SI.
Multinomial Logistic Regression Algorithms via Quadratic Gradient

Multinomial logistic regression, also known by other names such as multiclass logistic regression and softmax regression, is a fundamental classification method that generalizes binary logistic regression to multiclass problems. A recently work proposed a faster gradient called $\texttt{quadratic gradient}$ that can accelerate the binary logistic regression training, and presented an enhanced Nesterov's accelerated gradient (NAG) method for binary logistic regression. In this paper, we extend this work to multiclass logistic regression and propose an enhanced Adaptive Gradient Algorithm (Adagrad) that can accelerate the original Adagrad method. We test the enhanced NAG method and the enhanced Adagrad method on some multiclass-problem datasets. Experimental results show that both enhanced methods converge faster than their original ones respectively.
Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation

The architectural advancements in deep neural networks have led to remarkable leap-forwards across a broad array of computer vision tasks. Instead of relying on human expertise, neural architecture search (NAS) has emerged as a promising avenue toward automating the design of architectures. While recent achievements in image classification have suggested opportunities, the promises of NAS have yet to be thoroughly assessed on more challenging tasks of semantic segmentation. The main challenges of applying NAS to semantic segmentation arise from two aspects: (i) high-resolution images to be processed; (ii) additional requirement of real-time inference speed (i.e., real-time semantic segmentation) for applications such as autonomous driving. To meet such challenges, we propose a surrogate-assisted multi-objective method in this paper. Through a series of customized prediction models, our method effectively transforms the original NAS task into an ordinary multi-objective optimization problem. Followed by a hierarchical pre-screening criterion for in-fill selection, our method progressively achieves a set of efficient architectures trading-off between segmentation accuracy and inference speed. Empirical evaluations on three benchmark datasets together with an application using Huawei Atlas 200 DK suggest that our method can identify architectures significantly outperforming existing state-of-the-art architectures designed both manually by human experts and automatically by other NAS methods.
Sharp Frequency Bounds for Sample-Based Queries

A data sketch algorithm scans a big data set, collecting a small amount of data -- the sketch, which can be used to statistically infer properties of the big data set. Some data sketch algorithms take a fixed-size random sample of a big data set, and use that sample to infer frequencies of items that meet various criteria in the big data set. This paper shows how to statistically infer probably approximately correct (PAC) bounds for those frequencies, efficiently, and precisely enough that the frequency bounds are either sharp or off by only one, which is the best possible result without exact computation.
Generic Go to Go: Dictionary-Passing, Monomorphisation, and Hybrid

Go is a popular statically-typed industrial programming language. To aid the type safe reuse of code, the recent Go release (Go 1.18) published on 15th March 2022 includes bounded parametric polymorphism via generic types. Go 1.18 implements generic types using combination of monomorphisation and call-graph based dictionary-passing called hybrid. This hybrid approach can be viewed as an optimised form of monomorphisation that statically generates specialised methods and types based on possible instantiations. A monolithic dictionary supplements information lost during monomorphisation, and it is structured according to the program's call graph. Unfortunately, the hybrid approach still suffers from code bloat, poor compilation speed, and limited code coverage. In this paper we propose and formalise a new non-specialising call-site based dictionary-passing translation. Our call-site based translation creates individual dictionaries for each type parameter, with dictionary construction occurring in place of instantiation, overcoming the limitations of hybrid. We prove it correct using a novel and general bisimulation up to technique. To better understand how different generics translations approaches work in practice, we benchmark five translators, Go 1.18, two existing monomorphisation translators, our dictionary-passing translator, and erasure translator. Our findings reveal several suggestions for improvements for Go 1.18 -- specifically how to overcome the expressiveness limitations of generic Go, and improve compile time and compiled code size performance of Go 1.18.
TL;DW? Summarizing Instructional Videos with Task Relevance &amp; Cross-Modal Saliency

YouTube users looking for instructions for a specific task may spend a long time browsing content trying to find the right video that matches their needs. Creating a visual summary (abridged version of a video) provides viewers with a quick overview and massively reduces search time. In this work, we focus on summarizing instructional videos, an under-explored area of video summarization. In comparison to generic videos, instructional videos can be parsed into semantically meaningful segments that correspond to important steps of the demonstrated task. Existing video summarization datasets rely on manual frame-level annotations, making them subjective and limited in size. To overcome this, we first automatically generate pseudo summaries for a corpus of instructional videos by exploiting two key assumptions: (i) relevant steps are likely to appear in multiple videos of the same task (Task Relevance), and (ii) they are more likely to be described by the demonstrator verbally (Cross-Modal Saliency). We propose an instructional video summarization network that combines a context-aware temporal video encoder and a segment scoring transformer. Using pseudo summaries as weak supervision, our network constructs a visual summary for an instructional video given only video and transcribed speech. To evaluate our model, we collect a high-quality test set, WikiHow Summaries, by scraping WikiHow articles that contain video demonstrations and visual depictions of steps allowing us to obtain the ground-truth summaries. We outperform several baselines and a state-of-the-art video summarization model on this new benchmark.
A probabilistic view on predictive constructions for Bayesian learning

Given a sequence $X=(X_1,X_2,\ldots)$ of random observations, a Bayesian forecaster aims to predict $X_{n+1}$ based on $(X_1,\ldots,X_n)$ for each $n\ge 0$. To this end, in principle, she only needs to select a collection $\sigma=(\sigma_0,\sigma_1,\ldots)$, called ``strategy" in what follows, where $\sigma_0(\cdot)=P(X_1\in\cdot)$ is the marginal distribution of $X_1$ and $\sigma_n(\cdot)=P(X_{n+1}\in\cdot\mid X_1,\ldots,X_n)$ the $n$-th predictive distribution. Because of the Ionescu-Tulcea theorem, $\sigma$ can be assigned directly, without passing through the usual prior/posterior scheme. One main advantage is that no prior probability is to be selected. In a nutshell, this is the predictive approach to Bayesian learning. A concise review of the latter is provided in this paper. We try to put such an approach in the right framework, to make clear a few misunderstandings, and to provide a unifying view. Some recent results are discussed as well. In addition, some new strategies are introduced and the corresponding distribution of the data sequence $X$ is determined. The strategies concern generalized P\'olya urns, random change points, covariates and stationary sequences.
Teleportation of masked information

Abstract In this contribution, we investigated the possibility of teleporting classical/quantum masked information, which may coded either in a single qubit or qutrit. For this purpose, different systems are used as quantum channels; two-qubit, three-qubit, two qutrit systems, and different protocols are applied. All the teleported masked information are retrieved as masked states at the receiver station. The number of operations that may be performed by the receiver are limited. It is shown that, one can teleport masked classical information with maximum fidelity, while for quantum information the maximization depends on the weight parameter of the tele-ported state. Teleporting the total masked state is better than teleporting its marginals, where the fidelity of total masked state is maximum. However, the fidelity of teleporting masking quantum information via three qubit systems may be maximized by controlling the weight of the initial masked state and the polarization of the mediator. In some cases, the receiver need to diagonalize the final teleported state to maximize its fidelity, and consequently reduces the required local operations.
Multinomial Computation and Factorial Theorems for Artificial Intelligence and Cybersecurity

Combinatorial techniques with factorial and multinomial computation are used as algorithms for the programs development to apply in artificial intelligence and cybersecurity. Methodological advances in combinatorics and mathematics play a vital role in machine learning and cryptology for data analysis and artificial intelligence-based cybersecurity for protection of the computing systems, devices, networks, programs and data from cyber-attacks. In connection with these ideas, this article is prepared for applications in computing science and cybersecurity. This paper presents computing and combinatorial formulae such as theorems on factorials, binomial coefficients, multinomial computation and probability and binomial distributions.
A new way of video compression via forward-referencing using deep learning

To exploit high temporal correlations in video frames of the same scene, the current frame is predicted from the already-encoded reference frames using block-based motion estimation and compensation techniques. While this approach can efficiently exploit the translation motion of the moving objects, it is susceptible to other types of affine motion and object occlusion/deocclusion. Recently, deep learning has been used to model the high-level structure of human pose in specific actions from short videos and then generate virtual frames in future time by predicting the pose using a generative adversarial network (GAN). Therefore, modelling the high-level structure of human pose is able to exploit semantic correlation by predicting human actions and determining its trajectory. Video surveillance applications will benefit as stored big surveillance data can be compressed by estimating human pose trajectories and generating future frames through semantic correlation. This paper explores a new way of video coding by modelling human pose from the already-encoded frames and using the generated frame at the current time as an additional forward-referencing frame. It is expected that the proposed approach can overcome the limitations of the traditional backward-referencing frames by predicting the blocks containing the moving objects with lower residuals. Experimental results show that the proposed approach can achieve on average up to 2.83 dB PSNR gain and 25.93\% bitrate savings for high motion video sequences
Intertwining Ecosystems: A Large Scale Empirical Study of Libraries that Cross Software Ecosystems

An increase in diverse technology stacks and third-party library usage has led developers to inevitably switch technologies. To assist these developers, maintainers have started to release their libraries to multiple technologies, i.e., a cross-ecosystem library. Our goal is to explore the extent to which these cross-ecosystem libraries are intertwined between ecosystems. We perform a large-scale empirical study of 1.1 million libraries from five different software ecosystems, i.e., PyPI for Python, CRAN for R, Maven for Java, RubyGems for Ruby, and NPM for JavaScript to identify 4,146 GitHub projects that release libraries to these five ecosystems. Analyzing their contributions, we first find that a significant majority (median of 37.5%) of contributors of these cross-ecosystem libraries come from a single ecosystem, while also receiving a significant portion of contributions (median of 24.06%) from outside their target ecosystems. We also find that a cross-ecosystem library is written using multiple programming languages. Specifically, three (i.e., PyPI, CRAN, RubyGems) out of the five ecosystems has the majority of source code is written using languages not specific to that ecosystem. As ecosystems become intertwined, this opens up new avenues for research, such as whether or not cross-ecosystem libraries will solve the search for replacement libraries, or how these libraries fit within each ecosystem just to name a few.
Differentiable Inductive Logic Programming in High-Dimensional Space

Synthesizing large logic programs through symbolic Inductive Logic Programming (ILP) typically requires intermediate definitions. However, cluttering the hypothesis space with intensional predicates typically degrades performance. In contrast, gradient descent provides an efficient way to find solutions within such high-dimensional spaces. Neuro-symbolic ILP approaches have not fully exploited this so far. We propose extending the {\delta}ILP approach to inductive synthesis with large-scale predicate invention, thus allowing us to exploit the efficacy of high-dimensional gradient descent. We show that large-scale predicate invention benefits differentiable inductive synthesis through gradient descent and allows one to learn solutions for tasks beyond the capabilities of existing neuro-symbolic ILP systems. Furthermore, we achieve these results without specifying the precise structure of the solution within the language bias.
Development of Augmented Reality Application for Made-to-Order Furniture Industry in Pampanga, Philippines

The focus of the study was to develop a mobile application utilizing marker-less augmented reality for specific made-to-order products to support furniture and fixtures businesses. The study implemented mixed-methodology to properly identify the various stakeholders' considerations in developing the application. Interviews with key informants were conducted to ensure that the features were appropriate for the intended user needs, and selected ISO standards were used as evaluation criteria. The results indicate that the mobile application with marker-less AR technology was found to be highly acceptable by three evaluators (i.e., customers, owners, and IT experts). The study also highlighted the use of AR-related technology in this case, where marker-less has the potential to improve customer purchasing experience even further. Future studies may include using newer technologies to further improve the application. The study suggests that Augmented Reality technology could be used to connect specific businesses directly to consumers regardless of setting or context.
Memory Efficient Temporal &amp; Visual Graph Model for Unsupervised Video Domain Adaptation

Existing video domain adaption (DA) methods need to store all temporal combinations of video frames or pair the source and target videos, which are memory cost expensive and can't scale up to long videos. To address these limitations, we propose a memory-efficient graph-based video DA approach as follows. At first our method models each source or target video by a graph: nodes represent video frames and edges represent the temporal or visual similarity relationship between frames. We use a graph attention network to learn the weight of individual frames and simultaneously align the source and target video into a domain-invariant graph feature space. Instead of storing a large number of sub-videos, our method only constructs one graph with a graph attention mechanism for one video, reducing the memory cost substantially. The extensive experiments show that, compared with the state-of-art methods, we achieved superior performance while reducing the memory cost significantly.
Realistic quantum photonic neural networks

Quantum photonic neural networks are variational photonic circuits that can be trained to implement high-fidelity quantum operations. However, work-to-date has assumed idealized components, including a perfect $π$ Kerr nonlinearity. Here, we investigate the limitations of realistic quantum photonic neural networks that suffer from fabrication imperfections leading to photon loss and imperfect routing, and weak nonlinearities, showing that they can learn to overcome most of these errors. Using the example of a Bell-state analyzer, we demonstrate that there is an optimal network size, which balances imperfections versus the ability to compensate for lacking nonlinearities. With a sub-optimal $π/10$ effective Kerr nonlinearity, we show that a network fabricated with current state-of-the-art processes can achieve an unconditional fidelity of 0.891, that increases to 0.999999 if it is possible to precondition success on the detection of a photon in each logical photonic qubit. Our results provide a guide to the construction of viable, brain-inspired quantum photonic devices for emerging quantum technologies.