Reviewerly/exHarmony huggingface dataset:
Is Importance Weighting Incompatible with Interpolating Classifiers?

Importance weighting is a classic technique to handle distribution shifts. However, prior work has presented strong empirical and theoretical evidence demonstrating that importance weights can have little to no effect on overparameterized neural networks. Is importance weighting truly incompatible with the training of overparameterized neural networks? Our paper answers this in the negative. We show that importance weighting fails not because of the overparameterization, but instead, as a result of using exponentially-tailed losses like the logistic or cross-entropy loss. As a remedy, we show that polynomially-tailed losses restore the effects of importance reweighting in correcting distribution shift in overparameterized models. We characterize the behavior of gradient descent on importance weighted polynomially-tailed losses with overparameterized linear models, and theoretically demonstrate the advantage of using polynomially-tailed losses in a label shift setting. Surprisingly, our theory shows that using weights that are obtained by exponentiating the classical unbiased importance weights can improve performance. Finally, we demonstrate the practical value of our analysis with neural network experiments on a subpopulation shift and a label shift dataset. When reweighted, our loss function can outperform reweighted cross-entropy by as much as 9% in test accuracy. Our loss function also gives test accuracies comparable to, or even exceeding, well-tuned state-of-the-art methods for correcting distribution shifts.
Adjusting for Bias with Procedural Data

3D softwares are now capable of producing highly realistic images that look nearly indistinguishable from the real images. This raises the question: can real datasets be enhanced with 3D rendered data? We investigate this question. In this paper we demonstrate the use of 3D rendered data, procedural, data for the adjustment of bias in image datasets. We perform error analysis of images of animals which shows that the misclassification of some animal breeds is largely a data issue. We then create procedural images of the poorly classified breeds and that model further trained on procedural data can better classify poorly performing breeds on real data. We believe that this approach can be used for the enhancement of visual data for any underrepresented group, including rare diseases, or any data bias potentially improving the accuracy and fairness of models. We find that the resulting representations rival or even out-perform those learned directly from real data, but that good performance requires care in the 3D rendered procedural data generation. 3D image dataset can be viewed as a compressed and organized copy of a real dataset, and we envision a future where more and more procedural data proliferate while datasets become increasingly unwieldy, missing, or private. This paper suggests several techniques for dealing with visual representation learning in such a future.
A Rate-Distortion Perspective on Quantum State Redistribution

We consider a rate-distortion version of the quantum state redistribution task, where the error of the decoded state is judged via an additive distortion measure; it thus constitutes a quantum generalisation of the classical Wyner-Ziv problem. The quantum source is described by a tripartite pure state shared between Alice ($A$, encoder), Bob ($B$, decoder) and a reference ($R$). Both Alice and Bob are required to output a system ($\widetilde{A}$ and $\widetilde{B}$, respectively), and the distortion measure is encoded in an observable on $\widetilde{A}\widetilde{B}R$. It includes as special cases most quantum rate-distortion problems considered in the past, and in particular quantum data compression with the fidelity measured per copy; furthermore, it generalises the well-known state merging and quantum state redistribution tasks for a pure state source, with per-copy fidelity, and a variant recently considered by us, where the source is an ensemble of pure states [1], [2]. We derive a single-letter formula for the rate-distortion function of compression schemes assisted by free entanglement. A peculiarity of the formula is that in general it requires optimisation over an unbounded auxiliary register, so the rate-distortion function is not readily computable from our result, and there is a continuity issue at zero distortion. However, we show how to overcome these difficulties in certain situations.
Generalizing Interactive Backpropagating Refinement for Dense Prediction

As deep neural networks become the state-of-the-art approach in the field of computer vision for dense prediction tasks, many methods have been developed for automatic estimation of the target outputs given the visual inputs. Although the estimation accuracy of the proposed automatic methods continues to improve, interactive refinement is oftentimes necessary for further correction. Recently, feature backpropagating refinement scheme (f-BRS) has been proposed for the task of interactive segmentation, which enables efficient optimization of a small set of auxiliary variables inserted into the pretrained network to produce object segmentation that better aligns with user inputs. However, the proposed auxiliary variables only contain channel-wise scale and bias, limiting the optimization to global refinement only. In this work, in order to generalize backpropagating refinement for a wide range of dense prediction tasks, we introduce a set of G-BRS (Generalized Backpropagating Refinement Scheme) layers that enable both global and localized refinement for the following tasks: interactive segmentation, semantic segmentation, image matting and monocular depth estimation. Experiments on SBD, Cityscapes, Mapillary Vista, Composition-1k and NYU-Depth-V2 show that our method can successfully generalize and significantly improve performance of existing pretrained state-of-the-art models with only a few clicks.
On Minimal and Maximal High Utility Itemsets Mining using Propositional Satisfiability

Computing high utility motifs is a fundamental data mining method for discovering useful itemsets yielding high utility values. Minimal and maximal high utility itemsets are two examples of compact representations used to reduce the output size due to the large and incomprehensible number of patterns. In this paper, we present a novel method for mining minimal and maximal high utility itemsets using propositional satisfiability. First, we show that minimal and maximal high utility patterns are X-minimal models of a CNF formula. Then, to improve the scalability issue of our method, we harness a decomposition paradigm that splits the transaction database into smaller and independent transaction sub-bases, allowing an efficient enumeration of minimal and maximal high utility itemsets. Finally, through extensive evaluation studies on various real-world datasets, we demonstrate that our approach is very competitive w.r.t. to the state-of-the-art specialized solutions.
Online Continual Learning for Embedded Devices

Real-time on-device continual learning is needed for new applications such as home robots, user personalization on smartphones, and augmented/virtual reality headsets. However, this setting poses unique challenges: embedded devices have limited memory and compute capacity and conventional machine learning models suffer from catastrophic forgetting when updated on non-stationary data streams. While several online continual learning models have been developed, their effectiveness for embedded applications has not been rigorously studied. In this paper, we first identify criteria that online continual learners must meet to effectively perform real-time, on-device learning. We then study the efficacy of several online continual learning methods when used with mobile neural networks. We measure their performance, memory usage, compute requirements, and ability to generalize to out-of-domain inputs.
An Efficient and Robust Committee Structure for Sharding Blockchain

Nowadays, sharding is deemed as a promising way to save traditional blockchain protocols from their low scalability. However, such technique also brings several potential risks and huge communication overheads. An improper design may give rise to the inconsistent state among different committees. Further, the communication overheads arising from cross-shard transactions unfortunately reduce the system's performance. In this paper, we first summarize five essential issues that all sharding blockchain designers face. For each issue, we discuss its key challenge and propose our suggested solutions. In order to break the performance bottlenecks, we propose a reputation mechanism for selecting leaders. The term of reputation in our design reflects each node's honest computation resources. In addition, we introduce a referee committee and partial sets in each committee, and design a recovery procedure in case the leader is malicious. Under the design, we prove that malicious leaders will not hurt the system and will be evicted. Furthermore, we conduct a series of simulations to evaluate our design. The results show that selecting leaders by the reputation can dramatically improve the system performance.
The Shift-Dimension of Multipersistence Modules

We present the shift-dimension of multipersistence modules and investigate its algebraic properties. This gives rise to a new invariant of multigraded modules over the multivariate polynomial ring arising from the hierarchical stabilization of the zeroth total multigraded Betti number. We give a fast algorithm for the computation of the shift-dimension of interval modules in the bivariate case. We construct multipersistence contours that are parameterized by multivariate functions and hence provide a large class of feature maps for machine learning tasks.
Comparison of Segmentation-Based Image Compression Using Threshold, Region Growing, and Edge Detection

In the presenting paper we are dealing with to develop a lossless image compression (IC) method to utilize spatial redundancies inbuilt in image data which employs a best possible amount of segmentation information. To obtaining Multiscale segmentation we are using a earlier proposed transform which gives a tree-structured segmentation of the picture into regions which are identified by grayscale homogeneity. In the given proposed algor we have to shorten the tree to controlling the size and no of regions so that we can get a rate balance between the derived the coding gain and the operating cost inherent in coding the segmented data. Another uniqueness of the given proposed approach is that we are using an image model contain individual descriptions of the pixels lying close to the edges of a section and others lying in the center. In our results we can see that this proposed algorithm is providing better performance comparable to all the best available methods and it provides 15-20% better compression if we compare it with the JPEG lossless image compression standard for a enormous variety of images.
Knowledge-Enhanced Interactive Matching Network for Multi-turn Response Selection in Medical Dialogue Systems

Recently, the response selection for retrieval-based dialogue systems has gained enormous attention from both academic and industrial communities. Although the previous methods achieve promising results for intelligent customer service systems and open-domain chatbots, the response selection in medical dialogues suffers from lower performance because of the strong dependency on the domain knowledge. In this paper, we construct two specialized medical knowledge bases and propose a Knowledge-enhanced Interactive Matching Network (KIMN) for multi-turn response selection in medical dialogue systems. Compared with previous response selection approaches, the KIMN adopts pre-trained language model to alleviate the limited training data problem, and incorporates internal and external medical domain knowledge to perform interactive matching between responses and contexts. The experiments on a real-world medical dialogue dataset show that our proposed model consistently outperforms the strong baseline methods by large margins, which shows that our proposed model can retrieve more accurate responses for medical dialogue systems.
Almost balanced and uncorrelated quaternary sequence pairs of even length

Given a partition of &lt;inline-formula&gt;&lt;tex-math id="M1"&gt;\begin{document}$ \mathbb{Z}_N^* $\end{document}&lt;/tex-math&gt;&lt;alternatives&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="JUST-2021-0234_M1.jpg"/&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="JUST-2021-0234_M1.png"/&gt;&lt;/alternatives&gt;&lt;/inline-formula&gt; into four subsets, we present a generic construction of uncorrelated quaternary sequence pairs of length &lt;inline-formula&gt;&lt;tex-math id="M2"&gt;\begin{document}$ 2N $\end{document}&lt;/tex-math&gt;&lt;alternatives&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="JUST-2021-0234_M2.jpg"/&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="JUST-2021-0234_M2.png"/&gt;&lt;/alternatives&gt;&lt;/inline-formula&gt; using the interleaved technique based on this partition. By choosing partitions arising from cyclotomic classes of order 4 and 8 over &lt;inline-formula&gt;&lt;tex-math id="M5"&gt;\begin{document}$ \mathbb{Z}_p $\end{document}&lt;/tex-math&gt;&lt;alternatives&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="JUST-2021-0234_M5.jpg"/&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="JUST-2021-0234_M5.png"/&gt;&lt;/alternatives&gt;&lt;/inline-formula&gt;, we construct uncorrelated quaternary sequence pairs of length &lt;inline-formula&gt;&lt;tex-math id="M6"&gt;\begin{document}$ 2p $\end{document}&lt;/tex-math&gt;&lt;alternatives&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="JUST-2021-0234_M6.jpg"/&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="JUST-2021-0234_M6.png"/&gt;&lt;/alternatives&gt;&lt;/inline-formula&gt;, which are almost balanced and have low autocorrelation, except at a few positions.
Investigating Effect of Dialogue History in Multilingual Task Oriented Dialogue Systems

While the English virtual assistants have achieved exciting performance with an enormous amount of training resources, the needs of non-English-speakers have not been satisfied well. Up to Dec 2021, Alexa, one of the most popular smart speakers around the world, is able to support 9 different languages [1], while there are thousands of languages in the world, 91 of which are spoken by more than 10 million people according to statistics published in 2019 [2]. However, training a virtual assistant in other languages than English is often more difficult, especially for those low-resource languages. The lack of high-quality training data restricts the performance of models, resulting in poor user satisfaction. Therefore, we devise an efficient and effective training solution for multilingual task-orientated dialogue systems, using the same dataset generation pipeline and end-to-end dialogue system architecture as BiToD[5], which adopted some key design choices for a minimalistic natural language design where formal dialogue states are used in place of natural language inputs. This reduces the room for error brought by weaker natural language models, and ensures the model can correctly extract the essential slot values needed to perform dialogue state tracking (DST). Our goal is to reduce the amount of natural language encoded at each turn, and the key parameter we investigate is the number of turns (H) to feed as history to model. We first explore the turning point where increasing H begins to yield limiting returns on the overall performance. Then we examine whether the examples a model with small H gets wrong can be categorized in a way for the model to do few-shot finetuning on. Lastly, will explore the limitations of this approach, and whether there is a certain type of examples that this approach will not be able to resolve.
PARL: Enhancing Diversity of Ensemble Networks to Resist Adversarial Attacks via Pairwise Adversarially Robust Loss Function

The security of Deep Learning classifiers is a critical field of study because of the existence of adversarial attacks. Such attacks usually rely on the principle of transferability, where an adversarial example crafted on a surrogate classifier tends to mislead the target classifier trained on the same dataset even if both classifiers have quite different architecture. Ensemble methods against adversarial attacks demonstrate that an adversarial example is less likely to mislead multiple classifiers in an ensemble having diverse decision boundaries. However, recent ensemble methods have either been shown to be vulnerable to stronger adversaries or shown to lack an end-to-end evaluation. This paper attempts to develop a new ensemble methodology that constructs multiple diverse classifiers using a Pairwise Adversarially Robust Loss (PARL) function during the training procedure. PARL utilizes gradients of each layer with respect to input in every classifier within the ensemble simultaneously. The proposed training procedure enables PARL to achieve higher robustness against black-box transfer attacks compared to previous ensemble methods without adversely affecting the accuracy of clean examples. We also evaluate the robustness in the presence of white-box attacks, where adversarial examples are crafted using parameters of the target classifier. We present extensive experiments using standard image classification datasets like CIFAR-10 and CIFAR-100 trained using standard ResNet20 classifier against state-of-the-art adversarial attacks to demonstrate the robustness of the proposed ensemble methodology.
Towards Efficient Data-Centric Robust Machine Learning with Noise-based Augmentation

The data-centric machine learning aims to find effective ways to build appropriate datasets which can improve the performance of AI models. In this paper, we mainly focus on designing an efficient data-centric scheme to improve robustness for models towards unforeseen malicious inputs in the black-box test settings. Specifically, we introduce a noised-based data augmentation method which is composed of Gaussian Noise, Salt-and-Pepper noise, and the PGD adversarial perturbations. The proposed method is built on lightweight algorithms and proved highly effective based on comprehensive evaluations, showing good efficiency on computation cost and robustness enhancement. In addition, we share our insights about the data-centric robust machine learning gained from our experiments.
On the Effectiveness of Sparsification for Detecting the Deep Unknowns

Detecting out-of-distribution (OOD) inputs is a central challenge for safely deploying machine learning models in the real world. Previous methods commonly rely on an OOD score derived from the overparameterized weight space, while largely overlooking the role of sparsification. In this paper, we reveal important insights that reliance on unimportant weights and units can directly attribute to the brittleness of OOD detection. To mitigate the issue, we propose a sparsification-based OOD detection framework termed DICE. Our key idea is to rank weights based on a measure of contribution, and selectively use the most salient weights to derive the output for OOD detection. We provide both empirical and theoretical insights, characterizing and explaining the mechanism by which DICE improves OOD detection. By pruning away noisy signals, DICE provably reduces the output variance for OOD data, resulting in a sharper output distribution and stronger separability from ID data. DICE establishes superior performance, reducing the FPR95 by up to 24.69% compared to the previous best method.
Accountable Javascript Code Delivery

The internet is a major distribution platform for web applications, but there are no effective transparency and audit mechanisms in place for the web. Due to the ephemeral nature of web applications, a client visiting a website has no guarantee that the code it receives today is the same as yesterday, or the same as other visitors receive. Despite advances in web security, it is thus challenging to audit web applications before they are rendered in the browser. We propose Accountable JS, a browser extension and opt in protocol for accountable delivery of active content on a web page. We prototype our protocol, formally model its security properties with the Tamarin Prover, and evaluate its compatibility and performance impact with case studies including WhatsApp Web, AdSense and Nimiq. Accountability is beginning to be deployed at scale, with Meta's recent announcement of Code Verify available to all 2 billion WhatsApp users, but there has been little formal analysis of such protocols. We formally model Code Verify using the Tamarin Prover and compare its properties to our Accountable JS protocol. We also compare Code Verify's and Accountable JS extension's performance impacts on WhatsApp Web.
RB2: Robotic Manipulation Benchmarking with a Twist

Benchmarks offer a scientific way to compare algorithms using objective performance metrics. Good benchmarks have two features: (a) they should be widely useful for many research groups; (b) and they should produce reproducible findings. In robotic manipulation research, there is a trade-off between reproducibility and broad accessibility. If the benchmark is kept restrictive (fixed hardware, objects), the numbers are reproducible but the setup becomes less general. On the other hand, a benchmark could be a loose set of protocols (e.g. object sets) but the underlying variation in setups make the results non-reproducible. In this paper, we re-imagine benchmarking for robotic manipulation as state-of-the-art algorithmic implementations, alongside the usual set of tasks and experimental protocols. The added baseline implementations will provide a way to easily recreate SOTA numbers in a new local robotic setup, thus providing credible relative rankings between existing approaches and new work. However, these local rankings could vary between different setups. To resolve this issue, we build a mechanism for pooling experimental data between labs, and thus we establish a single global ranking for existing (and proposed) SOTA algorithms. Our benchmark, called Ranking-Based Robotics Benchmark (RB2), is evaluated on tasks that are inspired from clinically validated Southampton Hand Assessment Procedures. Our benchmark was run across two different labs and reveals several surprising findings. For example, extremely simple baselines like open-loop behavior cloning, outperform more complicated models (e.g. closed loop, RNN, Offline-RL, etc.) that are preferred by the field. We hope our fellow researchers will use RB2 to improve their research's quality and rigor.
Utilization of the Prim Algorithm to Determine the Nearest Path Car Transportation Problems of Goods Carrier Box

Prim’s algorithm has been proven to be able to be used in determining the closest path, minimum spanning tree in the problem of transporting box cars carrying goods. Based on the results of theoretical analysis tests that have been carried out from several experimental data, it can be obtained results showing that, the results of Prim’s algorithm complexity are as follows O(n2). By defining n here is the number of vertices contained in the graph. So it can be concluded that, empirically, the data from the average travel time of this prim algorithm shows a graph that has a quadratic nature. As for the large number of edges contained in a graph, it has no effect on the complexity of finding the nearest path, the problem of transporting box cars carrying goods is a minimum spanning tree using the Prims algorithm.