Reviewerly/exHarmony huggingface dataset:
Optimal training of integer-valued neural networks with mixed integer programming

Recent work has shown potential in using Mixed Integer Programming (MIP) solvers to optimize certain aspects of neural networks (NNs). However the intriguing approach of training NNs with MIP solvers is under-explored. State-of-the-art-methods to train NNs are typically gradient-based and require significant data, computation on GPUs, and extensive hyper-parameter tuning. In contrast, training with MIP solvers does not require GPUs or heavy hyper-parameter tuning, but currently cannot handle anything but small amounts of data. This article builds on recent advances that train binarized NNs using MIP solvers. We go beyond current work by formulating new MIP models which improve training efficiency and which can train the important class of integer-valued neural networks (INNs). We provide two novel methods to further the potential significance of using MIP to train NNs. The first method optimizes the number of neurons in the NN while training. This reduces the need for deciding on network architecture before training. The second method addresses the amount of training data which MIP can feasibly handle: we provide a batch training method that dramatically increases the amount of data that MIP solvers can use to train. We thus provide a promising step towards using much more data than before when training NNs using MIP models. Experimental results on two real-world data-limited datasets demonstrate that our approach strongly outperforms the previous state of the art in training NN with MIP, in terms of accuracy, training time and amount of data. Our methodology is proficient at training NNs when minimal training data is available, and at training with minimal memory requirements -- which is potentially valuable for deploying to low-memory devices.
Neural Network Weights Do Not Converge to Stationary Points: An Invariant Measure Perspective

This work examines the deep disconnect between existing theoretical analyses of gradient-based algorithms and the practice of training deep neural networks. Specifically, we provide numerical evidence that in large-scale neural network training (e.g., ImageNet + ResNet101, and WT103 + TransformerXL models), the neural network's weights do not converge to stationary points where the gradient of the loss is zero. Remarkably, however, we observe that even though the weights do not converge to stationary points, the progress in minimizing the loss function halts and training loss stabilizes. Inspired by this observation, we propose a new perspective based on ergodic theory of dynamical systems to explain it. Rather than studying the evolution of weights, we study the evolution of the distribution of weights. We prove convergence of the distribution of weights to an approximate invariant measure, thereby explaining how the training loss can stabilize without weights necessarily converging to stationary points. We further discuss how this perspective can better align optimization theory with empirical observations in machine learning practice.
Movement Penalized Bayesian Optimization with Application to Wind Energy Systems

Contextual Bayesian optimization (CBO) is a powerful framework for sequential decision-making given side information, with important applications, e.g., in wind energy systems. In this setting, the learner receives context (e.g., weather conditions) at each round, and has to choose an action (e.g., turbine parameters). Standard algorithms assume no cost for switching their decisions at every round. However, in many practical applications, there is a cost associated with such changes, which should be minimized. We introduce the episodic CBO with movement costs problem and, based on the online learning approach for metrical task systems of Coester and Lee (2019), propose a novel randomized mirror descent algorithm that makes use of Gaussian Process confidence bounds. We compare its performance with the offline optimal sequence for each episode and provide rigorous regret guarantees. We further demonstrate our approach on the important real-world application of altitude optimization for Airborne Wind Energy Systems. In the presence of substantial movement costs, our algorithm consistently outperforms standard CBO algorithms.
Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning

An oft-cited challenge of federated learning is the presence of heterogeneity. \emph{Data heterogeneity} refers to the fact that data from different clients may follow very different distributions. \emph{System heterogeneity} refers to the fact that client devices have different system capabilities. A considerable number of federated optimization methods address this challenge. In the literature, empirical evaluations usually start federated training from random initialization. However, in many practical applications of federated learning, the server has access to proxy data for the training task that can be used to pre-train a model before starting federated training. We empirically study the impact of starting from a pre-trained model in federated learning using four standard federated learning benchmark datasets. Unsurprisingly, starting from a pre-trained model reduces the training time required to reach a target error rate and enables the training of more accurate models (up to 40\%) than is possible when starting from random initialization. Surprisingly, we also find that starting federated learning from a pre-trained initialization reduces the effect of both data and system heterogeneity. We recommend that future work proposing and evaluating federated optimization methods evaluate the performance when starting from random and pre-trained initializations. We also believe this study raises several questions for further work on understanding the role of heterogeneity in federated optimization.
Graph Universal Cycles: Compression and Connections to Universal Cycles

Universal cycles, such as De Bruijn cycles, are cyclic sequences of symbols that represent every combinatorial object from some family exactly once as a consecutive subsequence. Graph universal cycles are a graph analogue of universal cycles introduced in 2010. We introduce graph universal partial cycles, a more compact representation of graph classes, which use "do not know" edges. We show how to construct graph universal partial cycles for labeled graphs, threshold graphs, and permutation graphs. For threshold graphs and permutation graphs, we demonstrate that the graph universal cycles and graph universal partial cycles are closely related to universal cycles and compressed universal cycles, respectively. Using the same connection, for permutation graphs, we define and prove the existence of an $s$-overlap form of graph universal cycles. We also prove the existence of a generalized form of graph universal cycles for unlabeled graphs.
Obstacle Identification and Ellipsoidal Decomposition for Fast Motion Planning in Unknown Dynamic Environments

Collision avoidance in the presence of dynamic obstacles in unknown environments is one of the most critical challenges for unmanned systems. In this paper, we present a method that identifies obstacles in terms of ellipsoids to estimate linear and angular obstacle velocities. Our proposed method is based on the idea of any object can be approximately expressed by ellipsoids. To achieve this, we propose a method based on variational Bayesian estimation of Gaussian mixture model, the Kyachiyan algorithm, and a refinement algorithm. Our proposed method does not require knowledge of the number of clusters and can operate in real-time, unlike existing optimization-based methods. In addition, we define an ellipsoid-based feature vector to match obstacles given two timely close point frames. Our method can be applied to any environment with static and dynamic obstacles, including the ones with rotating obstacles. We compare our algorithm with other clustering methods and show that when coupled with a trajectory planner, the overall system can efficiently traverse unknown environments in the presence of dynamic obstacles.
Physics-aware Differentiable Discrete Codesign for Diffractive Optical Neural Networks

Diffractive optical neural networks (DONNs) have attracted lots of attention as they bring significant advantages in terms of power efficiency, parallelism, and computational speed compared with conventional deep neural networks (DNNs), which have intrinsic limitations when implemented on digital platforms. However, inversely mapping algorithm-trained physical model parameters onto real-world optical devices with discrete values is a non-trivial task as existing optical devices have non-unified discrete levels and non-monotonic properties. This work proposes a novel device-to-system hardware-software codesign framework, which enables efficient physics-aware training of DONNs w.r.t arbitrary experimental measured optical devices across layers. Specifically, Gumbel-Softmax is employed to enable differentiable discrete mapping from real-world device parameters into the forward function of DONNs, where the physical parameters in DONNs can be trained by simply minimizing the loss function of the ML task. The results have demonstrated that our proposed framework offers significant advantages over conventional quantization-based methods, especially with low-precision optical devices. Finally, the proposed algorithm is fully verified with physical experimental optical systems in low-precision settings.
Describing realizable Gauss diagrams using the concepts of parity or bipartate graphs

Two recent publications describe realizable Gauss diagrams using conditions stating that the number of chords in certain sets of chords is even or odd. We demonstrate that these descriptions are incorrect by finding multiple counter-examples. However, the idea of having a parity-based description of realizable Gauss diagrams is attractive. We recall that realizability of Gauss diagrams as touch curves can be described via bipartite graphs. We show that realizable Gauss diagrams can be described via bipartite graphs.
An Efficient Incremental Simple Temporal Network Data Structure for Temporal Planning

One popular technique to solve temporal planning problems consists in decoupling the causal decisions, demanding them to heuristic search, from temporal decisions, demanding them to a simple temporal network (STN) solver. In this architecture, one needs to check the consistency of a series of STNs that are related one another, therefore having methods to incrementally re-use previous computations and that avoid expensive memory duplication is of paramount importance. In this paper, we describe in detail how STNs are used in temporal planning, we identify a clear interface to support this use-case and we present an efficient data-structure implementing this interface that is both time- and memory-efficient. We show that our data structure, called \deltastn, is superior to other state-of-the-art approaches on temporal planning sequences of problems.
FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation

The statistical heterogeneity of the non-independent and identically distributed (non-IID) data in local clients significantly limits the performance of federated learning. Previous attempts like FedProx, SCAFFOLD, MOON, FedNova and FedDyn resort to an optimization perspective, which requires an auxiliary term or re-weights local updates to calibrate the learning bias or the objective inconsistency. However, in addition to previous explorations for improvement in federated averaging, our analysis shows that another critical bottleneck is the poorer optima of client models in more heterogeneous conditions. We thus introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices. We provide theoretical analysis of the possible benefit from FedSkip and conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency. Source code is available at: https://github.com/MediaBrain-SJTU/FedSkip.
PENDIDIKAN LITERASI MEDIA TENTANG UJARAN KEBENCIAN DI KALANGAN ORMAS PEREMPUAN

Hate speech is one of the serious problems related to the presence of the internet, particularly social media. Media literacy education on hate speech is one of the efforts to overcome the problem. This community service was held in Klaseman Village, Sukoharjo. It aims to increase the community service target’s media literacy on hate speech. The community service target is expected to be able to access the internet wisely and avoid the practice of hate speech. The result achieved from the community service is that the target of community service had the knowledge and skills to prevent and counter the spread of hate speech on the internet.
ATIVIDADES FÍSICAS COM PACIENTES ONCOLÓGICOS

The INCA (National Cancer Institute) has been warning of the increase in cancer incidence and mortality both nationally and worldwide, the institute says that the factors that are contributing to this growth are: population growth, aging, as well as the change in the distribution and prevalence of cancer risk factors, especially those associated with socioeconomic development.Therefore, it is necessary to investigate treatment options that enable prevention as well as a good quality of life for patients.Therefore, this study is a literature review which aims to demonstrate and discuss some of the possible advantages of rehabilitation programs with cancer patients that involve exercises in order to highlight the importance of the physical education professional in the treatment.The research selected 36 studies ranging from articles, dissertations, tcc's, reviews, standardization and cataloging of the ministry of health for debates on the subject.With this, we could emphasize that there is growing evidence of the hypothesis that physical activity/physical exercise has a considerable benefit for cancer patients.However, it is up to future research to investigate this relationship between training and group viability so that there is a better clarification of its mechanisms.
Implemetasi algoritma fletcher-reeves dalam menganalisa nilai ekspor menurut golongan sitc

Indonesia has a lot of natural resources, so Indonesia uses them by conducting trade activities between countries. Export is a movement to sell or send goods from within the country to abroad. Conjugateigradient Fletcher-Reeves Algorithm According to some references, it is an appropriate development technique compared to the backpropagation strategy because this strategy can speed up the preparation time to arrive at the minimum convergence value. Furthermore, this research shows whether the algorithm performs well and can provide productive assembly results when used to solve problems because the value corresponds to the Standard International Trade Classification (SITC) class. The prediction models used are 5-10-1, 5-15-1, 5-20-1, 5-25-1 and 5-30-1. MSE of 0.00287273, the minimum among the other four models. The model can be used because it produces a fast combination and a fairly short period
SceDL: A Simultaneous Compression and Encryption Scheme for Deep Learning Models

&lt;p&gt;Efficiently securing and compressing neural network models is a problem of significant interest due to its high popularity in various machine learning and computer vision applications such as industrial automation, autonomous vehicles, surveillance, and medical imaging. Such technologies are necessary while running machine learning models in both resource-constrained edge devices as well as cloud-based servers. These models embody valuable intellectual property that must be protected. Traditional encryption ciphers can provide high security guarantees in order to secure the model, but their sizes are prohibitive for resource-constrained devices. In this paper, we present a simultaneous compression and encryption approach for deep learning models, where the model weights are encrypted using chaotic maps. We claim that employing multiple chaotic maps and a lossless compression method can help us create not only an efficient encryption scheme but also compress the models efficiently in a hardware-friendly manner. This reduces the model storage overheads by 1.51× as compared to the nearest computing work. Additionally, our method is 70% faster and provides much better security guarantees&lt;/p&gt;
Explainable Machine Learning: The Importance of a System-Centric Perspective

&lt;p&gt;This paper provides new perspectives on explainable machine learning and deep learning, from the view point of an explicit system-centric approach. Unlike most existing commentary on the said topic, this paper seeks to explain the fundamentals of the black box problem in machine learning by focusing on explicit system modeling. A convenient yet interesting example is used to illustrate the ideas in a tutorial-like manner thereby making the article accessible to readers from diverse backgrounds. &lt;/p&gt;
Feedback Particle Filter With Stochastically Perturbed Innovation And Its Application to Dual Estimation

Particle filters have, in recent years, been found to perform well in highly nonlinear problems as well as in estimation of parameters. However, there is still the problem of particle degeneracy in particle filters which has led to the invention of, among others, feedback particle filters. In this paper, we introduce a stochastically perturbed feedback particle filter and show that it is exact. The novelty is in the fact that the innovation process is stochastically perturbed. Resampled sinkhorn particle filter is also introduced. We then compare their performance with that of other filters in simultaneous state and parameter estimation.
Imitation Camouflage Synthesis based on Shallow Neural Network

Abstract Deep learning technology has been widely used in the military field, which have achieved great success. The traditional method for painting camouflage either using the background information or the artificial pattern. None of the traditional methods can both consider the background information and camouflage rules. In this paper, A new automatic camouflage generation framework is proposed. A method for generating camouflage pattern is designed. The imitation camouflage pattern is synthesized from the features of both background and artificial pattern. In our method, the texture feature of both background and traditional pattern patches are extracted from the feature maps of shallow neural network (SNN). Based on the feature maps, statistic information of second order differential and mean subtracted contrast normalized coefficients for texture and color is extracted. By iterating to optimize the imitation camouflage to be generated, the statistical information of the imitation camouflage can approximate the characteristic statistical information of the background and pattern. The new generated camouflage pattern can contain the color and texture information of background; besides, it can maintain the traditional patch camouflage criteria. Our approach makes camouflage painting more flexible and allows the target to better infuse into the background. And our method is designed for the preparation of painting camouflage.
Learning to Diversify for Product Question Generation

We address the product question generation task. For a given product description, our goal is to generate questions that reflect potential user information needs that are either missing or not well covered in the description. Moreover, we wish to cover diverse user information needs that may span a multitude of product types. To this end, we first show how the T5 pre-trained Transformer encoder-decoder model can be fine-tuned for the task. Yet, while the T5 generated questions have a reasonable quality compared to the state-of-the-art method for the task (KPCNet), many of such questions are still too general, resulting in a sub-optimal global question diversity. As an alternative, we propose a novel learning-to-diversify (LTD) fine-tuning approach that allows to enrich the language learned by the underlying Transformer model. Our empirical evaluation shows that, using our approach significantly improves the global diversity of the underlying Transformer model, while preserves, as much as possible, its generation relevance.
Preface to the Special Issue “Advancement of Mathematical Methods in Feature Representation Learning for Artificial Intelligence, Data Mining and Robotics”—Special Issue Book

The feature representation learning is the basic task that plays an important role in artificial intelligence, data mining and robotics [...]
Trans Perspective in Software Engineering

&lt;p&gt;The software industry employs few women and underrepresented minorities. Several initiatives from organizations, practitioners, and OSS communities address equity, diversity, and inclusion in software development. However, most of these initiatives deal in a generalist and simplistic way, often confusing diversity with gender diversity, reinforcing an industry with a homogeneous workforce, which has little room for the inclusion of sub represented communities. One of these communities is the transgender, who suffers from several intersections of oppression concerning gender, race, class, age, and disabilities, thus having voices erased in the industry.&lt;/p&gt; &lt;p&gt;Our research aims to identify the concepts of diversity in Software Engineering (SE) from a transgender perspective. We conduct a Systematic Grey Literature Review (GLR) to map the concepts around diversity in SE, focused on trans perspective. We detail our GLR, and inspired in Grounded Theory (GT) we have a coding protocol to build a concept map.&lt;/p&gt; &lt;p&gt;As a result, we obtained a concept map format, which illustrate some of the complexity of diversity issues in SE focused on trans people and how organizations and developers can contribute to trans inclusion in the software industry. &lt;/p&gt; &lt;p&gt;The paper starts a discussion regarding the perspective of trans people in the area. Research in diversity should observe the oppressions that exist and how they interact without simplifying and isolating issues such as treating gender in a binary way and not interconnected with other issues such as race. SE industry is strategic to the trans community. Although communities declare themselves to be diverse and open to entry, concrete actions are lacking. &lt;/p&gt;
Gromov-Wasserstein Autoencoders

Variational Autoencoder (VAE)-based generative models offer flexible representation learning by incorporating meta-priors, general premises considered beneficial for downstream tasks. However, the incorporated meta-priors often involve ad-hoc model deviations from the original likelihood architecture, causing undesirable changes in their training. In this paper, we propose a novel representation learning method, Gromov-Wasserstein Autoencoders (GWAE), which directly matches the latent and data distributions using the variational autoencoding scheme. Instead of likelihood-based objectives, GWAE models minimize the Gromov-Wasserstein (GW) metric between the trainable prior and given data distributions. The GW metric measures the distance structure-oriented discrepancy between distributions even with different dimensionalities, which provides a direct measure between the latent and data spaces. By restricting the prior family, we can introduce meta-priors into the latent space without changing their objective. The empirical comparisons with VAE-based models show that GWAE models work in two prominent meta-priors, disentanglement and clustering, with their GW objective unchanged.