OpenRaiser/Intern-Atlas huggingface dataset:
Variable Functioning and Its Application to Large Scale Steel Frame Design Optimization
Venue: arXiv cs.CE 2022
Year: 2022

To solve complex real-world problems, heuristics and concept-based approaches can be used in order to incorporate information into the problem. In this study, a concept-based approach called variable functioning Fx is introduced to reduce the optimization variables and narrow down the search space. In this method, the relationships among one or more subset of variables are defined with functions using information prior to optimization; thus, instead of modifying the variables in the search process, the function variables are optimized. By using problem structure analysis technique and engineering expert knowledge, the $Fx$ method is used to enhance the steel frame design optimization process as a complex real-world problem. The proposed approach is coupled with particle swarm optimization and differential evolution algorithms and used for three case studies. The algorithms are applied to optimize the case studies by considering the relationships among column cross-section areas. The results show that $Fx$ can significantly improve both the convergence rate and the final design of a frame structure, even if it is only used for seeding.
Classifiers are Better Experts for Controllable Text Generation
Venue: arXiv cs.CL 2022
Year: 2022

This paper proposes a simple method for controllable text generation based on weighting logits with a free-form classifier, namely CAIF sampling. Using an arbitrary text classifier, we adjust a small part of a language model's logits and guide text generation towards or away from classifier prediction. We experimented with toxicity avoidance and sentiment control tasks and showed that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and task accuracy metrics based on the external classifier of generated texts. In addition, compared to other approaches, it is easier to implement and tune and has significantly fewer restrictions and requirements.
cMelGAN: An Efficient Conditional Generative Model Based on Mel Spectrograms
Venue: arXiv cs.SD 2022
Year: 2022

Analysing music in the field of machine learning is a very difficult problem with numerous constraints to consider. The nature of audio data, with its very high dimensionality and widely varying scales of structure, is one of the primary reasons why it is so difficult to model. There are many applications of machine learning in music, like the classifying the mood of a piece of music, conditional music generation, or popularity prediction. The goal for this project was to develop a genre-conditional generative model of music based on Mel spectrograms and evaluate its performance by comparing it to existing generative music models that use note-based representations. We initially implemented an autoregressive, RNN-based generative model called MelNet . However, due to its slow speed and low fidelity output, we decided to create a new, fully convolutional architecture that is based on the MelGAN [4] and conditional GAN architectures, called cMelGAN.
Attack vs Benign Network Intrusion Traffic Classification
Venue: arXiv cs.CR 2022
Year: 2022

Intrusion detection systems (IDS) are used to monitor networks or systems for attack activity or policy violations. Such a system should be able to successfully identify anomalous deviations from normal traffic behavior. Here we discuss the machine learning approach to building an anomaly-based IDS using the CSE-CIC-IDS2018 dataset. Since the publication of this dataset a relatively large number of papers have been published, most of them presenting IDS architectures and results based on complex machine learning methods, like deep neural networks, gradient boosting classifiers, or hidden Markov models. Here we show that similar results can be obtained using a very simple nearest neighbor classification approach, avoiding the inherent complications of training such complex models. The advantages of the nearest neighbor algorithm are: (1) it is very simple to implement; (2) it is extremely robust; (3) it has no parameters, and therefore it cannot overfit the data. This result also shows that currently there is a trend of developing over-engineered solutions in the machine learning community. Such solutions are based on complex methods, like deep learning neural networks, without even considering baseline solutions corresponding to simple, but efficient methods.
What is an equivariant neural network?
Venue: arXiv cs.LG 2022
Year: 2022

We explain equivariant neural networks, a notion underlying breakthroughs in machine learning from deep convolutional neural networks for computer vision to AlphaFold 2 for protein structure prediction, without assuming knowledge of equivariance or neural networks. The basic mathematical ideas are simple but are often obscured by engineering complications that come with practical realizations. We extract and focus on the mathematical aspects, and limit ourselves to a cursory treatment of the engineering issues at the end.
Locating the Closest Singularity in a Polynomial Homotopy
Venue: arXiv cs.SC 2022
Year: 2022

A polynomial homotopy is a family of polynomial systems, where the systems in the family depend on one parameter. If for one value of the parameter we know a regular solution, then what is the nearest value of the parameter for which the solution in the polynomial homotopy is singular? For this problem we apply the ratio theorem of Fabry. Richardson extrapolation is effective to accelerate the convergence of the ratios of the coefficients of the series expansions of the solution paths defined by the homotopy. For numerical stability, we recondition the homotopy. To compute the coefficients of the series we propose the quaternion Fourier transform. We locate the closest singularity computing at a regular solution, avoiding numerical difficulties near a singularity.
What GPT Knows About Who is Who
Venue: arXiv cs.CL 2022
Year: 2022

Coreference resolution -- which is a crucial task for understanding discourse and language at large -- has yet to witness widespread benefits from large language models (LLMs). Moreover, coreference resolution systems largely rely on supervised labels, which are highly expensive and difficult to annotate, thus making it ripe for prompt engineering. In this paper, we introduce a QA-based prompt-engineering method and discern \textit{generative}, pre-trained LLMs' abilities and limitations toward the task of coreference resolution. Our experiments show that GPT-2 and GPT-Neo can return valid answers, but that their capabilities to identify coreferent mentions are limited and prompt-sensitive, leading to inconsistent results.
Roman: Making Everyday Objects Robotically Manipulable with 3D-Printable Add-on Mechanisms
Venue: arXiv cs.RO 2022
Year: 2022

One important vision of robotics is to provide physical assistance by manipulating different everyday objects, e.g., hand tools, kitchen utensils. However, many objects designed for dexterous hand-control are not easily manipulable by a single robotic arm with a generic parallel gripper. Complementary to existing research on developing grippers and control algorithms, we present Roman, a suite of hardware design and software tool support for robotic engineers to create 3D printable mechanisms attached to everyday handheld objects, making them easier to be manipulated by conventional robotic arms. The Roman hardware comes with a versatile magnetic gripper that can snap on/off handheld objects and drive add-on mechanisms to perform tasks. Roman also provides software support to register and author control programs. To validate our approach, we designed and fabricated Roman mechanisms for 14 everyday objects/tasks presented within a design space and conducted expert interviews with robotic engineers indicating that Roman serves as a practical alternative for enabling robotic manipulation of everyday objects.
Joint Distribution of Ages of Information in Networks
Venue: arXiv cs.IT 2022
Year: 2022

We study a general setting of status updating systems in which a set of source nodes provide status updates about some physical process(es) to a set of monitors. The freshness of information available at each monitor is quantified in terms of the Age of Information (AoI), and the vector of AoI processes at the monitors (or equivalently the age vector) models the continuous state of the system. While the marginal distributional properties of each AoI process have been studied for a variety of settings using the stochastic hybrid system (SHS) approach, we lack a counterpart of this approach to systematically study their joint distributional properties. Developing such a framework is the main contribution of this paper. In particular, we model the discrete state of the system as a finite-state continuous-time Markov chain, and describe the coupled evolution of the continuous and discrete states of the system by a piecewise linear SHS with linear reset maps. Using the notion of tensors, we first derive first-order linear differential equations for the temporal evolution of both the joint moments and the joint moment generating function (MGF) for an arbitrary set of age processes. We then characterize the conditions under which the derived differential equations are asymptotically stable. The generality of our framework is demonstrated by recovering several existing results as special cases. Finally, we apply our framework to derive closed-form expressions of the stationary joint MGF in a multi-source updating system under non-preemptive and source-agnostic/source-aware preemptive in service queueing disciplines.
Constant Power Root Market Makers
Venue: arXiv cs.CE 2022
Year: 2022

The paper introduces a new type of constant function market maker, the constant power root market marker. We show that the constant sum (used by mStable), constant product (used by Uniswap and Balancer), constant reserve (HOLD-ing), and constant harmonic mean trading functions are special cases of the constant power root trading function. We derive the value function for liquidity providers, marginal price function, price impact function, impermanent loss function, and greeks for constant power root market markers. In particular, we find that as the power q varies from the range of -infinity to 1, the power root function interpolates between the harmonic (q=-1), geometric (q=0), and arithmetic (q=1) means. This provides a toggle that trades off between price slippage for traders and impermanent loss for liquidity providers. As the power q approaches 1, slippage is low and impermanent loss is high. As q approaches to -1, price slippage increases and impermanent loss decreases.
Gradient Descent Optimizes Infinite-Depth ReLU Implicit Networks with Linear Widths
Venue: arXiv cs.LG 2022
Year: 2022

Implicit deep learning has recently become popular in the machine learning community since these implicit models can achieve competitive performance with state-of-the-art deep networks while using significantly less memory and computational resources. However, our theoretical understanding of when and how first-order methods such as gradient descent (GD) converge on \textit{nonlinear} implicit networks is limited. Although this type of problem has been studied in standard feed-forward networks, the case of implicit models is still intriguing because implicit networks have \textit{infinitely} many layers. The corresponding equilibrium equation probably admits no or multiple solutions during training. This paper studies the convergence of both gradient flow (GF) and gradient descent for nonlinear ReLU activated implicit networks. To deal with the well-posedness problem, we introduce a fixed scalar to scale the weight matrix of the implicit layer and show that there exists a small enough scaling constant, keeping the equilibrium equation well-posed throughout training. As a result, we prove that both GF and GD converge to a global minimum at a linear rate if the width $m$ of the implicit network is \textit{linear} in the sample size $N$, i.e., $m=Ω(N)$.
Manifold Characteristics That Predict Downstream Task Performance
Venue: arXiv cs.LG 2022
Year: 2022

Pretraining methods are typically compared by evaluating the accuracy of linear classifiers, transfer learning performance, or visually inspecting the representation manifold's (RM) lower-dimensional projections. We show that the differences between methods can be understood more clearly by investigating the RM directly, which allows for a more detailed comparison. To this end, we propose a framework and new metric to measure and compare different RMs. We also investigate and report on the RM characteristics for various pretraining methods. These characteristics are measured by applying sequentially larger local alterations to the input data, using white noise injections and Projected Gradient Descent (PGD) adversarial attacks, and then tracking each datapoint. We calculate the total distance moved for each datapoint and the relative change in distance between successive alterations. We show that self-supervised methods learn an RM where alterations lead to large but constant size changes, indicating a smoother RM than fully supervised methods. We then combine these measurements into one metric, the Representation Manifold Quality Metric (RMQM), where larger values indicate larger and less variable step sizes, and show that RMQM correlates positively with performance on downstream tasks.
Computing Bend-Minimum Orthogonal Drawings of Plane Series-Parallel Graphs in Linear Time
Venue: arXiv cs.CG 2022
Year: 2022

A planar orthogonal drawing of a planar 4-graph G (i.e., a planar graph with vertex-degree at most four) is a crossing-free drawing that maps each vertex of G to a distinct point of the plane and each edge of $G$ to a sequence of horizontal and vertical segments between its end-points. A longstanding open question in Graph Drawing, dating back over 30 years, is whether there exists a linear-time algorithm to compute an orthogonal drawing of a plane 4-graph with the minimum number of bends. The term "plane" indicates that the input graph comes together with a planar embedding, which must be preserved by the drawing (i.e., the drawing must have the same set of faces as the input graph). In this paper, we positively answer the question above for the widely-studied class of series-parallel graphs. Our linear-time algorithm is based on a characterization of the planar series-parallel graphs that admit an orthogonal drawing without bends. This characterization is given in terms of the orthogonal spirality that each type of triconnected component of the graph can take; the orthogonal spirality of a component measures how much that component is "rolled-up" in an orthogonal drawing of the graph.
Crowdsourced Hypothesis Generation and their Verification: A Case Study on Sleep Quality Improvement
Venue: arXiv cs.HC 2022
Year: 2022

A clinical study is often necessary for exploring important research questions; however, this approach is sometimes time and money consuming. Another extreme approach, which is to collect and aggregate opinions from crowds, provides a result drawn from the crowds' past experiences and knowledge. To explore a solution that takes advantage of both the rigid clinical approach and the crowds' opinion-based approach, we design a framework that exploits crowdsourcing as a part of the research process, whereby crowd workers serve as if they were a scientist conducting a "pseudo" prospective study. This study evaluates the feasibility of the proposed framework to generate hypotheses on a specified topic and verify them in the real world by employing many crowd workers. The framework comprises two phases of crowd-based workflow. In Phase 1 - the hypothesis generation and ranking phase - our system asks workers two types of questions to collect a number of hypotheses and rank them. In Phase 2 - the hypothesis verification phase - the system asks workers to verify the top-ranked hypotheses from Phase 1 by implementing one of them in real life. Through experiments, we explore the potential and limitations of the framework to generate and evaluate hypotheses about the factors that result in a good night's sleep. Our results on significant sleep quality improvement show the basic feasibility of our framework, suggesting that crowd-based research is compatible with experts' knowledge in a certain domain.
Residual Local Feature Network for Efficient Super-Resolution
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning based approaches has achieved great performance in single image super-resolution (SISR). However, recent advances in efficient super-resolution focus on reducing the number of parameters and FLOPs, and they aggregate more powerful features by improving feature utilization through complex layer connection strategies. These structures may not be necessary to achieve higher running speed, which makes them difficult to be deployed to resource-constrained devices. In this work, we propose a novel Residual Local Feature Network (RLFN). The main idea is using three convolutional layers for residual local feature learning to simplify feature aggregation, which achieves a good trade-off between model performance and inference time. Moreover, we revisit the popular contrastive loss and observe that the selection of intermediate features of its feature extractor has great influence on the performance. Besides, we propose a novel multi-stage warm-start training strategy. In each stage, the pre-trained weights from previous stages are utilized to improve the model performance. Combined with the improved contrastive loss and training strategy, the proposed RLFN outperforms all the state-of-the-art efficient image SR models in terms of runtime while maintaining both PSNR and SSIM for SR. In addition, we won the first place in the runtime track of the NTIRE 2022 efficient super-resolution challenge. Code will be available at https://github.com/fyan111/RLFN.
A Two-Phase Method for Production Planning and Machine Speed Optimization Problem
Venue: arXiv cs.CE 2022
Year: 2022

Textile industry is becoming a highly competitive area with the increase in demand for textile products. Since expanding the production capacity is not always feasible, optimizing the existing system is more practical. In particular, we consider a felt production system of a textile factory operating in Turkey in this study. We aim to minimize the production costs by optimizing machine operating speeds as well as building an efficient production lot sizing plan within the planning horizon. In this direction, we propose the Lot Sizing and Machine Speed (LSMS) nonlinear model to determine the optimal unit processing times and production quantities while minimizing the work-in-process and end item inventories by changing the machine operating speeds dynamically according to the demands. Since LSMS nonlinear optimization problem is NP-hard, we design a Two-Phase heuristic which iteratively processes a linear programming model by utilizing a commercial solver at each phase. We intensively test our Two-Phase heuristic via randomly generated demand, planning horizon and machine-hour capacity scenarios. Our computational experiments show that the introduced Two-Phase heuristic can find the local-optimal results in acceptable amount of time.
Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences
Venue: arXiv cs.CL 2022
Year: 2022

The increase in performance in NLP due to the prevalence of distributional models and deep learning has brought with it a reciprocal decrease in interpretability. This has spurred a focus on what neural networks learn about natural language with less of a focus on how. Some work has focused on the data used to develop data-driven models, but typically this line of work aims to highlight issues with the data, e.g. highlighting and offsetting harmful biases. This work contributes to the relatively untrodden path of what is required in data for models to capture meaningful representations of natural language. This entails evaluating how well English and Spanish semantic spaces capture a particular type of relational knowledge, namely the traits associated with concepts (e.g. bananas-yellow), and exploring the role of co-occurrences in this context.
Model Agnostic Local Explanations of Reject
Venue: arXiv cs.AI 2022
Year: 2022

The application of machine learning based decision making systems in safety critical areas requires reliable high certainty predictions. Reject options are a common way of ensuring a sufficiently high certainty of predictions made by the system. While being able to reject uncertain samples is important, it is also of importance to be able to explain why a particular sample was rejected. However, explaining general reject options is still an open problem. We propose a model agnostic method for locally explaining arbitrary reject options by means of interpretable models and counterfactual explanations.
Pareto-Optimal Fairness-Utility Amortizations in Rankings with a DBN Exposure Model
Venue: arXiv cs.IR 2022
Year: 2022

In recent years, it has become clear that rankings delivered in many areas need not only be useful to the users but also respect fairness of exposure for the item producers. We consider the problem of finding ranking policies that achieve a Pareto-optimal tradeoff between these two aspects. Several methods were proposed to solve it; for instance a popular one is to use linear programming with a Birkhoff-von Neumann decomposition. These methods, however, are based on a classical Position Based exposure Model (PBM), which assumes independence between the items (hence the exposure only depends on the rank). In many applications, this assumption is unrealistic and the community increasingly moves towards considering other models that include dependences, such as the Dynamic Bayesian Network (DBN) exposure model. For such models, computing (exact) optimal fair ranking policies remains an open question. We answer this question by leveraging a new geometrical method based on the so-called expohedron proposed recently for the PBM (Kletti et al., WSDM'22). We lay out the structure of a new geometrical object (the DBN-expohedron), and propose for it a Carathéodory decomposition algorithm of complexity $O(n^3)$, where $n$ is the number of documents to rank. Such an algorithm enables expressing any feasible expected exposure vector as a distribution over at most $n$ rankings; furthermore we show that we can compute the whole set of Pareto-optimal expected exposure vectors with the same complexity $O(n^3)$. Our work constitutes the first exact algorithm able to efficiently find a Pareto-optimal distribution of rankings. It is applicable to a broad range of fairness notions, including classical notions of meritocratic and demographic fairness. We empirically evaluate our method on the TREC2020 and MSLR datasets and compare it to several baselines in terms of Pareto-optimality and speed.