# Relaxed Rule-Based Learning for Automated Predictive Maintenance: Proof of Concept
## 1 Introduction  
Rule Learning (RL) is a well known methodology of Machine Learning (ML). By the Occam Razor principle [1], smaller models tend to make more accurate predictions. Based on this principle, RL algorithms should be designed to create small sets of small rules. Arguably, the best known specific method of RL is Decision Tree (DT) algorithm. However, existing literature highlights the fact that if we insist on the 'tree-likeness' of the rules set, rules become prohibitively long and complicated (Section 1.5.3. of [2]). This is due to the effect of _fragmentation_. A detailed discussion of this phenomenon is provided in Example 2, see also a related discussion in [3]. Therefore, an alternative approach has been actively investigated in which rules are created one by one without insisting on their set to fit a DT structure. In order to design an algorithm based on this approach, the following two questions must be answered. 1. How to create a single rule? 2. How to create a collection of rules? According to the Occam Razor principle, an algorithm for composition of a single rule must endeavor to make the rule as small as possible and as precise as possible. Thus, the task of a rule creation can be envisaged as an optimization problem with the objective function expressing a combination of these two criteria (plus the high coverage criteria to avoid overfitting). For a reasonably complex domain such an optimization problem is intractable [1], hence there is no hope to obtain an 'optimal' rule in a reasonable time. Consequently, the main methodology for obtaining a **single rule** are greedy local search (mainly Hill Climbing) algorithms [2]. The common feature of these algorithms is the absence of _backtracking_. In other words, a local search algorithm _grows_ a rule adding constraints on attributes one by one and cannot remove a constraint once it has been added. In the above local search framework, the main effort concentrates on the heuristics for choosing the next attribute constraint being added to a rule. Moreover, the constraints have a special form outlined in Example 1 below. Also, we identify a dataset with a table, an attribute (attr) with a column in the dataset, and an instance with the row of the dataset. Assume that our learning task is to learn a concept depending on \(5\) attributes \(attr_{1},\ldots,attr_{5}\) of a dataset. Assume that these attributes have integer values ranging between \(0\) and \(50\). Then the rules are of the form \(attr_{1}\in[10,\ldots,40]\wedge attr_{3}\in[20,\ldots 35]\to true\). The above rule states that all the instances of the dataset with the value of \(attr_{1}\) between \(10\) and \(40\) and the value of \(attr_{3}\) between \(20\) and \(35\) then this instance 'belongs' to the concept. Note that the constraints in Example 1 are given in the form of _intervals_. Moreover, the same attribute can occur more than once. In this case, the actual constraint on the attribute is the intersection of the intervals of all the occurrences of this attribute. For example, the rule as in Example 1 can be rewritten as follows \(attr_{1}\in[0,\ldots,40]\wedge attr_{3}\in[20,\ldots 35]\wedge attr_{1}\in[10,50] \to true\). The rule growth procedure starts from an empty rule and performs a number of iterations. Each iteration chooses an attribute and an interval and adds to the rule being formed the respective constraint, stating that the value of the chosen attribute belongs to the value of the chosen interval. The procedure also needs a terminating condition. An obvious one is that all the instances covered by the current rule are invariant w.r.t. the concept being studied (all belong or all do not belong to the concept). However, such a terminating condition may lead to long rules and potentially prone to overfitting. To avoid this situation, there are terminating conditions that stop the rule growth procedure even in case there is no full invariance. Let us now discuss approaches to forming **a collection of rules**.