# An Improved Data Generalization Model for Real-Time Data Analysis

Authors: A Srisaila; D Rajani; M V D N S Madhavi; G Jaya Lakshmi; K Amarendra; Narasimha Rao Dasari
Journal: Scientific Programming
 (2022)

---

## Abstract

This research proposes a maximum likelihood-Weibull distribution (WD) model for the generalized data distribution family. The distribution function of the anticipated maximum likelihood-Weibull distribution is defined where the statistical properties are derived. The data distribution is capable of modelling monotonically decreasing, increasing, and constant hazard rates. The proposed maximum likelihood-Weibull distribution is used for evaluated these parameters. The experimentation is done to evaluate the potential of the maximum likelihood-Weibull distribution estimated. Here, the online available dataset is adopted for computing the anticipated maximum likelihood-Weibull distribution performance. The outcomes show that the anticipated model is well-suited for computation and compared with other distributions as it possesses maximal and least value of some statistical criteria.


---

## Body

## 1. Introduction

Weibull distribution is a well-known distribution. It helps in observing the failures of various phenomena and components. Over the past few decades, various investigators concentrate on distribution. Alizadeh et al. [1] provide an extensive analysis of offering a number of historical facts, the various forms of this distribution considered by the practitioners, and probable errors and confusions that arise owing to the nonuniqueness. Bagheri et al. [2] perform a comprehensive analysis of distribution. It is the distribution that attains maximal attention. Moreover, Weibull distribution is applied to biological, medical, and earth sciences. Dattner et al. [3] discuss the effectual Weibull-based shape parameter estimation which relies on modified profile likelihood. The reliability analysis with an additive Weibull model based on failure rate function is considered by various approaches. Some consistent approaches are considered for the evaluation of various WD parameters [4, 5]. Assume that X is a random variable with WD with shape parameter α and scale parameter λ, and the probability distribution function (PDF) and cumulative distribution function (CDF) is expressed as follows:(1)fx=∝λx∝−1e−x∝/λ,x>0,∝>0,λ>0,(2)Fx=1−e−x∝/λ,x>0,∝>0,λ>0.It is essential to examine the productive estimation of CDF and PDF of Weibull distribution due to its significance over various Weibull distribution applications. There are various estimation approaches like maximal likelihood estimation, uniformly minimal variance unbiased estimation, least square estimation, percentile estimation, and weighted least square estimation [6]. Various investigations are noted that specifically concentrate on other distributions, for instance, the derive estimator of CDF and PDF of three generalized parameters with Poisson and Exponential distribution; however, its shape parameter is known to be assumed [7]. Some derived estimator of CDF and PDF of generalized parameter (Rayleigh) distribution when all parameters except shape parameter are considered to be known. Nagatsuka et al. [8] evaluate CDF and PDF of the Weibull extension model when all, however, the shape parameters are considered to be known. Some recent works, including exponential Weibull distribution, generalized exponential distribution, and exponential Gumbel distribution, are considered and analyzed [9]. The extension of this work includes the likelihood of the Weibull distribution that is unknown. Some works with CDF and PDF are evaluated and unknown parameters [10].


## 2. Maximal Likelihood-Weibull Estimator Model

ConsiderX1,…,Xn as the random sample acquired for evaluating the Weibull estimator, and the estimator λ describes λ˜ is λ˜=n−1∑i=1nxiα. Thus, this work attains the maximal likelihood of CDF and PDF using(3)f˜x=αλ˜xα−1e−xα/λ,(4)F˜x=1−e−xα/λ.The probability density ofT=∑i=1nxiα is expressed as(5)h∗t=tn−1e−t/λΓnλn.Fort>0 where some elementary algebra, this work attains the PDF of w=λ˜ as follows:(6)gw=nnwn−1Γnλne−nw/λ.Forw>0, this work computes Ef˜xr and computes EF˜xr.


## 3. Parametric Distribution of Weibull Distribution

The novelty behind Weibull distribution is its competency to examine the failure trends and failure prediction based on the provided dataset. The major advantage relies on its versatility, and it can be applied over smaller dataset samples. The Weibull distribution is shown in various forms [11]. The distribution functions are expressed as follows:(7)Ft=1−exp−t−τα−β,t≥r,(8)Ft=1−exp−λβ,t≥τ.The preliminary properties of Weibull distribution are discussed as follows:(9)ft=βα−βt−τβ−1exp−t−ταβ,t≥τ,(10)ft=βλt−τβ−1exp−t−τβ,t≥τ.The mode of MLWD is denoted ast=αβ−1/β1/β+τforβ>1 and τ=0<β≤1.The medium of MLWD is described atαlog21/β+τ.ConsiderT specifies the random variable based on the WD parameters [12]. The transformed variables T′=T−τ/α are measured as a standard form. The density function is expressed as follows:(11)ft=βtβ−1exp−tβ,x>0,β>0.The moment value ofT is expressed as follows:(12)μr′=ET′r=Γrβ+1,(13)VarT=Γ2β+1−ΓΓ1β+12.


## 4. Entropy

It is an uncertainty measure that helps in various applications like physics, statistical measure, hydrology, and engineering [13]. Generally, entropy facilitates us to provide proper statements and perform certain evaluations and factors related to pressing issues. It is generally a measure of uncertainty. Here, two diverse entropies are depicted as follows:(14)Irγ=11−γlog∫​gγxdx,γ≠0,1.Here, substitute the PDF of the WD in(15)Irγ=11−γγlogαlogαα−1+γ−1logλ−1−λλlogβ+log∑k=0∝γ1−1/λ+1/γλk!γ+kγ1−1/λ+1/γλ.The Shannon-based entropy model is expressed as(16)Hs=−Exlogfx.With PDF, the expression attained is(17)Hs=1−∝log∝∝−1−logβλlog∝∝−1+∝∝−1∑k=1∞−log∝^k+1k!1+k11+k−λ−1λ∅1−logβ−log1+k.When the distribution with provided with maximal entropy distribution when the mean and standard deviation is known for the provided real-time dataset, this makes the sense that the people use the distribution often as it is easier to evaluate the mean and SD of any provided dataset.


## 5. Maximal Likelihood Evaluator

This section attains the maximal likelihood estimator of various parameter distributions. Considerx¯=x1,x2,…,xn is a random sample with size n from the provided distribution with PDF [14]. The likelihood function is expressed as(18)lθ|x¯,c=∏i=1nfXxi=logαα−1nλβne−β∑i=1nxiλα∑i=1n1−e−βxiλ∏i=1nxiλ−1.The log-likelihood function is expressed as(19)logℓ=nloglog∝∝−1+nlogβ+nlogλ+λ−1∑i=1nlogxi−β∑i=1nxiλ+logα∑i=1n1−e−βxiλ.The log-likelihood estimator equations are expressed as(20)δloglδα=nαlogα−nα−1+∑i=1n1−e−βxiλα=0,(21)δloglδβ=nβ−∑i=1nxiλ+logα∑i=1nxiλe−βxiλ=0,(22)δloglδλ=nλ+∑i=1nlogxi−β∑i=1nxiλlogxi+βlogα∑i=1nxiλlogxie−βxiλ=0,where the maximal likelihood of α,β,λ is specified by ∝^,β^, and λ^, respectively, and attained by resolving the previous equations.


## 6. Applications of Maximal Likelihood Distribution

This section provides three diverse parameters for maximal likelihood distribution towards the real-time data for illustration purposes. It shows the feasibility of the new distribution while modelling the positive data [15]. The empirical hazard function of the provided dataset relies on the total time on the test. The distribution and the scaled transform are expressed as(23)H−1u=∫0F−1uSydy,0<u<1isexpressedasgu=H−1uH−11.The empirical version of total time on the test is expressed as(24)gnrn=Hn−1r/nHn−1n=∑i=1rxi:n−n−rxr:n∑i=1nxi:n,where r=1,2,..,n and xi:n,i=1,2,…,n, specifies the order statistics of various samples. It is convex, and the rate is increasing or decreasing. The maximal WD is considered the best choice for the provided datasets [16]. However, the distribution fits the prevailing parameters of Weibull distributions. The following are the corresponding distributions of Weibull and expressed as in equations (25) to (29).Generally, the WD model is expressed as(25)Fx=1−exp−βxλ;β>0,λ>0.The modified maximal WD model is expressed as(26)Fx=1−exp−βxλeαx;β>0,λ>0,α∈R.The modified WD model based on CDF is expressed as(27)Fx=1−exp−βxλ−∝x,β>0,λ>0,α≥0withβ+α>0.The exponential WD model based on CDF is expressed as(28)Fx=1−exp−βxλ;β>0,λ>0,α≥0withβ+α>0.The extended generalized gamma distribution is expressed as(29)fx=λααβλαΓαxλα−1exp−αβxλ;β>0,λ>0,α>0.Here, the formal fitness test is performed to verify the distribution, superior to the real-time dataset. However, it is not fulfilled that the anticipated model offers a superior fit to the other approaches. However, it can offer a better fit [17]. Thus, the likelihood distribution model is utilized as a substitute for the existing well-known distribution such as modified, exponential, and gamma [18]. Here, an online available UCI Machine Learning repository for disease prediction and emotion recognition datasets is considered [19, 20]. The dataset is taken from the online resource known as the Kaggle dataset for disease prediction. The standard error and p value computations are expressed in Tables 1 and 2. Figures 1 and 2 show the standard error comparison for datasets 1 and 2.Table 1
Standard error comparison (first dataset).
Distributionβ^λ^α^−Log (ℓ)p valueProbability-based MW0.194784.4837510.86413.4750.302WD0.059755.7807916.95015.2080.107EW0.019407.283850.6713014.6780.134MW0.008752.411252.154814.3570.197EGG0.601167.762600.6196014.5900.141Modified WD0.040726.381180.0311014.8960.215Table 2
Standard error comparison (second dataset).
Distributionβ^λ^α^−Log (ℓ)p valueProbability-based MW0.093701.590350.02118121.350.654W0.366401.32560—122.5300.296EW0.579601.101331.44260122.1600.423MW0.391161.55780−0.08775121.7200.554EGG0.501601.065341.47740122.2310.403Modified WD0.431331.28269−0.06318122.5130.311Figure 1
Standard error comparison (dataset 1).Figure 2
Standard error comparison (dataset 2).


## 7. Weighted Least Square Analysis

Various estimators recommend evaluating the beta parameter distribution. Here, a regression-based estimator of certain unknown parameters is derived. It is used for other distributions [21, 22]. Assume that X1,X2,…,Xn is a random sample with n size from the CDF with function F. and Xi,i=1,2,…,n specify the sample order in ascending order. The sample size n is expressed as (30)EFXj=jn+1,varFXj=jn−j+1n+12n+2,(31)CovFXj,FXl=jn−l+1n+12n+2,j<l.With the variance and expectation, two diverse variants of the least squares methods are given as follows.


### 7.1. Least Square Estimator

The minimal estimator is attained as follows:(32)∑j=1nFXj−jn+12.It is provided based on unknown parameters [23]. Thus, Weibull distribution with least square estimators of λ and λ˜ls and attained with minimized as(33)∑j=1n1−e−∝Xj/λ−jn+12.To attain the CDF ad PDF, we use the method for ML estimator. Thus,(34)f˜lsx=αf˜lsxα−1e−xα/λls,(35)F˜lsx=1−e−xα/λls.It is complex to predict the expectation and error rate of the estimators with mathematical modelling.


### 7.2. Weight-Based Least Square Estimators

The minimal weighted least square estimator is expressed as(36)∑j=1nwjFXj−jn+12.Based on all these analyses and unknown parameter estimation,wj=1/VarFXj=n+12n+2/jn−j+1. Thus, Weibull distribution for λ weighted square estimator is attained based on(37)∑j=1nwj1−e−∝Xj/λ−jn+12.With the weighted least estimator with CDF and PDF, the estimator is expressed as(38)f˜wlsx=∝λ˜xα−1e−xα/λ˜wls,(39)F˜wlsx=1−e−x∝/λwls.It is difficult to predict these weighted estimators‘ error rates and expectations with the mathematical modelling in Table3 and also the weighted estimation comparisons are depicted in Figure 3.Table 3
Weighted estimation comparison.
∝estimationλestimationLog-likelihoodMALE5.716.74−15.2PCE5.414.28−15.4LSE7.439.64−18.9WLSE7.01632.139−17.3Figure 3
Weighted estimation comparison.


## 7.1. Least Square Estimator

The minimal estimator is attained as follows:(32)∑j=1nFXj−jn+12.It is provided based on unknown parameters [23]. Thus, Weibull distribution with least square estimators of λ and λ˜ls and attained with minimized as(33)∑j=1n1−e−∝Xj/λ−jn+12.To attain the CDF ad PDF, we use the method for ML estimator. Thus,(34)f˜lsx=αf˜lsxα−1e−xα/λls,(35)F˜lsx=1−e−xα/λls.It is complex to predict the expectation and error rate of the estimators with mathematical modelling.


## 7.2. Weight-Based Least Square Estimators

The minimal weighted least square estimator is expressed as(36)∑j=1nwjFXj−jn+12.Based on all these analyses and unknown parameter estimation,wj=1/VarFXj=n+12n+2/jn−j+1. Thus, Weibull distribution for λ weighted square estimator is attained based on(37)∑j=1nwj1−e−∝Xj/λ−jn+12.With the weighted least estimator with CDF and PDF, the estimator is expressed as(38)f˜wlsx=∝λ˜xα−1e−xα/λ˜wls,(39)F˜wlsx=1−e−x∝/λwls.It is difficult to predict these weighted estimators‘ error rates and expectations with the mathematical modelling in Table3 and also the weighted estimation comparisons are depicted in Figure 3.Table 3
Weighted estimation comparison.
∝estimationλestimationLog-likelihoodMALE5.716.74−15.2PCE5.414.28−15.4LSE7.439.64−18.9WLSE7.01632.139−17.3Figure 3
Weighted estimation comparison.


## 8. Failure Data Analysis

There are two diverse kinds of mathematical modelling. The data failure is categorized into two diverse types: incomplete and complete. Concerning complete data, the actual data values realized are well-known for every observation, while in the case of incomplete data, the actual values are not known for certain observations [24]. The negative entropy value represents that something turns to be less disordered and huge energy is used. It may not occur spontaneously, and it reduces the randomness:(1)
Theory-based modelling relies on the well-established component failures, also called physics- or white-box-based models(2)
Empirical modelling: the available data relies on the model construction. It is also known as a black-box and data-dependent modelWith empirical modelling, the mathematical formulation is provided based on the preliminary analysis of available data. When the data analysis specifies a higher degree of variability, the model needs to capture the variability. It needs a stochastic and probabilistic way to express the provided dataset. The black-box modelling includes three diverse steps: (1) model assortment; (2) parameter evaluation; and (3) model validation.To choose various models with the probable model, the investigators need to understand various properties. It is often known as a trial and error model. The goodness fitting test is the key process for choosing the statistical distribution of the observed data’s best fits. The larger amount of the Weibull distribution model is known from various literature. The selection of an appropriate model from the WD is based on the probability plots. It also provides the crude evaluation of model parameters. There are various statistical testing processes for model validation.


## 9. Hypothesis Testing Analysis: Goodness Measure

There is another goodness fit-based model for the empirical distribution function, for instance, the Cramer von Mises test, KS test, chi-square test, and AD test. The goodness measure of the WD model is provided asH0, and the population is well-suited for WD vs. H1. There are various extensions like modification and generalization adopted by the WD model. It leads to the rise of model features of the provided empirical dataset, which is not determined by the three parameters of the WD model [25]. The monotonic properties of the Weibull are not capable of capturing the dataset‘s nature. It leads to failure rate as in Table 4.Table 4
Failure rate measure.
0.130.450.931.151.251.621.942.394.525.106.807.658.4311.9111.9513.0213.2614.3317.4818.1118.6719.2524.4025.0226.4226.8127.7626.6929.8531.6632.653540.7142.3543.0643.4044.3545.4248.1449.2049.4552.1858.6360.3072.2372.2272.2672.3086.2589.53Product reliability relies on the development, manufacturing, and design decisions made before the product launch. It influences the failures when the product is provided for launch. The prelaunching stage includes various phases. Some investigations indicate the target value of product reliability [26]. The product reliability is evaluated during the design phase based on the component reliability. Moreover, the enhancement is identified in the upper limit. When the target value is below the provided limit, the design with the components attains the desired target values. The program to enhance the reliability via the fix-test-cycle is done in the development phase [27]. The prototype is tested when the failure is noted, and the cause is examined. Based on these metrics, the design variations are done to handle the predicted failure causes. This process is continued until reliability is attained [28]. The item reliability during manufacturing intends to change from the manufacturing process. The quality control and proper process during the manufacturing phase and the variations are managed.During the postlaunch stage, the item reliability is reduced due to the deterioration and outcomes from the utilization. The deterioration is influenced by various factors, including the operating condition, environment, and maintenance. The deterioration rate is managed via the prevention maintenance phase. The reliability outcomes with higher maintenance costs for the buyer. Also, it leads to higher warranty costs and outcomes from the rectifying cost within the warranty period. Some products are composed of various components due to the component‘s failure. The WD model is used to deal with the failure of various components, and the literature is wider. During the development phase, it is essential to use acceleration testing to hasten the process of component failures. The enhancement in reliability during the development phase is modelled in various ways. The Weibull intensity model helps design the failure rate improvements as a development time function. The breakthroughs lead to enhancements as random points. During the manufacturing phase, the nonconforming fractions are lesser when the process is in-control, enhancing the process goes out of control. The likelihood-Weibull distribution is utilized to design the control duration in the control charts design to predict the variations from in-to out-control. When the failure rate is identified, it is prone to earlier failure. For the product that exhibits a failure rate, burn-in is utilized to eliminate these failures and enhance the product reliability before the release for sale (see Table5).Table 5
Advantages of various applications.
ReferencesAdvantagesNavid et al. [21]Yield steel strength and fatigue of steel lifeRana et al. [22]Fracture strength of glassShahbaz et al. [23]Pitting pipe corrosionSingh et al. [24]Adhesive wearable in metalsTahir et al. [25]Carbon-fibre composite failureTomer et al. [26]Coating failuresYousof et al. [28]Brittle material failureZografos et al. [29]Composite material failureYuri et al. [30]Concrete components


## 10. Likelihood Maximal Weibull Distribution and Other Applications

To perform statistical data analysis, computer software is needed. It can be extensively utilized in various other applications to examine various issues. There are enormous works that provide nominal outcomes to measure the distribution rate. Additionally, statistical software like SPSS, MiniTab, SAS, etc., is considered, and spreadsheet-like Excel evaluates the success and failure rate. The advantage of likelihood maximal Weibull distribution is its ability to solve the problem related to density evaluation. It performs maximization of likelihood function to predict the probability distribution and parameters of the finest observed data.


## 11. Conclusion

This research extensively analyses the maximal likelihood-Weibull distribution model for data generalization. The Weibull distribution is widely adopted for lifetime distribution evaluation in a reliable environment, and it is a versatile distribution that considers the characteristics of other distributions based on the provided scale and shape parameters. The parameter distribution and maximal likelihood estimator are considered to measure the varianceVarT. The uncertainty measure assists in statistical analysis with certain criteria. The maximal likelihood evaluator is measured with the PDF and CDF values. The proposed model can be used in various real-time applications where the analysis is done with dataset 1 and dataset 2. The standard errors from this dataset are explained with various metrics like p value. The weighted least square and failure data analysis is done with theoretical and empirical modelling, where the hypothesis for measuring the testing analysis is done with goodness measure. The model gives better outcomes than others, and some other distribution models will be examined in the future.



---

*Source: 4118371-2022-08-09.xml*