nagdmc linear reg

NAG DMCLinear Regression: nagdmc linear reg
Purposenagdmc linear reg computes a regression model with p parameters.

Declaration#include <nagdmc.h>void nagdmc linear reg(long rec1, long nvar, long nrec, long dblk, double data[],void (*dfun)(long, long, double [], char *, int *), char *comm,long chunksize, long nxvar, long xvar[], long yvar,long iwts, double *R2, double *rms, long *df, double b[],double se[], double cov[], double model[], double eps,
int *info);

Parameters1:

rec1 – long
On entry: the index in the data of the first data record used in the analysis.
Constraint: rec1 ≥ 0.

Input

2:

nvar – long
On entry: the number of variables in the data.
Constraint: nvar > 1.

Input

3:

nrec – long
On entry: the number of consecutive records, beginning at rec1, used in the analysis.
Constraint: nrec > 1.

Input

4:

dblk – long
On entry: the total number of records in the data block.
Constraint: dblk ≥ rec1 + nrec.

Input

5:

data[dblk ∗ nvar] – double
Input
On entry: the data values for the jth variable (for j = 0, 1, . . . , nvar−1) are stored in data[i∗nvar+j],
for i = 0, 1, . . . , dblk − 1. When the data function is used, data is not referenced.

6:

dfun – function supplied by user
On entry: the pointer to a data function supplied by the user.
Constraint: if dfun is a valid pointer, data must be 0.
The specification of dfun is:External Procedure

void dfun(long irec, long chunksize, double x[], char *comm, int *ierr)1:

irec – long
On entry: the index in the data of the first record returned.

Input

2:

chunksize – long
On entry: the number of consecutive records returned.

Input

3:

x[chunksize∗nvar] – double
OutputOn exit: data values for the jth variable (for j = 0, 1, . . . , nvar − 1) must be returned
in x[i ∗ nvar + j], for i = 0, 1, . . . , chunksize − 1.

4:comm – char *Input
On entry: a communication parameter allowing additional information to be passed
to dfun. This parameter is passed ‘as is’ through the calling function.

nagdmc linear reg.1

nagdmc linear reg

NAG DMC5:

7:ierr – int *OutputOn exit: if the value pointed to by ierr on return is greater than 100, the NAG DMC
function will terminate immediately and info will point to this value.

comm – char *Input

On entry: a communication parameter allowing additional information to be passed to dfun. Thisparameter is passed ‘as is’ through the calling function.
8:

chunksize – long

Input

On entry: if the data function is used, the function inputs no more than chunksize data records ata time; otherwise chunksize is not referenced.
Constraint: if dfun 6= 0, chunksize ≥ 1.
9:nxvar – long

Input

On entry: the number of independent variables. If nxvar = 0 then all variables in the data,excluding yvar and (if iwts ≥ 0) iwts, are treated as independent variables.
Constraint: 0 ≤ nxvar < nvar.
10:

xvar[nxvar] – long

Input

On entry: the indices indicating the position in data in which values of the independent variables
are stored. If nxvar = 0 then xvar must be 0, and the indices of independent variables are given by
j = 0, 1, . . . , nvar − 1; j 6= yvar and j 6= iwts.
Constraints: if nxvar > 0, 0 ≤ xvar[i] < nvar, for i = 0, 1, . . . , nxvar − 1; otherwise xvar must be0.
11:

yvar – long

Input

On entry: the index in data in which values of the dependent variable are stored.
Constraints: 0 ≤ yvar < nvar; if nxvar > 0, yvar 6= xvar[i], for i = 0, 1, . . . , nxvar − 1.
12:

iwts – long

Input

On entry: if iwts = −1, no weights are used; otherwise iwts is the index in data in which theweights are stored.
Constraints: −1 ≤ iwts < nvar; iwts 6= yvar; and if nxvar > 0, iwts 6= xvar[i], fori = 0, 1, . . . , nxvar − 1.
13:

R2 – double *Output2On exit: the R -value for the fitted model.
14:

rms – double *OutputOn exit: the residual mean square for the fitted model.
15:

df – long *OutputOn exit: the degrees of freedom for the residual mean square.
16:b[p] – double

OutputOn exit: the parameter estimates. b[0] is the mean parameter. b[i] is the coefficient of the ith
variable included in the model, for i = 1, 2, . . . , p − 1. If nxvar > 0 then the order the independent
variables are added to the model is defined by xvar, otherwise the order is defined by indices in thedata.
17:

se[p] – double

OutputOn exit: the standard errors of the parameters in b.
18:

cov[p ∗ (p + 1)/2] – double

OutputOn exit: the first p ∗ (p + 1)/2 elements of cov contain the upper triangular part of the variancecovariance matrix of the p parameters in b. They are stored packed by column, i.e., the covariance
between the parameter estimate given in b[i] and the parameter estimate given in b[j], j ≥ i, isstored in cov[j(j + 1)/2 + i], for i = 0, 1, . . . , p − 1 and j = i, i + 1, . . . , p − 1.

nagdmc linear reg.2

nagdmc linear reg

NAG DMC
19:

model[(3 ∗ p ∗ (p + 1))/2 + nvar + 14] – double

OutputOn exit: if not 0, information on the fitted model for use in the functions described in ‘See Also’.
20:

eps – double

Input

On entry: the value of the criterion used for model pruning. If eps = 0.0, a default value of 1e−10
is used.
Constraint: eps ≥ 0.0.
21:

info – int *OutputOn exit: info gives information on the success of the function call:
0: the function successfully completed its task.
i; i = 1, 2, . . . , 4, 6, 8, 9, . . . , 12, 20: the specification of the ith formal parameter wasincorrect.
57: there are no degrees of freedom for the error estimates.
58: the fit is exact, no error estimates.
98: there is an underlying computational problem (this is an unlikely error exit).
99: the function failed to allocate enough memory.
> 100: an error occurred in a function specified by the user.

Notation
nrec
nxvar
xvar
yvar
iwtsb

the number of data records, n.
the number of independent variables, p − 1.
the independent variables that take the values in X (except the mean).
the dependent variable that takes the values in y.
if iwts ≥ 0, iwts is the index in the data that defines the weights, W .
the parameter estimates, β̂.

Description
The general multiple linear regression model is defined byy = Xβ + ε,
where yXβand εis a vector of n observations on the dependent variable,
is an n by p matrix of the independent variables of column rank k,
is a vector of length p of unknown parameters,
is a vector of length n of unknown, Normally distributed, random errors
such that var ε = V σ 2 , where V is a known diagonal matrix of size n.

In most linear regression models the first term is taken as a mean term or an intercept, i.e., Xi,1 = 1,for i = 1, 2, . . . , n; this is assumed in the NAG DMC.
If V = I, the identity matrix, then least squares estimation is used. If V 6= I, then for a given
weight matrix W ∝ V −1 , weighted least squares estimation is used.
The least squares estimates β̂ of the parameters β minimize (y − Xβ)T (y − Xβ) while the weighted
least squares estimates minimize (y − Xβ)T W (y − Xβ).
NAG DMC uses a QR decomposition of X (or W 1/2 X in the weighted case), i.e.,
X = QR∗ (or W 1/2 X = QR∗ ),
R
where R =and R is a p by p upper triangular matrix and Q is an n by n orthogonal matrix.
0If R is of full rank, β̂ is the solution to∗Rβ̂ = c1 ,nagdmc linear reg.3

nagdmc linear reg

NAG DMC

where c = QT y (or QT W 1/2 y) and c1 is the first p elements of c. If R is not of full rank a solution
is obtained by means of a singular value decomposition (SVD) of R,
D 0
R = Q∗
PT,0 0where D is a k by k diagonal matrix with non-zero diagonal elements, k being the rank of R, andQ∗ and P are p by p orthogonal matrices. This gives the solutionβ̂ = P1 D−1 QT∗1 c1 ,P1 being the first k columns of P , i.e., P = (P1 P0 ), and Q∗1 being the first k columns of Q∗ .
The fit of the model can be examined by considering the residuals.
The residuals are defined as√
ri = yi − ŷi in the case of an unweighted analysis and as ri = wi (yi − ŷi ) in a weighted analysis. In
both cases ŷ = X β̂ and corresponds to the fitted values. These fitted values can be written as Hy
for an n by n matrix H. The ith diagonal element of H, hi , gives a measure of the influence of theith value of the independent variables on the fitted regression model. The values hi are sometimes
known as leverages.

References and Further Reading
Cook R D and Weisberg S (1982) Residuals and Influence in Regression Chapman and Hall.
Draper N R and Smith H (1985) Applied Regression Analysis (2nd Edition) Wiley.
Golub G H and van Loan C F (1996) Matrix Computations (3rd Edition) Johns Hopkins UniversityPress, Baltimore.
Hammarling S (1985) The singular value decomposition in multivariate statistics SIGNUM Newsl.
20 (3) 2-25.
McCullagh P and Nelder J A (1983) Generalized Linear Models Chapman and Hall.
Searle S R (1971) Linear Models Wiley.

See Also
nagdmc basic reg
nagdmc extr reg
nagdmc load reg
nagdmc predict reg
nagdmc save reg
nagdmc stepwise reg
linear reg ex.csimplified version of nagdmc linear reg using a restricted set of parameters.
computes fitted values, residuals and leverages for a regression.
reads a linear regression model from a binary file.
computes predictions given a fitted regression model.
writes a linear regression model to a binary file.
stepwise linear regression with Normal errors.
the example calling program.

nagdmc linear reg.4