CSIS Discussion Paper No. 46

An Empirical Evaluation of Hedonic Regression Models
Xiaolu Gao, Yasushi Asami, and Chang Jo Chung
MARCH, 2002

Center for Spatial Information Science
The University of Tokyo
Hongo 7-3-1, Bunkyo, Tokyo 113-8656, Japan1

An Empirical Evaluation of Hedonic Regression Models
Xiaolu Gaoa, Yasushi Asamib, and Chang Jo ChungcaDept. of Urban Research, National Research Institute for Land and InfrastructureManagement. Tachihara 1, Tsukuba, Ibaraki 305-0802, JAPAN gao-x925t@nilim.go.jp.
bCenter for Spatial Information Science, the University of Tokyo. Hongo 7-3-1, Bunkyo,
Tokyo 113-8656, JAPAN- asami@csis.u-tokyo.ac.jp.
cSpatial Data Analysis Lab., Geological Survey of Canada, Natural Resources Canada.
601 Booth Street, Ottawa, Ontario, CANADA K1A 0E8 - chung@gsc.nrcan.gc.ca.

KEY WORDS: Empirical evaluation, Cross-validation, Prediction, Spatial model

ABSTRACT:
Statistical criteria are poor at evaluating spatial exploratory models of hedonic
regression because they are heavily dependent on uncertain assumptions concerning
spatial relationships. To resolve this problem, empirical evaluation methods areproposed in this paper. A simple linear model and three spatial models for a housing andland price dataset in Tokyo are studied for illustration. The prediction power of models
is emphasized. With a cross-validation technique, the housing and land price at eachsample point are predicted, and then, numerical and graphical criteria are supposedusing these predicted values and real observed prices. The methods not only help us toselect suitable models for a dataset, but also provide an alternative for the significancetest of concerned spatial relationships.

2

1. INTRODUCTION
Recently, a variety of hedonic regression models have been proposed in addition to asimple formed model to study the spatial nature of variables. All of them proposed tomake use of the spatial characteristics of variables to improve models. Since they areoften exploratory models, it is critical issue as how well a model is, whether the spatial
characteristics identified by a model are convincible, and how to select the most suitablemodel for a dataset.
A difficulty in making such judgment is the lack of appropriate evaluation techniques.
This paper sets out to address this issue.
Traditionally, statistical testing criteria such as R-square, t-statistic, F statistic, AIC ∗ ,and so on, are often used to evaluate regression models. However, models are always
based upon such assumptions as linear assumption, normality, or non-collinearity, thus
statistical testing methods inevitably depend on these assumptions. For exploratorymodels, this creates problems because assumptions are open for test.
To overcome this problem, other evaluation methods have to be developed.

2. ALTERNATIVE REGRESSION MODELSIn this paper, we use a housing and land price dataset in Tokyo for illustration. It covers
the transaction price, structural attributes, environmental attributes, and x, y coordinatesof 190 detached housing lots. See Appendix A for details of it.
2.1 A Simple Linear Regression Model
Equation (1) shows the simplest form of hedonic regression models:y = a0 + ∑akx k+ ε,(1)where, y is unit price, x k for k= 1, 2, …, m are independent variables, ε is an error term,
and a0 and ak are parameters to be estimated.
An ordinary linear model in this form was developed in Gao and Asami (2001). It
explains 75.6% of unit price, has 16 independent variables, all of which statisticallysignificant at the level of F below 0.05. See Appendix B.
Since this model has not fully considered the impact of spatial location on unit price, to∗

Akaike Information Criterion, a tradeoff of likelihood and the number of estimatedparameters3study the spatial characters of dataset with location data is thought of a way to improveit.
One of the spatial relationships being explored is spatial dependency. Another is spatial
heterogeneity. The estimates of regression parameters in the presence of spatial
dependency have been discussed by a number of literatures, e.g., Dubin (1992; 1998)
and Can (1990; 1992), while, various localized modeling techniques were proposed tocapture spatial heterogeneity (Casetti, 1972; Getis and Ord, 1992; Fotheringham andBrunsdon, 1999). In addition, some models are developed to detect both of them(Anselin, 1988; Can, 1992).
In this paper, we choose three spatial regression models to explore the data.
2.2 A Spatial Dependency Model
In order to study spatial dependency effects, a prior probability method proposed by
Switzer et al. (1982) is applied. The model had been used to process satellite
classification maps. To evaluate the class at the center of a window using the data forthat location and the prior probability estimates obtained from the nearby observations in
the window area was shown to have increased classification accuracy. Similarly, theprior characteristic terms of sample lots, denoted by x k’ for k= 1, 2, …, m, are added tothe simple linear model:
y = a0 + ∑akx k + ∑ak’ x k’+ ε.

(2)For simplification, let x k’ take the value of x k that is associated with the nearest neighborof a sample. Appendix C gives the OLS estimates of this model, with an R-square of
0.801.
Although one might argue that correlations between x k and x k’ may lead to unstableestimates, the question is secondary to the concern whether spatial dependency is present.
If the absence of spatial dependency effects were demonstrated, the model itself would
be improper, let alone its estimates.
2.3 A Geographically Weighted Regression (GWR) Model
To identify spatial variations in relationships, a GWR model proposed in Fotheringham,
et al. (1998) is adopted. The model extends traditional regression framework byallowing parameters to be estimated locally so that the model in Equation (1) isrewritten as
yi= a0i + ∑akixki + εi,(3)4where, a0i and aki represent the values of a0 and ak at point i.
In order to estimate the model, an observation is weighted in accordance with its
proximity to point i. Let Wi be an n×n matrix whose diagonal elements wij denotes thegeographical weighting of all observed data for point i, and the off-diagonal elements
are zero. Data from observations close to i are weighted more than data fromobservations far away. Equation (4) gives the estimation of ai :aˆ i = ( X T Wi X ) −1 X T Wi Y ,(4)where, X and Y are the matrix of explanatory variables and unit prices, respectively.
A weighting function in Equation (5) is applied:

wij= exp(-

d ij

2

β2

).

(5)

where, dij is the Euclidean distance between point i and j, β is a bandwidth. By
minimizing (6), β is set to 1,250m.

∑ [ y i − yˆ ≠i ( β)] 2 .

(6)iNote that yˆ ≠i ( β) is the estimates at i with samples near to i but not i.∗
At the location of each of the 190 observations, regressions are run. The obtained
localized parameter estimates exhibit a high degree of variability over space anddemonstrate fairly complex spatial patterns of each variable.
The localized regressions result in high R-square fittings, beyond 0.95 at all sample
points. Apparently, it is the outcome of localized regression, so these values are notcomparable to that of the simple linear model.
2.4 A Spatial Dependency +GWR ModelIn the following, we use a specification in (7) to investigate whether spatial dependency
and heterogeneity effects are both present.
∗

Suppose that point i itself is also included to estimate a0i and aki. If β is very small, theweightings of all points except for i become negligible and the estimates will fluctuate
wildly throughout space.

5yi= a0i + ∑akixki + ∑aki’ x ki’ + εi,(7)Similar to the spatial dependency model, let x ki’ represent the corresponding values of
the nearest sample to i.
In addition, regressions are localized at each sample point with a GWR technique,adopting a weighting scheme of (5), and the optimal value of β being 1650m. The
regression yields the estimates of parameters at each sample points.

3. AN EMPIRICAL EVALUATION OF THE MODELS
Now, let us discuss how to select a best one from these models. The weakness of“black-box” statistical evaluation method is well aware of. For instance, models
supported by high R-square fitting or satisfying significance level are frequently
doubted because of weak assumptions or irrational estimates of parameters, thus a clear
conclusion can hardly be reached with these statistical criteria. This is an unavoidable
result of heavy dependency on statistical assumptions. The problem is especially
distinct in spatial exploratory models for the assumptions of models are to a large
degree uncertain. This prompts us to turn to an empirical evaluation approach. In
particular, the methods being presented in this paper focus on the prediction power ofmodels.
In the practice of hedonic regression, we often focus more on estimating unbiased
coefficients to identify the marginal prices of explanatory variables, rather than simply
see the prediction powers. Nonetheless, when spatial relationships are introduced to asimple ordinary model, we instinctively expect that this can improve the fitting of themodel. Thus conversely, if it does not work better at predicting prices, we may just be
satisfied with the simple model and think that its estimates are robust. From thisviewpoint, the improvement of fittings can be been seen as an alternative to test thesignificance of spatial relationships, too. (Can, 1992; Dubin, 1992 and 1998; Brunsdon,
et al., 1999)
A cross-validation approach is used to carry out the idea, i.e., to see how well the prices
already being observed can be predicted with the rest samples. An application of asimilar method can be seen in Bourassa et al. (2001), where, random 80% samples aremodeled to predict the rest 20% samples, and the sum of square of error is used as acriterion to evaluate proposed model. However, we are concerned tha t, to estimate ahedonic regression model, it is important to keep as large a sample size as possible.
Thus, we predict the price at every point with all the rest 189 samples.
3.1 Numerical Cross-validation Criteria

6First, consider some numerical criteria, for instance, ∑ ( y i − yˆ ≠ i ) 2 and ∑ yi − yˆ ≠ i ∗ ,where, yi and yˆ ≠i denote the observed and the predicted unit price at point i,
respectively.
Table 2 gives their calculations. The values of all the three spatial models are slightly
lower than that of the simple linear model. However, the falloffs are too small to suggest
any critically large improvements.

Table 2. A comparison of models by numerical criteria
Numerical criteria
2∑ yi − yˆ ≠ i∑ ( y i − yˆ ≠ i )Model
Simple linear model

1.96

15.29

Spatial dependency model

1.92

15.11

GWR model

1.81

14.60

Spatial dependency +GWR model

1.76

14.33

Centering on prediction error, more numerical criteria could be raised. Seriously though,numerical criteria just show a general aspect of model. Thus, some graphical criteria arefurther employed in order to illustrate broader and more complicated aspects of therelationships between model and data.
3.2 The Distribution of Observation and Prediction
On a scatter plot of observed unit price (yi ) and predicted unit price ( ˆy ≠ i ), the prediction
power of a model is indicated by the range occupied by the scatter points, in particular,
the short breadth of this range. Obviously, all points being on 45o diagonal line indicatesa perfect prediction, and higher prediction power is suggested by how slim and how
close the range is to the 45o line.
Following the idea, the three spatial models are compared with the simple linear modelin Fig. 1~3. The 95% confidence regions of the scatter points are plotted, respectively.
As results, again, all the spatial models are no much better than the simple linear modelfor this dataset.

∗This is deemed more robust than ∑ ( y i − yˆ ≠ i ) 2 .

7

1.5

Real observed unit price (million
yen/m 2 )

1.2

0.9

0.6

0.3

0
0

0.3

0.6

0.9

1.2

1.5
2

Predicted unit price (million yen/m )Simple linear model
95% confidence region
Spatial dependency model
95% confidence region

Real observed unit price (million yen/m2 )Figure 1. Simple linear model and spatial dependency model

1.5

1.2

0.9

0.6

0.3

0
0

0.3

0.6

0.9

1.2

1.5

Predicted unit price (million yen/m2 )Simple linear model
95% confidence region
GWR model
95% confidence region

Figure 2. Simple linear model and GWR model8

Real observed unit price (million
yen/m 2)

1.5

1.2

0.9

0.6

0.3

0
0

0.3

0.6

0.9

1.2

1.5

Predicted unit price (million yen/m2)
Simple linear model
95% confidence region
Spatial dependency +GWR model
95% confidence region

Figure 3. Simple linear model and spatial dependency +GWR model

3.3 Prediction Rate Curve
We further evaluate the models by the proportion of well-predicted samples with respect
to prediction error, which is referred to as prediction rate.
To do this, let äi = y i − yˆ ≠i and sort äi. Then plot the ratio of well-predicted samples
with sorted äi as horizontal axis. By this way, the prediction rate curve of a model isobtained.
The introduction of this criterion allows us to capture the comprehensive performanceof a model. In particular, it is helpful when we select models under a given level ofprediction error. Consider two models having prediction rate curves shown in Fig. 4.
Highly possibly, the sums of prediction errors of them are close. At tolerance level ξ 1 ,70% of samples are correctly predicted by model 1, while only 60% can be predicted bymodel 2. However, if tolerance ξ 2 is acceptable, more samples are correctly predicted bymodel 2.
From an empirical point of view, a dataset always has some poor-to-predict
observations. Thus, the performance of the prediction curves within a tolerable error
scope may be thought of more important.

9

The rate of well predicted samples

100%

70%

Model 260%

Model 10

ξ1

ξ2Prediction error δi

Figure 4. Evaluating models with prediction rate curves

The rate of well predicted samples

Prediction rate curve actually indicates the density of the scatter points in Fig. 1~3 withrespect to their distance to the 45o diagonal line. To illustrate, let us see the prediction
rate curves of the simple linear model and the spatial models in Fig. 5.

100
%
80%

60%

40%
20%
0
0

0.1

0.2

0.3

Prediction error δi (million yen/m2 )Simple linear
Spatial dependency
GWR
Spatial dependency +GWR

Figure 5. A comparison of prediction rate curves

The four curves twist together for most of the span. This implies that their performances
are fairly alike. All of them predict the unit price of about 70% of samples if a tolerance10

of 0.1 million yen/m2 is accepted. Within a tolerance of 0.2 million yen/m2 , about 95%
are correctly predicted. With the prediction error being larger than 0.2 million yen/m2 ,only about 5% of the sample can be predicted.
On the other hand, it might have been noted that the tail of the simple linear model
winds up a bit earlier than the other models. This is able to explain why this model
looks better than the others in Fig. 1~3. Nevertheless, we can still draw the conclusionthat they are similar, allowing that the tail part is not very important.
From the above empirical tests, it seems proper to say that, for this dataset, the proposed
spatial dependency model, GWR model, and the mixed model of the two are notsignificantly better than a simple linear model. In other words, the simple linear modelis fairly satisfactory. Moreover, we should not stick to the spatial relationships revealedby the above spatial models.
The failure of these three spatial models might either result from inappreciable spatial
relationships in the sample area, or result from inappropriate model specifications. In
the latter case, alternative specifications might be constructed and tested with similar
empirical techniques as described above.

4. CONCLUSION
In the sense that empirical criteria are not dependent on statistical assumptions, they fit
for any dataset and their application is not restricted to a certain kind of models.
Nonetheless, these methods are especially outstanding to evaluate spatial exploratorymodels of hedonic regression, where assumptions concerning the presence of spatial
relationships are uncertain. In such models, it is quite reasonable to believe that theadditional consideration on spatial relationships should improve the prediction power ofan ordinary model. Accordingly, we can simply focus on the prediction power ofmodels. This is plain and straightforward. These tests also provide an alternative for thesignificance test of spatial relationships. If a spatial model does not outperform an
ordinary model, we may just think that the estimates of the latter are robust enough.
Much extension can be made based on the empirical evaluation concept. For example,
we can judge the contribution of a variable by comparing the prediction powers ofmodels with and without the variable, or look for an optimal tolerance level for a dataset,and so forth. The empirical approach might become effective alternatives for statisticalevaluation methods.

REFERENCES
Anselin, L., 1988. Lagrange multiplier test diagnostics for spatial dependence andheterogeneity. Geographical analysis, 20, pp. 1-17.
Bourassa, S. C., M. Hoesli, and V. S. Peng, 2001. Do housing submarkets really matter?

11

AsRes Sixth Annual Conference, August 2001, Tokyo, Japan.
Brunsdon, C., A. S. Fotheringham, and M. Charlton, 1999. Some notes on parametricsignificance tests for geographically weighted regression. Journal of Regional Science,39(3), pp. 497-524.
Can, A. 1990. The measurement of neighborhood dynamics in urban house prices.
Economic Geography, 66, pp. 254-272.
Can, A., 1992. Specification and estimation of hedonic housing price models. Regional
Science and Urban Economics, 22, pp. 453-474.
Casetti, E., 1972. Generating models by the expansion method: applications togeographic research. Geographic Analysis, 4, pp. 81-91.
Dubin, R. A., 1992. Spatial autocorrelation and neighborhood quality. Regional Scienceand Urban Economics, 22, pp. 433-452.
Dubin R. A., 1998. Spatial autocorrelation: a primer. Journal of Housing Economics, 7,
pp. 304-327.
Fotheringham, A. S. and C. Brunsdon, 1999. Local forms of spatial analysis.
Geographical Analysis, 31(4), pp. 341-358.
Fotheringham, A. S., M. E. Charlton, and C. Brunsdon, 1998. Geographical weighted
regression: a natural evolution of the expansion method for spatial data analysis.
Environment and Planning A, 30, pp. 1905-1927.
Gao, X. and Y. Asami, 2001. The external effects of local attributes on living
environment in detached residential blocks in Tokyo. Urban Studies, 38(3), pp.
487-505.
Getis, A. and K. Ord, 1992. The analysis of spatial association by use of distancestatistics. Geographical Analysis, 24, pp. 189-206.
Switzer, P., W. S. Kowalik, and R. J. P. Lyon, 1982. A prior probability method forsmoothing discriminant analysis classification maps. Mathematical Geology, 14(5), pp.
433-444.

ACKNOWLEDGEMENTS
The authors gratefully acknowledge the comments by Atsuyuki Okabe and Yukio
Sadahiro. The present study is supported by the Grant- in-Aid for Scientific Research by
the Ministry of Education, Culture, Sports, Science and Technology.

12

APPENDIX A. A DESCRIPTION OF DATASET
The database used in this paper was developed in Gao and Asami (2001). It involves
190 detached housing lots in western Tokyo. Sample area is small, about 110 hectare.
Sample properties were drawn from the Oct. 1996 to Sep. 1997 issues of Weekly
Housing Information magazine, which provides transaction price and basic informationof land and houses in Japan.
In addition to the information provided by the magazine, details of sample properties,
such as road, sunshine, distances to public facilities, and neighborhood environments,
were obtained through site survey, GIS applications, and so on. The dataset also
includes the location information of sample lots, represented by the x and y coordinatesof their center point.

APPENDIX B. THE ESTIMATES OF A SIMPLE LINEAR MODEL

Constant
ActualFAR
T.station
W.roadResidual.b.age/SLandscapeT.Shinjuku
Frontage
Goodpavement
Parkinglot
B.quality1
Sunshine/S
Con.greenery/S
Con.greenery
Mix-use3/S
Mix-use3
Tree1

CoefficientStd.
(million
Coefficient
yen/m2 )/unit
0.91150.12760.182-0.0157- 0.4200.02090.137
0.56860.401-0.1726- 0.400-0.0168- 0.3100.00580.113
0.04200.114
0.03820.153
0.05750.150
0.94760.125
21.4547
0.268-0.1956-0.257-17.4766-0.3810.23840.404
0.03350.085

t-statistic

Sig. level

9.165
3.215
- 9.6122.855
6.419-8.463-6.596
2.383
2.798
3.536
3.507
2.669
3.138
-2.968-2.438
2.635
1.992

.000
.002
.000
.005
.000
.000
.000
.018
.006
.001
.001
.008
.002
.003
.016
.009
.048

R-square: 0.756, adjusted R-square: 0.734.
See Appendix D for the definition of variable names.
Statistical tests showed that the coefficients are quite stable, suggesting that the model
does not have serious multi-collinearity problems. In addition, the model was shown tobe superior to some other simple formed models such as log linear regression model.

13

APPENDIX C. THE ESTIMATES OF SPATIAL DEPENDENCY MODEL

Constant
ActualFAR
T.station
W.roadResidual.b.age/SLandscapeT.Shinjuku
Goodpavement
Frontage
Parkinglot
B.quality1
Sunshine/S
Con.greenery/S
Con.greenery
Mix-use3/S
Mix-use3
Tree1
NN-actualFAR
NN-t.station
NN-w.road
NN-residual.b.age/S
NN-landscapeNN-t.Shinjuku
NN-goodpavement
NN-frontage
NN-parkinglot
NN-b.quality1
NN-sunshine/S
NN-con.greenery/S
NN-con.greenery
NN-mix-use3/S
NN-mix-use3
NN-tree1

CoefficientStd.
t-statistic(million
Coefficient
yen/m2 )/unit
0.87207.464
0.09490.136
2.073-0.0117-0.311-3.5320.00480.069
1.154
0.66100.467
6.910-0.4670-1.082-4.453-0.0091-0.168-2.3630.04000.108
2.269
0.07790.151
3.102
0.02960.118
2.559
0.06860.179
3.637
0.63100.083
1.773
34.4830
0.431
4.172-0.2760-0.363-3.925-13.7990-0.301-1.8790.20700.351
2.147
0.02090.053
1.122
0.0299-0.0033
0.0099-0.10800.2780-0.0074
0.02420.00150.0080-0.0266
0.6290-13.7210-0.0011-8.82100.09140.00690.044-0.0850.068-0.0740.650-0.138
0.065
0.031
0.032
-0.0700.073-0.163-0.001-0.174
0.145
0.017

0.680
-0.9781.026-1.1272.629-1.941
1.338
0.572
0.725
-1.4331.582-1.435-0.014-0.7870.650
0.324

NN- indicates the prior characteristic terms estimatedfrom the nearest neighbor of sample lots.
R-square: 0.801, adjusted R-square: 0.761.
See Appendix D for the definition of variable names.

14

Sig.
level
.000
.040
.001
.250
.000
.000
.019
.025
.002
.011
.000
.078
.000
.000
.062
.033
.264
.497
.330
.306
.262
.009
.054
.183
.568
.469
.154
.116
.153
.989
.432
.517
.746

APPENDIX D. THE DEFINITION OF VA RIABLES
Variable name
ActualFAR
Con.greenery
Frontage
Goodpavement
LandscapeMix-use3
Parkinglot
Residual.b.age
S
Sunshine
Tree1
T.Shinjuku
T.station
W.roadMeaning (unit)
Building floor area / lot area (ratio)
Adjacent to public green space, if true, 1, otherwise 0
The frontage of a lot (m)The pavement of the road in front of a lot being good, if true, 1,
otherwise 0
Within designated landscape areas, if true, 1, otherwise 0
Intensive mixed land use, if true, 1, otherwise, 0
The count of parking lots (number)
Residual building age (year)
The size of a lot (m2 )Sunshine duration (hour)
Greenery in the district being good, if true, 1, otherwise 0Time distance to central city area (minute)Time distance to the nearest station (minute)
The width of the road in front of a lot (m)15