**Population estimates for very small areas in the UK: performance and
promise**

**Stephen Ludi Simpson, Estimating with Confidence Project**

**Paper given to the conference on population estimates methods, US
Bureau of the Census, Washington DC, June 8^th^ 1999.**

Correspondence: Stephen Simpson, City of Bradford Metropolitan District
Council, Policy and Research Unit, City Hall, Bradford, BD1 1HY. Email:
ludi.simpson@man.ac.uk

The major part of this paper reproduces a summary of the evidence and
conclusions from an extensive study of small area population estimates
in Britain. That study was the precursor of current plans for a UK-wide
research project producing small area population estimates, which in
1999 awaits a decision on funding.

The proposed project aims to demonstrate the feasibility of producing:

-   detailed age-sex profiles

-   annually updated

-   each figure with a quantified measure of reliability

-   using an accepted methodology

-   incorporating local datasets

-   on-line extraction and aggregation

-   satisfying all user's geographies of small areas.

The main features of these plans, which would be implemented in
2000-2001, are as follows.

**Political structure:**

Providing that the academic Economic and Social Research Council fund
the project, we have promise of data from various government agencies
and a strong dose of good will from users of population estimates, to
try out a method and some variations to produce population estimates by
age and sex for every small area in the UK.

The funders, data providers and users - from health, academic and
commercial sectors, local and central government - will each be
represented on an advisory group for the project.

We intend to put the results on the Internet and to actively invite
comment, particularly from local government. Outside of census year
there is no direct way of validating our results; consultation with
those who have local knowledge is a very important method of trapping
major errors in our estimates.

We will have a major challenge ourselves to distinguish what constitutes
good evidence for the alternative local estimates that we will be
offered. My own inclination at this research stage, is to substitute in
place of our estimates any alternative estimates based on local
evidence, even though we cannot confirm the quality of that evidence. I
can see little reason for local government to favour one of their
smaller areas over another. The adoption of minimum criteria for
accepting locally-produced population estimates will be useful, along
the lines of the ground-rules recently developed by the FSCPE program in
the USA.

I cannot say that we have yet won the argument for adopting
locally-produced estimates within a national set of statistics, even
when they are well-resourced and likely to more accurate than a
nationally-produced standard method. There is suspicion that this is not
consistent or even-handed, even when it is accepted that a standard
method will itself produce results whose quality varies between areas.

**Geography:**

We make the distinction between:

*Data units* - which are for a variety of different geographical
entities.

*Estimation Units* - the geographical units for which estimates are
made. These will for us in the first instance be electoral wards of
population 1-30 thousand population, but could be the much smaller
census output areas.

*Disaggregation units* - to which estimates are spread, not for
publication but to create building blocks for subsequent re-aggregation.
Ideally these disaggregation units will be geo-coded and postcoded
individual addresses, using the Addresspoint file (similar to the US
Master Area File but updated twice a year by government agencies). If
there are one thousand addresses in an estimation unit, we give each
0.1% of the estimate. Equally but with more approximation, the
disaggregation units could be unit postcodes, with an average of 15
addresses in each.

*Dissemination units* - the areas for which users want estimates. Users'
boundaries are generally unpredictable and overlap the estimation units,
but can be accurately constituted from disaggregation units using
look-up tables or GIS point-in-polygon routines.

This approach is very oriented toward users' needs for specific
geographies:

1.  It maintains consistency between outputs for each geography.

2.  It gives better accuracy for the users' area than approximating with
    whole estimation units.

And it is clearly suited to on-line extraction of estimates via a GIS
server.

**Datasets and methods:**

We will not substitute anything for the local government district
population estimates, already produced by national statistics offices.
These are the large areas, of 50,000 to 1 million population.

We will use two fairly straightforward methods to produce initial
estimates:

1.  Using administrative counts from patient and electorate registers,
    to apportion District populations to local areas, using what we know
    of the measured quality of the electorate at the last census year

2.  Rolling forward the past census.

For every small area, our consultation estimate will be the average of
the two initial estimates. The difference will be a measure of
reliability.

We will also publish locally-provided estimates whenever they are
offered with convincing evidence.

In parallel we will attempt other methods, including a component method
using births, deaths and migration for small areas, the latter based on
patient re-registration.

In all these methods we will attempt to deal with special populations -
armed forces and students in particular - with special datasets.

At the end of this process we intend to have developed candidate methods
suitable for use in the post-censal decade 2002-2011. The final choice
between candidate methods will await evaluation, including against the
2001 Census, as described at the end of this paper.

The remaining pages will also be published as "How can a national set of
small area population estimates gain acceptance? Lessons from the
*Estimating with Confidence* project", authors Stephen Simpson, Ian
Diamond, Elizabeth Middleton, David Lunn, Rachel Cossey, currently in
press as a chapter in *'Population counts in small area studies:
implications for studies of environment and health'* (Eds: Elliott,
Paul; Arnold, Richard) Studies on Medical and Population Subjects, The
Stationery Office, London, 1999.

**How can a national set of small area population estimates gain
acceptance? Lessons from the *Estimating with Confidence* project**

The *Estimating with Confidence* project was born from a network of
local authorities keen to evaluate their own demographic work; it
developed with academic support into a major research programme,
collated population estimates from 50 producers for over 5,000 small
areas and accounted for the variation in their accuracy^1-3^; most
recently it has produced a guide to the methods and data available to
make local population statistics in Britain^4^.

The focus of *Estimating with Confidence* was the accuracy of population
estimates for small areas made by local authorities between census
years, as measured against results from the 1991 national census. This
paper aims to assess the feasibility of a single national strategy to
produce updated population estimates between national census years for
all small areas in Britain. It examines the difficulties that such a
national strategy may have to overcome before they could be generally
accepted and used, and proposes promising lines for development. In so
doing the paper summarises many of the main results arising from the
*Estimating with Confidence* project.

**A national strategy producing local population estimates for all small
areas: who wants one?**

Among *Government Departments*, the Department of Health have identified
the need for a consistent set of population estimates by age and sex for
each electoral ward area in England and Wales. A measure of demand in
the catchment areas of the hospitals and other institutions is needed
for funding purposes. Such statistics would no doubt be also used in
fertility and mortality calculations for standard small areas. In a
separate development the Department of Health has funded research at the
Small Area Health Statistics Unit (SAHSU) at Imperial College, on the
impact of potential environmental health hazards. The calculation of
morbidity and mortality rates requires population estimates for each of
the past twenty years as part of its calculation of age-sex-specific
illness and mortality rates in ad hoc areas around the source of the
hazard. This research is described elsewhere in this publication.

British (formally, English, Welsh and Scottish) and European regional
policy requires local authorities to compete for resources by making the
case for the needs of specific local areas: those competitors which can
use up-to-date estimates are in a better position to make a convincing
case. Other Government requirements for regional planning, for example
to identify land for housing, employment and other uses, also create an
occasional demand for population statistics consistently for all small
areas in a region - usually counts of people or households.

The reorganisation of *local government* throughout Britain during
1996-1998 has abolished many of the larger authorities which have been
responsible for the small area population estimates used by local
authorities. In some authorities there is therefore a greater reliance
on commissioned work from commercial or other public bodies, and thus a
demand for standard demographic products.

In the early 1990s there were over one hundred different local
authorities undertaking demographic estimation work, each using local
administrative data sources and thus each with a different estimation
strategy^3^. While defending the relevance of their methods and work,
these producers often also seek a common understanding and
implementation of methods and data.

The *media industry* has expressed a need for a nationally consistent
and complete set of population estimates for small areas. It has founded
the Joint Industry Committee for Population Standards (JICPOPS), to
investigate how advertisers may have a reliable measure of the reach of
different media outlets as a percentage of the population of defined
areas. Two companies - Experian and CACI - were working for JICPOPS in
1997 and 1998 to provide common estimates of the population of each
postal sector in Britain, with emphasis on total population, number of
adults and number of households.

Thus although the purpose and detail of their requirements vary, there
is widespread demand for population estimates for small areas.

**A national strategy producing local population estimates for all small
areas: who has attempted to provide one?**

Although government offices were not in 1998 intending to produce
population estimates for areas smaller than local authority and health
districts, the Office for National Statistics (ONS) were actively
analysing the quality patient registers, a dataset with great potential
for local population statistics. They are held by local health
authorities to administer General Practitioner payments and the transfer
of medical records as patients change the doctor with whom they are
registered (also discussed elsewhere in this publication).

The box summarises four efforts to produce a national set of population
estimates for small areas. In addition to the three mentioned already,
the London Research Centre (LRC) produce small area statistics for all
of inner and outer London and for Berkshire, and will respond to
requests from other organisations.

The very fact that the four attempts use very different datasets and
approaches serves to emphasise the difficulty involved - there is no
methodology for producing population statistics at local level that has
been proven or accepted as cost efficient.

**Problems to be expected when providing population estimates for small
areas in Britain**

Accuracy has been measured here by the distance of the estimate from the
true population, expressed as percentage of the true population. The
direction of the inaccuracy is ignored - our studies found no tendency
to over- or under-estimate populations. The creation of a 'gold
standard' true population involved estimation of the non-response in the
1991 census and was a major preparatory exercise prior to the analyses
of accuracy reported here^5^.

*As finer age groups are estimated, the accuracy of population
statistics reduces:*

For a ward-sized area, generally of 3,000-15,000 population:

-   Total population:

> median accuracy 4%, 90% of small areas within 10% accuracy

-   5 broad age groups:

> median accuracy 9%, 50% of small areas within 10% accuracy

-   18 fine age groups:

> median accuracy 15%, 25% of small areas within 10% accuracy

Young adults (due to their mobility, see below), and elderly age groups
(partly due to their small numbers) are liable to be estimated with
greatest percentage inaccuracy, as also shown in figures 1 and 3 later
in this paper.

*Some types of area are more difficult to estimate:*

-   Small areas

> Percentage accuracy increases with the size of area estimated: this is
> the most consistent result from studies in Britain and abroad.

-   Fast changing areas

> Unfortunately the object of the exercise, to monitor changing
> population, is most difficult in precisely those areas with greatest
> change - where accurate monitoring is most needed.

-   City areas

> Inner city cores are more difficult to estimate, rather than urban
> suburbs and rural areas.

-   Mobile populations

> Areas with concentrations of students, armed forces, and young adults
> in general, change their size rapidly and are usually represented
> poorly by administrative indicators of population change such as the
> electoral and other registers.

These results are derived from multivariate statistical analysis - each
effect is additional to the other effects and applies to all methods.
There are interactions with age: all age groups are relatively difficult
to estimate in a local area with many students, although ages 15-24
particularly so^2^.

*There is only one strategy that is clearly more accurate than others -
a local census. But this is too expensive for anyone but government to
implement nationally.*

Large inaccuracies must be expected: Figure 1 displays results of an
experiment using eight different methods for 222 wards of Hampshire^6^.
No method estimated every 5-year age group in any ward-sized area to
within 10% accuracy ten years after the last census.

All methods of providing an updated 1991 population estimate gained, on
average, a better accuracy than relying on the 1981 census with no
update. There was not a great deal to choose between the six 'desktop'
methods (shown with thin lines in figure 1), which in various ways used
statistics from administrative registers, the previous census, and
government estimates of district population. The desktop methods that
did worse than no update at all, at young adult ages, were those that
aged on all areas the previous census age structure. This was a
particularly erroneous assumption for areas with many students and armed
forces which regenerate a stable age structure.

However, Figure 1 clearly shows that a local census is more successful
than other methods for estimating five-year age groups, by 5% to 10%
depending on the age group. In this case the local census consists of a
card from Hampshire County Council inserted, every other year since the
late 1970s, in the electoral canvass carried out by Hampshire's District
Councils.

In published studies using the data from Hampshire and four other local
censuses, the gain in accuracy was found to be less for broad age groups
(3% - 6%)^2^ and for the total population (1%)^1^. There is evidence
that some difficult areas - fast-changing ones - can particularly be
better estimated by local censuses than by other methods^2^.

These gains in accuracy are accompanied by tremendous flexibility of
geographical boundaries and tabulations of household composition gained
from a local census, even when recording only age and sex for each
resident. However, the gains must be weighed against the costs of a
census; although they have been considered worthwhile by some two dozen
local authorities at some stage during the past twenty years, they would
require national assistance if undertaken in all areas between the
current ten-yearly national censuses.

-   *Different areas require different treatment.*

The understanding of the *Estimating with Confidence* project is that
much of the difficulty in monitoring local populations is the local
specificity of the causes of population change, and the different impact
these causes have on the common methods of estimating population. Thus a
method appropriate in one area will not necessarily be appropriate in
another.

Discussion of Figure 1 has already drawn attention to transient special
groups, often composed of young adults in particular occupations and
housing. Local areas like these regenerate a stable age structure rather
than age over time as the occupants of a suburban housing estate might
be expected to.

Figure 2 shows a single ward where housing development in the 1980s
added very significantly to the total population, but where the new age
structure neither repeats the previous census (as assumed in the
estimate shown) nor reflects the previous census' structure aged by ten
years. The type of migrants to the housing development - tending in this
case to be young professionals, some with young children - would have to
be known before the age structure could be estimated correctly without a
census.

The practical relevance of any method depends on the available data.
More data about migration is available for districts than for wards, and
more electoral and vital event data are available for wards (postal
sectors in Scotland) than for other small areas. The implementation of
one method for all small areas is likely not to be a successful
approach. This constraint imposed by available data is perhaps less
permanent than may be imagined, with the increased availability of
postcoded datasets and computing power to manipulate them.

It has proved difficult for statistical analysis to identify exactly
which demographic circumstances favour one method rather than another.

Figure 3 points to one solution when confronted with such uncertainty.
The average of two methods estimate local populations more accurately
than either of the two methods individually; this is the case whenever
one of the methods is not consistently better than the other, and the
two are not too alike^6^. In the terms of Ray Chambers' contribution to
this publication the two are equally good estimation procedures and
somewhat independent of each other.

The logic here is that in some cases under-estimates will cancel
over-estimates to produce a better estimate. In addition, and equally
important, the largest inaccuracies will be avoided because for each
area the error of the worst estimate will be reduced.

Local twists and turns to population change highlight two other
difficulties for a national strategy for population estimates: the lack
of national data on local areas, and the existence of local knowledge
that would challenge nationally-produced estimates.

*There is a lack of relevant local data available nationally*

Only vital statistics and registers of dwellings are accessible to
government as individual postcoded demographic records; as tabulations
only electors in wards and pupils in schools are regularly available for
all areas of Britain between census years. At greater cost, registers of
postcoded electors and of patients can be attained^4^. These are
sufficient data to update population estimates, but require bold
assumptions about the consistent quality of each source as a demographic
indicator.

*There is a wealth of relevant local knowledge only available to some
agencies which will validate the quality of data and assumptions
employed by national producers of population estimates.*

The administrative registers nationally available for demographic work
depend on local agencies, particularly the local authority which
provides information for updated dwellings and electoral registers, and
the health authority which administers the patient registers. These
registers vary from area to area and from year to year as a result of
changed procedures (for better or worse), as well as from real
population changes. Experienced planners and demographers in those local
agencies are in a position to judge the local administrative turns that
may affect the quality of a register for demographic purposes. In
addition, local staff can use extra data sources (such as knowledge of
housing developments, local registers of school pupils, institutions or
dwellings, and occasional surveys), to help them judge the truth of
estimated changes in population.

Given the large inaccuracies that can be expected from local population
statistics, it is not difficult to foresee that local planners will
often be able to correctly challenge population estimates made from a
national strategy for all small areas.

**What hope for a national set of population estimates?**

In spite of the difficulties suggested above, a national government set
of population estimates for small areas may nonetheless be welcomed by
many local authorities and others. The public research involved in
providing them would reduce the current regular re-invention of methods
and knowledge. Both the methodology and the results would provide a
baseline that may be improved on in particular areas.

To have parallel official and unofficial sets of population statistics
is not necessarily an unstable situation nor unhelpful to planning. That
situation already exists for district population estimates, where
government work is shadowed by many local authorities to the benefit of
everyone's understanding of true population developments.

The problem is thus not only a technical one of achieving the most
accurate estimate for every small area, but also a political one of
finding an acceptable strategic approach.

*A successful strategy to produce population estimates for all small
areas is likely to allow different treatment of different areas.*

In particular, a successful strategy will cater for the way in which
transient special populations affect (or maintain) the local age
structure, and for the differential quality of datasets in each areas.

*A successful strategy to produce population estimates for all small
areas needs to incorporate local evidence.*

Government population statistics for local areas would be more
acceptable, and therefore more feasible, if a co-operative effort
substituted statistics for small areas that had been validated locally,
in place of draft estimates produced by government.

Such a co-operative effort is also not without precedent. The draft
migration assumptions in government population projections have for many
years been subject to discussion with local authorities who are
acknowledged to occasionally have better evidence for the likely future
trajectory of recent trends. In the USA, a Federal State Co-operative
Population Estimation programme has regularly incorporated data
validated by State agencies for localities into population estimates
used for Federal programmes^7^. As data available to Federal agencies
have improved, the use of local validation has lessened. In Britain, all
local data would need to be consistent with government estimates for
larger areas to gain acceptance, but simple scaling procedures can
ensure that this does occur.

*The 2001 Census provides an opportunity to test strategies for small
area estimation against what, hopefully, will be an accurate benchmark
of the true population in that year.*

There are four years before the results of the 2001 Census form another
opportunity against which to test our ability to estimate the population
of small areas between censuses. This is an ideal period to develop
existing and new estimates of the 2001 population without use of the
Census results, in order to compare the efficiency, with a view to
serious production of a regularly updated set of population estimates
for each small area of Britain during the following decade.

Such an assessment can gain from the experience of the *Estimating with
Confidence* project. In particular the successful isolation of local
factors associated with accuracy of population estimates in general,
suggests that each test strategy need not produce estimates for the same
set of areas, nor even named areas. Instead, the size and other
characteristics of the areas can be taken into account in order to then
fairly compare the accuracy of each strategy tested.

The dataset for each estimation strategy should comprise:

1.  For each of many small areas each of which is an aggregate of census
    output areas:

-   Population estimate for mid-2001.

-   This population estimate separately for consistent age groups:
    5-year age groups, or for age groups 0-14, 15-24, 25-44, 45-65, 65+
    as used in *Estimating with Confidence* (or for some other agreed
    but standard set)

-   Area characteristics from the 2001 data for each area, to include
    population size, student and armed forces population, unemployment
    and an agreed measure of city cores - perhaps the number of imputed
    households in the 2001 Census.

-   Population change between 1991 and 2001. To be agreed whether raw
    census data or adjusted census data should be used, the latter being
    preferred but possibly impractical.

1.  An account of the data sources and methodology employed, from which
    a timetable of production can be derived.

2.  An estimate of the resources required to update the estimates (a)
    annually and (b) at less frequent intervals, eg 2006.

**Conclusion**

A national strategy for producing population estimates for all small
areas in Britain is quite feasible. It requires acknowledgement of the
different strategies that should be employed in different types of area,
and the incorporation of knowledge available only to local demographers.

Small area population statistics are likely to be produced by a wide
variety of organisations for some time to come.

*Acknowledgements*

This report was first presented to a research workshop entitled
'Population counts in small area studies: implications for studies of
environment and health', at Imperial College Department of Epidemiology
and Public Health, Small Area Health Statistics Unit, November 11^th^
1997.

All the local authority and central government staff and many others who
contributed data and comments to the work of the *Estimating with
Confidence* project must bear some responsibility and credit for the
results reported in this paper. The research and dissemination was
funded by the Economic and Social Research Council, under its Analysis
of Large and Complex Datasets programme, with awards H519255057 and
H519255028.

Correspondence to Stephen Simpson, City of Bradford Metropolitan
District Council, Policy and Research Unit, City Hall, Bradford, BD1
1HY.

Authors' affiliations:

Stephen Simpson (Bradford Council and University of Manchester)

Ian Diamond, Southampton University

Liz Middleton, University of Manchester

David Lunn, Imperial College

Rachel Cossey, Stoke on Trent City Council.

*References:*

^1^ S Simpson, I Diamond, P Tonkin and R Tye (1996) 'Updating small area
population estimates in England and Wales', Royal Statistical Society
series A Vol 159 part 2, pages 235-47.

^2^ D Lunn, S Simpson, I Diamond, E Middleton (1998) 'The accuracy of
age-specific population estimates for small areas in Britain',
Population Studies 52: 327-344.

^3^ S. Simpson, E Middleton, I Diamond and D Lunn (1997) 'Small-area
population estimates: a review of methods used in Britain in the 1990s',
International Journal of Population Geography Vol 3 part 3, pages
265-280.

^4^ S Simpson (ed) (1998) 'Making local population statistics: a
practitioner's guide'. Local Authorities Research and Intelligence
Association: 2 Turnstone Close, Winnersh, Wokingham, RG41 5LQ.

^5^ S Simpson, R Cossey, I Diamond (1997) '1991 population estimates for
areas smaller than districts', Population Trends issue 90: 31-39.

^6^ E Middleton, S Simpson, I Diamond, and D Lunn (1996) 'An experiment
with methods of estimating the population of small areas in Britain'.
Working Paper 14, Estimating with Confidence Project. Department of
Social Statistics, Southampton University.

^7^ J Long (1993) 'Postcensal population estimates: states, counties,
and places'. Chapter 4 in 'Indirect estimates in federal programs',
Statistical policy working papers 21, US President's Office: Washington.