Measuring Bias in the LFS for Trade Union Membership Statistics MARCH 2013 

## Contents

Measuring bias in the Labour Force Survey for the Trade Union Membership Statistics ....... 3 
Introduction ................................................................................................................................... 3 
Proxy response ......................................................................................................................... 3 Whether pay and conditions are affected by collective agreements ......................................... 5 
Summary 
....................................................................................................................................... 7 

## Measuring Bias In The Labour Force Survey For The Trade Union Membership Statistics Introduction

The Office for National Statistics (ONS) has identified that there are uncertainties around the estimates derived from data collected in sample surveys, such as the Labour force Survey (LFS). The ONS's LFS User Guide, Volume 1 - LFS Background and Methodology 
20111, considers, in Section 7, the potential impact of non-sampling errors affecting the accuracy of LFS estimates. These include things like non-response bias (where responders to a survey differ from non-responders), non-sampling variance (for instance where different interviewers may cause respondents to answer questions  differently) and, for the LFS, potential respondent bias (where proxy respondents, providing answers for someone else, may consistently under- or over-estimate information about respondents). ONS sets out how the LFS at the survey level tackles these areas of potential nonsampling error, to help keep any uncertainties around population estimates to a minimum. 

This note looks at specific areas of uncertainty relating to the trade union membership statistics resulting from potential non-sampling error. 

In the annex to *Trade Union Membership Statistics 2011*, the technical note specified two potential areas of non-sampling error which may mean there is some bias impacting on the published TUM results. These are: 

1. Proxy response 
2. collective agreements - some employees, especially those in small workplaces 
within larger organisations, may not be aware that their wages and terms and conditions are subject to collective agreement negotiated by trade unions or staff associations at the organisation level. 

## Proxy Response

Trade Union Membership statistics are calculated from the Labour Force Survey (LFS). 

Respondents to the LFS can respond either in person, or through a family member responding on their behalf, called "proxy" respondents. Proxy respondents account for approximately one third of all respondents. As the trade union membership questions are only asked in one quarter a year, there is less opportunity to correct any inaccurate information provided by proxy respondents in future interview waves (each household –
identified by address - is interviewed for five consecutive quarters). However, In section 
                                            
11 of the LFS *User Guide*, ONS identify that there is some uncertainty due to respondent bias affecting a number of LFS variables.  
Annex 1 of the Trade Union Membership release states that there may be bias in the number of union members reported, as it appeared that a smaller proportion of proxy responses were members of Trade Unions than responses in person. Analysis shows that in 2010 19.6% of proxy respondents are members of Trade Unions compared with 26.0% of respondents in person. However, the likelihood of being a proxy respondent is affected by a person's characteristics, for example their occupation, age and sex. A younger male working in the skilled trade occupations is the most likely of all categories to be a proxy response. The likelihood of being a member of a trade union is also affected by these same variables. Therefore it is not possible to measure the level of bias by comparing the totals (19.6% and 26.0%) alone. The observed difference may be due to the difference in the underlying characteristics of the proxy respondents and not affected by the fact that they are a proxy response at all. In order to work out whether there is a bias in the data, and to assign a numerical value to the bias, a logistic regression was conducted on the weighted values of the respondents who responded in person. The regression modelled Trade Union Membership (albeit with a small R2) for all non proxy responses using variables: age, sex, employment status and occupation. These variables were chosen empirically and both significant and insignificant variables were kept in the equation for this task. It may be possible that other LFS variables correlate both with the proxy and trade union membership, but it was not possible to include everything in the regression.  
The regression equation2 was then applied to the weighted dataset for proxy respondents to produce a predicted probability of being a member of a trade union based on their age, sex, employment status and occupation alone. The results show that we would expect proxy respondents to have lower trade union membership that responses in person due to the demographic differences, however, these taken into account, we would expect union membership of proxy respondents to be 22.1%. However, only 19.6% of proxy respondents are reported to be members of trade unions. Therefore, bias in the proxy respondents for the 2010 Trade Union statistics can be estimated to be 2.5%. It is important to note that there may be other factors that also explain some of this bias; only 4 variables were tested in this analysis.  
In total, if we assume that there is a bias of 2.5% in the proxy respondents, this would increase the final proportion of trade union members in the population from 23.8% to 24.7%, an increase of almost 1%. It is unrealistic to attempt to remove this bias from the results as each of the LFS variables would have to be included in a regression analysis, which would also have to be carried out for the complete time series of TUM statistics. However, users should note that the respondent bias due to the use of proxy respondents in the LFS causes uncertainty in the published numbers of trade union members: In 2010, analysis showed that the published figures were potentially as much as 1% low due to the bias from proxy respondents.  

## Whether Pay And Conditions Are Affected By Collective Agreements

A second area of potential bias in the TUM statistics is found in the area of Collective Agreements. The LFS asks household members whether their pay and conditions are affected by collective agreements. However, there is an alternative annual source of information on collective agreements determining pay: the Annual Survey of Hours and Earnings (ASHE) also collects information on collective agreements, this time from employers. The two sources produce different results: 
For 2011, the LFS reports that 31.2% of UK employees had their wages and  conditions of employment affected by collective agreements between their employer and a trade union/staff association. This rose to 67.8% in the public sector, compared to only 16.9% in the private sector. 

In contrast, ASHE reports that 46.9% of employees had their pay set with reference to a collective agreement. This comprised 91.9% of public sector employees, and 27.0% of private sector employees. 

Therefore, there seems to be a difference between the LFS and ASHE data that jointly affects both the public and the private sectors. 

Two factors are likely to account for much of the differences. Firstly, the surveys ask different questions in relation to collective agreements: After a series of 2 other questions specifically about trade unions, the LFS asks respondents (household members): 

## Are Your Pay And Conditions Of Employment Directly Affected By Agreements Between Your Employer And Any Trade Union(S) Or Staff Association(S)?

On the other hand, with no previous mention of trade unions, ASHE asks  employers:  
Was the employee's pay set with reference to an agreement affecting more than one employee? For example, pay may be agreed collectively by a trade union or workers' committee. 

ASHE then goes on to ask those employers answering yes what type of agreement was made, offering five categories: national or industry, sub-national, organisational, workplace, national or industry supplemented by sub-national/organisational/workplace. 

 
The more detailed ASHE question underlies the complexity of information on collective agreements. The different questions may potentially suggest different interpretations, such that it may not be clear that some national agreements, for instance those determined by pay review bodies, were agreed between the employer and trade union/staff association. The LFS collective agreements question, which follows questions related to trade union membership and union presence in the workplace, may suggest to respondents that the question is asking about situations where unions are negotiating directly with the employer. ASHE, meanwhile, is focusing on those situations where employees pay is made with reference to a collective agreement - which may include situations where the employer itself does not negotiate with unions or staff association. This seemingly wider definition is likely to account for at least some of the difference. Another potential factor is that while the LFS is asking respondents about their pay (ie from the employee's perspective), ASHE is asking employers about whether the pay their employees receive is referenced to a collective agreement. Employers are more likely to know whether this is the case than employees - it is possible that a proportion of employees will not know how their pay is determined, especially where employers reference outside agreements (agreed at higher levels than the workplace, or organisation, or where firms make use of agreements where they haven't been involved in negotiation to provide a standard for their own pay agreements). Therefore, the different questions suggest that the LFS and ASHE provide differently defined indicators of whether employees have their pay affected by collective agreements, with the LFS definition more tightly focused, and the ASHE definition broader:  

- The LFS specifically asks employees about whether their pay and conditions of 
employment are affected by a collective agreement between employers and trade unions/staff associations 
-  ASHE asks whether an employee's pay is set with reference to an agreement 
affecting more than one employee - so this is much broader, to the extent that it 
could include agreements not involving trade unions or staff associations directly. 
 
The complexity of how to define collective agreements is demonstrated by the recent Workplace Employment Relations Survey (2011), which reports that in 2011 16% of private sector employees had their pay set by collective bargaining, compared to 44% of public sector workers. The reason given for the relatively low public sector figure was because an independent pay review body had taken responsibility for pay in the health industry following the Agenda for Change negotiations3. If pay review body awards are included within the collective agreement definition, which may be shown in a further WERS publication later in the year, then the percentage of employees covered by collective agreements will rise. Essentially this emphasises that ASHE and the LFS are producing different indicators of collective agreement coverage, and therefore the different results should not be unexpected. However, it is important to note that the LFS measure reflects the employees' perceptions of whether their pay or conditions are affected by a collective agreement between their employer and a trade union/staff association, and this will be affected partly by the employee's awareness, as well as the coverage of collective agreements across the UK workforce. 

## Summary

In summary, the 2011 Statistics on Trade Union Membership publication identified  two potential non-sampling errors which may have caused bias in the TUM statistics.  
Firstly, our analysis of 2010 data suggests that our estimates of the total number of union members may be low by almost 1% due to proxy respondents being less likely to be TU members than respondents in person. Secondly, we have determined that the ASHE survey provides a different measure of collective agreement coverage than the LFS, and that the different definitions (reflecting questionnaire design) are likely to account for much of the difference in their results on this issue (as suggested by the WERS results). The LFS definition is narrower than that for ASHE, so that should be taken account of when making use of the trade union membership statistics.  It should also be emphasised that the LFS measure reflects employees' perceptions as to whether their pay and conditions are affected by collective agreements between their employer and trade unions/staff associations. We do not intend to change the trade union membership statistics methodology on the basis of this analysis and therefore there will be no break in the time series. However, we will continue to monitor these two areas, and will consider this approach in light of emerging evidence.   
© Crown copyright 2013 You may re-use this information (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence. Visit www.nationalarchives.gov.uk/doc/open-government-licence, write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: 
psi@nationalarchives.gsi.gov.uk. 

This publication available from www.gov.uk/bis  
Any enquiries regarding this publication should be sent to: 
Department for Business, Innovation and Skills 
1 Victoria Street London SW1H 0ET 
Tel: 020 7215 5000 
If you require this publication in an alternative format, email enquiries@bis.gsi.gov.uk, or call 020 7215 5000.