Technical Regression Report 2024
Source: gov.uk (2025-03-13)

Delivering Better Value for Money

Contents

Introduction (#introduction)

VFM metrics data and explanatory tables (#vfmmetrics)

Regression analysis - headline results (#regressionresults)

Regression analysis - additional testing (#regressiontesting)

Explanatory variables excluded from the final model (#exclusions)

Annex A: VFM metrics definitions (#annexa)

Annex B: Explanatory variables definition (#annexb)

Annex C: Sample sizes in the final streamlined model (#annexc)

Annex D: Methodology (#annexd)

Annex E: Details of diagnostic testing (#annexe)

Annex F: Quality assurance (#annexf)

Annex G: Glossary of terms (#annexg)

Introduction

This technical report sets out the methodology and statistical process undertaken by the Regulator of Social Housing (RSH) to examine the factors influencing Value for Money (VFM) metrics of private registered providers (PRPs) in England. Key messages drawn from the analysis are set out in the accompanying VFM metrics summary report (https://www.gov.uk/government/publications/summary-regression-report-2024) which expands upon the observations published in 2018 (https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/744305/VfM_metrics___Technical_regression_report_-_Sept_2018.pdf).

Since 2018, the sector’s value for money metrics have radically changed. Challenges with rising essential expenditure on maintenance and major repairs, especially on levels of spending directed towards high rise buildings and compounded by high inflation and interest rates and below forecast rent increases have had a profound influence on sector performance. Moreover, in recent years, the regulator has begun to collect more detailed information on stock characteristics. The availability of this new data means that the impact of a wider range of factors can now be examined in a new regression model to enhance understanding of the differences in performance across the sector on the value for money metrics that landlords are required to publish.

Value for Money performance is driven by a range of factors, and without controlling for a sufficient range of these factors, simple comparisons around performance across groups of providers are unlikely to be meaningful. Regression analysis is a statistical method which overcomes this: it allows one to isolate the effects of a particular variable, holding all other explanatory variablefootnote 1 (#ftn1) constant. Regression analysis estimates how the value of the dependent variablefootnote 2 (#ftn2) changes given a unit change in the explanatory variable, while the other explanatory variables are held fixed. This models the relationship between the explanatory and dependent variable by determining a best-fit line to a dataset.

While it is a powerful tool for uncovering the relationships between variables observed in data, regression analysis on its own does not identify a specific cause of variation, which means that any inferences from the analysis must come with a degree of caution. The analysis however does provide valuable insights into factors which impact VFM performance and can help providers and other stakeholders understand some of the factors shaping differential performance on the metrics themselves.

This report presents detail of data definitions of the explanatory variables assessed and the diagnostic testing performed to assess the robustness of the analysis. The technical report is split into three main sections:

a. VFM metric data and explanatory variables – sources, definitions and descriptive statistics for key variables.

b. Regression analysis: headline results of the final model and interpretations between the explanatory variables and the VFM metrics.

c. Regression analysis: Additional testing – Economies of scale and explanatory variables excluded from the final model.

Figure 1: Schematic table outlining the statistical process

return to top (#top)

VFM metrics data and explanatory variables

This section describes the data used to run the regression analysis. This includes the definition of explanatory variables, the data sources and the data cleaning process performed. Full definitions of VFM metrics can be found in Annex A.

Data overview

The analysis is based on a cross-sectional dataset of VFM metrics and explanatory variables. This is complete for the vast majority of providers that own and/or manage at least 1,000 homes for 2022footnote 3 (#ftn3). The analysis is underpinned by 2022 data due to the timing of the analysis which was undertaken over an extended period.
- The VFM metrics data is primarily sourced from the Global Accounts (https://www.gov.uk/government/publications/2022-global-accounts-of-private-registered-providers) and most of the explanatory variables have been sourced from the [Statistical Data Return (SDR)]. A small number of variables relating to regional factors has been supplemented by other data sources including the Annual Survey of Hours and Earnings (ASHE) (https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/earningsandworkinghours/datasets/regionbyoccupation2digitsocashetable3) for regional wages, the Index of Multiple Deprivation (IMD) (https://www.gov.uk/government/statistics/english-indices-of-deprivation-2019) for local authority deprivation and the Office for National Statistics (ONS)footnote 4 (#ftn4) for population density.
Certain organisations affected by mergers that took place during the financial year ending 31st March 2022, which would otherwise affect results of some dependent variables due to one-off merger accounting treatment.

A very small number of providers who had missing data related to some explanatory variables reported in the 2022 SDR.

Value for Money metrics data

The VFM metrics are defined in the VFM Metrics Technical Note (https://assets.publishing.service.gov.uk/media/663a1ef34d8bb7378fb6c2fc/20230613_Value_for_Money_metrics_Technical_note_guidance_2024_-Final.pdf) and go some way to help measure performance on economy, efficiency and effectiveness on a comparable basis. As set out in the section above, the data used for the VFM metrics has been primarily derived from the FVA 2022 (electronic accounts data).

Significant value was derived from the observations published in 2018 (https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/744305/VfM_metrics___Technical_regression_report_-_Sept_2018.pdf) which have informed the basis of this updated analysis. Analysis has been focused on the VFM metrics which showed the most explanatory power in the analysis published 2018, with New Supply (non-social), EBTIDA MRI Interest Cover and Operating Margin (SHL) excluded from the analysis. The list of VFM metrics measured in the analysis include:

Reinvestment %

New supply (social) %

Gearing %

Headline Social Housing Cost £

Operating Margin (overall) %

Return on Capital Employed %

Data on explanatory variables

This section describes the key explanatory variables included in the final model, with the same explanatory variables included for each metric. Unless specified otherwise, data is drawn from SDR submissions by providers to the regulator and aggregated to group level. A full variable definition list can be found in Annex B. Descriptive data refers to groups with more than 1,000 units that complete electronic account returns and are hence included in this analysis.

Important new data related to stock characteristics collected within the SDR has been tested for the first time relating to stock age and stock height. These statistics are considered by the United Kingdom Statistics Authority (UKSA) and the regulatory arm Office for Statistics Regulation (OSR) to have met the highest standards of trustworthiness, quality and public value.

A selection criterion was established to assess eligibility for the inclusion of each explanatory variable based on previous statistical significance, data availability and their intuitive link with at least one VFM metric whilst avoiding the inclusion of variables that were likely to be highly correlated.

Regression analysis cannot be reliably run with missing data, which could potentially lead to inaccurate and misleading outputs. Data is complete for 198 PRPs, with a limited number of providers missing data for several variables excluded from the analysis. Three local authorities also had missing data as part of the IMD variablefootnote 6 (#ftn6).

Outliers

Outliers are observations which have ‘extreme’ characteristics and can disproportionately skew the analysis and produce misleading outputs. Providers that specialise in supported housingfootnote 7 (#ftn7) engage in a broad scope of activities which tend to result in higher costsfootnote 8 (#ftn8) with large degrees of variation depending on the level of support provided, meaning these providers often appear as outliers in the analysis. Several supported housing providers were identified in the outlier detection process and were excluded from certain metrics where relevant. The impact of outlier removal is commented on where relevant.

Selection of final model

In total 46 explanatory variables were tested as part of a default model. The variables included in the 2018 published final model and the most statistically significant variables were selected for further testing as part of a reduced model. One functional form of each explanatory variable was included when testing the reduced model to avoid multicollinearity, a required assumption for linear regression. Explanatory variables were subsequently added in a reduced model iteratively to assess the explanatory power of each model and any changes in statistical significance.

The aim of the process was to obtain consistent models across each of the metrics that included economically and statistically significant variables that are intuitive and easy to communicate. Using the full models as the basis of testing, decisions to omit or add variables were made both with reference to R2 and adjusted R2, statistical significance, Bayesian Information Criterion (BIC) and F-test statistics and the intuitive interpretation of results. While additional variables can increase the explanatory power, care was taken not to overfit the model which can lead to misleading results.

Final model explanatory variablesfootnote 9 (#ftn9)

This section outlines the explanatory variables selected in the final model. The full definition, calculation and data sources can be found in Table 7.

Total social stock (000s)

Total social stock (000s) reflects the total number of units owned and managed by each provider. The size of providers range from between 1,018 to 109,802 homes for 2022, with a median total social stock size of 6,831 homes and mean of 14,150 homes. Overall, 8% of groups have more than 40,000 units as of the 2022 dataset.

Figure 2: Number of PRPs by total social stock (000s) (2022 group level data)

Supported housing (% total)

This variable includes supported housing (SH) homes owned and managed as a proportion of total social housing stock owned and managed. Affordable rent stock is excluded from this calculation as AR SH homes are categorised as SH/HOP in the SDR. Registered providers who own and or manage supported housing stock have total supported housing proportions ranging from between 0% to 97.5%, with 20% of providers owning and or managing no supported housing homes.

13 (7%) providers in the dataset are defined as supported housing providers, where more than 30% of total social stock owned or managed is categorised as supported housing. Around 60% of providers own and manage less than 5% of their total social stock defined as supported housing.

Figure 3: Number of PRPs by percentage of SH (2022 group level data)

Housing for older people (% total)

The percentage of Housing for Older People (HOP) shows HOP owned and managed as a proportion of total social stock owned and managed. The measure includes care home units defined as social housing. AR homes are excluded from the calculation as AR HOP homes are categories as AR SH/HOP in the SDR and cannot be disaggregated.

HOP stock as a percentage of total social stock ranges from between 0% to 95.7%, with 15% of providers not owning or managing any HOP stock. Of the 198 providers included in the regression analysis, only six providers are defined as HOP providers with more than 30% of their homes classed as housing for older people. The median proportion of HOP % in the regression dataset is 6.4%.

Figure 4: Number of PRPs by percentage of HOP (2022 group level data)

Wage index

The Annual Survey of Hours and Earnings (ASHE) regional index was calculated for each provider, which provides an average wage for each region, divided by the average national wage (giving England a wage index of 1.00). The methodology is consistent with previous pieces of analysis published by the regulator. Regional wages have been calculated as one third skilled construction trades gross mean annual salary plus two thirds administrative occupations gross mean annual salary. A provider’s wage index was calculated by multiplying the provider’s share of stock owned in each region by the index. Out of the 198 providers included in the final dataset, 88% own or manage over 50% of homes which are concentrated in one region. The remaining 12% of providers are categorised as ‘Mixed’.

Table 1: Regional wage breakdown

Percentage of providers by regionfootnote 10 (#ftn10) and associated wage index

Large Scale Voluntary Transfer (LSVTs) organisations

LSVTs are included in the analysis as a dummy variable, with providers defined as an LSVT if over 50% of total homes were obtained through transfers from local authorities, with the age determined by the date of the largest transfer. The previously published analysis in 2018 included three time-dependent dummy variables in the final model which categorised the maturity of LSVTs based on whether the provider was transferred six or fewer years ago, seven to 12 years ago, or more than 12 years ago. The regression model published in 2018 found that LSVT organisations that transferred more than 12 years ago exhibit no statistically significant relationship when compared to traditional providers. Given there are also no LSVTs less than seven years old, the only dummy variable included is LSVTs less than 12 years old. Eight providers are classed as a LSVT less than 12 years old in the analysis.

Local authority deprivation (IMD)footnote 11 (#ftn11)

The deprivation variable is based on the 2019 Index of Multiple Deprivation’s local authority district summaries to assess the impact of a provider owning higher proportions of homes in more deprived local authorities. The measure maps a provider’s owned social homes per local authority to the deprivation percentile rank for each local authority. This provides a weighted deprivation percentile rank for each provider using the proportion of stock owned in each local authority.

The sector is generally concentrated in relatively deprived areas in England. The spread of deprivation amongst providers is skewed to the right with the median group of providers owning stock in the 38% (2018 published data: 30%) most deprived local authorities.

Figure 5: Number of PRPs by deprivation rank (2022 group level data)

Stock height

Since 2020, the regulator has collected additional data in relation to stock height. In 2022, data collected in the SDR relating to stock height was split into three discrete sections: houses/bungalows, homes in a block less than six storeys in height and homes in blocks of six or more storeys in heightfootnote 12 (#ftn12) {:#ft12}.

To avoid multicollinearity, the regression analysis includes % houses/bungalows and % homes in blocks six or more storeys only as separate variables, with % of homes in blocks less than six storeys excluded. Further information on selection of these variables and on additional functional forms tested in wider models can be found in the ‘explanatory variables excluded from the final model’ section.

The measurement relating to the variable houses/bungalows is calculated as the total number of low-cost rental and affordable rent houses or bungalows owned by each provider as reported in the SDR, divided by the provider’s total owned low- cost rental and affordable rent homes. Stock height data is not collected for Low-Cost Home Ownership (LCHO) and social leased homes.

The measurement relating to the variable homes in blocks of six or more storeys in height is calculated by number of low-cost rental and affordable rent homes owned in blocks six or more storeys in height divided by the total owned number of low-cost rental and affordable rent homes.

The median proportion of houses or bungalows in the dataset is 58%. For the majority of providers (72%), more than half of their total homes owned are classed as houses or bungalows. The median value for homes in blocks of six or more than six storeys is only 1%. There are seven providers who own more than 20% of their homes in blocks of six or more storeys as of 2022.

Figure 6: Number of PRPs by % houses/ bungalows

Figure 7: Number of PRPs by % homes in blocks less than six or more storeys

Average stock age

Average stock age is a new variable tested. Data related to this variable has been collected in the SDR since 2020. The measurement selects the midpoint of each age category set out in the SDR, subtracted from year 2022 and multiplied by the proportion of stock in each category. Stock age is limited to the age categories set out in the SDR, meaning stock constructed prior to the year 1919 have been set equal to 1918 and those constructed after the year 2020 has been set to 2021, affecting around 6% of total homes. The data is collected across a range of age categories as set out table 2 below. An example of the calculation is exhibited in table 3 below. The median average stock age in the dataset is 48 years and the mean age is 46 years as of 2022.

The date of construction refers to the date the building was originally constructed, and not the date of any major renovations that have subsequently taken place. Data relating to stock age includes owned low-cost rental and affordable rent homes, with homes classified as care homes, LCHO and social leased homes excluded.

Figure 8: Number of PRPs by average stock age

Table 2: Proportion of stock built in each period (SDR)

Table 3: Example of average age calculation:

Average property size

The average property size variable is based on the average number of bedrooms in a home and is categorised for, GN, SH and HOP units (LCHO units are excluded). The data is derived from the SDR and categorises data between 1-4 bedroomsfootnote 13 (#ftn13)

The average property size is calculated by multiplying the proportion of stock in each bedroom category by the number of bedrooms. An example is available in Table 4.

Table 4: Example of average property size calculation:

The median value for average property size is 2.043 bedrooms, with a mean of 2.01 bedrooms.

Figure 9: Number of PRPs by average property size

Non-Social Housing Lettings (SHL) income %

The percentage of income from non-SHL activity is a new variable tested. While the vast majority of providers are designated as not-for-profit, most large providers will generate income from non-social housing activities which allows them to support core social housing activities. These activities can include for example, student accommodation and outright market sales of homes. It can also include the provision of nursing homes and non-social support services.

This data is collected as part of the regulator’s global accounts dataset. The variable is based on the proportion of income generated by non-SHL income as a percentage of total turnover. The calculation of the variable and examples of non-SHL activities can be found in Annex B.

The median value of non-SHL income as a proportion of total turnover (net of grant) is 16%. A small minority of providers (13%), generate less than 5% of income from non-SHL activities.

Figure 10: Number of PRPs by % non-SHL income

return to top (#top)

Regression analysis - Headline results

This section summarises the results of the regression analysis based on the explanatory variables (11 variables)footnote 14 (#ftn14)tested in the final model. Commentary and inferences relate to statistically significant relationships found between the independent and dependent variables.

The baseline results referenced in this section are based on median values of all explanatory variables and 100% GN units. These outputs should be interpreted with care and must not be used as a precise calculation for business planning purposes. Outputs with references to the baseline are intended as a demonstration to show the impact of a variable’s relationship with the VFM metric only. The analysis should help to provide organisations with context on the factors impacting performance. The outputs reflect the relationships found on average across the sector, and individual comparisons with the outputs may be unsuitable due to factors uncaptured in the analysis and a provider’s unique circumstances. Although the explanatory power for five of the six metrics has increased, there remains a residual unexplained for each metric.

The results are drawn from standard Ordinary Least Square (OLS) regression analysis. The results of the OLS model were run on 1,135 observations – on average 189 providers per VFM metric. Outliers and overly influential observations that risk skewing the analysis were removed. These outliers were identified by using standard thresholds for studentised and standardised residuals and Cook’s distances. The wider context of a provider was also considered in determining whether a value was deemed to be an outlier. This resulted in the removal of 53 observations across the six metrics. The analysis shows statistically significantfootnote 15 (#ftn15) relationships at 80%, 90% and 95%, with enhanced commentary on relationships with at least 90% significance.

Default and final regression outputs were also run for models inclusive of outliers and influential variables to provide assurance on the robustness of results.

Baseline definitions

The baseline definition for results are as follows:

Traditional provider (non-LSVT)

Median total social stock units (6,830 homes)

100% General Needs stock

Operating in a median deprivation level based on the dataset of providers (median percentile: 0.63)

Operating in a region of the England average wage (index: 1.00)

Holding a median proportion of house/bungalows (58%)

Holding a median proportion of homes in blocks six or more storeys (1%)

Holding a median proportion of homes in blocks less than six storeys (41%)

Holding a median average property age (48 years)

Holding a median average property size (2.04 bedrooms)

Holding a median proportion of non-SHL income (16%)

Evaluating the performance of the VFM metrics when measured against a range of explanatory factors is complicated and likely to be affected by both the denominator and numerator of each measure – where this arises, we have further explained the effect.

Table 5: Final model outputs

Unless indicated otherwise, figures presented in the main body of the table are the regression coefficients. DV indicates dummy variable.

***Significant at 95% confidence level

**Significant at 90%

*Significant at 80% (standard t-tests).

Reinvestment

The baseline level of reinvestment is 6.1%. Reinvestment is measured as the total investment in social housing properties as a proportion of the net book value of total housing properties. Approximately 36% of variation in reinvestment can be explained by the explanatory variables.

LSVTs

LSVT providers less than 12 years old would expect to have a reinvestment level of 10.6% - 74% higher when compared to an equivalent traditional provider. This relationship is primarily being driven by LSVT providers averaging a lower net book value per unit - the denominator of the reinvestment measure.

The higher levels of reinvestment by LSVT organisations are also partly driven by an increased spend on maintenance and major repairs on existing stock due to post-transfer agreements. These providers were found to spend less per unit on development of new homes, as they focus on improving existing homes and on the delivery of services. Reinvestment for LSVT providers more than 12 years old typically reflect traditional providers following the completion of all committed major repair works in the early years post transfer.

Average stock age

The model shows that a provider with an average stock age of 62 yearsfootnote 16 (#ftn16) will expect to have a reinvestment level of 7.5%. This compares to a provider with an average stock age of 33 yearsfootnote 17 (#ftn17) who will expect to have a reinvestment level of 4.6%. A one-year increase in the average stock age is associated with 0.1 percentage point increase in the level of reinvestment.

Providers with an older average stock age average a lower net book value per unit, resulting in a lower denominator. There is no noticeable difference in actual reinvestment spend per unit between providers with different average stock ages.

Wage index

The model shows that the wage index has a statistically significant relationship with reinvestment. Providers operating with 100% of their stock in London will expect to have a reinvestment level of 3.4%, compared to the England average of 6.1% according to the model. Meanwhile, a provider operating with 100% of its homes in the North East of the country will expect to have a higher reinvestment level of 7.1%.

The difference in reinvestment at a regional level is driven by the difference in average property values between regions, the denominator of reinvestment. Providers in London average a significantly higher net book value per property than every other region whereas providers in the North East average the lowest net book value per property out of all regions. Providers operating solely in London, the region with the highest average wage, average a higher reinvestment spend per unit but the difference in net book value between regions is greater, which means London based providers are associated with lower levels of reinvestment.

% Supported Housing and Housing for Older People

A supported housing provider with 30% SH homes and 70% general needs homes, and a housing for older people provider with 30% HOP homes and 70% general needs homes, would both be expected to have a reinvestment level of 4.7% - 20% below a baseline provider. The model suggests a 10-percentage point increase in the proportion of supported housing or housing for older people are each associated with a 0.48 percentage point decrease in the level of reinvestment.

Providers with higher proportions of SH and HOP homes were found to spend less on new development per unit, leading to a negative relationship. Providers with higher proportions of SH and HOP homes provide additional services to tenants and as a result tend to have lower surplus funds to reinvest and limited capacity to borrow against.

Local Authority Deprivation

Providers that own a higher proportion of homes in more deprived local authorities are associated with lower levels of reinvestment. This result has changed compared to 2018 which found no significant relationship between these two variables. A provider operating solely in the 1% most deprived local authority of the country, assuming all other baseline assumptions, will expect to have a reinvestment level of 5.4%. In comparison, an equivalent provider operating solely in the 1% least deprived local authority of the country will expect to have a reinvestment level of 7.3%.

Providers with higher proportions of stock owned in more deprived local authorities were found to spend less on new development activity per unit, leading to a lower numerator. This was also despite having an average lower net book value per unit – the denominator of the measure.

New Supply

New supply is measured as new supply delivered as a proportion of total social stock, with the baseline level 1.5%, assuming the baseline definitions. Around a third (32%) of the variation in new supply is explained by the explanatory variables.

% Housing for Older People

Providers with higher proportions of HOP homes are associated with lower levels of new supply. The relationship is mainly driven by a group of providers with 0% HOP averaging a higher level of new supply compared to the sector average. A HOP provider with 30% HOP homes and 70% GN homes will expect to deliver 1% new supply compared to an equivalent baseline provider according to the model. A ten-percentage point increase in the proportion of housing for older people is associated with a 0.16-percentage point decrease in the proportion of new supply delivered.

Wage Index

There is a significant relationship between regional wages and new supply – this is also a change to the published analysis in 2018 which found no significant relationship between these two variables. A provider operating with 100% of their stock in the North East of the country will expect to deliver 1.2 percentage points more new supply than a provider with 100% of their stock in London according to the model. The relationship is driven by a group of providers based in London who have substantially lower new supply compared to the sector average.

Non-SHL income

Providers with higher proportions of non-SHL income on average deliver higher proportions of New Supply as a percentage of total social housing units. A 10-percentage point increase in the proportion of non-SHL income is associated with a 0.3 percentage point increase in the level of new supply delivered. A baseline provider who generates 36%footnote 18 (#ftn18) of their income from non-SHL income as a proportion of total turnover (net of grant) will expect to have a new supply delivery of 2.1% as a proportion of total social housing units. This compares to an equivalent provider who generates 3.5%footnote 19 (#ftn19) of their income from non-SHL income and would expect to deliver 1.1% of new supply as a proportion of total social housing units according to the model. This suggests that there is evidence of cross-subsidisation on a proportionate basis into new homes. However, there is insignificant evidence that income from non-social housing activity is invested into existing homes with no statistical significance found with the reinvestment metric.

Local Authority deprivation

Providers who operate in the most deprived local authorities in England are associated with delivering lower levels of new supply. The model shows that a provider only operating in the 1% most deprived local authority will on average, deliver approximately 1.3 percentage points fewer new homes compared to providers only operating in the 1% least deprived local authority.

Providers with higher proportions of stock owned in more deprived areas also average less turnover per unit due to lower levels of rental income, which restricts their capacity to invest into new homes. Equally, demand for housing in the most deprived areas outside of London tends to be lower due to social rent being closer to market rent and is likely to reduce a provider’s risk appetite to develop in these areas.

Gearing

The baseline level of gearing is 48% when measured as net debt as a proportion of the net book value of social homes, based on the characteristics outlined in the baseline definition. The model can explain around 30% of variation in net debt across the sector which represents a small increase compared to the analysis published in 2018.

% Supported Housing

Providers with higher proportions of supported housing homes are associated with lower levels of gearing - a similar relationship was found in the analysis published in 2018. A 10-percentage point increase in the proportion of supported housing is associated with a 5.7-percentage point decrease in gearing according to the model.

A SH provider, assuming 30% SH and 70% GN stock, will expect a gearing level of 31% - 35% lower compared to an equivalent baseline provider with 100% GN units. The relationship is driven by SH providers holding very low levels of debt (the numerator) and providers with higher proportions of SH averaging higher net book values per unit (the denominator of the measure). However, the disparity in gearing levels is less pronounced for providers who are not categorised as SH providers but who have between 5%-30% of SH homes.

Average stock age

The average age of stock is associated with marginally higher levels of gearing. A one-year increase in a provider’s average stock age is associated with a 0.2-percentage point decrease in a provider’s expected gearing level. A provider with an average stock age of 62 yearsfootnote 20 (#ftn20) will expect to have a gearing level of 50% - just two percentage points higher compared to a baseline provider. On the other hand, a provider with an average stock age of 33 yearsfootnote 21 (#ftn21) would expect to have a lower gearing level of 45%. Providers with a higher average stock age also have lower net debt per unit – impacting the numerator of the measure.

Non-SHL income

Providers who generate larger proportions of income from non-social housing activities as a percentage of turnover (net of grant) are associated with higher levels of gearing, driven by higher levels of debt per unit acquired to support these activities.

A 10-percentage point increase in the proportion of non-SHL income is associated with a 3.6-percentage point increase in the level of gearing according to the model. A baseline provider who generates 36%footnote 22 (#ftn22) of their income from non-social housing activities will expect to have a gearing level of 55%. This compares to a baseline provider who generates 3.5%footnote 23 (#ftn23) of their income from non-social housing activities who will expect to have a gearing level of 43%.

LSVT

An LSVT baseline provider would expect to have a gearing level of 32% which is a third lower compared to an equivalent traditional baseline provider according to the model. Most of these providers have low initial property valuations and tend to borrow more steadily against these lower valuations, resulting in a lower net debt per unit on average.

Local Authority Deprivation

Providers with higher proportions of stock owned in the most deprived local authorities are associated with lower gearing levels. Deprivation has a cancelling effect on the gearing metric, with providers owning higher proportions of homes in more deprived local authorities averaging lower property values but also averaging less net debt per home. However, the impact on debt is more substantial leading to a negative relationship. A provider only operating in the 1% most deprived local authority in England will expect a gearing level of 40% compared to a provider only operating in the 1% least deprived local authority which would expect a gearing level of 57%.

Headline Social Housing Cost

The additional explanatory variables have improved the explanatory power of the model for headline social housing costs by 15 percentage points compared to the regression output results published in 2018, with an r-squared value of 0.70. This means approximately 70% of the variation in cost performance is explained by the explanatory variables. However, there remains a considerable amount of unexplained variation due to uncaptured factors in the analysis.

The baseline level of HSHC is £3,850 assuming the characteristics outlined in the baseline definition, which represents an increase of 16% since 2018. The composition of the change in expenditure over this period is mainly attributed to increased levels of expenditure on major repairs and maintenance as set out in VFM metrics and reporting publications over that timeframe. However, we do not hold data on certain individual practices by providers for example on component lifecycles which could further explain differences in costs between similar types of providers.

Providers will take different approaches to the categorisation of costs in their accounts, meaning differences in cost apportionment and accounting policies in relation to capital expenditure are likely to have some impact on sub-components of the headline cost calculation and surrounding narrative as described in this section.

Total social stock

There is a statistically significant but small negative linear relationship between headline costs and the size of a provider. While the total social stock (000s) variable is significant, the response in change between the two variables is relatively minimal. The model shows that the headline cost is expected to fall by £13 per every 1,000-unit increase. A baseline provider with a total social housing stock of 35k homes would expect to have a headline cost of £3,480 per unit, compared to a smaller baseline provider with a total social housing stock of 5,000 homes who would expect to have a higher headline cost of £3,870 per unit according to the model. Additional analysis on economies of scale has been undertaken including testing other functional forms of total social stock. Findings on this can be found in the ‘Regression analysis – further testing’ section.

Smaller providers with less than 5,000 homes average higher management and service charge costs, whereas the largest providers with more than 35,000 homes average higher maintenance and major repair costs.

% Supported Housing

The provision of supported housing is a business activity undertaken by most providers. However, the precise cost for the provision of a supported housing unit is dependent on the level of service and types of support provided by different organisations and can vary widely depending on the support provided. Providers with higher proportions of supported housing average higher costs relating to charges for support services, management costs and service charge costs.

Each unit of supported housing is associated with an additional cost of £8,340 above a general needs unit - 217% higher than the baseline estimate. However, the estimated cost is sensitive to the removal of outliers. With the inclusion of outliers, the model estimates the additional cost to increase to £9,840 above a general needs home.

Wage index

Regional wage difference continues to have a significant impact on provider’s headline costs, albeit it is less pronounced when compared to the 2018 analysis, even when accounting for the addition of new explanatory variables that have a positive relationship with cost.

A baseline provider only operating in London would expect to have a headline cost of £4,660, which is 20% higher compared to the England average. A baseline provider with 100% of their stock in the North East will expect to have a headline cost of £3,550 which is £1,110 less than an equivalent provider with 100% of their stock based in London. All components relating to the HSHC measure in London were higher compared to elsewhere in the country.

% Houses or bungalows

Providers with higher proportions of houses or bungalows will expect to have a lower headline social housing cost. These homes are associated with lower service charge costs compared to homes in blocks less than six stories and homes in blocks of six stories or more. The headline cost of a house or bungalow is expected to be £2,850 per unit, assuming all other baseline assumptions.

Providers with more than 75% stock classed as houses or bungalows average lower service charge costs of £230 per unit, in comparison, providers with less than 25% houses or bungalows average service charges of £1,600 per unit. Providers with less than 25% houses or bungalows also average higher management and maintenance and major repair costs, but these cost differences are less pronounced compared to the differences in service charge costs per unit.

% Homes in a block more than six storeys

Providers with higher proportions of homes in a block more than six storeys are associated with significantly higher headline costs per unit. A home in a block of six or more stories is expected to have a headline cost of £10,710 – 275% higher than a house or bungalow for an equivalent provider.

Providers with over 25% of homes classed as homes in blocks more than six storeys average substantially higher maintenance and major repair costs of £4,300 per unit and service charge costs of £1,470 per unit, compared to providers with less than five percent of homes in blocks of six stories or more averaging £2,200 and £540 respectively. These higher costs are likely to reflect increased fire building and safety remediation work as well as Waking Watch costsfootnote 22 (#ftn24).

This expected headline cost of an additional home in a block of less than six stories is expected to be £5,130 according to the model, 80% higher than a house or bungalow and 52% lower than a home in a block of six or more storiesfootnote 25 (#ftn25) .

% Housing for Older People

Providers with higher proportions of HOP continue to be associated with higher headline costs per unit due to providing a mixed range of services such as social activities, provision of meals and 24-hour emergency help. The additional cost of a HOP unit is expected to be £3,420 above a general needs unit, 89% higher, assuming all other baseline assumptions. Similar to supported housing providers, there is considerable diversity with the cost associated with these types of homes. The estimation of these costs is influenced by the inclusion of outliers. Once all outlier providers are included in the model, the additional cost is expected to be £6,620 above the expected headline cost of a general needs home.

Providers who have higher proportions of HOP homes are associated with higher costs relating to charges for support services, management costs and service charges. However, there are no differences in maintenance and major repair costs across providers with various proportions of HOP homes.

Non-SHL income

The model shows that providers with higher proportions of non-SHL incomes are associated with higher headline costs, being driven by providers with higher development costs, service costs, charges for support services and ‘other social housing activities’ costs. A 10-percentage point increase in the proportion of non-SHL income is associated with a £240 increase in a provider’s headline social housing cost. A provider who generates 36%footnote 26 (#ftn26) of their income from non-SHL activities as a proportion of total turnover, will expect to have a headline cost of £4,320. This compares to a provider who generates only 3.5%footnote 27 (#ftn27) of their income from non-SHL activities as a proportion of total turnover (net of grant), who will expect to have a headline cost of £3,560 according to the model.

Operating Margin (Overall)

The baseline estimate for operating margin (overall) is 23%. Approximately 48% of the variation in overall operating margin can be explained by the explanatory variables, an increase compared to the analysis published in 2018.

Average stock age

Providers who have higher proportions of older stock are associated with lower operating margins. A one-year increase in a provider’s average age is associated with a 0.1-pecentage point increase in the overall operating margin. A provider with an average stock age of 62 yearsfootnote 28 (#ftn28) will expect to have an overall operating margin of 21.5% compared to a provider with an average stock age of 33 yearsfootnote 29 (#ftn29), who will expect to have an operating margin of 24.4%.

Providers with older stock are associated with a lower net operating surplus per unit which deflates the numerator. Providers with older stock also average a lower turnover per unit, with their Social Rent Ratefootnote 30 (#ftn30) impacted by these properties having lower valuations compared to newer properties. Providers with increased proportions of stock constructed post-2011 are also likely to have higher rental income generated from Affordable Rentsfootnote 31 (#ftn31) introduced in 2011.

Average property size

Providers who have larger properties are associated with a higher overall operating margin due to higher levels of rental income. A provider with an average property size of three-bedrooms will expect to have an overall operating margin of 30.1%. This compares to a provider with an average property size of one-bedroom homes who will expect to have an overall operating margin of 15% This relationship is also explicitly linked to the Social Rent Rate.

Non-SHL income

Providers with larger proportions of non-SHL incomes are associated with lower operating margins (overall). A ten-percentage point increase in the proportion of non-SHL income is associated with a 1.06-percentage point decrease in the overall operating margin. A baseline provider who generates 36%footnote 32 (#ftn32) of their income from non-SHL activities as a proportion of total turnover (net of grant), will expect to have an operating margin of 20.7%. However, a baseline provider who generates 3.5%footnote 33 (#ftn33) of their income from non-social housing activities as a proportion of total turnover, will expect to have an operating margin of 24.2%.

The relationship is affected by the additional costs of non-social housing activitiesfootnote 34 (#ftn34). The relationship is also being driven by higher levels of turnover being generated by providers with larger proportions of non-social housing income which is a fair reflection of income generated from these types of activities.

LSVT

LSVT organisations will expect to have an operating margin which is 24% lower compared to a baseline provider. This is linked to this group of providers having typically lower than average rental income, resulting in a lower net operating surplus per unit. These providers also spend slightly more on investment into existing homes, resulting in lower margins.

Local Authority deprivation

Providers that own higher proportions of homes in more deprived local authorities of the country are associated with lower rental incomes and lower overall operating margins. A baseline provider only operating in the 1% most deprived local authority of England, where demand for homes is typically low, will expect to have an overall operating margin of 18.8%, compared to an equivalent provider in the least deprived local authority who would expect to have an operating margin overall of 30.2%, a 61% increase in operating margin.

% Supported Housing and % Housing for Older People

Both the proportion of supported housing and housing for older people have insignificant relationships with operating margin, despite showing statistically significant negative relationships in the analysis published in 2018. Providers with increased proportions of HOP homes have both lower net surpluses per unit but also a lower turnover per unit, leading to a cancelling affect to the numerator and denominator, meaning no relationship could be determined. Although providers with increased proportions of SH homes have both a lower net surplus per unit and a higher turnover per unit, controlling for more variables such as property size, average stock age and the proportion of non-SHL income has led to the variable becoming statistically insignificant.

Return on Capital Employed (ROCE)

The baseline ROCE is 2.5% based on the baseline definition. The amount of variation that can be explained by the explanatory variables is 26% - a similar result published in 2018.

All statistically significant relationships with ROCE are impacted by the variable’s relationship with rental income which affects both net operating surplus (the numerator) and the location of the provider relating to the value of total assets (the denominator) of the measure.

Providers who have higher proportions of their stock in more deprived local authorities tend to have lower rental incomes and are associated with a lower ROCE. A provider only operating in the most deprived local authority of the country will expect to achieve a ROCE which is 26% lower than a provider only operating in the least deprived local authority due to these providers having a lower operating surplus. London based providers will expect a ROCE of 1.7% due to a higher value in total assets per unit - the denominator of the measure. Providers with larger average property sizes are associated with a larger ROCE, which is driven by higher net operating surpluses compared to providers with smaller average property sizes.

return to top (#top)

Regression analysis - additional testing

This section sets out the results of wider testing performed which extended to test the impact of other variables included in the default model which includes all 46 explanatory variables. The results of this testing are explored in detail below.

Economies of scale

Similar to previous regression analysis undertaken, significant testing was undertaken to assess whether there was any evidence of a relationship between the size of a provider and the VFM measures.

The total social stock (000s) variable has a statistically significant relationship but reports a small negative coefficient with headline cost, indicating a slightly negative relationship between the size of provider and headline costs. There were no other relationships between the size of provider and any other VFM metrics. Two dummy variables were also tested for mergers as part of the default model however neither of these variables were found to be statistically significant with any primary VFM metric.

Linear and quadratic variables (for total social stock (000s) plus total social stock (000s) squared) were also tested as part of the final model to test for non-linear relationships; again, no statistically significant relationship was found. Total social stock (000s) logged was also tested alongside the final model variables to test for any proportional relationships. This also found a statistically significant but weak relationship with HSHC. There was no other statistically significant relationship found with any other primary VFM metric.

The regression model suggests that headline costs fall by £13 per every 1,000 increase in homes, with smaller providers having higher management and service charge costs.

However, further inspection of the relationship between total social stock and headline costs shows that the largest providers, with more than 40,000 homes, exhibit higher costs, driven by higher maintenance and major repairs and ‘other’ costs. The negative relationship found in the regression analysis is driven mostly by providers with fewer than 40,000 homes. This is shown in figure 10, with just two out of the ten largest providers having a HSHC below the median for 2022.

Figure 10: Economies of scale breakdown between Total social size (000s) and HSHC 2022

Highlights from the default model and testing

The default model tested a significant number of explanatory variables, including several functional forms of the same variable which provided a basis to assess which functional forms were statistically significant and to help inform the selection of a final model. Additional functional forms can help to identify non-linear relationships which may exist.

Explanatory variables with multiple functional forms increase the risk of multicollinearity, which makes it difficult to determine the true impact of the explanatory variable on the dependent variable and violates an assumption of linear regression. Outputs of the primary regression should therefore not be used to make any inferences. Commentary explaining why variables were excluded from a final model and the inferences found from additional analysis that has been undertaken is outlined in the ‘Explanatory variables excluded from the final model’ section.

Table 6: Default Model Outputs

Unless indicated otherwise, figures presented in the main body of the table are the regression coefficients. DV indicates dummy variable.

***Significant at 95% confidence level

**Significant at 90%

*Significant at 80% (standard t-tests).

Explanatory variables excluded from the final model

Total General Needs (GN) Stock variables

Total GN Stock (000s) and its functional forms showed limited evidence of being statistically significant with the VFM metrics, whilst showing collinearity with Total Social Stock (000s). Total Social Stock (000s) was included in the final model to allow for consistency and longitudinal analysis with the 2018 publication.

Housing for Older People variables

The Total HOP stock (Logged) variable was found to be the most statistically significant functional form of the HOP variables in the default model. When tested as part of a reduced model, with one functional form for each stock type included, this was found to have the same statistical significance as the % HOP variable. % HOP was preferred in a final model due to being included in the 2018 final model to allow for longitudinal analysis and due to a preference of proportional variables, with inferences being simpler to communicate.

Supported Housing variables

The percentage of supported housing variable is the most statistically significant SH variable in the default model. This was also included in the 2018 final model and is preferred to allow for longitudinal analysis between the two pieces of analysis. There is increased interest in how proportions of SH impact cost as opposed to total SH numbers.

% Affordable Rent

The percentage of Affordable Rent is positively associated with new supply, which was also found in the 2018 analysis. Affordable Rent has been the predominant form of grant-funded development in the Affordable Homes Programme since 2011.This supports the delivery of new supply developed, which has led to providers with increased proportions being associated with higher levels of new supply. This means direct causation is complex when understanding this relationship. The percentage of Affordable Rent also had a positive statistically significant relationship with gearing.

Low-Cost Home Ownership (LCHO) variables

Total LCHO Stock (000s) Logged has a statistically significant positive relationship with reinvestment and new supply, whereas all other functional forms of LCHO were insignificant. The logged variable was included in further rounds of testing as the only LCHO variable as part of a reduced model which continued to have a statistically significant relationship with reinvestment and new supply.

Stock Height Variables

Data relating to stock height is now collected as part of the SDR and has been included in the regression analysis for the first time. Each of the three variables have been calculated as a percentage of total stock owned with different functional forms to test for potentially non-linear relationships that may exist. Variables were required to be transformed due to multicollinearity, as all three variables add up to 100%, meaning one variable would be automatically excluded from the analysis.

Functional forms included % squared, % cubed and % logged variables in the default model. Homes in a block six or more storeys was not transformed into a logged variable due to some providers having 0% homes in blocks six or more storeys, meaning the order of values would not be preserved.

Several stock height variables showed evidence of being statistically significant in the default model. The homes in blocks less than six storeys variable was found to be less significant as part of a reduced model compared to the other stock height variables. Preference was also given to proportional variables, for the purposes of simplifying the process of communicating outputs and inferences to an external audience. Inferences relating to the cost of homes in blocks less than six storeys can still be deduced when setting houses or bungalows and homes in blocks six or more storeys equal to zero where these variables are statistically significant.

% Reduction in DHS fails

Percent (%) reduction in DHS fails, which previously acted as a proxy for major repairs, was mostly insignificant across the VFM metrics. The level of non-decent stock failures reported in 2022 reduced significantly. The percentage reduction in homes made decent between 2021 and 2022 was only 0.3% (https://assets.publishing.service.gov.uk/media/634fb48fe90e0731ac4a565d/2022_PRP_STOCK_-briefing-notes_FINAL_V1.0_.pdf) meaning the variable was too low and not significant across any of the metrics.

Proportion of stock in pockets <50, 100, 250, 500 per Local Authority

The proportion of stock in pockets per Local Authority area measures the proportion of stock in units of, 50, 100, 250 and 500 or less, held in each local authority. This acts as a proxy to measure the dispersal of stock per local authority.

This variable calculates the proportion of GN, SH, HOP and LCHO units owned per local authority area. Units managed by providers are excluded from this calculation. The sum of total units held in pockets of less than 50, 100, 250 and 500 per local authority is divided by the total sum of GN, SH, HOP and LCHO units owned by a provider.

Additional testing on the default model found proportion of stock in pockets <100 per Local Authority to be the most statistically significant functional form. This found some significant relationships as part of a reduced model, showing evidence providers which hold increased proportions of stock in pockets of less than 100 per local authority are associated with lower reinvestment and lower levels of new supply. This was excluded from the final streamlined model, given other variables were found to be more statistically significant. Controlling for more factors, there was also a preference for a smaller model to avoid risk of overfitting. Further analysis also considered this to be an imperfect measure of geographical dispersal, with some local authorities in London being significantly closer together compared to elsewhere in the country.

Treatment of mergers

Two merger dummy variables were included due to an increased number of mergers that have taken place across the sector over the past ten years. The dummy variables included recently merged entities defined as mergers that took place in the five years up to 2022 and mergers which took place between five and ten years up to 2022. Where a provider merged more than once during these periods, the merger that resulted in the largest percentage increase in total units for the larger provider was included in the model.

Both merger variables were mostly statistically insignificant across most VFM metrics in both the default model and when included as part of a reduced model, demonstrating limited evidence they are influencing VFM metric performance. Mergers undertaken in the five years up to 2022, were found to have a positive relationship with overall operating margin only. Providers who merged five to ten years prior to 2022 were only statistically significantly associated with a higher ROCE. This lack of statistical significance meant merger variables were excluded from a final model.

Population density

The population density variable was compiled by mapping local authority district summaries data from the 2019 IMD against ONS population density data. This was found to be insignificant across the VFM metrics in the default model and when tested in a reduced model.

return to top (#top)

Annex A: VFM Metrics definitions

Metric 1 – Reinvestment %

The Reinvestment metric looks at the investment in properties (Existing stock and New Supply) as a percentage of the value of total properties heldfootnote 35 (#ftn35).

Measurement of VFM cost chain – efficiency

[Development of new properties (Total housing properties)

(+) Newly built properties acquired (Total housing properties)

(+) Works to existing (Total housing properties)

(+) Capitalised interest (Total housing properties)

(+) Schemes completed (Total housing properties)]

Divided byfootnote 36 (#ftn36).

[Tangible fixed assets: Housing properties at cost (Current period)

OR Tangible fixed assets: Housing properties at valuation (Current period)].

Metric 2 – New supply delivered %

The New supply metric sets out the number of new social housing units that have been newly constructed (acquired or developed) in the year as a proportion of total social housing units owned at period end.

Measurement of VFM cost chain – effectiveness
- New supply delivered (Social housing units) %
[Total social housing units developed, or newly built units acquired in-year (owned)

(Social rent general needs housing (excluding Affordable Rent), Affordable Rent general needs housing, social rent supported housing and housing for older people (excluding Affordable Rent), Affordable Rent supported housing and housing for older people, Low-Cost Home Ownership, care homes, other social housing units, Social leasehold)] Divided by

Total social housing units owned at period end (‘social units’ as defined in numerator).

Metric 3 – Gearing %

The gearing metric assesses how much of the adjusted assets are made up of debt and the degree of dependence on debt finance. It is often a key indicator of a registered provider’s appetite for growth.

Note: Registered providers can be restricted by lenders’ covenants and therefore may not have the ability in which to increase the loan portfolio despite showing a relatively average gearing result.

Measurement of VFM cost chain – efficiency

[Short-term loans

(+) Long-term loans

(-) Cash and cash equivalents

(+) Amounts owed to group undertakings

(+) Finance lease obligations]

Divided byfootnote 36 (#ftn37).

[Tangible fixed assets: Housing properties at cost (Current period)

OR Tangible fixed assets: Housing properties at valuation (Current period)].

Metric 4: Earnings Before Interest, Tax, Depreciation and Amortisation, Major Repairs Included (EBITDA MRI) Interest Cover %

The EBITDA MRI Interest Cover measure is a key indicator for liquidity and investment capacity. It seeks to measure the level of surplus that a registered provider generates compared to interest payable; the measure avoids any distortions stemming from the depreciation charge.

Note: Grants related to capitalised major repairs expenditure must be excluded.

Measurement of VFM cost chain – efficiency

[Operating surplus / (deficit) (overall)

(-) Gain/(loss) on disposal of fixed assets (housing properties)

(-) Gain/(loss) on disposal of other fixed assets

(-) Amortised government grant

(-) Government grants taken to income

(+) Interest receivable

(-( Capitalised major repairs expenditure for period

(+) Total depreciation charge for period]

Divided by

[Interest capitalised

(+) Interest payable and financing costs].

Metric 5 – Headline social housing cost per unit

The Headline social housing cost per unit metric assesses the headline social housing cost per unit as defined by the regulator. It is a proxy cash measure of a social housing cost per unit. This means it excludes non-cash items such as depreciation, amortisation and write downs.

Note: Grants related to capitalised major repairs expenditure must be excluded.

Measurement of VFM cost chain – economy

[Management costs

(+) Service charge costs

(+) Routine maintenance costs

(+) Planned maintenance costs

(+) Major repairs expenditure

(+) Lease costs

(+) Capitalised major repairs expenditure for period

(+) Other (social housing letting) costs

(+) Charges for support services (Operating expenditure)

(+) Development services (Operating expenditure)

(+) Community / neighbourhood services (Operating expenditure)

(+) Other social housing activities: Other (Operating expenditure)

Divided by

Total social housing units owned and/ or managed at period endfootnote 36 (#ftn37)

(Social rent general needs housing (excluding Affordable Rent), Affordable Rent general needs housing, social rent supported housing and housing for older people (excluding Affordable Rent), Affordable Rent supported housing and housing for older people, Low-Cost Home Ownership, care homes, other social housing units).

Metric 6 – Operating margin %

The Operating margin demonstrates the profitability of operating assets before exceptional expenses are considered. Increasing margins are one way to improve the financial efficiency of a business. In assessing this ratio, it is important that consideration is given to registered providers’ purpose and objectives (including their social objectives). Further consideration should also be given to SH/HOP providers who tend to have lower margins than average.

Measurement of VFM cost chain – efficiency

Operating margin (overall) %

[Operating surplus / (deficit) (overall)

(-) Gain/(loss) on disposal of fixed assets (housing properties)]

(-) Gain/(loss) on disposal of other fixed assets

Divided by

Turnover (overall).

Metric 7 – Return on capital employed %

The Return on capital employed (ROCE) compares the operating surplus to total assets less current liabilities and is a common measure in the commercial sector to assess the efficient investment of capital resources. The ROCE metric supports registered providers with a wide range of capital investment programmes.

Measurement of VFM cost chain – efficiency

[Operating surplus / (deficit) (overall) (including gain / (loss) on disposal of fixed assets)

(+) Share of operating surplus/(deficit) in joint ventures or associates] Divided by Total assets less current liabilities.

return to top (#top)

Annex B: Explanatory variables definitions

Table 7: Explanatory variable list and definitions

return to top (#top)

Annex C: Sample sizes in the final streamlined model

Table 8: Sample size for each metric in the final streamlined model

return to top (#top)

Annex D: Methodology

Outlier detection process

An outlier detection process was undertaken prior to the default model, which provided the dataset for further testing and was re-run ahead of producing the outputs from the final model. The outlier process was re-run due to data points having different influence in the regression analysis with fewer explanatory variables included in the final model. For example, the HOP variable had five related variables in the default model compared to one in the final model. Outliers and overly influential observations can disproportionately skew regression analysis outputs and can lead to inaccurate outputs and inferences.

Outliers were identified using standard thresholds for studentised and standardised residuals and Cook’s distances, which measures a data point’s influence in the regression. The data point’s distance to the interquartile range was also reviewed to assess whether the value was considered extreme, and the providers wider context was considered to assess whether the value could be explained and is in line to what would be expected. Large entities who merged in 2022 are excluded from the analysis due to one-off merger accounting treatment. In total, 60 observations were removed through the outlier detection process - on average ten removals (5% per metric). This compares to 13.3% removed in 2018.

The impact of removing outliers has been assessed for each primary VFM metric and commentary has been made on the impact of outlier removal where relevant. Results of the final model are robust to these alternative forms.

return to top (#top)

Annex E: Details of diagnostic testing

This section summarises the diagnostic testing on the final model and the outputs of testing. This is required to ensure the assumptions of linear regression have been followed and that inferences made from outputs are robust.

Heteroskedasticity and normality of residuals

Heteroskedasticity is where residuals are non-constant which violates an assumption of linear regression. The Breusch-Pagan test has been run for all primary VFM metrics which failed for five out of six metrics, with only operating margin (overall) passing. Where this test has been failed, outputs have been collected using robust standard errors which adjusts outputs for a model containing heteroskedasticity. Outputs are considered robust given the use of robust standard errors.

The Shapiro-Wilk test has been performed to test for normality, which assumes the residuals follow a normal distribution. Three of six VFM primary metrics passed the Shapiro-Wilk test, with reinvestment, new supply and HSHC metrics failing, impacted by the zero floor and lack of an upper limit. Inferences for these metrics remain valid due to the large sample size and use of robust standard errors.

Multicollinearity

Multicollinearity occurs where two or more explanatory variables are closely linearly related. This causes the regression model estimates of the coefficients to become unstable and inflates standard errors for the coefficients, making it difficult to distinguish the impact of an individual variable. Multicollinearity was tested using the Variance Inflation Factor (VIF) values for each of the explanatory variables in the full models. VIF values greater than ten are flagged to have multicollinearity. All VIF values were less than five in the final model, suggesting no presence of multicollinearity.

Model specification testing

Model misspecification was tested using the Ramsey RESET test. This tests whether alternative functional forms would produce a better fitted model. Five out of six models passed the RESET test, indicating a correctly specified model. HSHC is the only metrics to fail this test. However, preference to produce a consistent model for each of the primary VFM metrics that is intuitive and simple to communicate meant no further changes or transformations were applied.

return to top (#top)

Annex F – Quality Assurance

A thorough quality assurance process has been conducted at each stage of the analysis to verify calculations and outputs. This was undertaken during the calculation of each variable and throughout each stage of the regression analysis and diagnostic testing to ensure each stage of the analysis is correct and robust. The analysis has been performed in line with the guidance set out in The Aqua Book (https://assets.publishing.service.gov.uk/media/5a7f3bb8e5274a2e87db49be/aqua_book_final_web.pdf) and ensures the analysis follows the RIGOUR principles outlined in the framework. Additionally, the analysis has been externally verified by government analysts who reviewed the methodology and statistical processes followed in the analysis.

return to top (#top)

Annex G: Glossary of Terms

1: The variable which is changed to measure its relationship with the dependent variable. In this analysis this refers to the provider characteristics.↩ (#ft1)

2: The variable which is impacted by changes in the explanatory variable. In this analysis this refers to the VFM metrics.↩ (#ft2)

3: The regression analysis is based on historic data derived from the RSH 2022 Global Accounts (Electronic Annual Accounts) ↩ (#ft3)

4: Population and household estimates, England and Wales: Census data residents per square km 2021 ↩ (#ft4)

5: A categorical variable which is set = 1 if present or = 0 otherwise if a certain condition is met in the analysis. ↩ (#ft5)

6: Three local authorities were excluded as part of the local authority district summaries in the 2019 IMD. The median IMD dataset value was imputed where this data was missing which impacted less than 2% of homes in total.↩ (#ft6)

7: Defined as low-cost rent support, is accommodation that has been designed or altered in order to enable residents to live independently and/or is accommodation that has been designated as being available only to individuals within an identified group with specific support needs. Supported Housing or Housing for Older People providers are defined as having at least 30% of their owned stock classified as Supported Housing or Housing for Older People.↩ (#ft7)

8: Supported Housing providers reported a median HSHC of £9,220 in 2023 and £8,440 in 2022 compared to sector medians of £4,590 and £4,150 respectively.↩ (#ft8)

9: The full definition of variables, their calculations and data sources can be found in Annex B.↩ (#ft9)

10: The region of operation is used as an indicator and does not refer directly to the ASHE Regional Wage index of providers. For example, a London based provider will only have regional wage index of 1.24 if all of their stock holding is in London. The total wage index for each region is multiplied by the proportion of stock held in each region for each provider.↩ (#ft10)

11: Deprivation is measured at a local authority level rather than a lower super output area level as done in 2018, due to CORE lettings data being unavailable at the time of the analysis. CORE lettings data also only records new lettings in a given year, meaning a significant proportion of lettings will not be recorded. 2022 SDR data relating to stock location at a local authority was used as an alternative and considered a more accurate measure of owned stock location.↩ (#ft11)

12: Since 2023, data relating to stock height collected in the SDR has subsequently changed to homes in a block of less than 18 metres or fewer than seven storeys, and homes in a block of 18 metres or more or has at least seven storeys.↩ (#ft12)

13: Bedroom data for GN units are available for bedsits and 1-6+ bedrooms. Bedsits are assumed to be 1 bedrooms and 5-6+ bedroom homes are assumed to 4-bedroom properties in the analysis, as 4+bedrooms is the largest category collected for SH/HOP homes. Affordable rent and intermediate rent units are excluded from this calculation, as this would lead to an element of double counting due to the collection methodology in the SDR.↩ (#ft13)

14: The full models include 46 variables. These are shown in Table 6.↩ (#ft14)

15: Statistical significance refers to the probability the relationship found is not due to chance. 95% statistical significance is considered the standard threshold for assuming a relationship found is true. Given this analysis is exploratory, a 90% threshold has been used to provide further commentary on a relationship.↩ (#ft15)

16: 90th percentile for average stock age ↩ (#ft16)

17: 10th percentile for average stock age ↩ (#ft17)

18: 90th percentile ↩ (#ft18)

19: 10th percentile↩ (#ft19)

20: 90th percentile for average stock age↩ (#ft20)

21: 10th percentile for average stock age↩ (#ft21)

22: 90th percentile for non-SHL income↩ (#ft22)

23: 10th percentile for non-SHL income↩ (#ft23)

24: The hourly rate per person/ individual undertaking Waking Watch duties ranges from £12.00 per hour to £30.00 per hour. The median hourly rate per person/ individual undertaking Waking Watch duties is £13.99. The median monthly Waking Watch cost per building is £11, 361. Building Safety Programme: Waking Watch costs - GOV.UK (www.gov.uk) (https://www.gov.uk/government/publications/building-safety-programme-waking-watch-costs/building-safety-programme-waking-watch-costs)↩ (#ft24)

25: The expected cost of a home in blocks of less than six stories can be deducted from the model by setting the proportion of houses or bungalows and homes in blocks of six or more stories to zero. This variable was excluded from the analysis to avoid multicollinearity, which would have led to perfect correlation between the three stock height variables.↩ (#ft25)

26: 90th percentile↩ (#ft26)

27: 10th percentile↩ (#ft27)

28: 90th percentile ↩ (#ft28)

29: 10th percentile ↩ (#ft29)

30: Accommodation let at a social rent, where rent is set no higher than formula rent equal to 70% of the national average rent multiplied by relative county earnings multiplied by the bedroom weight plus 30% of the national average rent multiplied by relative property value. ↩ (#ft30)

31: Affordable rent housing is exempt from the social rent requirements, rent for affordable rent housing must not exceed 80% of gross market rent. ↩ (#ft31)

32: 90th percentile ↩ (#ft32)

33: 10th percentile ↩ (#ft33)

34: Development services, charges for support services and ‘other social housing activities ↩ (#ft34)

35: This metric is not based on cashflow data given the limitations on data collected as part of the FVA return. ↩ (#ft35)

36: Providers should use the measure agreed in their Statement of Financial Position / Balance Sheet. The figure should be either historic cost or valuation. ↩ (#ft36)

37: Providers should use the measure agreed in their Statement of Financial Position / Balance Sheet. The figure should be either historic cost or valuation. ↩ (#ft37)

38: Leasehold units, which for example include Right to Buy and fully stair-cased shared ownership units where the provider retains the freehold, are excluded from the denominator of this metric. ↩ (#ft38)