# The galaxy luminosity function at z similar or equal to 6 and evidence for rapid evolution in the bright end from z similar or equal to 7 to 5

R. A. A. Bowler, J. S. Dunlop, R. J. Mclure, H. J. Mccracken, B.

Milvang-Jensen, H. Furusawa, Y. Taniguchi, O. Le Fevre, J. P. U. Fynbo, M.

J. Jarvis, et al.

► To cite this version:

R. A. A. Bowler, J. S. Dunlop, R. J. Mclure, H. J. Mccracken, B. Milvang-Jensen, et al. The galaxy luminosity function at z similar or equal to 6 and evidence for rapid evolution in the bright end from z similar or equal to 7 to 5. Monthly Notices of the Royal Astronomical Society, 2015, 452 (2), pp.1817-1840. 10.1093/mnras/stv1403. hal-01434369

HAL Id: hal-01434369

https://hal.science/hal-01434369

Submitted on 15 Nov 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# The galaxy luminosity function at $z \simeq 6$ and evidence for rapid evolution in the bright end from $z \simeq 7$ to 5

R. A. A. Bowler,¹* J. S. Dunlop,¹ R. J. McLure,¹ H. J. McCracken,²

B. Milvang-Jensen,³ H. Furusawa,⁴ Y. Taniguchi,⁵ O. Le Fèvre,⁶ J. P. U. Fynbo,³

M. J. Jarvis⁷,⁸ and B. Häußler⁷,⁹

¹SUPA†, Institute for Astronomy, University of Edinburgh, Royal Observatory, Edinburgh EH9 3HJ, UK

²Institut d'Astrophysique de Paris, UMR 7095 CNRS, Université Pierre et Marie Curie, 98 bis Boulevard Arago, F-75014 Paris, France

³Dark Cosmology Centre, Niels Bohr Institute, University of Copenhagen, Juliane Maries Vej 30, DK-2100 Copenhagen, Denmark

⁴Astronomical Data Center, National Astronomical Observatory of Japan, Mitaka, Tokyo 181-8588, Japan

⁵Research Institute for Space and Cosmic Evolution, Ehime University, 2-5 Bunkyo-cho, Matsuyama 790-8577, Japan

⁶Laboratoire d'Astrophysique de Marseille, CNRS and Aix-Marseille Université, 38 rue Frédéric Joliot-Curie, F-13388 Marseille Cedex 13, France

⁷Subdepartment of Astrophysics, University of Oxford, Denys Wilkinson Building, Keble Road, Oxford OX1 2DL, UK

⁸Department of Physics, University of the Western Cape, Bellville 7535, South Africa

⁹Centre for Astrophysics, Science & Technology Research Institute, University of Hertfordshire, Hatfield, Herts AL10 9AB, UK

## ABSTRACT

We present the results of a search for bright ($-22.7 \le M_{UV} \le -20.5$) Lyman-break galaxies at $z \simeq 6$ within a total of 1.65 deg² of imaging in the UltraVISTA/Cosmological Evolution Survey (COSMOS) and United Kingdom Infrared Telescope Deep Sky Survey (UKIDSS) Ultra Deep Survey (UDS) fields. The deep near-infrared imaging available in the two independent fields, in addition to deep optical (including $z'$-band) data, enables the sample of $z \simeq 6$ star-forming galaxies to be securely detected longward of the break (in contrast to several previous studies). We show that the expected contamination rate of our initial sample by cool Galactic brown dwarfs is $\lesssim 3$ per cent and demonstrate that they can be effectively removed by fitting brown dwarf spectral templates to the photometry. At $z \simeq 6$, the galaxy surface density in the UltraVISTA field exceeds that in the UDS by a factor of $\simeq 1.8$, indicating strong cosmic variance even between degree-scale fields at $z > 5$. We calculate the bright end of the rest-frame Ultraviolet (UV) luminosity function (LF) at $z \simeq 6$. The galaxy number counts are a factor of $\sim 1.7$ lower than predicted by the recent LF determination by Bouwens et al. In comparison to other smaller area studies, we find an evolution in the characteristic magnitude between $z \simeq 5$ and $z \simeq 7$ of $\Delta M^* \sim 0.4$, and show that a double power law or a Schechter function can equally well describe the LF at $z = 6$. Furthermore, the bright end of the LF appears to steepen from $z \simeq 7$ to $z \simeq 5$, which could indicate the onset of mass quenching or the rise of dust obscuration, a conclusion supported by comparing the observed LFs to a range of theoretical model predictions.

**Key words:** galaxies: evolution – galaxies: formation – galaxies: high-redshift.

## 1 INTRODUCTION

The luminosity and mass functions of galaxies (i.e. the comoving number density as a function of intrinsic luminosity or stellar mass) are key observables in astronomy, as they trace the build-up and evolution of galaxies through cosmic time (Madau & Dickinson 2014). Through the comparison between the observed functions and the

predictions of theoretical models and simulations, it is possible to gain an insight into the dominant processes that control the formation and evolution of galaxies (e.g. Peng et al. 2010; Bower, Benson & Crain 2012). When observing high-redshift galaxies ($z \gtrsim 3$), the rest-frame UV emission is redshifted into the optical/near-infrared and galaxies can be efficiently selected via their strong Lyman-break (at $\lambda_{rest} = 1216 \AA$), as pioneered by Guhathakurta, Tyson & Majewski (1990) and Steidel & Hamilton (1992). When characterizing the number densities of these Lyman-break galaxies (LBGs) at the highest redshifts, the determination of the luminosity function (LF) of galaxies at rest-frame UV wavelengths has therefore become

<footnote>* E-mail: raab@roe.ac.uk</footnote>
a standard practice (Steidel et al. 1999). More recent progress in de-
termining the rest-frame UV LF at high redshift has been primarily
driven by data from the *Hubble Space Telescope (HST)*, where
the superior near-infrared sensitivity provided by the Wide Field
Camera 3 (WFC3) has enabled the detection of hundreds of galax-
ies at *z* > 6 since it was installed in 2009 (e.g. Bouwens et al. 2010;
McLure et al. 2010; Oesch et al. 2010), with samples of objects now
extending up to *z* ≃ 9 (Ellis et al. 2013; Oesch et al. 2013). The
tightest constraints on the LF have come primarily from a ‘wedding-
cake’ like combination of these *HST* surveys (McLure et al. 2013;
Oesch et al. 2013; Finkelstein et al. 2014; Bouwens et al. 2015), with
extremely deep, small-area surveys such as the *Hubble* Ultra Deep
Field (HUDF, area = 4.5 arcmin², typical limiting magnitude in the
near-infrared *m*<sub>AB</sub> ~ 29.5) and parallel fields detecting the faintest
objects, being combined with samples of brighter galaxies from
the wider area Cosmic Assembly Near-infrared Deep Extragalactic
Legacy Survey (CANDELS; Grogin et al. 2011; Koekemoer et al.
2011, area ≃ 0.2 deg², *m*<sub>AB</sub> ~ 26–27). Furthermore, specifically
at *z* ≃ 8, the Brightest of the Reionizing Galaxies (BoRG; Trenti
et al. 2011, area ≳ 350 arcmin², *m*<sub>AB</sub> ~ 27) pure-parallel survey has
enhanced the samples of *L* ≳ *L*<sup>*</sup> galaxies. Overall, there has been
a strong consensus between different analyses, using both the clas-
sical ‘colour–colour’ selection (Bouwens et al. 2011; Oesch et al.
2013; Schenker et al. 2013) and photometric redshift fitting ap-
proaches (McLure et al. 2009, 2010, 2013; Finkelstein et al. 2014).
At the bright end, determining the number of *L* ≫ *L*<sup>*</sup> galaxies
with *HST* surveys alone becomes challenging due to the declining
number counts of objects brighter than the characteristic luminos-
ity (*L*<sup>*</sup>), and the relatively small area provided by the HUDF and
CANDELS imaging. Consequently, the very bright end of the LF at
*z* = 5–7 has been successfully studied using ground-based surveys,
which provide degree-scale imaging albeit to shallower depths (e.g.
at *z* = 5–6; McLure et al. 2006, 2009; Willott et al. 2013, and at
*z* ≃ 7; Ouchi et al. 2009; Castellano et al. 2010a,b; Bowler et al.
2012, 2014). Previous comparisons between the ground- and space-
based determinations have generally shown good agreement (e.g.
Bouwens et al. 2007; McLure et al. 2009), however, the recent,
expanded analysis of ~0.2 deg² of *HST* imaging from the UDF and
CANDELS surveys by Bouwens et al. (2015) has revealed new ten-
sions between the different approaches. In particular, the results of
Bouwens et al. (2015) at *z* ≃ 6 and 7 are in excess of the previous
results at the bright end by a substantial factor (≈5 ×). Such a result
is surprising as the inferred high density of bright objects should
have been detected in the existing ground-based searches. While
Bouwens et al. (2015) note that the previous underestimation of
the absolute magnitudes of galaxies, uncertain contamination frac-
tions and overestimated selection volumes can mostly explain the
differences between previous *HST* based results (see Appendix F
of Bouwens et al. 2015), the disagreement with the ground-based
results lacks a clear explanation.
The inferred high number density of bright galaxies found
by Bouwens et al. (2015) and similarly by Finkelstein et al.
(2014, who used ≃ 300 arcmin² of data from the UDF and
the Great Observatories Origins Deep Surveys North and South
fields to select LBGs from *z* = 4–8) has changed the derived
form of the evolution in the rest-frame UV LF from *z* ≃ 8 to
*z* ≃ 4. The LF at high redshift is typically fitted using a Schechter
function (Schechter 1976), where the observed number density, *φ*,
follows a power law with slope *α* to faint luminosities and an ex-
ponential cut-off brightwards of the characteristic luminosity *L*<sup>*</sup>,
as *φ(L)* = *φ*<sup>*</sup>(*L*/*L*<sup>*</sup>)<sup>*α*</sup> e<sup>-*L*/*L*<sup>*</sup></sup>. The majority of previous studies of
the form of the evolution of the LF at high-redshift have tended

At the bright end, determining the number of *L* ≫ *L*<sup>*</sup> galaxies with *HST* surveys alone becomes challenging due to the declining number counts of objects brighter than the characteristic luminosity (*L*<sup>*</sup>), and the relatively small area provided by the HUDF and CANDELS imaging. Consequently, the very bright end of the LF at *z* = 5–7 has been successfully studied using ground-based surveys, which provide degree-scale imaging albeit to shallower depths (e.g. at *z* = 5–6; McLure et al. 2006, 2009; Willott et al. 2013, and at *z* ≃ 7; Ouchi et al. 2009; Castellano et al. 2010a,b; Bowler et al. 2012, 2014). Previous comparisons between the ground- and space-based determinations have generally shown good agreement (e.g. Bouwens et al. 2007; McLure et al. 2009), however, the recent, expanded analysis of ~0.2 deg² of *HST* imaging from the UDF and CANDELS surveys by Bouwens et al. (2015) has revealed new tensions between the different approaches. In particular, the results of Bouwens et al. (2015) at *z* ≃ 6 and 7 are in excess of the previous results at the bright end by a substantial factor (≈5 ×). Such a result is surprising as the inferred high density of bright objects should have been detected in the existing ground-based searches. While Bouwens et al. (2015) note that the previous underestimation of the absolute magnitudes of galaxies, uncertain contamination fractions and overestimated selection volumes can mostly explain the differences between previous *HST* based results (see Appendix F of Bouwens et al. 2015), the disagreement with the ground-based results lacks a clear explanation.

The inferred high number density of bright galaxies found
by Bouwens et al. (2015) and similarly by Finkelstein et al.
(2014, who used ≃ 300 arcmin² of data from the UDF and
the Great Observatories Origins Deep Surveys North and South
fields to select LBGs from z = 4–8) has changed the derived
form of the evolution in the rest-frame UV LF from z ≃ 8 to
z ≃ 4. The LF at high redshift is typically fitted using a Schechter
function (Schechter 1976), where the observed number density, φ,
follows a power law with slope α to faint luminosities and an ex-
ponential cut-off brightwards of the characteristic luminosity L*,
as φ(L) = φ*(L/L*)^α e^(-L/L*). The majority of previous studies of
the form of the evolution of the LF at high-redshift have tended

to favour a pure luminosity evolution, with an approximately con-
stant normalization, φ* (Bouwens et al. 2007, 2008, 2011; McLure
et al. 2009). Such an evolution is to be expected if the rest-frame
UV luminosity of galaxies follows approximately the hierarchical
assembly of the host dark matter haloes at high redshift (e.g. see
fig. 10 of Bouwens et al. 2008). In contrast, the new analyses by
both Bouwens et al. (2015) and Finkelstein et al. (2014) find an
approximately constant characteristic magnitude of the best-fitting
Schechter functions of M* ≃ −21 from z ≃ 4–7, and instead invoke
a strong evolution in the faint-end slope and the overall normal-
ization to reproduce the observed evolution. Whether the observed
evolution in the rest-frame UV LF from z ≃ 7 to z ≃ 4 occurs pri-
marily as density or luminosity evolution depends critically on the
combination of astrophysical processes with the underlying dark
matter halo mass function (HMF), for example the existence, origin
and onset of any cut-off luminosity or quenching mass (e.g. Peng
et al. 2010). Theoretical models tend to predict a more power-law
type form for the LF at high-redshift (e.g. in the Illustris simula-
tion; Genel et al. 2014), and the implementation of astrophysical
processes necessary to quench (e.g. feedback from accretion on to
a black hole) or obscure (e.g. dust production) star formation in the
most massive haloes is required to bring the LFs into agreement (e.g.
Bower et al. 2012; Cai et al. 2014). Hence the accurate determina-
tion of the evolution in the rest-frame UV LF at high redshifts is key
for constraining the implementation of such cut-off mechanisms in
these models.

Furthermore, at *z* ≃ 7, Bowler et al. (2014) found evidence for a
shallower drop-off in the number density of bright galaxies to that
expected from the standard Schechter-function fit, and instead found
that a double power-law (DPL) form was preferred. The observed
*z* ≃ 7 LF from Bowler et al. (2014) follows closely the form of
the HMF, suggesting that the onset of significant quenching or dust
obscuration occurs at *z* < 7. A detailed analysis of the bright end
of the *z* ≃ 6 LF is therefore essential to clarify the dominant form
of the evolution of the rest-frame UV LF at high redshift, and to
investigate how the functional form of the bright end changes as a
result of the potential build-up of dust or the role of feedback in
quenching the most massive galaxies.

In this work, we follow the methodology of Bowler et al. (2012, 2014) to perform a search for bright *z* ≃ 6 star-forming galaxies within the UltraVISTA and Ultra Deep Survey (UDS) fields. These fields contain a wealth of multiwavelength imaging (including deep z'-band and near-infrared data essential for the secure selection of *z* > 5 galaxies) covering an area over eight times greater than that analysed by Bouwens et al. (2015), and almost 20 times that utilized by Finkelstein et al. (2014). Recent improvements in the depth of imaging in both fields (e.g. ≳ 1 mag deeper in the *z'* and/or *Y*, *J*, *H* and *Ks* bands) also allows us to directly compare to, and reassess, previous results at *z* ≃ 6 determined using these survey fields by McLure et al. (2009) and Willott et al. (2013).

The paper is structured as follows. We start with a description of
the UltraVISTA and UDS data sets in Section 2, followed by the
details of our *z* ≃ 6 galaxy selection in Section 3. The potential
contamination of our sample by dwarf stars is quantified in Sec-
tion 4, and we detail our methodology to determine the rest-frame
UV LF in Section 5. In Section 6, we present the basic properties
of our sample of *z* ~ 6 LBGs, and compare our objects to previ-
ous *z* = 6 samples extracted from these survey fields by Willott
et al. (2013) and McLure et al. (2009). We present our rest-frame
UV LF results (including a comparison to previous studies) in Sec-
tion 7. Finally, we investigate the form of the *z* ≃ 6 LF and the
observed evolution from *z* ≃ 5 to *z* ≃ 7 in Section 8, and compare
<figure><img src="image_2.png" /><figcaption>**Figure 1.** The transmission curves for the filters used in this study in the UltraVISTA/COSMOS and UDS/SXDS fields are shown in the upper and lower plots, respectively. Each filter curve has been peak normalized to the median 5σ depth (calculated in 1.8 arcsec diameter circular apertures) and is shown as the response per unit wavelength. The two curves shown for the *Y*, *J*, *H* and *Ks* filters in the UltraVISTA/COSMOS field illustrate the depths from the two epochs of imaging. The deeper DR2 imaging exists over ~70 per cent of the UltraVISTA field with overlapping imaging from CFHTLS (for schematics see figs 1 and 2 of Bowler et al. 2014). In the upper plot an example redshifted galaxy SED from the Bruzual & Charlot (2003) library is shown (this SED assumes an exponentially declining star formation history with τ = 50 Myr, an age of 100 Myr and $A_V = 0.1$), in flux per unit frequency, where the mean IGM absorption from Madau (1995) has been applied. The Lyman-break can clearly be seen at ~0.9 µm in the SED, along with an example Lyman α emission line of $EW_0 = 10$ Å. The lower plot shows example M-, L- and T-dwarf spectra taken from the SpeX library.</figcaption></figure>

the observed evolution in the LF to the predictions from a range of theoretical models. We end with our conclusions in Section 9. Throughout we assume a standard Λ cold dark matter cosmology with $\Omega_m = 0.3$, $\Omega_\Lambda = 0.7$ and $H_0 = 70 \text{ km s}^{-1}$. All magnitudes are quoted in the AB system (Oke 1974; Oke & Gunn 1983) where $m_{AB} = -2.5 \log_{10}[F_v(Jy)/3631 Jy]$, and redshifts are photometric unless otherwise specified. The functional forms for the Schechter, DPL and Saunders functions in magnitudes are presented in Appendix A for reference.

## 2 DATA

The available optical/near-infrared imaging in the two extragalactic survey fields utilized here is summarized below, and further details of the data processing can be found in Bowler et al. (2012, 2014). Fig. 1 summarizes the photometric bands used and illustrates the relative depths of the imaging (calculated in 1.8 arcsec diameter circular apertures). The search for galaxies at $z \simeq 6$ requires deep z'-band imaging for selection, and the UltraVISTA and UDS fields both benefit from deep Subaru z'-band imaging extending faintward of $m_{AB} = 26$. The presence of imaging in filters longward of the Lyman-break, in the near-infrared at $z > 6$, allows the robust removal

of contaminant populations such as Galactic brown dwarfs and dusty low-redshift galaxies, while deep optical imaging is necessary to accurately determine the redshift defined by the strong Lyman-break (which moves through the *i*-band filter at these redshifts).

### 2.1 UltraVISTA/COSMOS

The UltraVISTA near-infrared imaging essential for this work lies within the Cosmological Evolution Survey (COSMOS) field (Scoville et al. 2007), which contains an abundance of multiwavelength imaging over 1–2 deg² on the sky.<fnref n="1" /> We obtained deep Subaru/SuprimeCam z'-band imaging of the central square degree of the field, which reaches a 5σ limit of $m_{AB} = 26.6–26.8$ (1.8 arcsec diameter circular aperture) over four separate SuprimeCam pointings. The COSMOS field also contains one of four ‘deep’ fields (D2) imaged as part of the Canada–France–Hawaii Legacy Survey (CFHTLS), each corresponding to a single 1 deg² pointing of CFHT/MegaCam. We use optical imaging in the *u**, *g*, *r* and *i* filters from the T0007 release of the CFHTLS. The near-infrared imaging

<footnote n="1">¹ http://cosmos.astro.caltech.edu/</footnote>
from UltraVISTA provides data in the *Y*, *J*, *H* and *K*<sub>s</sub> filters over a total of 1.5 deg<sup>2</sup> of the COSMOS field. The UltraVISTA survey consists of a 'deep' component with depths of m<sub>AB</sub> ~ 25 over the full 1.5 deg<sup>2</sup> (McCracken et al. 2012), superseded by the 'ultradeep' imaging that covers ~70 per cent of the full field in the form of four strips. Here, we utilize the second data release (DR2) of UltraVISTA (McCracken et al. 2013), in which the 'ultradeep' part has reached depths of m<sub>AB</sub> = 25.8 in the *Y* band (see table 1 of B14 and Fig. 1). The maximal area of overlapping optical/near-infrared imaging utilized here is defined by the region covered by the CFHTLS (1 deg<sup>2</sup>). Of this area, after the masking of large diffraction haloes, 0.62 deg<sup>2</sup> is covered by near-infrared imaging from the 'ultradeep' part of UltraVISTA, and the remaining 0.29 deg<sup>2</sup> has shallower imaging from the 'deep' component. Due to the difference in depth, we treat these two regions separately in the selection of LBGs and in the analysis of the LF.

## 2.2 UKIDSS UDS/SXDS

The United Kingdom Infrared Telescope Deep Sky Survey (UKIDSS) UDS field includes deep optical imaging taken as part of the Subaru *XMM-Newton* Deep Survey (SXDS) in the *B*, *V*, *R*, *i* and *z'* filters (Furusawa et al. 2008). Further to this public data, we obtained deep Subaru/SuprimeCam imaging in the *z'* band (see Bowler et al. 2014 for details) which reaches depths of m<sub>AB</sub> = 26.5–26.7 (1.8 arcsec diameter circular aperture) over four separate Suprime-Cam pointings. The DR10 of the UKIDSS UDS (Lawrence et al. 2007) provides the key near-infrared imaging over the field in the *J*, *H* and *K* filters. Finally, we use the DR2 of the VISTA Deep Extragalactic Observations survey (VIDEO; Jarvis et al. 2013), which has observed the UDS field in the *Y* band to a 5σ limiting depth of m<sub>AB</sub> = 25.3, extending ≈ 0.5 mag deeper than the previous imaging utilized in Bowler et al. (2014). The total area of overlapping optical and near-infrared imaging in the UDS/SXDS field was 0.74 deg<sup>2</sup>.

# 3 CANDIDATE SELECTION

## 3.1 Catalogue production and initial cuts

Candidate high-redshift galaxies were selected from the deep Subaru *z'*-band imaging in the UltraVISTA/COSMOS and UDS/SXDS fields using *SEXTRACTOR* v2.8.6 (Bertin & Arnouts 1996), with the fiducial photometry measured in 1.8 arcsec diameter circular apertures. Multiwavelength catalogues were produced using the 'dual-image' mode of *SEXTRACTOR*, with the Subaru *z'*-band used as the detection image and the other available bands used as the measurement images (e.g. *u*<sup>*</sup>*griYJHK*<sub>s</sub> for the UltraVISTA/COSMOS field and *BVRiYJHK* for the UDS/SXDS field).

The full catalogues were first cut at a *z'*-band magnitude of 26.0 in the UltraVISTA DR2 ('ultradeep') and UDS fields, and at 25.0 for the shallower UltraVISTA DR1 ('deep') regions, resulting in a minimum significance of 7σ in the shallowest Subaru/Suprime-Cam tiles for each field, extending to ~10σ in the deepest tiles. To ensure a non-detection in the bluest bands, we removed any object with a detection at greater than 2σ significance in either the *u* or *g* band in the UltraVISTA field, or the *B* or *V* band in the UDS, using local depth estimates within the field. Local depths were calculated at each point in the images from the median absolute deviation (MAD) estimator (using σ = 1.48 × MAD) of the counts within the closest 200, randomly placed, blank apertures. For the purpose of spectral energy distribution (SED) fitting, we corrected all magnitudes to total assuming a point-source correction. Many

of the candidate high-redshift galaxies are resolved in the ground-based imaging (see Bowler et al. 2014), however the size is still dominated by the seeing of the images and therefore the point source correction dominates any colour difference due to intrinsic size variation between the bands.

## 3.2 Photometric redshift fitting

The final sample of $z \simeq 6$ galaxies was defined by fitting Bruzual & Charlot (2003) SED models to the photometry, coupled with careful visual checks to remove artefacts and objects with low-level flux in the two bluest optical bands. Given the degeneracies between metallicity, age and dust reddening when using broad-band photometry, we fit our candidate galaxy photometry with a reduced set of model galaxy SEDs (numbering $\simeq 500$ models in total, before the application of dust attenuation). Exponentially declining ($\tau$) models with characteristic time-scales in the range $50 \text{ Myr} \le \tau \le 10 \text{ Gyr}$ were used, and the Calzetti et al. (2000) attenuation law was assumed in all cases. The 'high-redshift' model set, designed to identify good high-redshift candidates, consisted of models with ages from 10 Myr to the age of the Universe at $z=5$, $A_V = 0.0–2.0$ and a single metallicity of $0.2 Z_\odot$ motivated by recent measurements of the metallicity in low-redshift LBG analogues (Stanway & Davies 2014). The photometric redshifts for our sample of $z \simeq 6$ LBGs have typical errors of $\Delta z \simeq 0.1–0.2$, depending on observed magnitude. The 'contaminant' model set was designed to provide red and dusty galaxy SEDs in the range $z=1–3$, with strong Balmer or 4000 Å breaks, and therefore we allow ages up to the present age of the Universe, $A_V = 0.0–6.0$ and a single metallicity of $1.0 Z_\odot$. In addition to the suite of galaxy SEDs, we also fit standard stellar templates with types M4 to T8 (taken from the SpeX library<sup>2</sup>), as cool Galactic brown dwarfs can mimic the colours of high-redshift LBGs with $z=5–7$. We discuss and quantify potential brown dwarf contamination of our sample in Section 4.

For inclusion in our final $z \simeq 6$ galaxy sample, we required that the object had a best-fitting SED at $5.5 < z < 6.5$ with an acceptable $\chi^2$ (≤11.3; calculated as the $\chi^2$ value that corresponds to 2σ significance given the number of degrees of freedom in the fitting), and a $\Delta\chi^2 > 4$ between the high-redshift solution and the next best-fitting $z < 5.5$ model (a 2σ condition when marginalizing over all parameters but the redshifts; see Press et al. 1992). This step reduced the sample of several thousand objects that passed the magnitude cuts described above to a total sample of 335 objects, with 205, 3 and 127 objects from the UltraVISTA DR2 strips, DR1 and UDS, respectively. The sample was then carefully visually checked to remove single-band detections (including *z'*-band CCD bleeds) and to identify false optical non-detections due to the negative haloes around stars in the CFHTLS and Subaru optical imaging, which resulted in the removal of a further 26 objects (15 in the UltraVISTA DR2 and 11 in the UDS). At this point, we also removed the extreme LAE 'Himiko' from our UDS sample, because of the known spectroscopic redshift of $z = 6.595$, which places it outside our desired redshift range (see Section 6.4.1). The result of the SED fitting and visual checks described was a sample of 309 objects (190, 3 and 116 in the UltraVISTA DR2, DR1 and UDS, respectively) that are consistent with being $5.5 < z < 6.5$ LBGs. However, one final source of contamination must be considered, namely cool Galactic brown dwarfs. The removal of candidates consistent with being a brown dwarf using the criterion described in detail in the next section, resulted in the removal of 37 and 10 objects in the UltraVISTA

<footnote n="2">² http://pono.ucsd.edu/~adam/browndwarfs/spexprim/</footnote>
DR2 and UDS samples, respectively, producing a final sample of 266 galaxies with photometric redshifts in the range $5.5 < z < 6.5$.

# 4 CONTAMINATION BY BROWN DWARFS

Cool Galactic brown dwarfs (with spectral types M, L and T) have SEDs that peak in the near-infrared and drop steeply towards the optical bands (e.g. the M8 type dwarf shown in Fig. 1), potentially mimicking the colours of $z > 5$ galaxies. The number density of brown dwarfs begins to drop at $J > 25$ (Ryan et al. 2011) as the number counts of galaxies rapidly rise (e.g. see Fig. 2), and therefore they are in practice a negligible contaminant for extremely deep, small-area imaging programmes such as the HUDF. In the search for the brightest high-redshift $z \simeq 6$ LBGs however, the number densities of brown dwarfs can begin to dominate (see Fig. 2), and the high-redshift galaxy samples become increasingly susceptible to contamination by the relatively more numerous M- and L-type dwarfs. The comparatively poor resolution of the ground-based imaging surveys utilized here compared to *HST* imaging, coupled with the small measured sizes of LBGs (Ono et al. 2012; Curtis-Lake et al. 2014), precludes any discrimination based on size, and hence we must carefully assess the available multiwavelength information to remove brown dwarfs from the $z \simeq 6$ sample. The UltraVISTA/COSMOS and UDS/SXDS fields utilized here contain the best deep optical to near-infrared photometric data available on the degree scale, and hence, as we show quantitatively in the following section, the removal of brown dwarfs using SED fitting of standard spectral templates can be cleanly performed. The possibility of photometric scattering of brown dwarfs (which are considerably more numerous than $z > 5$ LBGs at $m_{AB} \lesssim 25$) into our sample must be carefully considered however, and we use a simple model of the Galactic stellar distribution to estimate the likely number of contaminant brown dwarfs in Section 4.2.

<figure><img src="image_3.png" /><figcaption>**Figure 2.** The total number of dwarf stars (with spectral types M4–T8) predicted in the $0.62 \deg^2$ UltraVISTA/COSMOS DR2 field is shown as the grey shaded region, and the corresponding values for the $0.74 \deg^2$ UDS/SXDS field are shown as the black lines. The lower and upper curves for each field were calculated assuming a Galactic scaleheight of $h_Z = 300$ pc and $h_Z = 400$ pc, respectively, using the exponential disc model of Caballero, Burgasser & Klement (2008). The predicted number densities of $z = 6$ LBGs determined from the LF of McLure et al. (2009) is shown as the dashed line, for an example survey area of $0.7 \deg^2$. The upper axis shows an estimate of the absolute UV magnitude corresponding to the continuum magnitude on the lower axis, assuming the object is at $z = 6$.</figcaption></figure>

## 4.1 Injection and recovery simulations

We quantify the potential brown dwarf contamination of our sample by injecting and recovering synthetic dwarf star photometry into the UltraVISTA/COSMOS and UDS/SXDS images, passing the fake stars through an identical selection procedure as described above for our $5.5 < z < 6.5$ LBGs. Each standard spectral template for dwarf star types M4–T8 from the SpEX library was integrated through the appropriate filters for the UltraVISTA/COSMOS and UDS/SXDS fields, scaled to a total $z'$-band magnitude in the range $m_{AB} = 23-26$ and injected into the images using the model PSF (determined using the method described in Bowler et al. 2014) at a random position. As described further in Section 5.1, we used representative subsections of the full UltraVISTA and UDS mosaics to reduce computing time. If the object was recovered ($\sim 20$ per cent of injected objects are lost due to blending with other sources), and passed the $z'$-band magnitude cut and the optical non-detection conditions (Section 3.1), the photometry was corrected to a total magnitude in each band and fitted with the galaxy and stellar spectral templates used in the LBG selection. The injected stellar photometry was then classified as an LBG contaminant if $\chi^2_{gal} < 11.3$ and $5.5 < z_{phot} < 6.5$, following exactly our LBG selection procedure.

The resulting fraction of each stellar type classified as an LBG contaminant depended strongly on the assumed magnitude, with injected stars brighter than $z' \simeq 25$ rarely being classified as an LBG. We find that up to $\sim 10-20$ per cent of the input dwarf stars with $z' = 25-26$ are classified as $5.5 < z < 6.5$ galaxies, depending on subtype and field (where the differing relative depths of the imaging results in different vulnerabilities). The predicted number and redshift distribution of dwarf stars is discussed in the next section.

Crucially, the simulations show that the overwhelming majority ($>95$ per cent) of injected stellar templates that are recovered as LBG candidates remain good stellar fits. We also find that we can exclude the majority of the contaminant brown dwarfs from our galaxy sample by requiring that the object has a poor stellar fit quantified as $\chi^2_* > 10.0$. A small number of genuine LBGs will also be excluded as a result of this criterion, however we account for the additional incompleteness of our sample when calculating the LF, by recreating this selection criterion in our galaxy injection and recovery simulations. By applying such a selection criterion, we remove 37 objects from the UltraVISTA DR2 sample, 0 from the DR1 and 10 from the UDS sample. In Appendix B, we present the 47 potential high-redshift galaxies that were excluded from the original sample as possible dwarf star contaminants based on a good stellar fit with $\chi^2_* < 10.0$.

## 4.2 Number density model

Despite the apparent success of our stellar fitting method for removing dwarf stars masquerading as high-redshift LBGs, even a small contamination rate at the bright end could be significant for the determination of the $z \simeq 6$ LF (Fig. 2). Hence to constrain the likely contamination rate, an estimate of the number of each stellar type as a function of magnitude is required.

The dwarf stars relevant for high-redshift galaxy studies are typically distant objects in Galactic terms, e.g. an M4 dwarf star with observed magnitude $z' = 26-24$ probes the Galaxy at a radius of $1.5-4$ kpc (or $250-600$ pc for a T8 dwarf), and determining the scaleheight of the disc components is challenging. Previous searches for M-, L- and T-type stars have commonly assumed a single disc model (e.g. Ryan et al. 2011; Holwerda et al. 2014) to describe the
observed number of stars, as the small samples of objects preclude a more complicated analysis. We follow such an approach using a single exponential disc model as described in Caballero et al. (2008). Here, the number density of each dwarf star type s, as a function of Galactic longitude and latitude (l, b) and heliocentric distance (d), is given by

$$n_s = n_s(d=0) e^{-\frac{R(d,l,b)-R_{\odot}}{h_R}} e^{-\frac{|Z_{\odot}+dsinb|}{h_Z}}, \quad (1)$$

where R⊙ and Z⊙ denote the Galactocentric solar radius and height above the Galactic disc, and hR and hZ are the radial scalelength and scaleheight for the model. We use the approximations to equation (1) outlined in Caballero et al. (2008) relevant for deep extragalactic survey fields, and predict the number density of each spectral type as a function of magnitude by integrating along a line of sight through the Galactic disc. The local number densities and stellar absolute magnitudes (Ml) were taken from Caballero et al. (2008), converting the magnitudes from the Vega to the AB system according to Frei & Gunn (1994). We then calculated the absolute z'-band magnitude from Ml using the i-J colours from Caballero et al. (2008) and the standard spectral templates described in Section 3.2 to convert from Ml to Mz. We adopt the main parameters of the Galactic thin disc model from Chen et al. (2001), with Z⊙ = 27 ± 4 pc, R⊙ = 8600 ± 200 pc and hR = 2250 ± 1000 pc. The most relevant quantity for this study is the Galactic vertical scaleheight, where we use two values at hZ = 300 pc and hZ = 400 pc to reflect the uncertainty in this quantity (Pirzkal et al. 2009; Ryan et al. 2011; Holwerda et al. 2014). Fig. 2 shows the predicted number of dwarf stars as a function of magnitude in the two fields. The analytic model shows that we expect more dwarf stars in the UltraVISTA/COSMOS field (Galactic coordinates b = 42.1, l = 236.8) despite the smaller area covered (area of DR2 = 0.62 deg² versus UDS = 0.74 deg²), due to the line of sight intersecting with more of the Galactic disc and being at a lower Galactic latitude to the UDS/SXDS (b = -60.0, l = 169.9). The intersection of a pencil-beam survey with this exponentially declining distribution results in each dwarf star spectral type (which has a corresponding intrinsic absolute magnitude, Ml,AB ~ 9-17 for spectral types M4-T8) having a peak in number density at an increasingly faint apparent magnitude, with M-dwarfs peaking at z' < 23, L-dwarfs at z' ≈ 25 and T-dwarfs at z' > 26.

## 4.3 Predicted number of contaminant brown dwarfs

By combining the predicted number densities of dwarf stars illustrated in Fig. 2 with the probability of a given dwarf stellar type of a given magnitude passing the LBG selection criterion (from the simulations described in Section 4.1), we can predict the number of contaminant brown dwarfs expected in our z ≈ 6 sample. The expected pseudo-redshift distribution for these stars can also be calculated and is shown in Fig. 3, assuming a vertical scaleheight of hZ = 400 pc to provide an upper limit on the number of objects. Within the UltraVISTA DR2 area, we would predict a total of 2-7 dwarf stars with hZ = 300-400 pc, and in the UDS this drops to 0.1-0.4. Applying a χ² > 10.0 condition to exclude objects with good stellar fits, the number of contaminant stars drops dramatically to 0.1-0.4 in UltraVISTA DR2 and 0.02-0.06 in the UDS samples as shown in the lower panel of Fig. 3. The predicted redshift distribution for the contaminant stars also illustrates the differences between the selection functions of the two fields. Most notably, the UDS field is more prone to contamination by late-type dwarfs, such as L- and T-types, as a result of the deep optical imaging excluding

<figure><img src="image_4.png" /><figcaption>Figure 3. The expected photometric redshift distribution of dwarf stars that are acceptable z > 5 LBG candidates in the UltraVISTA DR2 and UDS fields. The UltraVISTA DR2 results are shown as the blue histograms with a thick black outline, where each stellar type is highlighted in a different shade of blue, whereas the distribution for the UDS is shown in yellow and orange. A vertical Galactic scaleheight of hZ = 400 pc is assumed to illustrate an upper limit on the number of potential stellar contaminants. The upper panel shows the predicted number of brown dwarfs in our sample when no attempt is made to remove objects with good stellar fits. By requiring that robust high-redshift galaxies have a poor stellar fit, with χ² > 10.0, the expected number of contaminant brown dwarfs in the sample drops essentially to zero as shown in the lower panel.</figcaption></figure>

early-type M-dwarfs, and the shallower Y-band imaging making the separation of a sharp break and the gentle rise of a dwarf star through the z', Y and J filters difficult to distinguish. These late-type dwarfs have higher photometric redshifts when identified as LBG candidates, and hence in any case are excluded as part of our selection procedure for 5.5 < z < 6.5 objects. The UltraVISTA/COSMOS field is more prone to contamination by M-dwarf stars, due to the shallower optical data available in the field. The lack of dwarf stars showing a best-fitting redshift in the range 5.7 < zphot < 6.1 arises due to stellar templates being unable to reproduce the resulting large i-z' colour produced from an LBG in this redshift range, this is clearly seen in the example SEDs shown in Fig. 4. McLure et al. (2009) restricted the redshift range of their sample of z ~ 6 LBGs to the range 5.7 < z < 6.3, which our simulations show is also a relatively clean dwarf star region.

In summary, our simulations would suggest a ~3 per cent contamination of our initial sample, which can be reduced to ≪1 object by ensuring all galaxy candidates have a bad stellar fit (with χ² > 10.0). Imposing a χ² > 10.0 condition on our penultimate sample resulted in the removal of 37 and 10 objects from the UltraVISTA DR2 and UDS fields, respectively, corresponding to ~20 per cent of the initial sample. Inspection of the χ² values for the stellar and galaxy fits for objects removed from the sample as potential stars (shown
<figure><img src="image_5.png" /><figcaption>**Figure 4.** SED fitting results and postage-stamp images of two objects from our penultimate sample of $z \simeq 6$ galaxies. For each object cut-outs (10 arcsec on the side) from the images are shown in the upper panels. The two plots show the observed photometry as the black points, with the best-fitting high- and low-redshift galaxy templates shown as the blue and red lines in the left-hand panel, and the best-fitting stellar template shown in the right-hand panel. The inset in each plot shows the variation of $\chi^2$ with either redshift or stellar spectral-type. The upper object at $z \simeq 6$ shows a clear break between the *i* and *z'*-bands that cannot be recreated by the low-redshift galaxy or stellar templates. The lower object however, is both a good stellar and high-redshift galaxy fit, illustrating the difficulty in removing stars in particular at $z > 6.3$ and $z < 5.7$, where the optical to near-infrared break appears more gradual. The lower object was removed from our final sample based on a good stellar fit (taken as $\chi_*^2 < 10.0$).</figcaption></figure>

in Table B1), reveals that many of these objects are relatively poor stellar fits with $\chi_*^2 \sim 5-9$. Therefore, we are likely excluding some genuine galaxies by imposing such a cut, a hypothesis supported by the redshift distribution of our sample shown in Fig. 6 which, after the imposition of a $\chi_*^2 > 10.0$ condition, shows a deficit of objects at $5.5 < z < 5.7$ as compared to our expected distribution. The deficit is particularly obvious in the UDS field, and inspection of the $\chi_*^2$ values for the objects classified as stars reveals that they are all $\chi_*^2 > 5$. As the key aim of this work is the accurate determination of the LF at $z \simeq 6$, free from contamination by brown dwarfs that can dominate the number counts of the brightest LBGs, we choose to apply such a cut with the acknowledgment that genuine galaxies will be excluded at this step. This incompleteness is taken into account in our injection and recovery simulations. In addition,

to ensure that dwarf star contamination and our removal methodology has a minimum impact on our LF determination, we choose to restrict our redshift range to $5.7 < z < 6.3$ in the LF analysis. Such a redshift restriction has the additional benefit of reducing the impact of the evolving LF on our analysis and makes the median redshift ($z_{med} \simeq 5.9$) more in-line with previous determinations (McLure et al. 2009; Willott et al. 2013).

## 5 DETERMINATION OF THE LF

In the calculation of the rest-frame UV LF from an observed galaxy sample, it is necessary to account for the impact of photometric scatter and the particular selection methodology implemented, as these effects, unless corrected for, can strongly affect the derived
intrinsic number density of galaxies. The high signal-to-noise ratio of our galaxies (>7σ in the z' band) strongly reduces the occurrence of spurious detections, and the application of our careful SED fitting procedure can remove low-redshift interlopers and Galactic brown dwarfs. Our samples will, however, still suffer incompatibility from blending with foreground objects and from misidentification as dwarf stars or low-redshift galaxies at the faint end of our sample. In the following section, we describe our injection and recovery simulations that quantify our completeness and the methodology we use to calculate the binned LF including this correction. Our rest-frame UV LF results are presented in Section 7. For both synthetic and observed galaxies in this work, we calculate the rest-frame UV absolute magnitude at a central wavelength of λ<sub>rest</sub> = 1500 Å, in concordance with previous work (McLure et al. 2013; Bowler et al. 2014) by integrating the best-fitting SED in the rest-frame with a top-hat filter of width 100 Å.

## 5.1 Completeness simulations

We estimate the completeness of our sample following the methodology presented previously in McLure et al. (2009), McLure et al. (2013) and Bowler et al. (2014), by injecting synthetic z = 5–7 LBGs into the UltraVISTA/COSMOS and UDS/SXDS data sets, and attempting to recover them using an identical procedure as for the real z = 6 objects selected in this work. Due to the differing depths of available optical/near-infrared imaging in the two fields, and the different selection criteria used for the ‘deep’ and ‘ultradeep’ parts of the UltraVISTA survey, separate simulations for each region were performed to provide field dependent completeness correction factors. In each case, the z'-band magnitude cut applied (z' < 26.0 or z' < 25.0 in the ‘ultradeep’ and ‘deep’ components, respectively) was recreated in the simulations.

Photometric errors can scatter injected objects in magnitude and redshift, and therefore we inject galaxies from z = 5 to 7 and with magnitudes as faint as 1 mag below the z'-band cut we use for each field. Whereas the number of objects scattered out of a bin is symmetric, the number scattered into a given bin depends on the underlying LF (e.g. a Schechter function exponential decline will result in more objects up scattered into a given bin than a shallower function). Since we do not know a priori the form of the bright end of the LF, we calculate the incompatibility using a range of functional forms for the injected galaxy population and compare the results. In addition, the LF of LBGs is evolving with time, which must be taken into account and could potentially affect the derived parameters at each redshift (Muñoz & Loeb 2008). In the Schechter function case, we assume the parameters from McLure et al. (2009) at z = 5 with ([M*, φ*, α] = [-20.73, 0.00094/Mpc³, -1.66]) and McLure et al. (2013) at z = 7 ([M*, φ*, α] = [-19.90, 0.0011/Mpc³, -1.90]), with a simple linear evolution in all the Schechter parameters between these redshifts. We use an analogous approach for the DPL, using the fitted parameters from Bowler et al. (2014) at z = 5 ([M*, φ*, α] = [-21.0, 0.00039/Mpc³, -1.9]) and z = 7 ([M*, φ*, α] = [-20.4, 0.00031/Mpc³, -2.2]) with a fixed bright-end slope (β = -4.4). Using the evolving LFs, we then randomly populated an input M<sub>UV</sub>-z plane, assigning each galaxy a rest-frame UV slope, β<sub>UV</sub>, drawn at random from a Gaussian distribution with mean β<sub>UV</sub> = -1.8 and standard deviation σ = 0.3, to mimic the slightly red β<sub>UV</sub> and intrinsic scatter found for bright galaxies at z = 5 by Rogers et al. (2014). The objects were then selected using S<sub>EXTRACTOR</sub> and the high-redshift candidates extracted using the magnitude cuts and SED fitting analysis as described in Section 3.

We find that ≈ 20 per cent of injected objects are not recovered due to blending by foreground images in the crowded optical bands. Of the objects that are recovered, we find a completeness of ≈80 per cent, where objects here are lost either because they do not pass the optical drop-out criterion or because they were misclassified as a dwarf star or low-redshift galaxy contaminant.

## 5.2 The $1/V_{max}$ estimator

We used the $1/V_{max}$ estimator (Schmidt 1968) to derive the LF, where the number density of objects in a given magnitude bin, φ(M), depends on the maximum volume (V<sub>max</sub>) each galaxy could have been selected in, modulated by a completeness correction factor (C<sub>f</sub>) which accounts for the impact of both object blending and photometric scatter on the observed number of galaxies in a given bin,

$$ \phi(M) = \sum_{i=1}^{N} \frac{C_f(M_i, z_i)}{V_{\text{max},i}}. \qquad (2) $$

Here, the sum is over the N galaxies in the magnitude bin in question, where we chose magnitude bins of width ΔM = 0.25–0.5 mag depending on the number of objects available. The V<sub>max</sub> was calculated by artificially redshifting the best-fitting SED of each galaxy to find the maximum redshift it could have been observed at and still be included in the sample (e.g. by passing the initial z'-band magnitude cut). The comoving volume was then calculated from this z<sub>max</sub> and the minimum allowed redshift of z<sub>min</sub> = 6.7, and the survey area. The completeness correction factor depended on the field or region of the imaging in which the galaxy was selected, which has been denoted here as f. The three regions in which objects were selected were the UltraVISTA/COSMOS DR2 (or ‘ultradeep’ strips) comprising 0.62 deg² of imaging, the shallower UltraVISTA/COSMOS DR1 (or ‘deep’ region) comprising 0.29 deg² or the UDS/SXDS field which provided 0.74 deg². The errors on the number densities are assumed to be Poissonian, however there is also an additional error in the derived number density resulting from the error in the completeness value. Hence, we include an estimate of this error by bootstrap resampling the galaxies within each bin. The Poisson error dominates in the bright bins, however the error in the completeness becomes comparable for the faintest bins.

## 5.3 The binned LF

We restrict the redshift range to 5.7 < z < 6.3 to enable direct comparison to the work of McLure et al. (2009) and to ensure we are not influenced by any residual brown dwarf contamination or any incompatibility due to our dwarf star removal methodology. Our results are robust to the underlying function assumed in our completeness simulations, due to the high signal to noise of our galaxies which reduces the effect of up-scattering. We assume the DPL parametrization as in Bowler et al. (2014) in the final LFs presented.

We determine the LF in the range -22.625 < M<sub>UV</sub> < -21.125, using 0.25 or 0.5 mag bins depending on the number of objects available. The faintest bin is defined by the point at which our observed absolute magnitude counts begin to drop, as shown in Fig. 6, indicating that our sample is becoming increasingly incomplete. The maximum volume available to our brightest objects is 7 × 10⁶ Mpc³ in the UltraVISTA DR2 and UDS fields combined, with an additional 1 × 10⁶ Mpc³ available for the brightest objects that could have been selected in the 0.29 deg² of UltraVISTA DR1
<figure><img src="image_6.png" /><figcaption>**Figure 5.** The *z' - *Y* versus *i* − *z'* colours of the full 5.5 < *z* < 6.5 sample. The colours derived from high-redshift galaxy SEDs (shown as grey tracks) and the colours of Galactic dwarf stars (star symbols) are shown for comparison. The galaxy sample is shown as the black points, with the UltraVISTA/COSMOS and UDS/SXDS galaxies shown as the filled and open circles, respectively. Objects that were excluded from these samples as possible brown dwarfs are shown as the red points. The typical error bar is shown in the lower left corner, and where an object is detected at less than the 2σ-level in a given filter the magnitude here is set to the local 2σ depth, and we display the colour as a limit using an arrow. The M-, L- and T-dwarf star colours were calculated from spectra taken from the compilation of stellar spectra described in Findlay et al. (2012). High-redshift LBG colours taking from Bruzual & Charlot (2003) models are shown as the grey lines (constant star formation history, *Z* = 0.2*Z*⊙, *A*<sub>V</sub> = 0.0 − 0.5, age = 50–500 Myr).</figcaption></figure>

imaging. Our LF results calculated following the described method-
ology are presented and discussed in Section 7.

6 THE SAMPLE

The final sample of 5.5 < *z* < 6.5 galaxies consists of 266 objects, with 156, 3 and 107 coming from the UltraVISTA/COSMOS DR2, DR1 and UDS/SXDS fields, respectively. In the reduced redshift range 5.7 < *z* < 6.3 used by McLure et al. (2009), we find 105, 2 and 70 objects in these fields. We postpone a full discussion of the SED properties and sizes of these objects to a future paper (Bowler et al., in preparation), however the basic sample properties are discussed below.

6.1 Galaxy colours

Galaxies at *z* ~ 6 are typically selected as *i*-band dropout objects,
and in Fig. 5 we show the *i* − *z'* colour (which straddles the break at
*z* ≃ 5.8) against the *z'* − *Y* colour (which determines the rest-frame
UV colour) of our sample of galaxies selected by their photometric
redshift. Both the UltraVISTA/COSMOS and UDS/SXDS samples
occupy a similar region of colour space, indicating no strong biases
in the galaxy colours due to the different relative imaging depths.
The colours of the objects can be reproduced by Bruzual & Charlot
(2003) models with *A*<sub>V</sub> = 0.0–0.5 within the errors (Calzetti et al.

2000 attenuation law), with no strong evidence for extremely red
objects [to be compared with the stack of z ~ 6 galaxies found
by Willott et al. (2013) which had a best-fitting A<sub>V</sub> = 0.75, as
discussed further in Section 6.5]. We also show the colours of the
possible dwarf stars excluded from our sample, along with the stellar
locus as derived from stellar spectra compiled by Findlay et al.
(2012). Several of these objects do lie on the stellar locus, however
a considerable fraction are found with colours differing by up to
~0.5 mag. The results of the injection and recovery simulations
described in Section 4 showed that the majority of dwarf stars
enter our LBG sample as a result of scattering of the photometry,
hence a wide range of colours for potential dwarf stars is to be
expected. The identification of potential stellar contaminants that
would have identical colours to LBGs illustrates the power of using
the full multiwavelength photometry, as several of the likely dwarf
stars would be indistinguishable from LBGs based on a simple
colour selection. Furthermore, Fig. 5 clearly shows that if a strict
colour selection for z ~ 6 objects, designed to remove dwarf star
candidates, was implemented, the true colour distribution of LBGs
would have been biased to bluer objects.

6.2 Redshift, M<sub>UV</sub> and m<sub>AB</sub> distributions

In Fig. 6, we show the redshift distribution for our full sample of 5.5 < *z* < 6.5 LBGs, and the *M*<sub>UV</sub> and *m*<sub>AB</sub> distributions in the restricted redshift range (5.7 < *z* < 6.3) to allow a direct comparison with our LF determination. In each panel, predicted distributions from the simulations described in Section 5.1 for an evolving LF model are shown. For each simulated distribution, the number of galaxies predicted by a linearly evolving model according to a Schechter function (derived from Bouwens et al. 2015 or McLure et al. 2009) or a DPL function (derived from Bowler et al. 2014) were injected into the images, and the resulting *z*<sub>phot</sub>, *M*<sub>UV</sub> and *m*<sub>AB</sub> histograms are displayed. The LF determination of Bouwens et al. (2015) overpredicts the number of galaxies we should find by around a factor of ≃1.7, whereas the LF determination of McLure et al. (2009) is in better agreement, underpredicting the observed number by ≃20 per cent. The results of the model using the DPL fits from Bowler et al. (2014) agree well with the final sample of galaxies we find. Splitting the sample by field (middle and lower panel for the UltraVISTA/COSMOS and UDS/SXDS samples, respectively) reveals an excess of galaxies in the UltraVISTA/COSMOS field as opposed to the McLure et al. (2009) and Bowler et al. (2014) models, which is present over a range of redshifts and absolute magnitudes. In both fields there exist more *M*<sub>UV</sub> < −22.0 galaxies than predicted by the Schechter function model of McLure et al. (2009).

The redshift distribution differs from the model prediction, with
a flatter distribution than expected. Some of the flattening could be
a result of too strictly removing galaxy candidates that have good
stellar fits, which would cause a drop in the number of objects in the
range 5.5 < z < 5.7. However, the precise details of the form and
evolution of the LF in the range z = 5–7 are not well constrained,
and therefore the model predictions shown are rough estimates of
the predicted distributions.

6.3 Cosmic variance between the fields

The galaxies in our sample are not uniformly spread between
the UltraVISTA/COSMOS and UDS/SXDS fields. Given that the
UltraVISTA DR2 region is only ~84 per cent of the area of the
<figure><img src="image_7.png" /><figcaption>**Figure 6.** The distributions of the $z \simeq 6$ galaxy sample with best-fitting photometric redshift, $M_{1500}$ and observed $z'$-band aperture magnitude. The upper row of plots shows the full sample, with the middle and lower rows showing the UltraVISTA/COSMOS and UDS/SXDS samples separately. The lines show the predicted distributions from the injection and recovery simulations described in Section 5.1, for evolving LF models derived from the Schechter function fits of McLure et al. (2009) and Bouwens et al. (2015) and the DPL fits from Bowler et al. (2014) in blue, purple and orange, respectively. In the $M_{UV}$ and $m_{AB}$ plots, we show only the objects in the restricted redshift range $5.7 < z < 6.3$.</figcaption></figure>

UDS data, the discrepancy becomes more significant, with the ratio of the surface density between the UltraVISTA DR2 and the UDS =1.7 (or 1.8 in the redshift range $5.7 < z < 6.3$). Here, we compare the smaller DR2 region of the UltraVISTA field to the UDS, because they have similar depths in the near-infrared and were cut at an identical $z'$-band magnitude ($z' = 26.0$). The difference is present over a full range of redshifts and magnitudes as shown in Fig. 6. Both Willott et al. (2013) and Bouwens et al. (2015) also note an overdensity of galaxies in the UltraVISTA/COSMOS field. Furthermore, when comparing the derived $\phi^*$ between the five CANDELS fields at $z \simeq 6$, Bouwens et al. (2015) finds the largest discrepancy occurs between the COSMOS and UDS fields, with the UDS appearing marginally underdense.

Correcting for the different areas of the surveys (0.62 deg² versus 0.74 deg²), we find a surface density of 169 ± 13 and 95 ± 10 galaxies per deg² in the UltraVISTA/COSMOS and UDS/SXDS fields, respectively (Poisson errors). We calculated the predicted cosmic

variance from the Cosmic Variance Calculator v1.02³ (Trenti & Stiavelli 2008) with a Sheth-Tormen halo MF, $\sigma_8 = 0.9$ and a unity halo filling factor. The result of the calculation was an expected error on the average density of 132 ± 21 galaxies per deg², where both cosmic variance and Poisson errors are included. Hence the difference in number counts just exceeds that expected due to the large-scale structure variations over fields of this size.

The predicted number counts of brown dwarfs are higher in the UltraVISTA/COSMOS field as a result of the Galactic coordinates which, if dwarf star contamination was significant, could produce a higher number of objects in our sample in this field. The simulations described in Section 4 show that dwarf stars can be effectively identified and removed using our stellar fitting procedure and furthermore, the expected pseudo-redshift distribution of dwarf stars (shown in Fig. 3) does not match that observed in our sample. The

<footnote n="3">³ http://casa.colorado.edu/~trenti/CosmicVariance.html</footnote>
strongest evidence for a genuine discrepancy between the number of objects between the two fields comes from considering the reduced redshift range, which cannot be dominated by stellar contamination due to the large *i* − *z*′ colour required. Restricting the redshift range of our sample to the dwarf star free region with 5.7 < *z* < 6.1 results in 82, 2 and 61 objects in the UltraVISTA DR2, DR1 and UDS fields, producing a number density ratio of ~1.6.

We consider the possibility of gravitational lensing and further
large-scale structure effects that could be responsible for the field-
to-field variance in Section 7.

6.4 Overlap with previous studies

6.4.1 McLure et al. (2009) and Curtis-Lake et al. (2013)

The first search for *z* ≳ 5 galaxies using the UKIDSS UDS near-infrared imaging was undertaken by McLure et al. (2006), who found nine massive *z* ≃ 5 galaxy candidates by combining the SXDS optical imaging with the *J*, *H* and *K* images from the UDS early data release. Using the subsequent release of the UKIDSS UDS imaging (DR1, which is ~1.5 mag shallower than the data utilized in this work), McLure et al. (2009) were able to calculate the bright end of the LF at *z* = 5 and 6. In addition, the brightest 14 galaxies from the *z* ≃ 6 sample of McLure et al. (2009) in the range 6.0 *z* < 6.5 were targeted spectroscopically and the results were presented by Curtis-Lake et al. (2013) who detected Lyman α emission in 11 of the objects. Comparing our sample of *z* ≃ 6 galaxies in the UDS, we find 8 of the 10 galaxies presented in Curtis-Lake et al. (2013). Inspection of our initial catalogues reveals that the two excluded objects, UUDS_J021922.01−045536.3 and UUDS_J021701.44−050309.4, were both removed because they are fainter than our imposed *z*'-band magnitude limit, with 1.8 arcsec aperture magnitudes of *z*′ = 26.2 and *z*′ = 26.4, respectively. The brightest object targeted by Curtis-Lake et al. (2013) was identified as a faint AGN based on the broad Ly α line (Willott et al. 2009). This object was excluded from our galaxy sample based on a χ² = 12.6, which slightly exceeds our acceptable criterion. Closer inspection of the SED reveals that the poor galaxy fit is driven by an enhanced *z*'-band flux, as a result of strong Ly α emission at the very blueward edge of the *z*'-band filter (at *z* = 6.01), which contributes ~70 per cent of the *z*'-band flux (Willott et al. 2009). The presence of strong Ly α emission sufficient to significantly change the redshift of a galaxy candidate in our sample is unlikely, as the space density of quasars is extremely low (e.g. one object in the full UDS area) and the most luminous LBGs exhibit significantly lower EWs (EW₀ ≪ 100 Å; Stark, Ellis & Ouchi 2011; Curtis-Lake et al. 2013). We note that the one exception to this rule is the extreme LAE ‘Himiko’, which, consistent with its discovery in a narrow-band survey, stands out as unusual in our SED fitting analysis.

6.4.2 Willott et al. (2013)

Using the 4deg² of multiwavelength imaging from the CFHTLS
‘deep’ component, Willott et al. (2013) found 40 i-band dropout
 galaxies at *z* ~ 6. The CFHTLS data consisted of *u*#, *g*, *r*, *i* and
 z-band imaging in four separate MegaCam pointings, including the
 COSMOS field (D2) which is utilized in this work. When available,
further near-infrared data from a variety of different observing pro-
grammes in each field was utilized (e.g. from the UltraVISTA DR1
and VISTA VIDEO surveys). Willott et al. (2013) were sensitive
to the very brightest *z* ≃ 6 galaxies, with a z-band magnitude limit

of 25.3 (in a 2 arcsec diameter circular aperture), and the impo-
sition of a strict *i* − *z* > 2 criterion resulted in a higher median
redshift of the candidates with *z* ≳ 5.8 (see Fig. 5). In our final
*z* ≃ 6 sample, we find 7 of the 15 galaxies presented in Willott
et al. (2013) in the COSMOS/UltraVISTA field. One object was re-
moved only in the final stage of potential star removal, where it was
classified as a possible brown dwarf with χ<sub>*</sub><sup>2</sup> = 9.1. Another ob-
ject in the sample, WMH18, was not selected in our initial z’-band
catalogue and inspection of the imaging shows that it is heavily
blended with a low-redshift galaxy 3 arcsec away. The six further
missing objects however, were initially selected in our sample but
subsequently removed.

The removal of four of the objects (WMH11, WMH12, WMH19 and WMH21) is simply because they lie in the shallower DR1 region of UltraVISTA, where we applied a conservative magnitude limit of $z' < 25.0$. We nevertheless extracted the photometry for these objects and performed SED fitting as for our $z \simeq 6$ objects, finding that all the objects excluding WHM11 would indeed have been selected as high-redshift galaxies. WMH11 has low-level flux in the optical bands and was excluded based on our local depth cuts in the $u^*$ and $g$ bands.

Of the final two objects, WMH23 was also removed based on a
marginal detection in the *u*\* band, although the results of SED fitting
of this object also show it to be a plausible high-redshift candidate.
We note here that occasionally genuine *z* ≃ 6 objects will be lost
during our selection process as a result of our optical drop-out
criterion, however this incompleteness is carefully simulated and
included in our LF analysis. Finally, object WMH14 was excluded
because it has a marginally unacceptable χ² = 12.6, which exceeds
our formal good fit criterion (χ² < 11.3). Inspection of the imaging
reveals the object to be close to a low-redshift companion, which is
likely contaminating the photometry for this galaxy.

Two of the 15 objects presented in the *z* ≃ 6 sample from Willott et al. (2013) in the COSMOS field have spectroscopic confirmations, WMH13 at *z*<sub>spec</sub> = 5.983 and WMH15 at *z*<sub>spec</sub> = 5.847. Reassuringly, we find photometric redshifts of *z* = 5.92<sup>+0.13</sup><sub>-0.08</sub> and *z* = 5.99<sup>+0.09</sup><sub>-0.23</sub>, respectively.

We therefore find good agreement with the bright sample of *z* ≃ 6 galaxies found in Willott et al. (2013) with 12 out of the 15 objects present being consistently classified as high-redshift galaxies here. However as presented in Section 5, the derived rest-frame UV LF from Willott et al. (2013) falls below our determination, with the difference in derived number densities suggesting that a large proportion (~60 per cent at *M*<sub>UV</sub> ≃ −22.0) of objects selected in this work are not present in the Willott et al. (2013) sample. Using the new, deeper, optical and near-infrared photometry available for the UltraVISTA/COSMOS field in this study we applied the Willott et al. (2013) selection criterion to the full *z* ≃ 6 sample we derive using photometric redshift fitting. Of the 159 LBGs we find in the UltraVISTA/COSMOS field, 31 are sufficiently bright (*z* < 25.3, 2-arcsec diameter circular apertures) to have been included by Willott et al. (2013), and of these, 16 objects now also pass the *i* − *z* > 2 criterion imposed by Willott et al. (2013). Although these 16 objects pass the required selection criterion, only five were previously found by Willott et al. (2013), suggesting that the origin of the discrepancy between the *z* ≃ 6 LF determinations by Willott et al. (2013) and this work is a result of the selection procedure employed using the shallower CFHTLS z-band and near-infrared imaging. In particular, the strict *i* − *z* > 2 colour selection criterion applied by Willott et al. (2013) was not robustly applicable given the relative depths of the *i*- and *z*-band imaging available, likely resulting in the exclusion of many genuine LBGs at *z* ≃ 6 as demonstrated above.
<figure><img src="image_8.png" /><figcaption>**Figure 7.** The rest-frame UV slope ($\beta$) measured for our sample in the reduced redshift range $5.7 < z < 6.3$, plotted against absolute UV magnitude ($M_{UV}$). The galaxies shown have a detection in the Y, J, H or Ks bands at greater than $5\sigma$ significance. Objects in the UltraVISTA/COSMOS field are shown as grey circles, with the galaxies previously detected by Willott et al. (2013) highlighted in red, and the objects in the UDS/SXDS field are shown as dark grey squares. The colour–magnitude relation at $z \simeq 5$ determined by Rogers et al. (2014) is shown as the blue line, and the derived $\beta_{UV}$ values for the two brightest $z \simeq 7$ galaxies from Bowler et al. (2014) are shown as open blue circles.</figcaption></figure>

### 6.4.3 Bowler et al. (2014)

In Bowler et al. (2014), we presented a sample of $z \simeq 7$ LBGs found within the UltraVISTA/COSMOS and UDS/SXDS data sets utilized in this work. As part of the selection procedure, which was primarily aimed at finding $6.5 < z < 7.5$ galaxies, candidates with $z > 6$ were retained and presented if the presence of Ly $\alpha$ emission in the spectrum could shift the object into the primary redshift window. We therefore expect some overlap with the sample presented in Bowler et al. (2014). Taking the subset of the 34 objects presented in Bowler et al. (2014) that have $z' < 26.0$, we find six galaxies in the UltraVISTA field and 1 in the UDS (the spectroscopically confirmed LAE ‘Himiko’ at $z = 6.595$). Comparing to the $z \simeq 6$ sample, we find all seven objects (Himiko was identified and removed from the final sample), with photometric redshifts that agree within the errors. The full recovery of these objects in this work is a strong vindication of our selection methodology, as the samples presented in Bowler et al. (2014) were selected in a different band (Y and J band selected) and refined using a slightly different SED fitting analysis.

## 6.5 Rest-frame UV slope ($\beta_{UV}$)

The rest-frame UV slope, $\beta_{UV}$, of each galaxy was measured by fitting a power law (parametrized as $F_{\lambda} \propto \lambda^{\beta_{UV}}$) to the YJHK photometry for each object. The $z'$ band was excluded from the fitting process, as at $z \gtrsim 5.8$ the Lyman-break is moving through the filter and furthermore there could be contamination by Lyman $\alpha$ emission. In Fig. 7, we show the derived $\beta_{UV}$ values for the 87 LBGs in the $5.7 < z < 6.3$ range that have detections in one near-infrared band at greater than $5\sigma$ significance, where the reduced redshift range was chosen to allow more direct comparison with the results of Willott et al. (2013). Using a subset of the 40 objects covered by sufficiently deep near-infrared data, Willott et al. (2013) was able to measure $\beta_{UV}$ using an identical method, finding a mean value of $\beta_{UV} = -1.38 \pm 0.2$, which is redder than that found for fainter

galaxies which tend to exhibit $\beta_{UV} \simeq -2.0$ (Dunlop et al. 2013; Rogers, McLure & Dunlop 2013; Bouwens et al. 2014). We measured $\beta_{UV}$ for the seven objects from the Willott et al. (2013) that are present in our sample, using the deeper near-infrared imaging now available, and highlight these objects as red points in Fig. 7. Excluding the faintest object (WHM22) that has a poorly constrained $\beta_{UV}$ value (and is not included in Fig. 7 due to low significance near-infrared detections), we find a mean $\beta_{UV} = -1.55 \pm 0.05$ for the subset of the Willott et al. (2013) sample, where the error is the standard error on the mean. For the galaxies in our sample with $M_{UV} < -22.0$, we find on average slightly bluer values with a mean $\beta_{UV} = -1.8 \pm 0.1$ (excluding the brightest object as discussed below). For the brightest galaxies in our sample therefore, we do find redder rest-frame UV slopes than in similarly bright galaxies at $z \simeq 7$ (by $\Delta\beta_{UV} \simeq 0.2$), however our full sample does not show the particularly red average $\beta_{UV}$ found by Willott et al. (2013). At $M_{UV} \lesssim -22$ where our $\beta_{UV}$ values are sufficiently accurate, our results follow the colour–magnitude relation derived at $z \simeq 5$ by Rogers et al. (2014) well, and tentatively show an increased scatter as expected from their analysis, although careful modelling of potential biases in the selection process are required to show this quantitatively.

The very brightest object in our sample lies within the UltraVISTA/COSMOS field and shows a particularly red slope ($\beta_{UV} = -1.1 \pm 0.2$), in contrast to the bluer values ($\beta_{UV} \simeq -2.0$) found for the very brightest $z \simeq 7$ galaxies (Bowler et al. 2014). Larger samples are clearly needed, however an increase in dust obscuration for the brightest objects in our sample is one theoretical process by which the number density at the bright end of the rest-frame UV LF could be suppressed (e.g. see comparison with theoretical models in Section 8.3). The rest-frame UV slope can be linked to the predicted attenuation in the rest-frame UV according to the ‘IRX–$\beta_{UV}$ relation’ (Meurer, Heckman & Calzetti 1999), where the IRX is the ‘infrared excess’ and is given by $L_{IR}/L_{UV}$. While a $\beta_{UV} \simeq -2.0$ as observed at $z \simeq 7$ suggest little or no dust attenuation, if the updated relation of Takeuchi et al. (2012) is assumed, we predict an attenuation in the range $A_{UV} \simeq 1.5–0.7$ for rest-frame UV slopes of $\beta_{UV} = -1.0$ to $-1.5$ (as we observe in some of the brightest galaxies in our sample). The predicted total far-infrared luminosity implied for objects in this $\beta_{UV}$ range according to the IRX–$\beta_{UV}$ relation is $L_{TIR} \simeq 0.4–1.5 \times 10^{11}$ L⊙, similar to the observed luminosity of two $z \simeq 6$ galaxies found by Willott et al. (2015).

# 7 THE LUMINOSITY FUNCTION

In Fig. 8 and Table 1, we present our measured rest-frame UV LF at $z \simeq 6$. The binned LF points were derived from 127 luminous LBGs with $M_{UV} \le -21.125$ found within the combined UltraVISTA and UDS imaging, in the redshift range $5.7 < z_{phot} < 6.3$. The median redshift of the galaxies included in our LF determination is $z_{med} = 5.9$. Comparing to previous determinations of the $z \simeq 6$ LF from a compilation of HST imaging from Bouwens et al. (2007, 2015), the larger area available from the combined UltraVISTA/COSMOS and UDS/SXDS fields allows us to more accurately probe lower space densities of objects (down to $\sim 1 \times 10^{-7}$ mag$^{-1}$ Mpc$^{-3}$). Furthermore, the error bars on our brightest points show that we are able to probe the number densities of the brightest galaxies more accurately than the previous determinations using ground-based imaging surveys from Willott et al. (2013) and McLure et al. (2009).
<figure><img src="image_9.png" /><figcaption>**Figure 8.** The rest-frame UV LF at $z \simeq 6$, showing our results from the combined UltraVISTA/COSMOS and UDS/SXDS data sets as the black circles. The redshift range is restricted to $5.7 < z < 6.3$ as in McLure et al. (2009), which results in a median redshift of $z_{med} \sim 5.9$. The inset plot shows our results from the UltraVISTA/COSMOS and UDS/SXDS fields separately as the red and orange points, respectively, with the best-fitting Schechter functions overplotted in identical colours. Here, we have not corrected the absolute magnitudes for dust extinction. We have shifted the Willott et al. (2013) points by 0.05 mag faintward for clarity.</figcaption></figure>

<table><caption>**Table 1.** The binned rest-frame UV LF points at $z_{med} \sim 5.9$ from this work, as shown in Fig. 8. The upper section of the table shows the results from the UltraVISTA/COSMOS and UDS/SXDS fields combined, with the middle and lower sections showing the results including only the UltraVISTA/COSMOS and UDS/SXDS fields, respectively (e.g. the inset in Fig. 8). Columns 1 and 2 show the central $M_{UV}$ of the bin and the width, where we calculate the $M_{UV}$ by integrating the best-fitting SED through a top-hat filter centred on 1500 Å with a width of 100 Å. The weighted bin centre is shown in Column 3, given by the median completeness corrected $M_{UV}$ of the galaxies in that bin. The number density is shown in Column 4 and the number of galaxies in each bin is shown in Column 5.</caption><thead><tr><th>M<sub>UV</sub><br>(mag)</th><th>ΔM<sub>UV</sub><br>(mag)</th><th>M<sub>UV,w</sub><br>(mag)</th><th>φ<br>(mag<sup>-1</sup> Mpc<sup>-3</sup>)</th><th>#</th></tr></thead><tbody><tr><td>-22.625</td><td>0.500</td><td>-22.52</td><td>1.16 ± 0.67 × 10<sup>-6</sup></td><td>3</td></tr><tr><td>-22.125</td><td>0.500</td><td>-22.08</td><td>5.98 ± 1.64 × 10<sup>-6</sup></td><td>17</td></tr><tr><td>-21.750</td><td>0.250</td><td>-21.74</td><td>1.90 ± 0.41 × 10<sup>-5</sup></td><td>23</td></tr><tr><td>-21.500</td><td>0.250</td><td>-21.49</td><td>3.92 ± 0.70 × 10<sup>-5</sup></td><td>35</td></tr><tr><td>-21.250</td><td>0.250</td><td>-21.22</td><td>9.14 ± 1.39 × 10<sup>-5</sup></td><td>49</td></tr><tr><td>-22.625</td><td>0.500</td><td>-22.52</td><td>2.20 ± 1.27 × 10<sup>-6</sup></td><td>3</td></tr><tr><td>-22.125</td><td>0.500</td><td>-22.11</td><td>7.60 ± 2.92 × 10<sup>-6</sup></td><td>10</td></tr><tr><td>-21.750</td><td>0.250</td><td>-21.75</td><td>2.92 ± 0.76 × 10<sup>-5</sup></td><td>16</td></tr><tr><td>-21.500</td><td>0.250</td><td>-21.48</td><td>4.76 ± 1.17 × 10<sup>-5</sup></td><td>19</td></tr><tr><td>-21.250</td><td>0.250</td><td>-21.22</td><td>1.34 ± 0.25 × 10<sup>-4</sup></td><td>31</td></tr><tr><td>-22.125</td><td>0.500</td><td>-22.04</td><td>4.54 ± 1.74 × 10<sup>-6</sup></td><td>7</td></tr><tr><td>-21.625</td><td>0.500</td><td>-21.57</td><td>2.15 ± 0.47 × 10<sup>-5</sup></td><td>23</td></tr><tr><td>-21.250</td><td>0.250</td><td>-21.22</td><td>5.54 ± 1.45 × 10<sup>-5</sup></td><td>18</td></tr></tbody></table>

The use of two independent fields in the present analysis also allows us to probe the cosmic variance and potential large-scale structure effects in the number counts of bright objects. Willott et al. (2013) pointed out an overdensity of $z \simeq 6$ galaxies in the COSMOS/UltraVISTA (CFHTLS D2) field, an observation that we are able to confirm using $\times 10$ the number of LBGs. The inset plot in Fig. 8 shows our results at $z \simeq 6$ determined from the two fields separately. There is a clear excess of galaxies in the COSMOS/UltraVISTA survey as compared to the UDS/SXDS field, which is present over the full magnitude range probed. The discrepancy is most noticeable in the faintest bin at $M_{UV} = -21.5$ where the number counts differ by a factor of $\gtrsim 2$. The discrepancy between the fields is also evident in the observed $M_{UV}$ histograms in Fig. 6. We note that our faintest bin is our most uncertain, however if lensing by foreground structures (as discussed in Section 7.3) was a factor in the increased number counts in UltraVISTA/COSMOS then we would expect the largest difference at the faint end of our sample, due to the rapidly increasing number counts of objects faintward of the limiting magnitude.

## 7.1 Comparison to previous work

Previous determinations of the $z \simeq 6$ rest-frame UV LF from McLure et al. (2009), Willott et al. (2013), Bouwens et al. (2007, 2015) and Finkelstein et al. (2014) are shown in Fig. 8, with the best-fitting Schechter functions from McLure et al. (2009) and Bouwens et al. (2015) also shown. The derived data points faintward of $M_{UV} = -20.5$ show good agreement (with the exception of Finkelstein et al. 2014 in the range $-20.0 \gtrsim M_{UV} \gtrsim -19.0$), although there is tension between the Schechter function fits derived by McLure et al. (2009) and Bouwens et al. (2015). At
$M_{UV} \lesssim -21$ however, large discrepancies between the determination of Bouwens et al. (2015) and the ground-based results of McLure et al. (2009) and Willott et al. (2013) become evident. Using our wide-area and deep data set, we find a $z \simeq 6$ LF that lies approximately mid-way between the previous determinations.

### 7.1.1 McLure et al. (2009)

Using the DR1 of the UKIDSS UDS near-infrared data in the UDS/SXDS field (Section 6.4.1), McLure et al. (2009) determined the $z \simeq 5$ and 6 LF following an analogous methodology to this work. Comparing our determination of the LF from the combined UltraVISTA/COSMOS and UDS/SXDS data sets to the results of McLure et al. (2009) however, we find a significantly higher number density of bright LBGs. The difference is further highlighted because the Schechter function fit of McLure et al. (2009) undershoots the brightest two binned points from their work. Considering our derived LF from the two fields separately (inset of Fig. 8) sheds light on the discrepancy, as the results from this work exclusively in the UDS/SXDS field are in fact in fair agreement (within $\simeq 1\sigma$) with the data points of McLure et al. (2009). Overall there is still a lower number of objects found by McLure et al. (2009), which is likely due to the shallower near-infrared photometry available, resulting in a more conservative selection procedure being employed to ensure the removal of low-redshift interlopers or dwarf stars without secure near-infrared colours.

### 7.1.2 Willott et al. (2013)

The results of Willott et al. (2013), which were derived from 40 galaxies at $z \simeq 6$ found in the four CFHTLS fields (as described in Section 6.4.2), are in good agreement with those of McLure et al. (2009) and therefore fall below our determination. The four independent fields analysed should make the Willott et al. (2013) result more robust to the potentially large cosmic variance in the number counts of bright LBGs as found in this study. Furthermore, one of the CFHTLS fields analysed by Willott et al. (2013) overlaps with the UltraVISTA/COSMOS field, which we find to be overdense compared to the UDS/SXDS, a result also corroborated by Willott et al. (2013) who found 15 galaxies in UltraVISTA/COSMOS as compared to ~8 in each of the three other fields. We note that two of the four fields used only had shallow J-band data available (D3 and D4), and hence the number counts here are the most uncertain (8 and 9 objects, respectively). Taking these uncertainties into account, it remains possible that either the UltraVISTA/COSMOS field is overdense or the true global number densities of these galaxies was not adequately probed by CFHTLS due to the inhomogeneous data sets (e.g. the four CFHTLS fields should have had ~11.5 galaxies in each). Potential biases in the Willott et al. (2013) selection were highlighted in the comparison between the Willott et al. (2013) sample and that derived from this work, and in our rest-frame UV slope measurements (see Section 6).

In particular, the very brightest data point derived by Willott et al. (2013) at $M_{UV} = -22.5$ disagrees strongly with our derived number density of galaxies. However, as discussed in Bowler et al. (2014), Willott et al. (2013) do not directly measure such a low space density, rather they infer $\phi(M)$ using a maximum likelihood approach. In Bowler et al. (2014), we estimated the space density of galaxies in the Willott et al. (2013) analysis at $M_{UV} = -22.5$ using the brightest two objects presented (WHM5 and WHM29, neither of which is in the COSMOS/UltraVISTA field). The result ($\phi \simeq$

$2.0 \pm 1.4 \times 10^{-7} \text{ mag}^{-1} \text{ Mpc}^{-3}$) is still in tension with our brightest point, however the deeper z'-band and near-infrared imaging utilized here would suggest that our results are more reliable.

### 7.1.3 Bouwens et al. (2015)

Using a combination of the HUDF and two parallel fields, the Early Release Strip (ERS) and the CANDELS survey fields, Bouwens et al. (2015) selected a large sample of LBGs at $z \simeq 6$. The HST surveys used in the analysis covered a total of 0.2 deg$^2$ on the sky. The LF determination derived from the full set of fields available is included in Fig. 8 along with the best-fitting Schechter function. The data points and the fit lie above our determination from the UltraVISTA/COSMOS and UDS/SXDS fields, and furthermore the simple evolving LF model we use assuming the Bouwens et al. (2015) LFs at $z=5$ and 6 predicts approximately double the number of galaxies than we find in these fields (see Fig. 6). Due to the relatively small area of the fields used by Bouwens et al. (2015), the number of $z \simeq 6$ galaxies at $M_{UV} \simeq -22.5$ is poorly constrained, however at $M_{UV} = -22.0$ there is a clear tension with our results.

At the very bright end of the LF however, the small area probed by Bouwens et al. (2015) results in the samples being vulnerable to strong cosmic variance, as is evident from the distribution of the number counts of bright objects across the fields used (fig. 14 and table 8 of Bouwens et al. 2015), which can vary by up to 50 per cent. Furthermore, the small number statistics result in a large Poisson error as demonstrated by the brightest point shown in Fig. 8. Hence, the results of Bouwens et al. (2015) cannot be relied upon in this magnitude regime ($M_{UV} \lesssim -21$). The area, depth and homogeneity of the data sets utilized in this work enables a significantly improved determination of the bright end of the LF at $z \simeq 6$ than can be provided by the combination of current and future HST surveys. Finally, for the Bouwens et al. (2015) results to be correct at the bright end of the LF we would expect to find roughly double the number of LBGs, which at $z' \sim 25.0$ would be individually detected at a significance of $>20\sigma$.

### 7.1.4 Finkelstein et al. (2014)

From a combination of the UDF and parallels, the two GOODS fields and parallel imaging taken as part of the Hubble Frontier Fields programme, Finkelstein et al. (2014) selected a sample of $z = 4-8$ LBGs using a photometric redshift fitting methodology. In total, the area included was $\simeq 300 \text{ arcmin}^2$, and the derived LF points are shown in Fig. 8. The results of Finkelstein et al. (2014) are in excellent agreement with our determination of the bright end of the LF, however they appear to diverge from the results of Bouwens et al. (2007, 2015) at fainter magnitudes, and furthermore show a step at $M_{UV} \simeq -19.0$. Although the Finkelstein et al. (2014) analysis used a subset of the larger area of imaging used by Bouwens et al. (2015), the results are in good agreement with the ground-based analysis presented here, and could indicate large overdensities in the additional fields incorporated by Bouwens et al. (2015) or contamination of the Bouwens et al. (2015) sample by brown dwarfs (see the discussion in Finkelstein et al. 2014).

## 7.2 Gravitational lensing by foreground galaxies

As in Bowler et al. (2014), to determine whether the bright galaxies we find are only present in our sample as a result of ‘moderate’
gravitational lensing by foreground galaxies, we estimated the ex-
pected magnification due to galaxies along the line of sight to each
z ≃ 6 object in our sample. Note that strong gravitational lens-
ing by galaxies directly along the line of sight is ruled out by our
deep optical non-detections. The full details of our approach are
described in Appendix C. We find that the galaxies in our sam-
ple show magnifications in the range Δm<sub>AB</sub> = 0.0–0.6 mag, with
a median magnification of Δm<sub>AB</sub> ≃ 0.1–0.2. However, the sample
does not show ‘excess’ lensing compared to random positions in the
UltraVISTA/COSMOS or UDS/SXDS imaging, and no correlation
between the magnification and the absolute magnitude of the galaxy
was identified. As the median magnification is relatively small (less
than the LF bins we use) and likely an upper limit as a consequence
of the assumed Faber–Jackson relation in the calculation of the
line-of-sight mass distribution, we do not correct for the lensing
effect in our LF analysis. Furthermore, previous LF determinations
at high redshift do not generally correct for the magnification, and
recent work by Mason et al. (2015) has shown that the effect is
small in the magnitude range currently probed by observations,
and that the corrected Schechter function parameters agree with the
results assuming no lensing effect within the errors. The accurate
determination of the lensing magnification and the impact on the
observed LF will be extremely important however for future, wider
area surveys, such as those performed by Euclid.

7.3 The potential effect of large-scale structure

As described in Section 6, we find a difference in the number
counts of galaxies between the two fields studied, with the UltraV-
ISTA/COSMOS field containing ≃1.8 times the number of objects
found in the UDS/SXDS field. Cosmic variance struggles to account
for the difference, and hence further investigation is warranted. The
UltraVISTA/COSMOS field has been known to harbour an unusual
richness of structure as measured by clustering analyses, particu-
larly at z ≃ 1 (McCracken et al. 2007; Meneux et al. 2009; Skibba
et al. 2014). The z ≃ 6 objects here show no evidence for ‘excess’
lensing by foreground galaxies as compared to a random position
on the sky, and the UltraVISTA/COSMOS fields shows a simi-
lar predicted magnification distribution to the UDS/SXDS field.
However, such a calculation does not include gravitational lens-
ing by galaxy clusters. A full analysis of the lensing cross-section
for the two fields is beyond the scope of this work, however an
estimate of the magnification by the total matter in clusters can
be made using the X-ray observations available in the fields. The
UltraVISTA/COSMOS and UDS/SXDS fields have been observed
to similar depths (≈2–3 × 10<sup>-15</sup> ergs cm<sup>-2</sup> s<sup>-1</sup> at 0.5–2 keV) with
*XMM–Newton*, and the X-ray bright clusters have been identified
by Finoguenov et al. (2007, 2010) in each field, respectively. From
the X-ray luminosity, we calculated M<sub>200</sub><sup>4</sup> using the correlation pre-
sented by Rykoff et al. (2008). The magnification due to the total
matter present in clusters was then calculated using the singular
isothermal sphere (SIS) approximation as described in Section 7.2,
summing the contribution from the clusters at the position of each
z ≃ 6 galaxy in our sample.

The cluster X-ray LF reveals a higher density of X-ray lumi-
nous clusters in the UltraVISTA/COSMOS field compared to the
UDS/SXDS (Finoguenov et al. 2010), and calculating the predicted
magnification from the clusters using the simple method described

shows that the additional lensing magnification due to the dif-
ference in number density of high-mass clusters is of the order
of ~0.05 mag. Although small, such a magnification could have a
significant effect on the determination of the bright end of the LF
due to the declining number counts, and could be the origin of the
discrepancy we find close to the 5σ limit of our survey. Correcting
our derived LF points by ~0.05 mag faintward would not impact
on our conclusions described below, and would further strengthen
the derived evolution in M*.

Conversely, narrow-band studies of the UDS/SXDS field have
revealed large voids in the distribution of z = 5.7 galaxies (Ouchi
et al. 2005), with comoving sizes of the order of 10–40 Mpc which
corresponds to ~4–17 arcmin on the sky (see fig. 2 of Ouchi et al.
2005). Hence, it remains possible that the z ≃ 6 LF derived from
the UDS/SXDS field is biased low as a result of these voids at
z = 5.7 ± 0.1.

The discovery of cosmic variance between degree-scale fields at
z ≃ 6 in this work further highlights the necessity of using multi-
ple large fields to robustly determine the number density of bright
galaxies. A single, or even a collection of CANDELS fields could be
strongly influenced by this large-scale structure as a result of their
small size. Given the potential foreground structure in the COS-
MOS/UltraVISTA field and/or lack of structure in the UDS/SXDS
field discussed above, future work on additional fields will be re-
quired to shed light on the origin of the discrepancy between them.

8 FORM AND EVOLUTION OF THE UV LF

8.1 The functional form of the z ≃ 6 LF

The galaxy LF at high redshift is commonly fitted with a Schechter
function, with the resulting best-fitting parameters used to determine
the dominant form of the evolution(e.g. Bouwens et al. 2007, 2012,
2015; McLure et al. 2013; Finkelstein et al. 2014). A Schechter
function (functional form detailed in Appendix A) tends to describe
well the mass and LFs at low redshift (e.g. Baldry et al. 2012; Love-
day et al. 2012; Kelvin et al. 2014; Mortlock et al. 2014). Similarly
at higher redshifts, a Schechter function has also provided a good fit
to the observations (e.g. McLure et al. 2013; Schenker et al. 2013;
Finkelstein et al. 2014; Bouwens et al. 2015), although qualitatively
this is not surprising given the often reduced dynamic range and
increased errors in LF determinations of z > 4 LBGs. In contrast,
at z ≃ 7 Bowler et al. (2014) showed that a DPL provides a better
description of the rest-frame UV LF, from measurements of the
number of bright galaxies with M<sub>UV</sub> ≲ −21.5. Furthermore, theo-
retical models do not generally predict a Schechter function-type
form without the addition of uncertain dust or feedback processes
(e.g. Cai et al. 2014; Dayal et al. 2014, see Section 8.3). It is there-
fore important to consider alternatives to a Schechter function, if
the data warrant such a conclusion, and to be aware that assuming
a given functional form for the LF can potentially hide subtleties in
any derived LF evolution (Jaacks, Thompson & Nagamine 2013).

Indeed at lower redshift, a broken power law or DPL has been
shown to provide a better description of the LF of galaxies in groups
(e.g. Tempel et al. 2014, 2009), the far-infrared galaxy LFs (e.g.
Soifer et al. 1987) and the LFs of quasars (e.g. McGreer et al. 2013).
A shallower decline at the bright end of the LF than expected from
a Schechter function has also been found in the NUV LF from the
Wiggles survey (Jurek et al. 2013) and in the H α LF (Gunawardhana
et al. 2013). From a theoretical standpoint, Salim & Lee (2012)
has shown that scatter in the mass-to-light ratio of galaxies will
naturally lead to a shallower function when measuring the LF. If the

<footnote n="4">⁴ The mass enclosed within a sphere of radius $R_{200}$, which contains a density of 200 $\rho_{\text{critical}}$ at that redshift.</footnote>
mass function is well described by a Schechter function, then one would expect any LF measurement that directly traces the galaxy mass (e.g. the rest-frame optical) would also follow such a form (Bernhard et al. 2014). However, when the luminosity of the galaxies in question is measured from wavelength regimes that are dominated by recent star formation, and hence trace the star formation rate (SFR) rather than the mass directly, then the shape of the observed LF will be convolved with the mass-to-SFR relation of the galaxies (Salim & Lee 2012). The scatter in this relation tends to flatten the slope of the LF, resulting in a shallower function. Salim & Lee (2012) showed that a Saunders functional form (detailed in Appendix A, derived originally by Saunders et al. 1990 to model the 60 µm LF), where the bright end of the LF declines as a lognormal, provides an improved fit to an LF that follows the SFR of the galaxies.

In addition, there are several observations effects and biases that can also lead to an apparent deviation from a Schechter function form. Flux boosting of galaxies close to the limiting depth of a survey can cause inaccuracies in the derived LF unless properly accounted for with simulations, and gravitational lensing can strongly affect the observed LF (Wyithe et al. 2011) at the very bright end. The number counts of bright galaxies can also be affected by quasar contamination (e.g. Bian et al. 2013), although the number densities of quasars appear too low to strongly influence the $z > 5$ galaxy LF (Bowler et al. 2014). Furthermore, we consider and rule out a strong gravitational lensing effect on our derived LF, and the effect of photometric scatter is accounted for by our completeness simulations. Finally, cosmic variance and small number statistics can influence the derived LF and best-fitting functional parameters (e.g. Trenti & Stiavelli 2008; Eardley et al. 2015). By analysing two independent, degree-scale, fields, we are able to directly measure the effect of cosmic variance on the bright-end of the LF using a larger sample of bright galaxies than previously obtained at $z \simeq 6$.

The bright end of the LF is sensitive to astrophysical effects such as dust obscuration and feedback mechanisms (we discuss theoretical predictions in Section 8.3), and it is observations of the number density of rare and bright galaxies that place the tightest constraints on the position of the knee in the LF. The typical errors on the determination of the bright end of the LF at high redshift, and more severely the systematic difference between different studies, make a secure determination of the form of the LF challenging (see Fig. 12 or the compilation of results by Bouwens et al. 2015). We therefore investigate the functional form of the rest-frame UV LF using a subset of the observed LF points at $z \simeq 6$, using the determination from this work at $M_{UV} \lesssim -21.5$ due to the superior depth and/or area of the combined UltraVISTA/COSMOS and UDS/SXDS fields when compared to previous studies. For the faint end of the LF, we used the results derived by Bouwens et al. (2007) given the slight discrepancy we find with the Bouwens et al. (2015) results at $z \simeq 6$. We fit these determinations of the LF with a Schechter, DPL and Saun-

<figure><img src="image_10.png" /><figcaption>**Figure 9.** The $z \simeq 6$ LF points from this work and Bouwens et al. (2007), with the best-fitting Schechter (red solid line), DPL (black dashed line) and Saunders function (blue dot-dashed line) shown. The grey shaded region shows the $1\sigma$ confidence interval on the DPL fit. The dark matter HMF, scaled as described in the text, is shown as the solid grey line.</figcaption></figure>

ders functional form and show the resulting fits in Fig. 9. The DPL provides a slightly better fit to the data, even when corrected for the additional parameter available in the fitting, although a Schechter function also provides a good fit to the data. As can be seen from the best-fitting parameters shown in Table 2, the assumed functional form changes the derived characteristic magnitude $M^*$. Although there is clearly a change in slope of the LF at brighter magnitudes, the exact position of the ‘break’ is not clear from the current data and therefore depends strongly on the function assumed. The Saunders function also provides a good fit to the data, however the function poorly constrains the characteristic magnitude $M^*$. The uncertainty is a result of the parameter $\sigma$, which provides additional freedom in the shape of the bright end of the function. Hence, we only present the results of fitting with the Schechter function and DPL in the next section.

In Bowler et al. (2014), we found good agreement between the observed rest-frame UV LF of galaxies at $z \simeq 7$ and the shape of the HMF when scaled using a constant mass-to-light ratio. Evolving the HMF according to the Reed et al. (2007) model using the online tool ‘HMFcalc’ (Murray, Power & Robotham 2013), and using the same scaling as at $z \simeq 7$, we find the curve shown in Fig. 9. Again, there is a good agreement between the simple LF predicted from the HMF, especially considering that the only evolution incorporated is due to dark matter halo build-up. There is

<table><caption>**Table 2.** The best-fitting parameters derived from fitting the observed $z \simeq 6$ LF. The results are shown for a Schechter function, double power-law and Saunders function, which are displayed in Fig. 9. The errors are the $1\sigma$ errors on that parameter, where the $\chi^2$ value has been minimized over all other parameters.</caption><thead><tr><th>Function</th><th>φ*<br>(mag<sup>-1</sup> Mpc<sup>-3</sup>)</th><th>M*<br>(mag)</th><th>α</th><th>β</th><th>σ</th></tr></thead><tbody><tr><td>Schechter function</td><td>5.7<sup>+2.7</sup><sub>-2.0</sub> × 10<sup>-4</sup></td><td>-20.77<sup>+0.18</sup><sub>-0.19</sub></td><td>-1.88<sup>+0.15</sup><sub>-0.14</sub></td><td>—</td><td>—</td></tr><tr><td>Double power law</td><td>1.9<sup>+1.2</sup><sub>-0.8</sub> × 10<sup>-4</sup></td><td>-21.20<sup>+0.22</sup><sub>-0.22</sub></td><td>-2.10<sup>+0.16</sup><sub>-0.14</sub></td><td>-5.1<sup>+0.5</sup><sub>-0.6</sub></td><td>—</td></tr><tr><td>Saunders function</td><td>3.0<sup>+6.1</sup><sub>-2.3</sub> × 10<sup>-4</sup></td><td>-21.04<sup>+0.91</sup><sub>-1.12</sub></td><td>-2.01<sup>+0.21</sup><sub>-0.17</sub></td><td>—</td><td>0.2<sup>+0.1</sup><sub>-0.1</sub></td></tr></tbody></table>
<figure><img src="image_11.png" /><figcaption>**Figure 10.** The best-fitting DPL and Schechter function fits to a selection of observations of the rest-frame UV LF at $z = 5, 6$ and $7$ (green, black and blue lines/points, respectively). For each redshift, the best-fitting function is shown as the solid line and the second best-fitting function is shown as a dotted line. The inset table shows the reduced $\chi^2$ for each fit. At $z \simeq 5$, the fitted points are from van der Burg, Hildebrandt & Erben (2010), Bouwens et al. (2015) and Finkelstein et al. (2014), and at $z \simeq 6$ the LF determination from this work is combined with the results from Bouwens et al. (2007). Finally, at $z \simeq 7$ we fit to the LF determined by Bowler et al. (2014) and McLure et al. (2013). The best-fitting parameters are presented in Table 3.</figcaption></figure>

a clear deficit of galaxies at the faint end, as would be expected from models of supernova feedback which rapidly quench the star formation in low-mass galaxies. In contrast to the results at $z \simeq 7$ from Bowler et al. (2014), the bright end of the rest-frame UV LF at $z \simeq 6$ also shows a deficit of objects compared to the underlying halo distribution. Regardless of the exact scaling of the HMF into luminosity space, the comparison indicates that the bright-end slope of the LF is now steeper than the HMF at $z \simeq 6$. Although tentative, this steepening could indicate that we are now observing the build-up of dust or the onset of AGN feedback (or some other mass quenching mechanism) in the brightest galaxies at $z \simeq 6$ as discussed further in Section 8.3.

## 8.2 Evolution of the LF from $z \simeq 5-7$

Observations of the rest-frame UV galaxy LF between $z \simeq 7$ and $z \simeq 5$ such as those shown in Fig. 10 reveal a strong evolution in the number densities of galaxies at high redshift. This work at $z \simeq 6$ and the analogous work at $z \simeq 7$ presented in Bowler et al. (2014) allow the form and evolution of the bright end of the LF to be tightly constrained, and potential evolution of the functional form to be investigated. As can be seen in Fig. 10, the observed LFs from $z \simeq 5$ to $z \simeq 7$ show little evolution faintward of $M_{UV} \simeq -19.0$, however brighter than $M_{UV} \simeq -20.0$ there is clear evolution in the number densities of galaxies, with bright LBGs ($M_{UV} \simeq -21.5$) at $z \simeq 7$ being an order of magnitude less numerous than $z \simeq 5$ galaxies of the same luminosity. Determining the exact evolution of the LFs however is not straightforward, as although the general agreement between different studies is good, there are systematic differences between the results that are larger than the errors estimated by each individual study. Furthermore, the methodology used to derive the

Schechter function parameters (i.e. $\chi^2$ versus maximum likelihood fitting) can also introduce systematic differences between studies based on small samples. Hence, the exact parametrization of the LF derived from each analysis can disagree; for example at $z \simeq 5$ van der Burg et al. (2010) find a best-fitting characteristic magnitude $M^* = -20.93^{+0.10}_{-0.11}$ from an analysis of the CFHTLS data using a colour-colour selection, whereas the fit to the McLure et al. (2009) results gives $M^* = -20.73 \pm 0.11$. We therefore only fit to a subset of the available rest-frame UV LF points from different studies as motivated below, although we display a large compilation of studies in Fig. 12.

To attempt to quantify the evolution of the bright end of the LF, we fit DPL and Schechter functions to a subset of derived rest-frame UV LF points from $z \simeq 5-7$. A simple $\chi^2$ minimization method was used, and the errors were found as the value of the parameter which gives a $\Delta\chi^2 = 1.0$, minimized over all other parameters. We combine our results with those of Bouwens et al. (2007) at $z \simeq 6$, again excluding the Bouwens et al. (2015) points due to the uncertainties in the bright end of the LF where the sample is sensitive to cosmic variance (see Section 6.3). At $z \simeq 5$, we choose to fit to the van der Burg et al. (2010) results at the bright end, excluding the results of McLure et al. (2009) as they were based on a single field, and hence, as we have found in this study at $z \simeq 6$, could be vulnerable to cosmic variance. The results of Bouwens et al. (2015) and Finkelstein et al. (2014), although excluded in the fitting process at $z \simeq 6$ and $z \simeq 7$, agree well at $z \simeq 5$ at faint magnitudes and hence faintwards of $M_{UV} = -20.0$ we use the $z \simeq 5$ points from Bouwens et al. (2015). At $z \simeq 7$, we use the McLure et al. (2013) determination of the LF, which follows a similar methodology to this work. Bouwens et al. (2015) showed that the total magnitudes of the brightest galaxies found by McLure et al. (2013) were underestimated by
<table><caption>**Table 3.** The best-fitting DPL and Schechter-function parameters derived from fitting the selection of observations of the rest-frame UV LF at $z \simeq 5, 6$ and $7$ as described in the text and displayed in Fig. 10. The results from the combined fields, the UltraVISTA/COSMOS field alone and the UDS/SXDS field alone are displayed in the upper, middle and lower parts of the table, respectively. For each field combination, the DPL results are shown above the Schechter function fits, and are identifiable by the presence of the bright-end slope ($\beta$) value. Column 1 gives the approximate redshift, with the characteristic number density and absolute magnitude shown in Columns 2 and 3. The faint and bright-end slope (for the DPL) are displayed in Columns 4 and 5.</caption><thead><tr><th>z</th><th>φ*<br>(mag<sup>-1</sup> Mpc<sup>-3</sup>)</th><th>M*<br>(mag)</th><th>α</th><th>β</th></tr></thead><tbody><tr><td>5.0</td><td>2.5<sup>+0.6</sup><sub>-0.4</sub> × 10<sup>-4</sup></td><td>-21.40<sup>+0.13</sup><sub>-0.12</sub></td><td>-2.00<sup>+0.05</sup><sub>-0.05</sub></td><td>-4.8<sup>+0.3</sup><sub>-0.4</sub></td></tr><tr><td>6.0</td><td>1.9<sup>+1.2</sup><sub>-0.8</sub> × 10<sup>-4</sup></td><td>-21.20<sup>+0.22</sup><sub>-0.22</sub></td><td>-2.10<sup>+0.16</sup><sub>-0.14</sub></td><td>-5.1<sup>+0.5</sup><sub>-0.6</sub></td></tr><tr><td>7.0</td><td>2.2<sup>+1.7</sup><sub>-0.9</sub> × 10<sup>-4</sup></td><td>-20.61<sup>+0.31</sup><sub>-0.26</sub></td><td>-2.19<sup>+0.12</sup><sub>-0.10</sub></td><td>-4.6<sup>+0.4</sup><sub>-0.5</sub></td></tr><tr><td>5.0</td><td>6.4<sup>+1.1</sup><sub>-0.9</sub> × 10<sup>-4</sup></td><td>-21.07<sup>+0.09</sup><sub>-0.09</sub></td><td>-1.81<sup>+0.06</sup><sub>-0.05</sub></td><td>—</td></tr><tr><td>6.0</td><td>5.7<sup>+2.7</sup><sub>-2.0</sub> × 10<sup>-4</sup></td><td>-20.77<sup>+0.18</sup><sub>-0.19</sub></td><td>-1.88<sup>+0.15</sup><sub>-0.14</sub></td><td>—</td></tr><tr><td>7.0</td><td>3.7<sup>+1.5</sup><sub>-1.1</sub> × 10<sup>-4</sup></td><td>-20.56<sup>+0.17</sup><sub>-0.17</sub></td><td>-2.09<sup>+0.10</sup><sub>-0.09</sub></td><td>—</td></tr><tr><td colspan="5" style="text-align: center;">UltraVISTA/COSMOS field only</td></tr><tr><td>6.0</td><td>1.6<sup>+1.1</sup><sub>-0.7</sub> × 10<sup>-4</sup></td><td>-21.35<sup>+0.25</sup><sub>-0.23</sub></td><td>-2.08<sup>+0.15</sup><sub>-0.14</sub></td><td>-5.1<sup>+0.6</sup><sub>-0.8</sub></td></tr><tr><td>6.0</td><td>4.8<sup>+2.7</sup><sub>-1.9</sub> × 10<sup>-4</sup></td><td>-20.95<sup>+0.21</sup><sub>-0.23</sub></td><td>-1.88<sup>+0.16</sup><sub>-0.15</sub></td><td>—</td></tr><tr><td colspan="5" style="text-align: center;">UDS/SXDS field only</td></tr><tr><td>6.0</td><td>2.7<sup>+3.1</sup><sub>-1.3</sub> × 10<sup>-4</sup></td><td>-20.88<sup>+0.45</sup><sub>-0.30</sub></td><td>-2.08<sup>+0.20</sup><sub>-0.17</sub></td><td>-4.8<sup>+0.7</sup><sub>-0.8</sub></td></tr><tr><td>6.0</td><td>6.3<sup>+4.0</sup><sub>-2.8</sub> × 10<sup>-4</sup></td><td>-20.60<sup>+0.22</sup><sub>-0.25</sub></td><td>-1.92<sup>+0.19</sup><sub>-0.18</sub></td><td>—</td></tr></tbody></table>

assuming a point-source correction when using small apertures, as these objects are marginally extended. We therefore boost the magnitudes of the points at $M_{UV} = -21.0$ and $-20.5$ from the McLure et al. (2013) analysis by 0.15 and 0.1 mag, respectively, when fitting, to account for the underestimation of the magnitudes here. Uncertainties in the Bouwens et al. (2015) analysis at the bright end of the $z \simeq 7$ LF (which we exclude from the fitting process) are evident in Fig. 12, where the implied number density of galaxies at $M_{UV} = -21.86$ is comparable to that of $z \simeq 5$ galaxies at the same luminosity.

Fig. 10 shows the result of fitting a DPL and Schechter function to the described subset of the observed LF points at $z = 5-7$, with the best-fitting function parameters presented in Table 3. The reduced $\chi^2$ values for the fits are shown in Fig. 10, where it is evident that the current error bars available on the observed rest-frame UV LF result in an ‘overfitting’ of the data with the multivariable fit provided by the Schechter or DPL functions, resulting in $\chi^2_{red} < 1$.

Reassuringly, our fits recover the steep faint-end slopes found in previous studies (Bouwens et al. 2012; McLure et al. 2013; Schenker et al. 2013), showing that our measurement of $\alpha$ is not being strongly influenced by any tension in the fitting process. For the DPL fit, the recovered bright-end slope values are relatively uncertain at $z = 6$ and $7$, however there is tentative evidence for a steepening of $\beta$ from $z \simeq 7$ to $z \simeq 6$. At $z = 5$, the errors on $\beta$ are much smaller, however as we have only fitted to the van der Burg et al. (2010) data, the derived value and uncertainty does not include the systematic error between the van der Burg et al. (2010) and McLure et al. (2009) results (which can be seen in Fig. 12). Fitting the two studies separately, we find $\beta = -4.8^{+0.4}_{-0.5}$ and $-4.4^{+0.3}_{-0.3}$ for the van der Burg et al. (2010) and McLure et al. (2009) results, respectively, and

<figure><img src="image_12.png" /><figcaption>**Figure 11.** The evolution in the characteristic magnitude derived from fitting a Schechter function to the $z \simeq 5-7$ data shown in Fig. 10 are shown as the black circles. The results derived from the UltraVISTA/COSMOS and UDS/SXDS fields separately at $z \simeq 6$ are shown in dark red and orange, respectively. The results of primarily HST surveys from Finkelstein et al. (2014), Bouwens et al. (2015, 2007), Schmidt et al. (2014), McLure et al. (2013), Schenker et al. (2013) and Oesch et al. (2012) are shown, with additional results from wider area ground-based imaging from van der Burg et al. (2010) and McLure et al. (2009). The constant characteristic magnitude of $M^* = -21.0$ proposed by Bouwens et al. (2015) and Finkelstein et al. (2014) is shown as a horizontal dashed line, and the solid line shows a simple linear fit to our $M^*$ results from $z \simeq 5-7$. For clarity, in some cases the plotted redshift of a point has been shifted by $\Delta z = 0.05-0.1$.</figcaption></figure>

hence it remains possible that a further steepening of the bright end of the LF continues to $z = 5$ and this is not excluded by the data. Furthermore, at $z \simeq 5$, the Schechter function formally becomes the best-fitting function, demonstrating the steepening of the bright-end slope that is observed in the data.

The effects of cosmic variance on our derived LF at $z \simeq 6$ can be clearly seen in Fig. 8, where the LF was derived in the UltraVISTA/COSMOS and UDS/SXDS fields separately. To quantify the differences, we fit Schechter and DPL functions to the separate determinations and present the results in Table 1. We find that both the best-fitting $M^*$ and $\phi^*$ differ between the individual degree scale fields, although they are consistent within the errors. A deviation in these parameters with environments is to be expected theoretically (Trenti & Stiavelli 2008), and has been observed at lower redshift (e.g. McNaught-Roberts et al. 2014; Eardley et al. 2015). The combined LF we present here therefore represents a closer approximation to the underlying bright end of the LF at $z \simeq 6$ than that obtainable in an individual degree scale field, although additional sight-lines and wider area imaging will be necessary to constrain the LF and impact of cosmic variance further.

### 8.2.1 Evolution in $M^*$ from $z = 5-7$?

The evolution in the characteristic magnitude from $z \simeq 4-8$ is displayed in Fig. 11, including the results derived from HST surveys (Bouwens et al. 2007, 2015; Oesch et al. 2012; McLure et al. 2013; Schenker et al. 2013; Finkelstein et al. 2014; Schmidt et al. 2014) and ground-based analyses (McLure et al. 2009; van der Burg et al. 2010). The results from this work were taken from the fitting of Schechter and DPL functions to the data shown in Fig. 10, and are displayed in Table 3. While the error bars at $z \simeq 6$ are large,
at $z \simeq 5$ and $z \simeq 7$ the data we fit to more completely fills the available magnitude space and hence $M^*$ is more securely defined. The functional form assumed changes the derived values, as can be seen from comparing the DPL and Schechter function results, with the DPL fit tending to produce brighter characteristic magnitudes. The DPL results cannot therefore be directly compared with the results from other studies that exclusively derive $M^*$ assuming a Schechter function, however as a DPL provides a better fit to the observed LF at $z \simeq 6$ and 7 we include these results for comparison in Table 3. Fig. 11 illustrates the uncertainty in deriving the bright end of the LF from small fields such as those provided by the CANDELS survey, as the implied evolution from Bouwens et al. (2015) changes substantially depending on whether the full or a reduced set of the CANDELS fields are included. Furthermore, the errors on $M^*$ derived by Finkelstein et al. (2014) are large at $z > 6$ where the small area of the imaging used results in weak constraints on the break luminosity. The effect of cosmic variance in degree-scale fields is also evident from the faint inferred $M^*$ found in the potentially underdense UDS/SXDS field by McLure et al. (2009) at $z \simeq 6$, and by the difference in the $M^*$ derived from the two separate fields in our analysis.

In contrast to the work by Bouwens et al. (2015) and Finkelstein et al. (2014), we find an evolution in $M^*$ between $z \simeq 5$ to $z \simeq 7$ of $\Delta M^* \simeq 0.4-0.5$ mag. While our $M^*$ results agree with the determinations from Bouwens et al. (2015) and Finkelstein et al. (2014) within the errors, this is predominantly due to the large errors on $M^*$ derived by these studies, a consequence of the increased uncertainty in the number counts of bright galaxies when relatively small-area surveys are used. Both Bouwens et al. (2015) and Finkelstein et al. (2014) found little evolution in the characteristic magnitude over the same redshift range, with both studies suggesting that an approximately constant $M^* \simeq -21.0$ provides a good fit at $z \le 7$. Instead, as shown in Fig. 11, our results rule out a constant $M^*$ between $z \simeq 5$ and 7. The results of fitting the $z \simeq 7$ rest-frame UV LF, including the results of Bowler et al. (2014), with both a DPL and a Schechter function show a best-fitting $M^* > -21.0$; a result which, when combined with the faint characteristic magnitude found at $z \simeq 8$ (Oesch et al. 2012; McLure et al. 2013; Schenker et al. 2013; Schmidt et al. 2014; Bouwens et al. 2015) suggest a smooth brightening of $M^*$ from $z \simeq 8$ to $z \simeq 5$. Although the error bars on $M^*$ are relatively large at $z \simeq 6$, they follow a smooth decline to the observed $M^* \simeq -20.5$ observed at $z \simeq 7$. Such an evolution, primarily in the characteristic magnitude of galaxies, is qualitatively to be expected from the hierarchical coalescence and growth of the underlying HMF. We caution however, that condensing any evolution in the LF to a single parameter is very uncertain and may be missing subtleties in the form of the evolution as illustrated by the potential change in functional form from $z \simeq 7$ to $z \simeq 5$ hinted at in Fig. 10. For example, we also find an evolution in $\phi^*$, however it is weaker than that obtained by Bouwens et al. (2015) and Finkelstein et al. (2014). Instead a detailed comparison of the full observed LF with the predictions of theoretical models is necessary to provide a more complete view of the evolution, and we compare the observed UV LFs at $z \simeq 5, 6$ and 7 to a compilation of semi-analytic and hydrodynamical models in Section 8.3. The evolution we observe over $z = 5-7$ is occurring in only 400 Myr and hence represents apparently rapid evolution in the characteristic magnitude of LBGs in the first billion years of cosmic time (however such evolution is arguably expected given the rapid evolution in the underlying HMF). Improved constraints on the rest-frame UV LF around the apparent break magnitude from reconciling the various HST-based determinations, combined with future constraints on the form of the

extreme bright end of the LF from wider area imaging (e.g. VISTA VIDEO; Jarvis et al. 2013), will reduce the current errors on the determination of the form and evolution of the LF.

## 8.3 Comparison to theory

In Fig. 12, we present a comparison of the latest observational data on the rest-frame UV galaxy LF at $z \simeq 5, 6$ and 7 (including the new results on the bright end presented here and in Bowler et al. 2014) with the predictions of several of the latest semi-analytic and hydrodynamical models of galaxy formation. This comparison is not completely fair, as some of the models have been (to some extent) tuned to explicitly match existing high-redshift data (generally at the faint end of UV LF; e.g. Dayal et al. 2014), while others have not been tuned at all (e.g. the FiBY simulations; Paardekooper, Khochfar & Dalla 2013; Khochfar et al., in preparation). Moreover some models include the effects of dust obscuration (e.g. the smoothed particle hydrodynamics simulations of Jaacks et al. 2013 and the new Munich models of Clay et al. 2015; Henriques et al. 2015) while others have yet to implement any form of dust obscuration at these redshifts (e.g. the Illustris simulation predictions from Genel et al. 2014, the FiBY simulations from Paardekooper et al. 2013 and the semi-analytic results from Dayal et al. 2014). Instructively, the predictions of the Munich models (Clay et al. 2015), GALFORM (Gonzalez-Perez et al. 2013) and the Cai et al. (2014) model were made available to us both with and without dust obscuration. Finally, it should be noted that several of the models do not cover large enough cosmological volumes for very useful comparison with the very bright end as derived from the degree-scale ground-based surveys (e.g. Kimm & Cen 2013; Cen & Kimm 2014).

Despite these complications, some useful conclusions can still be drawn from this figure. First, it is clear that while most models do a reasonable job of reproducing the fainter end of the LF, there is a general problem of overpredicting the bright end (with the sole exception of the revised Munich models; Clay et al. 2015; Henriques et al. 2015), even though the actual data produced by the work presented here indicate a shallower bright-end slope than would be inferred from a Schechter function fit to the fainter data. Secondly, with the possible exception of Dayal et al. (2014, although this dust free model seems to start to struggle at $z \simeq 5$), those models which do provide a satisfactory fit to the LF over this large dynamic range include substantial dust obscuration. In particular, the model which apparently performs 'best' in this comparison is the Cai et al. (2014) model after application of dust obscuration, but it can be seen that the impact of this dust obscuration is enormous, equivalent to either an average depression of UV luminosity by $A_{1500} \simeq 2$ mag at a number density of $10^{-5} \text{ mag}^{-1} \text{Mpc}^{-3}$, or a depression in observed number density by $\simeq 2$ orders of magnitude at $M_{1500} \simeq -22.5$.

Thus, while much attention has been focused on the faint end of the high-redshift galaxy LF in recent years (quite reasonably, especially given the important implications for reionization; Robertson et al. 2013) it is clear that the full shape of the LF, extended to the brightest magnitudes through large-area ground-based surveys, has the potential to differentiate between alternative models of early galaxy formation and evolution. Moreover, while it currently remains unclear whether the shape of the bright end of the LF at $z \simeq 5-7$ is really driven by evolution in dust properties or by mass quenching (e.g. Peng et al. 2010), or early AGN feedback (or indeed by some other as yet poorly understood mechanism for regulating star formation), forthcoming observations have the potential to clarify and quite possibly resolve these issues. For example,
<figure><img src="image_13.png" /><figcaption>**Figure 12.** A comparison of the latest observational data on the rest-frame UV galaxy LF at $z \simeq 5, 6$ and $7$ (including the new results on the bright end presented here and in Bowler et al. 2014) with the predictions of several of the latest semi-analytic and hydrodynamical models of galaxy formation. The sources of the data points are indicated in each panel, with the various model references provided in the central ($z \simeq 6$) panel. The implications of this comparison are discussed in the text (Section 8.3) but in general it can be seen that most of the models struggle to reproduce the observations over the redshift range $z \simeq 5-7$ when faced with the large dynamic range now made possible by the combined ground-based and HST data set. See the text for full references to the models displayed here.</figcaption></figure>

pointed Atacama Large Millimeter/submillimeter Array (ALMA) follow-up of bright UV-selected galaxies can address the prevalence of dust in such objects, while improved measurements of the stellar mass function at these early times (e.g. through improved deconfusion of deep *Spitzer* IRAC data, and ultimately with *JWST* observations) will provide another important reference point for comparison with theoretical predictions. At the same time, UltraVISTA DR3 (expected 2015 July) should be deep enough to enable the work presented here at $z \simeq 6-7$ to be extended out to $z \simeq 8$ (with potentially useful constraints also at $z \simeq 9$), while wider-area surveys (e.g. with VISTA VIDEO at near-infrared wavelengths, and Subaru/Hyper-SuprimeCam at red optical wavelengths) culminating in the *Euclid* Deep Survey (Laureijs et al. 2011) should remove any remaining ambiguity over the shape of the bright end of the galaxy UV LF in the first billion years of cosmic history.

## 9 CONCLUSION

We have selected a sample of star-forming galaxies at $z \simeq 6$ from the COSMOS/UltraVISTA and UDS/SXDS fields, which in total provide an area of $1.65 \text{ deg}^2$ of deep multiwavelength imaging in the optical/near-infrared. The galaxies were selected using a full photometric redshift analysis, which allows the removal of low-redshift dusty galaxies and cool Galactic brown dwarf stars. The main findings of our work are as follows.

(i) Using a simple thin-disc galaxy model, we find that the expected number of brown dwarf stars in each field greatly exceeds the number of LBGs at the very bright-end ($m_{AB} < 25$), however brown dwarfs can be cleanly removed using fitting of stellar templates to the multiwavelength optical/near-infrared photometry.

(ii) We measure the rest-frame UV slope of the galaxies in our sample, finding that the derived values follow the colour–magnitude relation found at $z \simeq 5$ by Rogers et al. (2014), showing a mean $\beta_{UV} = -1.8 \pm 0.1$ at $M_{UV} < -22.0$, in contrast to the redder slopes found by Willott et al. (2013).

(iii) The number density of $z \simeq 6$ galaxies we find a factor of $\sim 1.8$ more galaxies in the UltraVISTA/COSMOS field than in the UDS/SXDS, a deviation that just exceeds that predicted from cosmic variance between fields of this size. We consider the effect of gravitational lensing of our objects by galaxies close to the line of sight, finding no evidence that the objects in our sample have preferential boosting over random positions in the field.

(iv) We calculate the rest-frame UV galaxy LF from our sample, using the restricted redshift range $5.7 < z < 6.3$ to compare directly with the work of McLure et al. (2009) and to ensure minimal contamination by brown dwarfs. Our determination of the LF lies mid-way between previous determinations from HST surveys by Bouwens et al. (2015) and the ground-based analysis by McLure et al. (2009). In particular, the recent determination of the UV LF from Bouwens et al. (2015) overpredicts the expected number of LBGs in the fields we analyse by approximately a factor of 2. We find a good agreement with the results of Finkelstein et al. (2014) in the magnitude range where our results overlap.

(v) By comparing the LF derived from the COSMOS/UltraVISTA and UDS/SXDS fields separately, we conclude that part of the discrepancy between the results of Bouwens et al. (2015) and McLure et al. (2009) at the bright end of the LF is a result of the UDS/SXDS (analysed by McLure et al. 2009) appearing underdense at $z \simeq 6$. Our results show that cosmic variance on scales of $1 \text{ deg}^2$ can be significant and therefore determining the bright-end of the LF from relatively small area
data sets such as the 200 arcmin² CANDELS fields could be highly
uncertain.

(vi) We fit Schechter and DPL functions to the observed $z \simeq 6$ rest-frame UV LF, showing that a DPL is marginally preferred, although an exponential decline also provides an acceptable description of the current data. The fits show that the bright-end slope of the LF appears to steepen from $z \simeq 7$ to $z \simeq 5$ suggesting we may be observing the onset of feedback (e.g. from AGN or some other form of mass quenching, e.g. Peng et al. 2010) or the build-up of dust in the brightest LBGs (Rogers et al. 2014).

(vii) In contrast to Bouwens et al. (2015) and Finkelstein et al. (2014), we find clear evidence for a brightening of the characteristic magnitude of $\Delta M^* \sim 0.4–0.5$ between $z \simeq 7$ and 5. Our results show that the evolution can still be well described as predominately luminosity evolution, as expected if the star formation of the galaxies follows the hierarchical build-up of the underlying dark matter haloes. We caution however that there still exist unexplained systematic errors between LF determinations at $z \simeq 5–7$ that can impact the results of functional fitting, and the future analysis of wider area imaging along independent sight-lines is required to further quantify these systematics and similarly the effect of cosmic variance at the bright-end.

(viii) Finally, comparison of a collection of the latest semi-analytical and hydrodynamical models of galaxy formation to the observed rest-frame UV galaxy LF at $z \simeq 5, 6$ and 7 reveals that most models tend to overpredict the number density of bright galaxies and substantial attenuation is required ($A_{1500} \simeq 1.5–2.0$) to bring the models into agreement with the data.

## ACKNOWLEDGEMENTS

RAAB and JSD acknowledge the support of the European Research Council via the award of an Advanced Grant to JSD. JSD also acknowledges the contribution of the EC FP7 SPACE project ASTRODEEP (Ref. No: 312725). RJM acknowledges the support of the European Research Council via the award of a Consolidator Grant (PI McLure). JPUF and BM acknowledge the support of the European Research Council via the award of a Consolidator Grant (PI Fynbo). We thank Zhen-Yi Cai, Cen Renyue, Taysun Kimm, Sadegh Khochfar, Shy Genel, Mark Vogelsberger, Bruno Henriques, Scott Clay, Simon White, Jason Jaacks, Violeta Gonzalez-Perez and Pratika Dayal for generously supplying their theoretical results, in some cases in advance of publication. We thank the anonymous referee for comments that improved this paper.

This work is based on data products from observations made with ESO Telescopes at the La Silla Paranal Observatory as part of programme ID 179.A-2005, using data products produced by TERAPIX and the Cambridge Astronomy Survey Unit on behalf of the UltraVISTA consortium, and as part of the VIDEO survey, under programme ID 179.A-2006 (PI: Jarvis). This study was based in part on observations obtained with MegaPrime/MegaCam, a joint project of CFHT and CEA/DAPNIA, at the CFHT which is operated by the National Research Council (NRC) of Canada, the Institut National des Sciences de l'Univers of the Centre National de la Recherche Scientifique (CNRS) of France, and the University of Hawaii. This work is based in part on data products produced at TERAPIX and the Canadian Astronomy Data Centre as part of the CFHTTLS, a collaborative project of NRC and CNRS. This research has benefitted from the SpeX Prism Spectral Libraries, maintained by Adam Burgasser at http://pono.ucsd.edu/~adam/browndwarfs/spexprism.

## REFERENCES

* Baldry I. K. et al., 2012, MNRAS, 421, 621

* Bernardi M. et al., 2003, AJ, 125, 1849

* Bernhard E., Bethermin M., Sargent M., Buat V., Mullaney J. R., Pannella M., Heinis S., Daddi E., 2014, MNRAS, 442, 509

* Bertin E., Arnouts S., 1996, A&AS, 117, 393

* Bian F. et al., 2013, ApJ, 774, 28

* Bouwens R. J., Illingworth G. D., Franx M., Ford H., 2007, ApJ, 670, 928

* Bouwens R. J., Illingworth G. D., Franx M., Ford H., 2008, ApJ, 686, 230

* Bouwens R. J. et al., 2010, ApJ, 708, L69

* Bouwens R. J. et al., 2011, Nature, 469, 504

* Bouwens R. J. et al., 2012, ApJ, 752, L5

* Bouwens R. J. et al., 2014, ApJ, 793, 115

* Bouwens R. J. et al., 2015, ApJ, 803, 34

* Bower R. G., Benson A. J., Crain R. A., 2012, MNRAS, 422, 2816

* Bowler R. A. A. et al., 2012, MNRAS, 426, 2772

* Bowler R. A. A. et al., 2014, MNRAS, 440, 2810

* Bruzual G., Charlot S., 2003, MNRAS, 344, 1000

* Caballero J. A., Burgasser A. J., Klement R., 2008, A&A, 488, 181

* Cai Z.-Y., Lapi A., Bressan A., De Zotti G., Negrello M., Danese L., 2014, ApJ, 785, 65

* Calzetti D., Armus L., Bohlin R. C., Kinney A. L., Koornneef J., Storchi-Bergmann T., 2000, ApJ, 533, 682

* Castellano M. et al., 2010a, A&A, 511, A20

* Castellano M. et al., 2010b, A&A, 524, A28

* Cen R., Kimm T., 2014, ApJ, 782, 32

* Chen B. et al., 2001, ApJ, 553, 184

* Clay S., Thomas P., Wilkins S., Henriques B., 2015, MNRAS, 451, 2692

* Curtis-Lake E. et al., 2013, MNRAS, 429, 302

* Curtis-Lake E. et al., 2014, preprint (arXiv:1409.1832)

* Dayal P., Ferrara A., Dunlop J. S., Pacucci F., 2014, MNRAS, 445, 2545

* Dunlop J. S. et al., 2013, MNRAS, 432, 3520

* Eardley E. et al., 2015, MNRAS, 448, 3665

* Ellis R. S. et al., 2013, ApJ, 763, L2

* Findlay J. R., Sutherland W. J., Venemans B. P., Reylé C., Robin A. C., Bonfield D. G., Bruce V. A., Jarvis M. J., 2012, MNRAS, 419, 3354

* Finkelstein S. L. et al., 2014, preprint (arXiv:1410.5439)

* Finoguenov A. et al., 2007, ApJS, 172, 182

* Finoguenov A. et al., 2010, MNRAS, 403, 2063

* Frei Z., Gunn J. E., 1994, AJ, 108, 1476

* Furusawa H. et al., 2008, ApJS, 176, 1

* Genel S. et al., 2014, MNRAS, 445, 175

* Gonzalez-Perez V., Lacey C. G., Baugh C. M., Frenk C. S., Wilkins S. M., 2013, MNRAS, 429, 1609

* Grogin N. A. et al., 2011, ApJS, 197, 35

* Guhathakurta P., Tyson J. A., Majewski S. R., 1990, ApJ, 357, L9

* Gunawardhana M. L. P. et al., 2013, MNRAS, 433, 2764

* Henriques B., White S., Thomas P., Angulo R., Guo Q., Lemson G., Springel V., Overzier R., 2015, MNRAS, 451, 2663

* Holwerda B. W. et al., 2014, ApJ, 788, 77

* Ilbert O. et al., 2008, in Kodama T., Yamada T., Aoki K., eds, ASP Conf. Ser. Vol. 399, Panoramic Views of Galaxy Formation and Evolution. Astron. Soc. Pac., San Francisco, p. 169

* Ilbert O. et al., 2009, ApJ, 690, 1236

* Ilbert O. et al., 2013, A&A, 556, A55

* Jaacks J., Thompson R., Nagamine K., 2013, ApJ, 766, 94

* Jarvis M. J. et al., 2013, MNRAS, 428, 1281

* Jurek R. J. et al., 2013, MNRAS, 434, 257

* Kelvin L. S. et al., 2014, MNRAS, 439, 1245

* Kimm T., Cen R., 2013, ApJ, 776, 35

* Koekemoer A. M. et al., 2011, ApJS, 197, 36

* Laureijs R. et al., 2011, preprint (arXiv:1110.3193)

* Lawrence A. et al., 2007, MNRAS, 379, 1599
Loveday J. et al., 2012, MNRAS, 420, 1239
McCracken H. J. et al., 2007, ApJS, 172, 314
McCracken H. J. et al., 2012, A&A, 544, A156
McCracken H. J. et al., 2013, The Messenger, 154, 29
McGree I. D. et al., 2013, ApJ, 768, 105
McLure R. J. et al., 2006, MNRAS, 372, 357
McLure R. J., Cirasuolo M., Dunlop J. S., Foucaud S., Almaini O., 2009, MNRAS, 395, 2196
McLure R. J., Dunlop J. S., Cirasuolo M., Koekemoer A. M., Sabbi E., Stark D. P., Targett T. A., Ellis R. S., 2010, MNRAS, 403, 960
McLure R. J. et al., 2013, MNRAS, 432, 2696
McNaught-Roberts T. et al., 2014, MNRAS, 445, 2125
Madau P., 1995, ApJ, 441, 18
Madau P., Dickinson M., 2014, ARA&A, 52, 415
Mason C. A. et al., 2015, ApJ, 805, 79
Meneux B. et al., 2009, A&A, 505, 463
Meurer G. R., Heckman T. M., Calzetti D., 1999, ApJ, 521, 64
Mortlock A. et al., 2014, MNRAS, 447, 2
Muñoz J. A., Loeb A., 2008, MNRAS, 386, 2323
Murray S., Power C., Robotham A., 2013, Astron. Comput., 3, 23
Oesch P. A. et al., 2010, ApJ, 709, L16
Oesch P. A. et al., 2012, ApJ, 759, 135
Oesch P. A. et al., 2013, ApJ, 773, 75
Oke J. B., 1974, ApJS, 27, 21
Oke J. B., Gunn J. E., 1983, ApJ, 266, 713
Ono Y. et al., 2012, ApJ, 744, 83
Ouchi M. et al., 2005, ApJ, 620, L1
Ouchi M. et al., 2009, ApJ, 706, 1136
Paardekooper J.-P., Khochfar S., Dalla C. V., 2013, MNRAS, 429, L94
Peng Y. et al., 2010, ApJ, 721, 193
Pickles A. J., 1998, PASP, 110, 863
Pirzkal N. et al., 2009, ApJ, 695, 1591
Press W. H., Teukolsky S. A., Vetterling W. T., Flannery B. P., 1992, Numerical Recipes in FORTRAN. The Art of Scientific Computing. Cambridge Univ. Press, Cambridge
Reed D. S., Bower R., Frenk C. S., Jenkins A., Theuns T., 2007, MNRAS, 374, 2
Robertson B. E. et al., 2013, ApJ, 768, 71
Rogers A. B., McLure R. J., Dunlop J. S., 2013, MNRAS, 429, 2456
Rogers A. B. et al., 2014, MNRAS, 440, 3714
Ryan R. E. et al., 2011, ApJ, 739, 83
Rykoff E. S. et al., 2008, MNRAS, 387, L28
Salim S., Lee J. C., 2012, ApJ, 758, 134
Saunders W., Rowan-Robinson M., Lawrence A., Efstathiou G., Kaiser N., Ellis R. S., Frenk C. S., 1990, MNRAS, 242, 318
Schechter P., 1976, ApJ, 203, 297
Schenker M. A. et al., 2013, ApJ, 768, 196
Schmidt M., 1968, ApJ, 151, 393
Schmidt K. B. et al., 2014, ApJ, 786, 57
Scoville N. et al., 2007, ApJS, 172, 38
Skibba R. A. et al., 2014, ApJ, 784, 128
Soifer B. T., Sanders D. B., Madore B. F., Neugebauer G., Danielson G. E., Elias J. H., Lonsdale C. J., Rice W. L., 1987, ApJ, 320, 238
Stanway E. R., Davies L. J. M., 2014, MNRAS, 439, 2474
Stark D. P., Ellis R. S., Ouchi M., 2011, ApJ, 728, L2
Steidel C. C., Hamilton D., 1992, AJ, 104, 941
Steidel C. C., Adelberger K. L., Giavalisco M., Dickinson M., Pettini M., 1999, ApJ, 519, 1
Takeuchi T. T., Yuan F.-T., Ikeyama A., Murata K. L., Inoue A. K., 2012, ApJ, 755, 144
Tempel E., Einasto J., Einasto M., Saar E., Tago E., 2009, A&A, 495, 37
Tempel E. et al., 2014, A&A, 566, A1
Trenti M., Stiavelli M., 2008, ApJ, 676, 767
Trenti M. et al., 2011, ApJ, 727, L39
van der Burg R. F. J., Hildebrandt H., Erben T., 2010, A&A, 523, A74
Willott C. J. et al., 2009, AJ, 137, 3541
Willott C. J. et al., 2013, AJ, 145, 4

Willott C. J., Carilli C. L., Wagg J., Wang R., 2015, preprint (arXiv:1504.05875)
Wyithe J. S. B., Yan H., Windhorst R. A., Mao S., 2011, Nature, 469, 181

## APPENDIX A: SCHECHTER, DPL AND SAUNDERS FUNCTIONAL FORMS

For reference, the Schechter function parametrization of the LF in magnitudes is

$$ \phi(M) = 0.4 \ln 10 \phi^*[10^{-0.4(M-M^*)}]^{(1+\alpha)} e^{(-10^{0.4(M-M^*)})} \quad (A1) $$

where $M^*$ and $\phi^*$ are the characteristic magnitude and number density, respectively, and $\alpha$ denotes the faint-end slope. The DPL function parametrization of the LF is

$$ \phi(M) = \frac{\phi^*[10^{0.4(\alpha+1)(M-M^*)} + 10^{0.4(\beta+1)(M-M^*)}]}{10^{0.4(\alpha+1)(M-M^*)} + 10^{0.4(\beta+1)(M-M^*)}} \quad (A2) $$

Here, an additional parameter $\beta$ determines the slope of the bright-end of the LF, as opposed to the assumed exponential decline in the Schechter function. Finally, the Saunders function (which has been highlighted as appropriate choice to fit the LF by Salim & Lee 2012) is parametrized as

$$ \phi(M) = 0.4 \ln 10 \phi^*[10^{-\Delta M'}]^{(\alpha+1)} \exp\left(-\frac{\log^2(1 + 10^{-\Delta M'})}{2\sigma^2}\right) \quad (A3) $$

with $\Delta M' = 0.4(M - M^*)$.

## APPENDIX B: POTENTIAL BROWN DWARFS

In this Appendix, we present the high-redshift galaxy candidates that were excluded from the final sample of $z \simeq 6$ objects based on a good stellar fit to the photometry.

## APPENDIX C: GRAVITATIONAL LENSING BY FOREGROUND GALAXIES

Following the approach presented in McLure et al. (2009) and Bowler et al. (2014), we estimated the magnification, $\mu$, from foreground galaxies at a separation, $\theta$, from our high-redshift galaxy as

$$ \mu = \frac{\theta}{\theta_{\text{E}}} \quad (C1) $$

using the SIS approximation to describe the dark matter halo of each foreground galaxy. Here, $\theta_E$ denotes the Einstein radius which depends on the velocity dispersion, $\sigma_V$, of the dark matter halo as

$$ \theta_E = \frac{4\pi(\sigma_V/c)^2 D_{LS}}{D_S} \quad (C2) $$

in the SIS model, where $D_{LS}$ denotes the luminosity distance from the lens object the source and $D_S$ denotes the luminosity distance to the source.

In each field, we created a K-band (mass selected) catalogue using the MAG_AUTO from SEXTRACTOR as an estimate for the total magnitude of the foreground galaxies, which then allowed an estimate of the velocity dispersion of the dark matter halo from the i-band absolute magnitude using the Faber-Jackson relation from Bernardi et al. (2003). The photometric redshifts of the foreground K-band selected objects were calculated using the LE PHARE code using 3 arcsec diameter circular aperture photometry following Ilbert et al. (2009, 2013). The Ilbert et al. (2009) SED template set was
<table><caption>**Table B1.** High-redshift galaxy candidates that were excluded based on a good fit to a stellar template (defined as $\chi^2_{\star} < 10.0$). Objects have been ordered by their best-fitting galaxy photometric redshift and separated by field (UltraVISTA/COSMOS DR2 in the upper part of the table, followed by objects in the UDS/SXDS). Note that the brightest object (S1) formally has a somewhat poor stellar fit, however this object is clearly stellar from inspection of the SED fit. The large $\chi^2_{\star}$ here is a result of the limited template set available and high-S/N photometry.</caption><thead><tr><th rowspan="2">ID</th><th rowspan="2">RA<br>(J2000)</th><th rowspan="2">Dec.<br>(J2000)</th><th rowspan="2">i</th><th colspan="5">UltraVISTA/COSMOS field</th><th rowspan="2">Star<br>type</th><th rowspan="2">χ<sup>2</sup><sub>★</sub></th><th rowspan="2">z<sub>phot</sub></th><th rowspan="2">χ<sup>2</sup><sub>gal</sub></th></tr><tr><th>z'<sub>★</sub></th><th>Y</th><th>J</th><th>H</th><th>K</th></tr></thead><tbody><tr><td>S1</td><td>10:00:10.71</td><td>+02:06:37.1</td><td>27.2<sup>+0.6</sup><sub>-0.4</sub></td><td>23.8<sup>+0.1</sup><sub>-0.1</sub></td><td>22.6<sup>+0.1</sup><sub>-0.1</sub></td><td>21.6<sup>+0.1</sup><sub>-0.1</sub></td><td>21.0<sup>+0.1</sup><sub>-0.1</sub></td><td>20.7<sup>+0.1</sup><sub>-0.1</sub></td><td>L8</td><td>27.5</td><td>6.4</td><td>11.1</td></tr><tr><td>S2</td><td>10:00:23.90</td><td>+01:59:05.6</td><td>27.1<sup>+0.7</sup><sub>-0.4</sub></td><td>25.1<sup>+0.1</sup><sub>-0.1</sub></td><td>24.3<sup>+0.1</sup><sub>-0.1</sub></td><td>23.9<sup>+0.1</sup><sub>-0.1</sub></td><td>23.7<sup>+0.1</sup><sub>-0.1</sub></td><td>23.6<sup>+0.1</sup><sub>-0.1</sub></td><td>M8</td><td>3.9</td><td>6.3</td><td>2.9</td></tr><tr><td>S3</td><td>10:00:09.93</td><td>+02:22:07.2</td><td>>27.0</td><td>25.5<sup>+0.1</sup><sub>-0.1</sub></td><td>25.1<sup>+0.3</sup><sub>-0.2</sub></td><td>24.5<sup>+0.3</sup><sub>-0.3</sub></td><td>25.1<sup>+0.5</sup><sub>-0.4</sub></td><td>>25.5</td><td>M4</td><td>6.7</td><td>6.3</td><td>4.8</td></tr><tr><td>S4</td><td>10:02:14.87</td><td>+02:11:04.9</td><td>26.8<sup>+0.4</sup><sub>-0.3</sub></td><td>24.9<sup>+0.1</sup><sub>-0.1</sub></td><td>24.2<sup>+0.1</sup><sub>-0.1</sub></td><td>23.9<sup>+0.1</sup><sub>-0.1</sub></td><td>24.0<sup>+0.1</sup><sub>-0.1</sub></td><td>23.7<sup>+0.1</sup><sub>-0.1</sub></td><td>M7</td><td>9.0</td><td>6.2</td><td>4.9</td></tr><tr><td>S5</td><td>09:58:56.08</td><td>+02:35:08.1</td><td>>27.3</td><td>24.9<sup>+0.1</sup><sub>-0.1</sub></td><td>24.1<sup>+0.1</sup><sub>-0.1</sub></td><td>23.2<sup>+0.1</sup><sub>-0.1</sub></td><td>22.7<sup>+0.1</sup><sub>-0.1</sub></td><td>22.3<sup>+0.1</sup><sub>-0.1</sub></td><td>L8</td><td>4.6</td><td>6.2</td><td>9.6</td></tr><tr><td>S6</td><td>09:58:42.41</td><td>+02:26:06.7</td><td>26.9<sup>+0.5</sup><sub>-0.3</sub></td><td>24.8<sup>+0.1</sup><sub>-0.1</sub></td><td>24.3<sup>+0.1</sup><sub>-0.1</sub></td><td>23.9<sup>+0.1</sup><sub>-0.1</sub></td><td>24.0<sup>+0.2</sup><sub>-0.1</sub></td><td>23.9<sup>+0.1</sup><sub>-0.1</sub></td><td>M7</td><td>7.0</td><td>6.2</td><td>5.4</td></tr><tr><td>S7</td><td>10:02:07.38</td><td>+02:25:44.1</td><td>>27.4</td><td>25.6<sup>+0.1</sup><sub>-0.1</sub></td><td>25.3<sup>+0.3</sup><sub>-0.2</sub></td><td>24.7<sup>+0.2</sup><sub>-0.2</sub></td><td>24.6<sup>+0.3</sup><sub>-0.2</sub></td><td>25.0<sup>+0.3</sup><sub>-0.2</sub></td><td>M7</td><td>3.0</td><td>6.2</td><td>4.1</td></tr><tr><td>S8</td><td>10:02:02.78</td><td>+02:24:00.0</td><td>27.1<sup>+0.7</sup><sub>-0.4</sub></td><td>25.0<sup>+0.1</sup><sub>-0.1</sub></td><td>24.5<sup>+0.2</sup><sub>-0.1</sub></td><td>24.4<sup>+0.1</sup><sub>-0.1</sub></td><td>24.1<sup>+0.2</sup><sub>-0.1</sub></td><td>24.4<sup>+0.2</sup><sub>-0.2</sub></td><td>M7</td><td>6.5</td><td>6.2</td><td>3.5</td></tr><tr><td>S9</td><td>10:00:53.64</td><td>+02:09:45.5</td><td>>27.9</td><td>25.7<sup>+0.1</sup><sub>-0.1</sub></td><td>25.3<sup>+0.2</sup><sub>-0.2</sub></td><td>24.8<sup>+0.2</sup><sub>-0.2</sub></td><td>25.3<sup>+0.5</sup><sub>-0.3</sub></td><td>25.2<sup>+0.4</sup><sub>-0.3</sub></td><td>M7</td><td>8.6</td><td>6.2</td><td>3.7</td></tr><tr><td>S10</td><td>10:00:45.18</td><td>+02:31:40.3</td><td>>27.3</td><td>25.6<sup>+0.1</sup><sub>-0.1</sub></td><td>25.1<sup>+0.1</sup><sub>-0.1</sub></td><td>25.1<sup>+0.4</sup><sub>-0.3</sub></td><td>25.1<sup>+0.6</sup><sub>-0.4</sub></td><td>25.1<sup>+0.3</sup><sub>-0.2</sub></td><td>M7</td><td>9.7</td><td>6.2</td><td>0.1</td></tr><tr><td>S11</td><td>09:58:45.50</td><td>+02:23:24.7</td><td>>27.6</td><td>25.4<sup>+0.2</sup><sub>-0.1</sub></td><td>25.2<sup>+0.3</sup><sub>-0.2</sub></td><td>24.5<sup>+0.2</sup><sub>-0.2</sub></td><td>24.8<sup>+0.3</sup><sub>-0.3</sub></td><td>25.1<sup>+0.4</sup><sub>-0.3</sub></td><td>M7</td><td>9.1</td><td>6.2</td><td>5.7</td></tr><tr><td>S12</td><td>09:58:45.02</td><td>+02:29:04.3</td><td>>26.2</td><td>25.7<sup>+0.2</sup><sub>-0.1</sub></td><td>>25.4</td><td>>25.6</td><td>>25.2</td><td>>25.0</td><td>M4</td><td>5.6</td><td>5.9</td><td>1.2</td></tr><tr><td>S13</td><td>10:01:55.84</td><td>+02:37:51.9</td><td>>27.7</td><td>25.7<sup>+0.1</sup><sub>-0.1</sub></td><td>25.7<sup>+0.4</sup><sub>-0.3</sub></td><td>24.9<sup>+0.3</sup><sub>-0.2</sub></td><td>24.7<sup>+0.2</sup><sub>-0.2</sub></td><td>24.6<sup>+0.2</sup><sub>-0.2</sub></td><td>M7</td><td>6.5</td><td>5.7</td><td>1.9</td></tr><tr><td>S14</td><td>09:59:29.41</td><td>+01:46:40.3</td><td>27.4<sup>+0.7</sup><sub>-0.4</sub></td><td>25.4<sup>+0.1</sup><sub>-0.1</sub></td><td>25.1<sup>+0.3</sup><sub>-0.2</sub></td><td>25.1<sup>+0.4</sup><sub>-0.3</sub></td><td>25.1<sup>+0.5</sup><sub>-0.3</sub></td><td>25.1<sup>+0.5</sup><sub>-0.4</sub></td><td>M5</td><td>9.4</td><td>5.7</td><td>1.1</td></tr><tr><td>S15</td><td>10:02:00.39</td><td>+02:23:52.3</td><td>>27.1</td><td>25.6<sup>+0.1</sup><sub>-0.1</sub></td><td>25.8<sup>+0.6</sup><sub>-0.4</sub></td><td>25.5<sup>+0.4</sup><sub>-0.3</sub></td><td>>25.7</td><td>>25.5</td><td>M4</td><td>8.2</td><td>5.7</td><td>0.5</td></tr><tr><td>S16</td><td>09:58:53.79</td><td>+02:22:37.1</td><td>>26.8</td><td>25.2<sup>+0.1</sup><sub>-0.1</sub></td><td>24.9<sup>+0.2</sup><sub>-0.2</sub></td><td>24.6<sup>+0.2</sup><sub>-0.2</sub></td><td>24.3<sup>+0.2</sup><sub>-0.2</sub></td><td>24.2<sup>+0.1</sup><sub>-0.1</sub></td><td>M5</td><td>5.9</td><td>5.7</td><td>0.5</td></tr><tr><td>S17</td><td>09:59:15.77</td><td>+01:51:06.2</td><td>>27.5</td><td>25.7<sup>+0.1</sup><sub>-0.1</sub></td><td>25.9<sup>+0.4</sup><sub>-0.3</sub></td><td>25.3<sup>+0.3</sup><sub>-0.2</sub></td><td>24.9<sup>+0.3</sup><sub>-0.3</sub></td><td>25.0<sup>+0.3</sup><sub>-0.3</sub></td><td>M5</td><td>9.9</td><td>5.7</td><td>2.0</td></tr><tr><td>S18</td><td>09:59:07.45</td><td>+02:35:03.2</td><td>27.0<sup>+0.4</sup><sub>-0.3</sub></td><td>25.3<sup>+0.1</sup><sub>-0.1</sub></td><td>25.0<sup>+0.2</sup><sub>-0.1</sub></td><td>24.9<sup>+0.2</sup><sub>-0.2</sub></td><td>24.3<sup>+0.2</sup><sub>-0.2</sub></td><td>24.8<sup>+0.2</sup><sub>-0.2</sub></td><td>M5</td><td>9.0</td><td>5.7</td><td>6.2</td></tr><tr><td>S19</td><td>10:00:07.62</td><td>+02:21:25.0</td><td>26.7<sup>+0.5</sup><sub>-0.3</sub></td><td>25.4<sup>+0.1</sup><sub>-0.1</sub></td><td>25.4<sup>+0.4</sup><sub>-0.3</sub></td><td>>25.2</td><td>>25.1</td><td>>25.3</td><td>M4</td><td>8.9</td><td>5.7</td><td>0.9</td></tr><tr><td>S20</td><td>10:02:18.39</td><td>+02:20:25.9</td><td>26.9<sup>+0.4</sup><sub>-0.3</sub></td><td>25.2<sup>+0.1</sup><sub>-0.1</sub></td><td>24.8<sup>+0.2</sup><sub>-0.1</sub></td><td>24.6<sup>+0.2</sup><sub>-0.1</sub></td><td>24.4<sup>+0.2</sup><sub>-0.2</sub></td><td>24.6<sup>+0.2</sup><sub>-0.2</sub></td><td>M5</td><td>3.4</td><td>5.7</td><td>6.0</td></tr><tr><td>S21</td><td>10:01:42.49</td><td>+02:38:24.0</td><td>26.5<sup>+0.3</sup><sub>-0.2</sub></td><td>24.8<sup>+0.1</sup><sub>-0.1</sub></td><td>24.6<sup>+0.2</sup><sub>-0.2</sub></td><td>24.3<sup>+0.1</sup><sub>-0.1</sub></td><td>24.3<sup>+0.2</sup><sub>-0.2</sub></td><td>24.4<sup>+0.2</sup><sub>-0.2</sub></td><td>M5</td><td>7.6</td><td>5.6</td><td>4.0</td></tr><tr><td>S22</td><td>10:01:47.42</td><td>+02:06:18.9</td><td>27.0<sup>+0.6</sup><sub>-0.4</sub></td><td>25.5<sup>+0.1</sup><sub>-0.1</sub></td><td>25.6<sup>+0.2</sup><sub>-0.2</sub></td><td>25.2<sup>+0.3</sup><sub>-0.3</sub></td><td>25.4<sup>+0.6</sup><sub>-0.4</sub></td><td>25.4<sup>+0.4</sup><sub>-0.3</sub></td><td>M4</td><td>8.3</td><td>5.6</td><td>1.0</td></tr><tr><td>S23</td><td>10:01:54.85</td><td>+02:33:33.4</td><td>27.3<sup>+0.7</sup><sub>-0.4</sub></td><td>25.5<sup>+0.1</sup><sub>-0.1</sub></td><td>25.2<sup>+0.3</sup><sub>-0.2</sub></td><td>25.1<sup>+0.4</sup><sub>-0.3</sub></td><td>25.0<sup>+0.4</sup><sub>-0.3</sub></td><td>24.6<sup>+0.3</sup><sub>-0.2</sub></td><td>M5</td><td>6.4</td><td>5.6</td><td>0.7</td></tr><tr><td>S24</td><td>09:58:44.72</td><td>+02:25:37.2</td><td>27.1<sup>+0.5</sup><sub>-0.3</sub></td><td>25.6<sup>+0.2</sup><sub>-0.1</sub></td><td>25.5<sup>+0.3</sup><sub>-0.3</sub></td><td>25.4<sup>+0.4</sup><sub>-0.3</sub></td><td>>25.5</td><td>25.4<sup>+0.4</sup><sub>-0.3</sub></td><td>M4</td><td>5.5</td><td>5.6</td><td>0.6</td></tr><tr><td>S25</td><td>10:00:38.46</td><td>+01:56:22.3</td><td>26.6<sup>+0.2</sup><sub>-0.2</sub></td><td>25.2<sup>+0.1</sup><sub>-0.1</sub></td><td>24.8<sup>+0.1</sup><sub>-0.1</sub></td><td>24.5<sup>+0.2</sup><sub>-0.1</sub></td><td>24.5<sup>+0.2</sup><sub>-0.2</sub></td><td>24.8<sup>+0.2</sup><sub>-0.2</sub></td><td>M5</td><td>4.9</td><td>5.6</td><td>9.9</td></tr><tr><td>S26</td><td>09:58:41.39</td><td>+02:23:03.7</td><td>26.9<sup>+0.3</sup><sub>-0.2</sub></td><td>25.4<sup>+0.1</sup><sub>-0.1</sub></td><td>25.2<sup>+0.3</sup><sub>-0.2</sub></td><td>25.5<sup>+0.7</sup><sub>-0.4</sub></td><td>25.1<sup>+0.4</sup><sub>-0.3</sub></td><td>25.2<sup>+0.4</sup><sub>-0.3</sub></td><td>M4</td><td>7.7</td><td>5.5</td><td>1.6</td></tr><tr><td>S27</td><td>10:01:35.94</td><td>+02:23:03.3</td><td>27.0<sup>+0.4</sup><sub>-0.3</sub></td><td>25.5<sup>+0.1</sup><sub>-0.1</sub></td><td>25.2<sup>+0.3</sup><sub>-0.3</sub></td><td>24.5<sup>+0.2</sup><sub>-0.2</sub></td><td>24.9<sup>+0.4</sup><sub>-0.3</sub></td><td>>25.1</td><td>M5</td><td>4.8</td><td>5.5</td><td>7.7</td></tr><tr><td>S28</td><td>10:02:15.54</td><td>+02:36:46.9</td><td>26.8<sup>+0.3</sup><sub>-0.2</sub></td><td>25.7<sup>+0.1</sup><sub>-0.1</sub></td><td>25.4<sup>+0.2</sup><sub>-0.2</sub></td><td>25.8<sup>+0.6</sup><sub>-0.4</sub></td><td>25.1<sup>+0.7</sup><sub>-0.4</sub></td><td>>25.8</td><td>M4</td><td>7.5</td><td>5.5</td><td>5.2</td></tr><tr><td>S29</td><td>10:02:25.51</td><td>+02:33:32.1</td><td>26.7<sup>+0.3</sup><sub>-0.2</sub></td><td>25.4<sup>+0.1</sup><sub>-0.1</sub></td><td>24.9<sup>+0.3</sup><sub>-0.2</sub></td><td>24.5<sup>+0.3</sup><sub>-0.2</sub></td><td>24.7<sup>+0.3</sup><sub>-0.2</sub></td><td>>25.3</td><td>M5</td><td>7.3</td><td>5.5</td><td>10.0</td></tr><tr><td>S30</td><td>10:02:21.87</td><td>+02:24:01.0</td><td>26.7<sup>+0.4</sup><sub>-0.3</sub></td><td>25.3<sup>+0.1</sup><sub>-0.1</sub></td><td>25.2<sup>+0.4</sup><sub>-0.3</sub></td><td>25.4<sup>+0.7</sup><sub>-0.4</sub></td><td>25.1<sup>+0.5</sup><sub>-0.4</sub></td><td>24.7<sup>+0.3</sup><sub>-0.2</sub></td><td>M4</td><td>7.7</td><td>5.5</td><td>1.8</td></tr><tr><td>S31</td><td>10:00:08.43</td><td>+02:20:49.1</td><td>26.8<sup>+0.4</sup><sub>-0.3</sub></td><td>25.3<sup>+0.1</sup><sub>-0.1</sub></td><td>25.6<sup>+0.5</sup><sub>-0.3</sub></td><td>25.2<sup>+0.4</sup><sub>-0.3</sub></td><td>25.0<sup>+0.6</sup><sub>-0.4</sub></td><td>25.1<sup>+0.4</sup><sub>-0.3</sub></td><td>M4</td><td>7.5</td><td>5.5</td><td>0.9</td></tr><tr><td>S32</td><td>10:02:05.41</td><td>+02:14:46.4</td><td>26.9<sup>+0.5</sup><sub>-0.3</sub></td><td>25.5<sup>+0.1</sup><sub>-0.1</sub></td><td>25.5<sup>+0.3</sup><sub>-0.2</sub></td><td>25.0<sup>+0.2</sup><sub>-0.2</sub></td><td>25.3<sup>+0.6</sup><sub>-0.4</sub></td><td>25.1<sup>+0.3</sup><sub>-0.3</sub></td><td>M4</td><td>5.6</td><td>5.5</td><td>2.1</td></tr><tr><td>S33</td><td>10:01:40.07</td><td>+02:11:51.1</td><td>27.0<sup>+0.3</sup><sub>-0.3</sub></td><td>25.7<sup>+0.1</sup><sub>-0.1</sub></td><td>25.6<sup>+0.4</sup><sub>-0.3</sub></td><td>25.5<sup>+0.5</sup><sub>-0.4</sub></td><td>>25.4</td><td>>25.6</td><td>M4</td><td>6.7</td><td>5.5</td><td>1.0</td></tr><tr><td>S34</td><td>10:02:22.64</td><td>+01:51:57.1</td><td>26.7<sup>+0.4</sup><sub>-0.3</sub></td><td>25.5<sup>+0.2</sup><sub>-0.2</sub></td><td>>25.3</td><td>>25.5</td><td>>25.1</td><td>>24.9</td><td>M4</td><td>4.8</td><td>5.5</td><td>1.5</td></tr><tr><td>S35</td><td>10:00:24.21</td><td>+02:39:07.2</td><td>26.8<sup>+0.4</sup><sub>-0.3</sub></td><td>25.6<sup>+0.1</sup><sub>-0.1</sub></td><td>25.5<sup>+0.3</sup><sub>-0.2</sub></td><td>25.0<sup>+0.2</sup><sub>-0.2</sub></td><td>>25.6</td><td>>25.6</td><td>M4</td><td>6.5</td><td>5.5</td><td>4.9</td></tr><tr><td>S36</td><td>10:01:33.84</td><td>+02:03:03.9</td><td>26.7<sup>+0.5</sup><sub>-0.3</sub></td><td>25.7<sup>+0.1</sup><sub>-0.1</sub></td><td>>25.8</td><td>>25.5</td><td>>25.3</td><td>>24.7</td><td>M4</td><td>7.8</td><td>5.5</td><td>2.2</td></tr><tr><td>S37</td><td>09:59:08.92</td><td>+01:50:50.7</td><td>24.7<sup>+0.1</sup><sub>-0.1</sub></td><td>23.5<sup>+0.1</sup><sub>-0.1</sub></td><td>23.1<sup>+0.1</sup><sub>-0.1</sub></td><td>22.8<sup>+0.1</sup><sub>-0.1</sub></td><td>22.9<sup>+0.1</sup><sub>-0.1</sub></td><td>22.9<sup>+0.1</sup><sub>-0.1</sub></td><td>M5</td><td>9.9</td><td>5.5</td><td>10.3</td></tr></tbody></table><table><thead><tr><th colspan="10">UDS/SXDS field</th></tr></thead><tbody><tr><td>S38</td><td>02:18:01.62</td><td>-04:52:22.3</td><td>27.4<sup>+0.4</sup><sub>-0.3</sub></td><td>25.4<sup>+0.1</sup><sub>-0.1</sub></td><td>24.6<sup>+0.2</sup><sub>-0.2</sub></td><td>24.1<sup>+0.1</sup><sub>-0.1</sub></td><td>24.2<sup>+0.1</sup><sub>-0.1</sub></td><td>24.0<sup>+0.1</sup><sub>-0.1</sub></td><td>M8</td><td>6.8</td><td>6.4</td><td>6.1</td></tr><tr><td>S39</td><td>02:16:13.17</td><td>-04:51:40.5</td><td>27.4<sup>+0.4</sup><sub>-0.3</sub></td><td>25.4<sup>+0.1</sup><sub>-0.1</sub></td><td>24.8<sup>+0.2</sup><sub>-0.2</sub></td><td>24.1<sup>+0.1</sup><sub>-0.1</sub></td><td>24.1<sup>+0.1</sup><sub>-0.1</sub></td><td>23.9<sup>+0.1</sup><sub>-0.1</sub></td><td>M8</td><td>5.4</td><td>6.3</td><td>9.3</td></tr><tr><td>S40</td><td>02:19:36.16</td><td>-05:03:16.2</td><td>>27.8</td><td>25.5<sup>+0.1</sup><sub>-0.1</sub></td><td>24.9<sup>+0.2</sup><sub>-0.2</sub></td><td>23.8<sup>+0.1</sup><sub>-0.1</sub></td><td>23.7<sup>+0.1</sup><sub>-0.1</sub></td><td>23.2<sup>+0.1</sup><sub>-0.1</sub></td><td>L4</td><td>7.9</td><td>6.3</td><td>11.0</td></tr><tr><td>S41</td><td>02:16:50.79</td><td>-05:28:44.5</td><td>26.8<sup>+0.3</sup><sub>-0.2</sub></td><td>25.6<sup>+0.1</sup><sub>-0.1</sub></td><td>25.1<sup>+0.4</sup><sub>-0.3</sub></td><td>25.3<sup>+0.2</sup><sub>-0.1</sub></td><td>25.6<sup>+0.5</sup><sub>-0.3</sub></td><td>25.5<sup>+0.4</sup><sub>-0.3</sub></td><td>M4</td><td>9.7</td><td>5.6</td><td>3.5</td></tr><tr><td>S42</td><td>02:17:40.34</td><td>-04:40:06.4</td><td>26.5<sup>+0.1</sup><sub>-0.1</sub></td><td>25.1<sup>+0.1</sup><sub>-0.1</sub></td><td>25.0<sup>+0.3</sup><sub>-0.2</sub></td><td>24.6<sup>+0.1</sup><sub>-0.1</sub></td><td>24.5<sup>+0.2</sup><sub>-0.2</sub></td><td>24.4<sup>+0.1</sup><sub>-0.1</sub></td><td>M5</td><td>8.2</td><td>5.6</td><td>1.3</td></tr><tr><td>S43</td><td>02:18:17.95</td><td>-05:25:47.9</td><td>26.8<sup>+0.2</sup><sub>-0.2</sub></td><td>25.7<sup>+0.2</sup><sub>-0.1</sub></td><td>>25.6</td><td>25.6<sup>+0.3</sup><sub>-0.3</sub></td><td>>25.6</td><td>26.1<sup>+0.6</sup><sub>-0.4</sub></td><td>M4</td><td>7.1</td><td>5.5</td><td>0.6</td></tr><tr><td>S44</td><td>02:17:06.25</td><td>-04:49:21.6</td><td>26.7<sup>+0.2</sup><sub>-0.2</sub></td><td>25.6<sup>+0.1</sup><sub>-0.1</sub></td><td>25.6<sup>+0.5</sup><sub>-0.3</sub></td><td>25.7<sup>+0.3</sup><sub>-0.3</sub></td><td>>25.5</td><td>25.8<sup>+0.6</sup><sub>-0.4</sub></td><td>M4</td><td>9.5</td><td>5.5</td><td>0.3</td></tr></tbody></table>
<table><caption>Table B1 – continued</caption><thead><tr><th rowspan="2">ID</th><th rowspan="2">RA<br>(J2000)</th><th rowspan="2">Dec.<br>(J2000)</th><th colspan="7">UDS/SXDS field</th><th rowspan="2">Star<br>type</th><th rowspan="2">χ<sub>*</sub><sup>2</sup></th><th rowspan="2">z<sub>phot</sub></th><th rowspan="2">χ<sub>gal</sub><sup>2</sup></th></tr><tr><th>i</th><th>z'</th><th>Y</th><th>J</th><th>H</th><th>K</th></tr></thead><tbody><tr><td>S45</td><td>02:16:38.53</td><td>-05:02:08.2</td><td>26.8<sup>+0.2</sup><sub>-0.2</sub></td><td>25.7<sup>+0.1</sup><sub>-0.1</sub></td><td>25.4<sup>+0.3</sup><sub>-0.2</sub></td><td>25.7<sup>+0.4</sup><sub>-0.3</sub></td><td>25.4<sup>+0.4</sup><sub>-0.3</sub></td><td>25.6<sup>+0.5</sup><sub>-0.3</sub></td><td>M4</td><td>7.1</td><td>5.5</td><td>0.9</td></tr><tr><td>S46</td><td>02:19:09.36</td><td>-04:40:00.4</td><td>26.3<sup>+0.1</sup><sub>-0.1</sub></td><td>25.2<sup>+0.1</sup><sub>-0.1</sub></td><td>25.5<sup>+0.5</sup><sub>-0.3</sub></td><td>25.0<sup>+0.2</sup><sub>-0.2</sub></td><td>24.9<sup>+0.3</sup><sub>-0.2</sub></td><td>25.2<sup>+0.3</sup><sub>-0.2</sub></td><td>M4</td><td>8.1</td><td>5.5</td><td>2.2</td></tr><tr><td>S47</td><td>02:17:32.33</td><td>-04:39:36.4</td><td>26.9<sup>+0.2</sup><sub>-0.2</sub></td><td>25.7<sup>+0.1</sup><sub>-0.1</sub></td><td>25.7<sup>+0.5</sup><sub>-0.3</sub></td><td>25.7<sup>+0.4</sup><sub>-0.3</sub></td><td>24.9<sup>+0.3</sup><sub>-0.2</sub></td><td>25.4<sup>+0.4</sup><sub>-0.3</sub></td><td>M4</td><td>8.6</td><td>5.5</td><td>3.4</td></tr></tbody></table>

<figure><img src="image_14.png" /><figcaption>Figure C1. The magnification distribution of the full $z \simeq 6$ sample due to gravitational lensing by the mass associated with foreground galaxies close to the line of sight. The observed magnification distribution is shown in grey, for the UltraVISTA/COSMOS and UDS/SXDS samples in the upper and lower plots, respectively. The predicted magnification distribution for random positions in the field is shown as the blue points, where we plot the median and 68-percentiles. The inset plot shows the gravitational magnification plotted against the $z'$-band magnitude for the sample.</figcaption></figure>

used in the SED fitting process rather than the high- and low-redshift model subsets utilized in our sample selection, to better represent the range of galaxies found at $z < 4$. We select galaxies based on an acceptable galaxy solution and a superior galaxy fit over that from stars (using the PICKLES library of stellar templates; Pickles 1998).

The resulting magnification distributions for our samples of objects in the UltraVISTA/COSMOS and UDS/SXDS fields are shown in Fig. C1, where we also plot the magnification against the $z'$-band magnitude of the galaxy. The magnification sums the contribution

from all foreground galaxies closer than 10 arcsec to the high-redshift galaxy. We find that the majority of galaxies have some magnification of the order of ~0.1 mag (median values are 0.11 in the UltraVISTA/COSMOS field and 0.16 in the UDS/SXDS field), with several objects showing magnifications as large as ~0.6 mag. If we use the Ilbert et al. (2008) photometric redshifts in the UltraVISTA/COSMOS field, we find a similar shape of magnification distribution as shown in Fig. C1 with an identical median magnification. The inset plot of magnification against $z'$-band magnitude shows no evidence for the brightest objects having the largest magnification, which, if true, could influence the derived shape of the LF.

The magnification of the objects we derive could impact the measured LF, however we must determine if this magnification is unusual given that all astronomical imaging surveys show foreground objects close to the line of sight of the background high-redshift galaxies. Hence, we calculated the expected magnification for random positions in the field, using the full K-band catalogues used to determine the magnification of our sample. A minimum separation of 1 arcsec was applied when calculating the lensing at a given position, to exclude very high magnification of objects directly along the line of sight. The resulting distribution of magnification values, for a randomly drawn sample of 159 or 107 objects for the UltraVISTA/COSMOS and UDS/SXDS fields, respectively, is shown in Fig. C1. The simulated samples of objects were run 1000 times for each field, and the median and 68-percentiles were calculated from the derived magnification distributions. The random distributions are very similar to those observed in our samples, showing median values of 0.11 and 0.18, indicating that modest gravitational lensing of our objects is not unusual for high-redshift sources in the field. We therefore do not correct the absolute magnitudes of our objects for this magnification when determining the LF.

We find a slight difference in the observed and predicted magnification distributions between the UltraVISTA/COSMOS and UDS/SXDS fields. The K-band data in the UDS/SXDS field is deeper than that in UltraVISTA/COSMOS ($m_{AB} = 24.6$ as compared to $m_{AB} = 24.2$), which would imply a higher surface density of sources in the UDS/SXDS field and hence a higher derived magnification. Inspection of the distribution of the number of sources within a 10 arcsec radius of the sample of high-redshift galaxies shows that this is indeed the case, and the magnification distributions can be brought into closer agreement if we cut the UDS/SXDS catalogue at the same depth as the UltraVISTA/COSMOS data, which results in a shift in the peak of the magnification distribution faintwards by ~0.05 mag. Any residual difference is likely due to the slightly different redshift distribution between the two fields, a result of large-scale structure in the fields or the different relative depths in the multiwavelength images which can subtly bias the photometric redshifts.