# APOGEE Data Releases 13 and 14: Stellar Parameter and Abundance Comparisons with Independent Analyses

Henrik Jönsson, Carlos Allende Prieto, Jon Holtzman, Diane Feuillet, Keith

Hawkins, Katia Cunha, Szabolcs Mészáros, Sten Hasselquist, J.

Fernández-Trincado, D. García-Hernández, et al.

► To cite this version:

Henrik Jönsson, Carlos Allende Prieto, Jon Holtzman, Diane Feuillet, Keith Hawkins, et al. APOGEE Data Releases 13 and 14: Stellar Parameter and Abundance Comparisons with Independent Analyses. The Astronomical Journal, 2018, 156 (3), pp.126. 10.3847/1538-3881/aad4f5. hal-02122966

HAL Id: hal-02122966

https://hal.science/hal-02122966v1

Submitted on 5 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

APOGEE Data Releases 13 and 14: Stellar Parameter and Abundance Comparisons with
Independent Analyses

Henrik Jönsson¹,²,³ Ⓓ, Carlos Allende Prieto¹,² Ⓓ, Jon A. Holtzman⁴ Ⓓ, Diane K. Feuillet⁵ Ⓓ, Keith Hawkins⁶,⁷, Katia Cunha⁸,⁹,
Szabolcs Mészáros¹⁰,<fnref n="23" />, Sten Hasselquist⁴, J. G. Fernández-Trincado¹¹,¹², D. A. García-Hernández¹,², Dmitry Bizyaev¹³,¹⁴,
Ricardo Carrera¹⁵ Ⓓ, Steven R. Majewski¹⁶ Ⓓ, Marc H. Pinsonneault¹⁷ Ⓓ, Matthew Shetrone¹⁸ Ⓓ, Verne Smith¹⁹,
Jennifer Sobeck²⁰, Diogo Souto⁹ Ⓓ, Guy S. Stringfellow²¹ Ⓓ, Johanna Teske²²,<fnref n="24" />,<fnref n="25" />, and Olga Zamora¹,²

¹Instituto de Astrofísica de Canarias (IAC), E-38205 La Laguna, Tenerife, Spain; henrikj@astro.lu.se

²Universidad de La Laguna, Dpto. Astrofísica, E-38206 La Laguna, Tenerife, Spain

³Lund Observatory, Department of Astronomy and Theoretical Physics, Lund University, Box 43, SE-22100 Lund, Sweden

⁴New Mexico State University, Las Cruces, NM 88003, USA

⁵Max Planck Institute for Astronomy, Königstuhl 17, D-69117 Heidelberg, Germany

⁶Columbia University, 550 West 120th Street, New York, NY10027, USA

⁷Department of Astronomy, the University of Texas at Austin, 2515 Speedway Boulevard, Austin, TX 78712, USA

⁸Steward Observatory, The University of Arizona, 933 North Cherry Avenue, Tucson, AZ 85721-0065, USA

⁹Observatório Nacional, Rua General José Cristino, 77, 20921-400 São Cristóvão, Rio de Janeiro, RJ, Brazil

¹⁰ELTE Eötvös Loránd University, Gothard Astrophysical Observatory, Szombathely, Hungary

¹¹Departamento de Astronomía, Casilla 160-C, Universidad de Concepción, Concepción, Chile

¹²Institut Utinam, CNRS UMR6213, Univ. Bourgogne Franche-Comté, OSU THETA, Observatoire de Besançon, BP 1615, F-25010 Besançon Cedex, France

¹³Apache Point Observatory and New Mexico State University, P.O. Box 59, Sunspot, NM 88349-0059, USA

¹⁴Stemberg Astronomical Institute, Moscow State University, Moscow, Russia

¹⁵INAF-Osservatorio astronomico di Padova, Vicolo dell'Osservatorio 5, I-35122 Padova, Italy

¹⁶Department of Astronomy, University of Virginia, Charlottesville, VA 22904-4325, USA

¹⁷Department of Astronomy, The Ohio State University, 140 West 18th Avenue, Columbus, OH 43210, USA

¹⁸University of Texas at Austin, McDonald Observatory, Fort Davis, TX 79734, USA

¹⁹National Optical Astronomy Observatory, 950 North Cherry Avenue, Tucson, AZ 85719, USA

²⁰Department of Astronomy, Box 351580, University of Washington, Seattle, WA 98195, USA

²¹Center for Astrophysics and Space Astronomy, Department of Astrophysical and Planetary Sciences, University of Colorado, Boulder, CO 80309-0389, USA

²²Carnegie Department of Terrestrial Magnetism, 5241 Broad Branch Road, NW, Washington, DC 20015-1305, USA

Received 2018 May 3; revised 2018 June 30; accepted 2018 July 3; published 2018 August 29

Abstract

Data from the SDSS-IV/Apache Point Observatory Galactic Evolution Experiment (APOGEE-2) have been released as part of SDSS Data Releases 13 (DR13) and 14 (DR14). These include high-resolution H-band spectra, radial velocities, and derived stellar parameters and abundances. DR13, released in 2016 August, contained APOGEE data for roughly 150,000 stars, and DR14, released in 2017 August, added about 110,000 more. Stellar parameters and abundances have been derived with an automated pipeline, the APOGEE Stellar Parameter and Chemical Abundance Pipeline (ASPCAP). We evaluate the performance of this pipeline by comparing the derived stellar parameters and abundances to those inferred from optical spectra and analysis for several hundred stars. For most elements—C, Na, Mg, Al, Si, S, Ca, Cr, Mn, Ni—the DR14 ASPCAP analyses have systematic differences with the comparisons samples of less than 0.05 dex (median), and random differences of less than 0.15 dex (standard deviation). These differences are a combination of the uncertainties in both the comparison samples as well as the ASPCAP analysis. Compared to the references, magnesium is the most accurate alpha-element derived by ASPCAP, and shows a very clear thin/thick disk separation, while nickel is the most accurate iron-peak element (besides iron itself).

**Key words:** Galaxy: abundances – stars: abundances – surveys

**Supporting material:** machine-readable tables

1. Introduction

The Apache Point Observatory Galactic Evolution Experi-
ment (APOGEE-2) is an ongoing project within SDSS-IV
(Gunn et al. 2006; Eisenstein et al. 2011; Wilson et al. 2012;
Zasowski et al. 2013, 2017; Nidever et al. 2015; Albareti
et al. 2017; Blanton et al. 2017; Majewski et al. 2017;
Abolfathi et al. 2018), analyzing spectroscopically stars of all
major Galactic components using H-band spectra (R ~ 22,500).
The reduced spectra, information about the observations, and

the stellar parameters and stellar abundances determined from
the spectra are periodically released to the public. In Holtzman
et al. (2018), the two most recent data releases—DR13 from
2016 July and DR14 from 2017 August—are presented. DR13
contains 164,562 stars observed between 2011 April and 2014
July, while DR14 contains 277,371 stars observed between
2011 April and 2016 January. Currently, the APOGEE analysis
include 22 elements, meaning that the APOGEE data set is a
unique data set for astronomical research based on tracing
chemical evolution of stellar populations and/or chemical
tagging of stars. In this paper we attempt to assess the accuracy
of the DR13 and DR14 APOGEE stellar parameters and
abundances through a comparison with alternative analyses of a
subset of APOGEE stars.

<footnote n="23">²³Premium Postdoctoral Fellow of the Hungarian Academy of Sciences.</footnote>

<footnote n="24">²⁴Hubble Fellow.</footnote>

<footnote n="25">²⁵Carnegie Origins Fellow, jointly appointed to Carnegie DTM/OBS.</footnote>
The accuracy of the APOGEE stellar parameters and abundances has been examined several times before.

Hawkins et al. (2016b) made an independent spectroscopic analysis of the APOGEE spectra to determine abundances of C, N, O, Na, Mg, Al, Si, S, K, Ca, Ti, V, Cr, Mn, Co, and Ni in the 2012 giant stars from DR12 that also have asteroseismic analysis from *Kepler* light curves, which provides constraints on the stellar parameters. Their determined abundances show small differences to those of DR12 for many elements, but significant differences regarding Si, S, Ti, and V.

Souto et al. (2016) used the DR13 APOGEE spectra to determine C, N, O, Na, Mg, Al, Si, K, Ca, Ti, V, Cr, Mn, Co, and Ni in 12 giant stars within the open cluster NGC 2420 ([Fe/H] ~ −0.16). The results from this manual analysis compare well with DR13 abundances for most of the elements studied, although for Na, Al, and V there are larger differences.

Fernández-Trincado et al. (2016) manually re-analyzed the DR12 APOGEE spectrum of one peculiar metal-poor field giant star with a globular cluster (GC) second-generation (SG) abundance pattern. They derived the stellar parameters using iSpec (Blanco-Cuaresma et al. 2014) and found values very close to the APOGEE DR12 values. For the abundances of C, N, O, Mg, and Al, they found differences of about 0.3 dex when comparing their manually, MOOG<fnref n="26" />-derived abundances to those of DR12. Subsequently, the same star was analyzed using an optical spectrum by Pereira et al. (2017), confirming its SG abundance pattern, but their derived stellar parameters differ significantly from those of the APOGEE analysis pipeline: they arrive at an effective temperature that is 300 K lower than DR14, and a surface gravity that is 0.9 dex lower.<fnref n="27" /> The metallicity is, however, very similar in Pereira et al. (2017) and DR14. More DR13 APOGEE spectra of SG-type field stars have been analyzed manually by Fernández-Trincado et al. (2017), and the derived abundances generally agree within 0.2 dex with the abundances in DR13.

Souto et al. (2017) used the DR13 APOGEE spectra to determine C, O, Na, Mg, Al, Si, K, Ca, Ti, V, Cr, and Mn in two planet-hosting M-dwarf stars. This work shows that APOGEE spectra can be analyzed to determine detailed chemical compositions of M-dwarfs, if FeH is included in the analysis. Since this molecule is not included in the DR13 or DR14 line lists, this work concludes that no results from these data releases regarding M-dwarfs can be fully trusted, but that this issue may be solved in upcoming data releases; the plan is to use FeH in future APOGEE analysis.

Wilson et al. (2018) compared the effective temperatures and metallicities as derived by the APOGEE Stellar Parameter and Chemical Abundances Pipeline (ASPCAP, DR14) for 221 dwarf stars from the *Kepler* Object of Interest catalog to those derived from independent, optical analyses, and found the DR14 effective temperatures to be around 60 K lower than the optically determined effective temperatures in all their reference samples, with a spread of around 130 K. Regarding [Fe/H], they found a zero mean offset and a spread of 0.09 dex.

There have also been several other works in which one or more stellar parameter(s) have been independently determined and either have been, or could be, compared to those derived by APOGEE; for example, the APOKASC-project (Pinsonneault et al. 2014) has derived asteroseismic surface gravities

for 1916 red giants based on data from *Kepler* and APOGEE (as used by Hawkins et al. 2016b). This catalog has recently been updated to include 6681 targets (Pinsonneault et al. 2018). However, these surface gravities are subsequently used to calibrate the surface gravities in DR13 and DR14, and therefore cannot be considered as an independent analysis. The same type of asteroseismic-spectroscopic analysis has been made on 606 stars observed by *CoRoT* and APOGEE within the CoRoGEE project (Anders et al. 2017). Also, short-time variations—"flicker"—in the light-curves of *Kepler* targets have been used as a basis for determining surface gravities (Bastien et al. 2016), and there are of course several photometric calibration relations designed to estimate effective temperatures of stars (e.g., González Hernández & Bonifacio 2009).

Furthermore, the data-driven analysis code called the Cannon (Ness et al. 2015; Casey et al. 2016) has been used on APOGEE spectra to determine stellar parameters and stellar abundances for a majority of the DR14 APOGEE sample of stars (see Holtzman et al. 2018 for details). However, since this analysis is based on a training set from the ASPCAP DR14 results, it cannot be considered independent of the DR14 values. In fact, these results will be evaluated in this paper in addition to the ASPCAP-derived stellar parameters and abundances.

None of the works above has compared a large number of stars of any APOGEE release to reference analyses independently determining all the classical spectroscopic stellar parameters—effective temperature, surface gravity, and metallicity—as well as at least some elemental abundances. This paper conducts a deeper analysis of the accuracy of the APOGEE DR13/DR14 stellar parameters and abundances by comparing these results to those of sizable independent studies.

## 2. The APOGEE DR13 and DR14 Samples

The spectral analysis that determines the stellar parameters and chemical abundances is performed automatically by ASPCAP (García Pérez et al. 2016). The stellar parameters of a particular star are determined by optimization using a large library of pre-computed synthetic spectra with different stellar parameters, C, N, and overall alpha elemental abundances covering the entire APOGEE wavelength range 15140–16940 Å (Zamora et al. 2015). The same spectral library is then used with the determined stellar parameters fixed (to the uncalibrated values, see Section 4.1) to derive the abundances of the individual elements using windows corresponding to spectral lines that are sensitive to the element of interest. The determination of stellar parameters and abundances is made with the code FERRE<fnref n="28" /> (Allende Prieto et al. 2006) and the model atmospheres used are MARCS (Gustafsson et al. 2008) plane-parallel/spherical models (for high/low log g) for T<sub>eff</sub> < 3500 K and ATLAS9 (Kurucz 1979, and updates) plane-parallel models for T<sub>eff</sub> ≥ 3500 K (Mészáros et al. 2012). The spectral libraries in DR13 and DR14 were calculated with Turbospectrum (Alvarez & Plez 1998; Plez 2012) using plane-parallel or spherical radiative transfer consistently with the stellar model in question. The same line list is used to construct the synthetic spectral libraries for both DR13 and DR14 (internally tagged as 20150714). It is based on a thorough literature review in combination with astrophysical

<footnote n="26">²⁶ http://www.as.utexas.edu/~chris moog.html</footnote>

<footnote n="27">²⁷ In fact, the APOGEE analysis pipeline is concluded to be not very precise for SG stars with extreme "non-standard" abundance patterns in Section 4.2.</footnote>

<footnote n="28">²⁸ Available at http://github.com/callendeprieto/ferre.</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 1.** HR diagrams for the ASPCAP analyses of DR13 and DR14, as well as the DR14-based Cannon analysis. Stars that have calibrated values for all three “classical” spectroscopic stellar parameters are plotted color-coded according to their metallicity, while stars with one or more uncalibrated stellar parameter are plotted in gray. Crude regions of different types of stars are marked: hot stars (a), GK-giants (b), M-giants (c), FGK-dwarfs (d), and KM-dwarfs (e). As a guide for the eye, isochrones with [Fe/H] = 0.0 and ages 1–10 Gyr are plotted using solid dark gray lines. One isochrone with [Fe/H] = −1.0 and age 10 Gyr, and one with [Fe/H] = +0.5 and age 10 Gyr are plotted using dotted dark gray lines (Bressan et al. 2012).</figcaption></figure>

log gf-values determined using high-resolution atlas-spectra of the Sun and Arcturus (Livingston & Wallace 1991; Hinkle et al. 1995); for more details, see Shetrone et al. (2015). The elements analyzed and released in DR13 and DR14 are the same: C, N, O, Na, Mg, Al, Si, P, S, K, Ca, Ti, V, Cr, Mn, Co, Ni, Cu, Ge, Rb, Nd, and Yb. However, no comparisons for Cu, Ge, Rb, Nd, and Yb are made in this paper, mainly since the determination of these elements is a work in progress, and still not fully reliable; see Section 5.19.

For some of the observed stars, ASPCAP fails to determine the stellar parameters for one reason or another (the signal-to-noise ratio, S/N, could be very low, the star could be too cool/ hot, the star could be a spectroscopic binary, etc.), which means that 152,641 stars (93%) have stellar parameters in DR13 and 264,078 stars (95%) have stellar parameters in DR14.

The S/N distributions in the two data releases are similar, with DR14 having a slightly higher fraction of high-S/N spectra: in DR13 29% have S/N < 100 and for DR14 the fraction is 26%. The flag SN_WARN is triggered for stars with S/N < 70 and the flag SN_BAD is triggered for stars with S/N < 30.

The “raw” output from FERRE is calibrated to reproduce surface gravities determined by asteroseismology, to yield homogeneous abundances in clusters of stars, and to reproduce solar abundance ratios for stars with near-solar metallicity in the solar vicinity (for details, see Holtzman et al. 2018). Because of the lack of asteroseismic surface gravities for dwarfs at the time of data release calibration for DR13 and DR14, only stars with log g < 4.0 have calibrated surface gravities. As a consequence the giant and subgiant stars with log g < 4.0 (105,599 in DR13 and 159,047 in DR14) have more accurate ASPCAP-parameters than the dwarf stars with log g > 4.0.

The HR diagrams based on the DR13 and DR14 ASPCAP analyses as well as the DR14-based Cannon analysis are shown in Figure 1. The stars with calibrated values for all three “classical” spectroscopic stellar parameters (effective temperature, surface gravity, and metallicity) are color-coded with respect to their iron abundances, and the giant branch lines up just as expected from isochrones in effective temperature, surface gravity, and metallicity. Stars with one or more uncalibrated stellar parameter(s) are shown in gray. Also shown is a very crude division into different types of stars based on “typical,” optical spectroscopic reference samples as described in Section 3. From this division, the DR13 sample consists of 6% hot stars, 46% GK-giants, 12% M-giants, 26% FGK-dwarfs, and 10% KM-dwarfs (using uncalibrated T<sub>eff</sub> and log g); the same values for DR14 are 6%, 42%, 9%, 34%, and 10%, respectively. Hence, DR14 has a larger fraction of FGK-dwarfs and a lower fraction of GK-giants as compared to DR13.

To make the comparison between ASPCAP and other independent analyses in this paper as relevant as possible, we have chosen to compare only stars that are not flagged to have an uncertain or bad ASPCAP analysis (see Holtzman et al. 2015 for a description of the APOGEE flags), and which have calibrated values for all three stellar parameters in DR13/ DR14. For example, we know from Souto et al. (2017) that the DR13 and DR14 versions of ASPCAP are not producing reliable results for M-dwarfs due to a lack of FeH molecular lines in the adopted synthetic spectra, and from Figure 1, we see that the results for dwarfs from ASPCAP are not following the main sequence expected from isochrones. This means that we are left with the subgiant and GKM-giant stars, but this is still a majority of the APOGEE sample: 105,599 stars of the 152,641 stars with determined parameters in DR13 have calibrated values for all three stellar parameters (69%), while
159,047 of the 264,078 stars with determined parameters in DR14 have calibrated values for all three stellar parameters (60%).

### 3. Independent Analyses

We have surveyed the literature for suitable independent works with which to compare the ASPCAP-determined stellar parameters and abundances. In this section we describe the reference samples. We focus mainly on works that have a significant number of stars ($\gtrsim$100), and which have all classical stellar parameters—effective temperature, surface gravity, and metallicity—determined together with as many elemental abundances as possible, but we also discuss some other smaller works of special interest.

#### 3.1. Field Star Samples

When comparing stellar parameters and abundances between different works it is hard to say which one is most accurate, since “ground truth” usually is not available. Therefore, comparing two samples where the overlap is just a few stars is not expected to say much of interest on a statistical basis: it is desirable to be able to distinguish possible systematic trends of differences as functions of, for example, metallicity, effective temperature, and so on, from differences stemming from the combined random uncertainties. Due to the small number of overlapping stars with APOGEE, we did not use the samples of stars analyzed by Sousa et al. (2008, 2011b, 2011a), Adibekyan et al. (2012), Tsantaki et al. (2013), Suárez-Andrés et al. (2016), Delgado-Mena et al. (2017), Bensby et al. (2014), Battistini & Bensby (2015, 2016), Fuhrmann (1998, 2004, 2008, 2011), Fuhrmann & Chini (2012, 2015), Fuhrmann et al. (2017), Ivanyuk et al. (2017), Reddy et al. (2003, 2006), and Valenti & Fischer (2005) in the comparison. The small number of targets overlapping between these studies and APOGEE is not surprising, since these works are all mainly based on dwarf stars, and APOGEE is mainly targeting giant stars.<fnref n="29" /> Furthermore, we considered comparing to the GALAH survey, but it was not used because the overlap with their first data release included only 23 stars (Martell et al. 2017). A very interesting study for comparison with APOGEE is the metal-poor giant star sample of Ruchti et al. (2011), but unfortunately only one star from this sample is currently among the APOGEE-observed stars. Interesting samples, in spite of their small numbers, are the Gaia benchmark stars (Jofré et al. 2014, 2015; Heiter et al. 2015a), but since there are only four stars in this sample observed with APOGEE that have all three parameters calibrated in DR14, this sample was also not used.

A total of 342 of the 1304 stars analyzed by Petigura et al. (2017) are among the APOGEE targets, but only 20 of them are subgiants and in the regime with calibrated surface gravities.

Based on these lines of reasoning, the five comparison samples we have found most relevant for our purposes are the samples of Brewer et al. (2016), da Silva et al. (2015), the Gaia-ESO DR3, the sample of Jönsson et al. (2017), and our own sample based on analysis of optical spectra from the ARCES spectrometer. They are all described in more detail in Sections 3.1.1–3.1.5 below, and their HR diagrams are shown

in Figure 2. All these analyses have been made with 1D local thermal equilibrium (LTE) models, except in the case of iron, for which (very small) non-LTE (NLTE) corrections have been applied in Jönsson et al. (2017).

#### 3.1.1. BACCHUS-analyzed ARCES Stars

We have observed a sample of 100 stars using the optical spectrometer ARCES ($R \sim 32,000$) on the Apache Point 3.5 m telescope. The stars were chosen from the APOGEE catalog to have a spread in stellar parameters, and include both dwarfs and giants with a wide range of metallicities. The stars have $0.0 < V < 11.1$ and the spectra have an S/N that ranges from $50 \le S/N \le 300$, with a median S/N of 115 around 6000 Å.

For determination of the stellar parameters as well as the abundances of O, Na, Mg, Al, Si, S, K, Ca, Ti, V, Cr, Mn, Co, Ni, Cu, Rb, and Y, we used the Brussels Automatic Code for Characterizing High AccUracy Spectra (henceforth BACCHUS; Masseron et al. 2016). BACCHUS is a stellar parameter and abundance analysis pipeline that uses Turbospectrum in combination with MARCS spherical 1D LTE models. The model atmosphere grid is alpha-enhanced for the lower metallicities according to the “standard” MARCS scheme. The stellar parameters are determined in the classical way, demanding excitation and ionization equilibrium using a set of Fe I and Fe II lines. The analysis performed is similar to that described in Hawkins et al. (2015), with the exception of the line list used: here we used the Gaia-ESO line list (v.5, U. Heiter et al. 2015b, 2018, in preparation), complemented with line information from the VALD database (Kupka et al. 2000; Ryabchikova et al. 2015) for the non-covered wavelength regimes in the Gaia-ESO list.

The performance of BACCHUS has been thoroughly tested against a set of well-known Gaia benchmark stars (e.g., Jofré et al. 2014, 2015; Heiter et al. 2015b; Hawkins et al. 2016a) and found to be both accurate and precise. One particular strength of BACCHUS is that it uses spherical radiative transfer in the spectral synthesis, something that is recommended when analyzing giants (Heiter & Eriksson 2006). We refer the reader to Section 4.3.3 of Jofré et al. (2014) and Section 2.2 of Hawkins et al. (2015) for more details about BACCHUS and how it is employed for stellar parameters and abundance analysis.

Three tables present the results from this analysis: Table 1 presents the atomic data used, Table 2 presents abundances from individual spectral lines, and Table 3 summarizes the adopted stellar parameters and abundances for each star. These tables are given in their entirety online.

The elements that overlap with DR13/DR14 are O, Na, Mg, Al, Si, S, K, Ca, Ti, V, Cr, Mn, Co, Ni, Cu, and Rb and the number of non-flagged stars overlapping with DR13 are 83/98 (here and subsequently, these two numbers give the number of stars with all calibrated parameters/number with at least one uncalibrated parameter), and the number of non-flagged stars overlapping with DR14 are 81/96.

#### 3.1.2. Brewer et al. (2016)

Brewer et al. (2016) determined the abundances of C, N, O, Na, Mg, Al, Si, Ca, Ti, V, Cr, Mn, Ni, and Y in a sample of 1615 stars with $0.0 < V < 16.4$. The optical spectra used were recorded using the spectrometer HIRES ($R \sim 70,000$) at the 10 m telescope at Keck Observatory. They found that their

<footnote n="29">²⁹ Evaluating the ASPCAP performance in the uncalibrated regime of the FGK-dwarfs by comparing to reference samples containing a high degree of such stars will be performed in the future (J. Teske et al. 2018, in preparation).</footnote>
<figure><img src="image_4.png" /><figcaption>**Figure 2.** HR diagrams for the reference samples used in the comparisons in Sections 4.1–6. In the comparisons made in this paper, only stars with calibrated values for all three stellar parameters in APOGEE are used, and these stars (for DR14) are plotted color-coded in metallicity, the stars that are not in DR14 are plotted in gray, and the stars that do overlap with DR14 but only have uncalibrated values for one or more stellar parameter are plotted in darker gray. The elements overlapping between APOGEE and the reference in question are listed in each panel. As a guide for the eye, the same isochrones and regions as in Figure 1 are plotted in every panel.</figcaption></figure>

<table><caption>**Table 1**
Line Data Used in the BACCHUS Analysis</caption><thead><tr><th>Element</th><th>Wavelength (Å)</th><th>log gf</th><th>E<sub>low</sub> (eV)</th></tr></thead><tbody><tr><td>[O I]</td><td>6300.3038</td><td>-9.7150</td><td>0.0000</td></tr><tr><td>Na I</td><td>5682.6333</td><td>-0.7060</td><td>2.1020</td></tr><tr><td>Mg I</td><td>5711.0880</td><td>-1.7240</td><td>4.3460</td></tr><tr><td>Mg I</td><td>8712.6890</td><td>-1.2130</td><td>5.9320</td></tr><tr><td>Mg I</td><td>8717.8250</td><td>-0.8660</td><td>5.9330</td></tr></tbody></table>

(This table is available in its entirety in machine-readable form.)

<table><caption>**Table 2**
Line-to-line Abundances from the BACCHUS Analysis</caption><thead><tr><th>Star</th><th>Element</th><th>Line (Å)</th><th>Abundance</th></tr></thead><tbody><tr><td>2MASSJ00002012+5612368</td><td>Al</td><td>5557.1</td><td>6.62</td></tr><tr><td>2MASSJ00002012+5612368</td><td>Al</td><td>6696.0</td><td>6.68</td></tr><tr><td>2MASSJ00002012+5612368</td><td>Al</td><td>6698.7</td><td>6.76</td></tr><tr><td>2MASSJ00002012+5612368</td><td>Al</td><td>7835.3</td><td>6.72</td></tr><tr><td>2MASSJ00002012+5612368</td><td>Al</td><td>7836.1</td><td>6.70</td></tr></tbody></table>

(This table is available in its entirety in machine-readable form.)

precision decreased significantly for the 424 stars with S/N < 100 compared to the 1191 stars with S/N > 100.

The elements that overlap between this study and DR13/DR14 are C, N, O, Na, Mg, Al, Si, Ca, Ti, V, Cr, Mn, and Ni, and the number of non-flagged stars overlapping with DR13 are 75/187, and the number of non-flagged stars overlapping with DR14 are 50/225.

The stellar parameters were spectroscopically determined using the χ²-minimizing spectral synthesis code Spectroscopy

Made Easy (SME; Valenti & Piskunov 1996) on several spectral features, for example Fe I and Fe II lines, and the wings of the Mg I b triplet. The same code was used to determine the stellar abundances. They used ATLAS plane-parallel model atmospheres, and the line list used was originally from VALD, but the wavelengths, transition probabilities, and van der Waals broadening were changed to fit a spectrum of the Sun.
<table><caption>**Table 3**
Stellar Parameters and Abundances from the BACCHUS Analysis</caption><thead><tr><th>Star</th><th><i>T</i><sub>eff</sub></th><th>log g</th><th>[Fe/H]</th><th><i>v</i><sub>mic</sub></th><th>[O/Fe]</th><th>[Na/Fe]</th><th>[Mg/Fe]</th></tr></thead><tbody><tr><td>2MASSJ00002012+5612368</td><td>4751 ± 75</td><td>2.67 ± 0.49</td><td>0.22 ± 0.09</td><td>1.18 ± 0.07</td><td>-0.20 ± 0.05</td><td>0.04 ± 0.08</td><td>-0.06 ± 0.03</td></tr><tr><td>2MASSJ00012723+8520108</td><td>5956 ± 16</td><td>4.14 ± 0.06</td><td>0.20 ± 0.07</td><td>1.15 ± 0.04</td><td>...</td><td>0.19 ± 0.04</td><td>0.06 ± 0.03</td></tr><tr><td>2MASSJ00041502+5614532</td><td>4596 ± 57</td><td>2.74 ± 0.12</td><td>0.24 ± 0.09</td><td>0.97 ± 0.07</td><td>...</td><td>-0.07 ± 0.07</td><td>-0.17 ± 0.03</td></tr><tr><td>2MASSJ00100473+8601230</td><td>6565 ± 13</td><td>4.29 ± 0.29</td><td>0.06 ± 0.12</td><td>1.18 ± 0.07</td><td>...</td><td>0.14 ± 0.12</td><td>0.08 ± 0.07</td></tr><tr><td>2MASSJ00202846+6238519</td><td>4825 ± 27</td><td>3.02 ± 0.29</td><td>0.17 ± 0.08</td><td>1.16 ± 0.05</td><td>...</td><td>0.05 ± 0.04</td><td>-0.07 ± 0.02</td></tr></tbody></table>

**Note.** All abundances are relative to the solar abundances of Grevesse et al. (2007).

(This table is available in its entirety in machine-readable form.)

### 3.1.3. da Silva et al. (2015)

da Silva et al. (2015) determined the abundances of C, N, O, Na, Mg, Si, Ca, Ti, V, Mn, Ni, Cu, and Ba in a sample of 309 stars with $1.1 < V < 9.7$. The optical spectra were recorded using the spectrometer ELODIE ($R \sim 42,000$) at the Haute Provence Observatory. A subsample of 172 of the stars were previously analyzed in da Silva et al. (2011), but in the 2015 paper the sample was expanded, and more elemental abundances were determined.

The elements that overlap between this study and DR13/ DR14 are C, N, O, Na, Mg, Si, Ca, Ti, V, Mn, Ni, and Cu, and the number of non-flagged stars overlapping with DR13 are 33/38, and the number of non-flagged stars overlapping with DR14 are 30/43.

The stellar parameters were determined using the automatic equivalent width measurement code ARES (Sousa et al. 2007) and MOOG on Fe I and Fe II lines. ATLAS plane-parallel model atmospheres were used. The same method was used when determining the elemental abundances, except in the cases of C, N, O, and Na, where the spectral synthesis mode in MOOG was used instead of the equivalent width method. The original line list used was taken from VALD, but then it was astrophysically calibrated to fit the solar spectrum.

### 3.1.4. The Gaia-ESO Survey

The Gaia-ESO survey (Gilmore et al. 2012) is an ongoing optical spectroscopic survey that so far has observed more than 83,000 stars in which they intend to determine abundances of Li, C, N, O, Na, Mg, Al, S, Ca, Sc, Ti, V, Cr, Co, Ni, Zn, Y, Zr, Ba, La, Ce, Nd, and Eu. In their latest data release (DR3, from 2017 May), they present stellar parameters and abundances of 25,533 stars.<fnref n="30" /> The spectrometers FLAMES-GIRAFFE ($R \sim 20,000$) and FLAMES-UVES ($R \sim 47,000$) at the Very Large telescope are used for carrying out the observations, and the targeted stars have $12 \le J \le 17.5$.

The elements that overlap between DR3 of Gaia-ESO and DR13/DR14 are C, N, O, Na, Mg, Al, S, Ca, Ti, V, Cr, Co, Ni, and Nd, and the number of non-flagged stars overlapping with DR13 are 139/152, and the number of non-flagged stars overlapping with DR14 are 244/278.

The analysis of the Gaia-ESO spectra is done by several research groups—nodes—using their own preferred method. Some nodes use equivalent width methods and others spectral synthesis. In the end, all these results are averaged using an elaborate scheme based on the performance of the particular nodes for different types of stars (Smiljanic et al. 2014). To try to minimize the systematic differences between the nodes, they

all use the same model atmospheres (spherical MARCS for giants, and plane-parallel MARCS for dwarfs) and line list. A significant amount of work has been devoted to finding and vetting atomic data (U. Heiter et al. 2015b, 2018, in preparation), a task that has benefited many independent optical stellar spectroscopic works.

### 3.1.5. Jönsson et al. (2017)

Jönsson et al. (2017) determined the abundances of O, Mg, Ca, and Ti in a sample of 291 stars with $0.0 < V < 11.9$. The optical spectra used were recorded with the spectrometers FIES ($R \sim 67,000$) at the Nordic Optical telescope, NARVAL ($R \sim 65,000$) at the Télescope Bernard Lyot, and ESPaDOnS ($R \sim 65,000$) at the Canada-France-Hawaii Telescope. The spectra have $30 < S/N < 250$, but most are close to 100.

The elements that overlap between this study and DR13/ DR14 are O, Mg, Ca, and Ti, and the number of non-flagged stars overlapping with DR13 are 106/106, and the number of non-flagged stars overlapping with DR14 are 120/120.

The stellar parameters were determined using the SME code on Fe I, Fe II lines and wings of strong Ca I lines. The analysis was made using MARCS spherical 1D LTE models that were alpha-enhanced for the lower metallicities according to the “standard” MARCS scheme, and a slightly updated version of the Gaia-ESO line list (v.5, see their Section 3.1). NLTE corrections for Fe were applied (Lind et al. 2012).

## 3.2. GC Star Samples

Multiple populations in GCs are extensively studied in the literature using both photometric and spectroscopic data. To date, almost all GCs have been found to have multiple main sequences and/or subgiant and/or giant branches, (e.g., Piotto et al. 2007, 2015; Milone et al. 2008). These different populations in metal-poor clusters have different chemical compositions. For example, sodium and oxygen are found to vary such that the stars that were formed first—the first-generation (FG) stars—are sodium poor and oxygen rich, while the SG stars are sodium rich and oxygen poor (e.g., Carretta et al. 2009a, 2009b, 2009c).

As mentioned in Holtzman et al. (2015), the ASPCAP team has long suspected that the pipeline is not performing optimally for stars with extreme types of “non-standard” elemental abundance patterns, like SG GC stars. In Section 4.2, we attempt to quantize these problems by comparing APOGEE DR13, DR14, and the Cannon results to those of independent analyses of cluster stars.

Mészáros et al. (2015) analyzed 428 giant stars in 10 northern GCs using actual APOGEE DR10 spectra, photometric effective temperatures, and surface gravities determined

<footnote n="30">³⁰ http://www.eso.org/rm/api/v1/public/releaseDescriptions/92</footnote>
<figure><img src="image_5.png" /><figcaption>**Figure 3.** Differences in $T_{\text{eff}}$ and [Fe/H] for DR13, DR14, the Cannon, and the references. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, and the Jönsson et al. (2017) stars are marked with red dots.</figcaption></figure>

by isochrone fitting. They also made an extensive cross-match between the APOGEE-observed GC stars and previous works (in this paper we use Sneden et al. 1991, 1992, 1997, 2000, 2004; Kraft et al. 1992; Minniti et al. 1996; Shetrone 1996; Cavallo & Nagar 2000; Ivans et al. 2001; Kraft & Ivans 2003; Ramirez & Cohen 2003; Cohen & Meléndez 2005; Johnson et al. 2005; Yong et al. 2006, 2008; Carretta et al. 2009b; Koch & McWilliam 2010; Lai et al. 2011; O'Connell et al. 2011; Johnson & Pilachowski 2012). We have chosen to compare the APOGEE results to those of the optical references in Mészáros et al. (2015), and not the actual results of that work, to be consistent with the rest of the paper, where optical reference works are used. However, we have checked and found that the conclusions would be the same if the values from the independent H-band analysis of Mészáros et al. (2015) were used to compare to the APOGEE abundances.

# 4. Comparing the Stellar Parameters

## 4.1. Field Star Samples

Figure 3 shows the comparison between the ASPCAP DR13, DR14, and the Cannon parameters and those of the reference samples, as a function of both effective temperature and metallicity. Table 4 summarizes the mean differences and scatter. We find, in most cases, scatter that is consistent with the combined uncertainties of the samples being compared.

In DR14, calibrated surface gravities were not provided for warmer stars, hence these stars appear in the panels for DR13 but not for DR14.

The DR14-calibrated values and the values as determined from the Cannon are expected to be similar, since the Cannon has been trained on DR14-calibrated results, which is the case both when looking at Figure 3 and Table 4. The calibrated DR13 effective temperatures show a systematic offset of only −14 K and scatter of 115 K when compared to the references; for DR14 the same values are +53 and 108 K, and for the DR14 Cannon analysis the values are +30 and 132 K, indicating a slightly higher scatter for the Cannon effective
<table><caption>**Table 4**
Median and Standard Deviation for the Differences in Stellar Parameters between APOGEE and the References, in the Sense APOGEE – Reference</caption><thead><tr><th></th><th></th><th>BACCHUS</th><th>Brewer+(2016)</th><th>da Silva+(2015)</th><th>Gaia-ESO DR3</th><th>Jönsson+(2017)</th><th>All</th></tr></thead><tbody><tr><td rowspan="3"><i>T</i><sub>eff</sub></td><td>DR13</td><td>-31 ± 79 (83)</td><td>-62 ± 157 (75)</td><td>-84 ± 85 (33)</td><td>10 ± 106 (139)</td><td>34 ± 75 (106)</td><td>-14 ± 115 (436)</td></tr><tr><td>DR14</td><td>3 ± 45 (81)</td><td>34 ± 259 (50)</td><td>-35 ± 47 (30)</td><td>72 ± 70 (244)</td><td>67 ± 61 (120)</td><td>53 ± 108 (525)</td></tr><tr><td>Cannon</td><td>4 ± 53 (79)</td><td>-26 ± 303 (55)</td><td>-67 ± 87 (27)</td><td>40 ± 76 (248)</td><td>53 ± 74 (108)</td><td>30 ± 132 (517)</td></tr><tr><td rowspan="3"><i>log g</i></td><td>DR13</td><td>-0.10 ± 0.19 (83)</td><td>-0.05 ± 0.21 (75)</td><td>-0.23 ± 0.16 (33)</td><td>-0.00 ± 0.22 (139)</td><td>-0.04 ± 0.12 (106)</td><td>-0.06 ± 0.20 (436)</td></tr><tr><td>DR14</td><td>-0.13 ± 0.21 (81)</td><td>-0.08 ± 0.26 (50)</td><td>-0.28 ± 0.14 (30)</td><td>-0.03 ± 0.20 (244)</td><td>-0.05 ± 0.13 (120)</td><td>-0.08 ± 0.20 (525)</td></tr><tr><td>Cannon</td><td>-0.15 ± 0.17 (79)</td><td>-0.12 ± 0.25 (55)</td><td>-0.18 ± 0.14 (27)</td><td>-0.02 ± 0.22 (248)</td><td>-0.04 ± 0.14 (108)</td><td>-0.07 ± 0.21 (517)</td></tr><tr><td rowspan="3">[Fe/H]</td><td>DR13</td><td>-0.11 ± 0.08 (83)</td><td>-0.14 ± 0.06 (75)</td><td>-0.07 ± 0.05 (33)</td><td>-0.00 ± 0.13 (139)</td><td>0.04 ± 0.05 (106)</td><td>-0.04 ± 0.11 (436)</td></tr><tr><td>DR14</td><td>-0.03 ± 0.07 (81)</td><td>-0.07 ± 0.06 (50)</td><td>0.01 ± 0.06 (30)</td><td>0.05 ± 0.12 (244)</td><td>0.11 ± 0.05 (120)</td><td>0.04 ± 0.10 (525)</td></tr><tr><td>Cannon</td><td>-0.04 ± 0.06 (79)</td><td>-0.07 ± 0.09 (55)</td><td>0.01 ± 0.08 (27)</td><td>0.07 ± 0.11 (248)</td><td>0.12 ± 0.06 (108)</td><td>0.05 ± 0.11 (517)</td></tr></tbody></table>

**Note.** These values encompass systematic and random uncertainties in the APOGEE analysis as well as the reference work. The number in parenthesis is the number of stars used in the comparison. The values for effective temperatures are not representative, since there is a trend of effective temperature in the APOGEE data. For more information, see Section 4.1.

temperatures. From the bottom panel in the leftmost column of plots, it is apparent that the scatter is increasing in the Cannon analysis for effective temperatures around 5000 K. This can possibly be traced to the upwards “flare” left of the red clump in the Cannon HR diagram in Figure 1.

In the second column of panels in Figure 3, a more or less clear trend is seen in the effective temperature difference between DR13 and all comparison samples as a function of [Fe/H]. This trend was also found when comparing the ASPCAP effective temperatures to effective temperatures derived from photometry. While no effective temperature calibration was applied in DR13, Holtzman et al. (2018) suggest a relation to be applied to DR13 to remove this effect. A similar effective temperature correction was applied as part of the DR14 calibrations, and consequently, the trend is much less pronounced in the plot showing the DR14-calibrated parameters (fourth row, second column panel in Figure 3). However, a weak residual is still present in DR14 (and the DR14 Cannon values) for the most metal-rich stars, in the sense that the calibrated DR14 ASPCAP effective temperatures are approximately 100 K higher than the optical effective temperatures.

Even if the ASPCAP trend of effective temperature with metallicity is reduced with the calibrated effective temperatures presented in DR14, any trend can potentially have far-reaching consequences for the derived abundances, since the uncalibrated stellar parameters (effective temperature, surface gravity, and metallicity) are used when determining the abundances in ASPCAP. This methodology is motivated by the fact that in H-band spectra of giants at the resolution of APOGEE, many of the spectral lines of interest are somewhat blended by the vast amount of molecular lines present, and using the stellar parameters that—on a global level—best fit the spectrum will do the best job at synthesizing, and hence removing the impact of, blending lines. However, some elements whose spectral lines show a large dependence on the adopted effective temperature might be more precisely determined if calibrated stellar parameters were to be used instead (see discussion in Sections 5.1–5.19).

In the two rightmost columns of panels in Figure 3 and in Table 4, systematic zero-point differences regarding metallicity scales can be seen: for example, DR13 is systematically ~0.04 dex higher in [Fe/H] than what is found in Jönsson et al. (2017) for the very same stars, while ~0.11 dex lower than the BACCHUS-analyzed ARCES stars. [Fe/H] in DR14 is ~0.11

dex higher than Jönsson et al. (2017), and ~0.03 dex lower than the BACCHUS-analyzed ARCES stars. There is a possible negative trend for $\Delta[\text{Fe/H}]$ as a function of $T_{\text{eff}}$ for the calibrated DR13 values (surprisingly not as obvious for the DR13 uncalibrated values), but in no other panels in the two rightmost columns can obvious trends be seen when all reference values are taken into account. The fact that different reference samples do not always agree with each other highlights the challenges of determining accurate stellar parameters.

## 4.2. GC Star Samples

In Figure 4 we have plotted differences of effective temperature and surface gravities as determined by DR13, DR14, and the Cannon as compared to the optical references for FG and SG GC stars.

From this plot, one can draw the conclusion that the APOGEE analyses are over-estimating (with respect to the optical studies taken as references) effective temperatures and surface gravities for SG stars, and especially so for stars with extreme SG-type abundance patterns (the oxygen-poor, blue points). For FG stars, the spread in effective temperatures is about the same as when comparing to the disk-type abundance-pattern stars in Figure 3.

It is also clear that the APOGEE stellar parameters for the extreme SG-type field star from Fernández-Trincado et al. (2016) actually are expected to show the large deviations found in Pereira et al. (2017) ($\Delta T_{\text{eff}} = +317$ K and $\Delta \log g = +0.89$ in the sense APOGEE – Pereira et al. 2017). These large inaccuracies in stellar parameters obviously heavily influence all the APOGEE-determined abundances for the extreme SG-type stars. This is also easily confirmed from Pereira et al. (2017) where abundance differences of up to 0.5 dex compared to DR14 can be seen. Unfortunately, all inaccuracies in the APOGEE abundances seem to work in the sense that they tend to erase the SG abundance pattern of C, N, O, Na, and Al of the star, i.e., compared to the references, the SG-typical low carbon and oxygen abundances are determined higher by ASPCAP, and vice versa for the SG-typical high nitrogen, sodium, and aluminum abundances that are determined lower by ASPCAP.

In the rest of this paper we avoid using comparison works with an expected high ratio of “non-standard” abundance-pattern stars. However, from the design and target selection of the APOGEE survey, the number of stars with these types of
<figure><img src="image_6.png" /><figcaption>**Figure 4.** Differences of effective temperature and surface gravities as determined by DR13, DR14, and the Cannon as compared to the optical references for first- and second-generation globular cluster stars, color-coded with [O/Fe] from the reference in question. The stars have been randomly spread out somewhat along the x-axis to make the plot clearer and show all points.</figcaption></figure>

extreme SG abundance patterns is believed to be very small. For example, Fernández-Trincado et al. (2017) conducted a search for such stars within DR13 and found 260 stars. However, as described above, ASPCAP analysis of stars with atypical abundance patterns leads to systematic errors in the stellar parameters that in turn result in systematic errors in chemical abundances such that these stars appear less atypical, so the actual number of stars in the DR13 sample with SG-type abundance pattern might be much higher.

## 5. Comparing the Abundances

In this section, we assess the APOGEE/ASPCAP chemical abundances element by element. Table 5 presents a summary, showing the median difference and spread between the ASPCAP analysis and the literature values for elements that, as discussed below, do not show systematic trends with any stellar parameter. Table 6 shows comparable results for other elements, although these may be less meaningful because there are systematic trends for these. Generally speaking, one can see from Table 5 that DR14 appears to have more precise and accurate abundances than DR13. The spread of the Cannon abundances is greater than that of ASPCAP DR14 abundances for all elements but silicon. It is important to note that while, for example, there are 50 stars in Brewer et al. (2016) that have calibrated stellar abundances in DR14, there are not necessarily 50 Brewer points in every element comparison: for some stars either Brewer et al. (2016) or ASPCAP might have failed to determine the abundance in question. The same is true for all comparison samples and elements. This and the fact that not all APOGEE abundances are derived in all comparison works results in some elements being more thoroughly evaluated than others, something that can be seen in the figures in this section as well as in Tables 5 and 6 and.

### 5.1. Carbon, C

In ASPCAP, the carbon abundance is determined in three ways: first, [C/M] is determined as one of the stellar parameters from fitting of the entire spectra. Second, once the stellar parameters have been fixed, the carbon abundance is determined from wide windows of the spectra covering numerous CN and CO molecular lines, among these the $^{13}$CO 3-0 molecular band head and several $^{12}$CO band heads (4-1, 5-2, 6-3, 7-4, 8-5, 9-6), that are sensitive to the derived effective temperature as well as surface gravity. Third, the carbon abundance is derived using mainly six regions with C I lines (the lines at 15784.5, 16004.9, 16021.7, 16333.9, 16505.2, and 16890.4 Å). The last two methods are expected to give more accurate carbon abundances, and calibrated values are only given for those. Therefore, they are used in the comparison below. However, the differences between the carbon abundances derived from the initial “parameter” run based on the entire spectra, and the subsequent run using solely features from carbon-bearing molecules, are only a few hundredths of a dex for the stars considered here.

When comparing the references with carbon abundances determined to the molecular carbon abundances of APOGEE (see the top row of panels in Figure 5), the DR13 [C/H] are systematically 0.02 dex lower than the references and the abundance difference shows a spread of 0.13 dex. In DR14, the systematic shift is 0.01 dex and the spread is 0.15 dex. The carbon abundances from the Cannon show a surprisingly large spread of 0.29 dex.

The third row of panels in Figure 5 shows the same comparison, but using the carbon abundance as derived from the C I lines instead. The results are similar, with DR13 showing a systematic shift of −0.10 dex and a spread of 0.14 dex, DR14 shows a shift of −0.06 dex and a spread of 0.15 dex.
<caption>Table 5</caption>

Median and Standard Deviation for the Differences in Abundances between APOGEE and the References, in the Sense APOGEE – Reference, for Elements Not Showing Any Trends with Stellar Parameters When Comparing to the References

<table><thead><tr><th></th><th></th><th>BACCHUS</th><th>Brewer+(2016)</th><th>da Silva+(2015)</th><th>Gaia-ESO DR3</th><th>Jönsson+(2017)</th><th>All</th></tr></thead><tbody><tr><td rowspan="3">C</td><td>DR13</td><td>...</td><td>-0.04 ± 0.09 (32)</td><td>-0.12 ± 0.06 (23)</td><td>0.12 ± 0.12 (35)</td><td>...</td><td>-0.02 ± 0.13 (90)</td></tr><tr><td>DR14</td><td>...</td><td>0.00 ± 0.12 (34)</td><td>-0.08 ± 0.06 (24)</td><td>0.15 ± 0.12 (37)</td><td>...</td><td>0.01 ± 0.15 (95)</td></tr><tr><td>Cannon</td><td>...</td><td>0.03 ± 0.12 (54)</td><td>-0.09 ± 0.08 (27)</td><td>0.07 ± 0.45 (48)</td><td>...</td><td>-0.00 ± 0.29 (129)</td></tr><tr><td rowspan="3">C I</td><td>DR13</td><td>...</td><td>-0.09 ± 0.08 (36)</td><td>-0.18 ± 0.07 (23)</td><td>0.07 ± 0.12 (35)</td><td>...</td><td>-0.10 ± 0.14 (94)</td></tr><tr><td>DR14</td><td>...</td><td>-0.06 ± 0.11 (35)</td><td>-0.16 ± 0.08 (25)</td><td>0.06 ± 0.13 (39)</td><td>...</td><td>-0.06 ± 0.15 (99)</td></tr><tr><td>Cannon</td><td>...</td><td>0.00 ± 0.14 (54)</td><td>-0.18 ± 0.08 (27)</td><td>0.02 ± 0.32 (48)</td><td>...</td><td>-0.03 ± 0.22 (129)</td></tr><tr><td rowspan="3">Na</td><td>DR13</td><td>-0.15 ± 0.12 (73)</td><td>-0.13 ± 0.09 (34)</td><td>-0.26 ± 0.28 (21)</td><td>-0.27 ± 0.26 (34)</td><td>...</td><td>-0.18 ± 0.19 (162)</td></tr><tr><td>DR14</td><td>-0.00 ± 0.11 (70)</td><td>-0.04 ± 0.10 (25)</td><td>-0.14 ± 0.21 (20)</td><td>-0.10 ± 0.22 (29)</td><td>...</td><td>-0.03 ± 0.16 (144)</td></tr><tr><td>Cannon</td><td>0.01 ± 0.17 (78)</td><td>0.01 ± 0.29 (55)</td><td>-0.07 ± 0.16 (25)</td><td>-0.07 ± 0.31 (41)</td><td>...</td><td>-0.02 ± 0.24 (199)</td></tr><tr><td rowspan="3">Mg</td><td>DR13</td><td>-0.11 ± 0.07 (79)</td><td>-0.04 ± 0.06 (36)</td><td>-0.16 ± 0.06 (23)</td><td>-0.12 ± 0.12 (114)</td><td>-0.05 ± 0.07 (105)</td><td>-0.09 ± 0.10 (357)</td></tr><tr><td>DR14</td><td>-0.03 ± 0.08 (79)</td><td>0.04 ± 0.06 (49)</td><td>-0.08 ± 0.07 (30)</td><td>-0.02 ± 0.10 (222)</td><td>0.02 ± 0.08 (117)</td><td>-0.01 ± 0.09 (497)</td></tr><tr><td>Cannon</td><td>-0.03 ± 0.08 (79)</td><td>0.02 ± 0.11 (55)</td><td>-0.11 ± 0.07 (27)</td><td>0.00 ± 0.16 (244)</td><td>0.00 ± 0.09 (108)</td><td>-0.01 ± 0.13 (513)</td></tr><tr><td rowspan="3">Al</td><td>DR13</td><td>-0.18 ± 0.06 (78)</td><td>-0.04 ± 0.25 (36)</td><td>...</td><td>-0.09 ± 0.15 (111)</td><td>...</td><td>-0.12 ± 0.15 (225)</td></tr><tr><td>DR14</td><td>-0.06 ± 0.07 (78)</td><td>0.05 ± 0.12 (34)</td><td>...</td><td>0.04 ± 0.16 (172)</td><td>...</td><td>0.01 ± 0.14 (284)</td></tr><tr><td>Cannon</td><td>-0.04 ± 0.09 (79)</td><td>-0.05 ± 0.16 (54)</td><td>...</td><td>0.03 ± 0.26 (234)</td><td>...</td><td>-0.01 ± 0.22 (367)</td></tr><tr><td rowspan="3">Si</td><td>DR13</td><td>-0.13 ± 0.08 (79)</td><td>0.01 ± 0.08 (36)</td><td>-0.15 ± 0.05 (23)</td><td>...</td><td>...</td><td>-0.12 ± 0.10 (138)</td></tr><tr><td>DR14</td><td>-0.04 ± 0.08 (79)</td><td>0.06 ± 0.14 (49)</td><td>-0.07 ± 0.05 (30)</td><td>...</td><td>...</td><td>-0.03 ± 0.11 (158)</td></tr><tr><td>Cannon</td><td>-0.04 ± 0.09 (79)</td><td>0.07 ± 0.12 (55)</td><td>-0.07 ± 0.05 (27)</td><td>...</td><td>...</td><td>-0.03 ± 0.10 (161)</td></tr><tr><td rowspan="3">S</td><td>DR13</td><td>-0.08 ± 0.14 (65)</td><td>...</td><td>...</td><td>-0.03 ± 0.19 (33)</td><td>...</td><td>-0.06 ± 0.16 (98)</td></tr><tr><td>DR14</td><td>-0.06 ± 0.12 (66)</td><td>...</td><td>...</td><td>0.00 ± 0.15 (37)</td><td>...</td><td>-0.03 ± 0.13 (103)</td></tr><tr><td>Cannon</td><td>-0.05 ± 0.13 (65)</td><td>...</td><td>...</td><td>-0.06 ± 0.16 (37)</td><td>...</td><td>-0.05 ± 0.14 (102)</td></tr><tr><td rowspan="3">Ca</td><td>DR13</td><td>-0.18 ± 0.08 (79)</td><td>-0.12 ± 0.05 (36)</td><td>-0.07 ± 0.05 (22)</td><td>0.04 ± 0.12 (84)</td><td>0.01 ± 0.06 (105)</td><td>-0.04 ± 0.12 (326)</td></tr><tr><td>DR14</td><td>-0.11 ± 0.08 (79)</td><td>-0.04 ± 0.13 (35)</td><td>-0.01 ± 0.07 (24)</td><td>0.12 ± 0.14 (138)</td><td>0.05 ± 0.07 (120)</td><td>0.03 ± 0.14 (396)</td></tr><tr><td>Cannon</td><td>-0.10 ± 0.16 (79)</td><td>-0.05 ± 0.15 (55)</td><td>-0.05 ± 0.10 (26)</td><td>0.14 ± 0.20 (173)</td><td>0.04 ± 0.10 (108)</td><td>0.01 ± 0.19 (441)</td></tr><tr><td rowspan="3">Ti II</td><td>DR13</td><td>-0.11 ± 0.19 (78)</td><td>-0.13 ± 0.19 (36)</td><td>-0.11 ± 0.12 (22)</td><td>0.06 ± 0.17 (107)</td><td>0.04 ± 0.14 (101)</td><td>-0.03 ± 0.19 (344)</td></tr><tr><td>DR14</td><td>-0.01 ± 0.16 (78)</td><td>-0.19 ± 0.21 (31)</td><td>-0.11 ± 0.16 (25)</td><td>0.15 ± 0.22 (197)</td><td>0.16 ± 0.14 (116)</td><td>0.09 ± 0.21 (447)</td></tr><tr><td>Cannon</td><td>0.02 ± 0.20 (79)</td><td>-0.08 ± 0.25 (55)</td><td>-0.07 ± 0.28 (27)</td><td>0.11 ± 0.35 (233)</td><td>0.24 ± 0.23 (108)</td><td>0.09 ± 0.31 (502)</td></tr><tr><td rowspan="3">Cr</td><td>DR13</td><td>-0.05 ± 0.09 (79)</td><td>-0.10 ± 0.07 (36)</td><td>...</td><td>0.03 ± 0.12 (82)</td><td>...</td><td>-0.03 ± 0.11 (197)</td></tr><tr><td>DR14</td><td>0.02 ± 0.09 (80)</td><td>-0.03 ± 0.07 (35)</td><td>...</td><td>0.09 ± 0.14 (131)</td><td>...</td><td>0.04 ± 0.12 (246)</td></tr><tr><td>Cannon</td><td>0.01 ± 0.10 (79)</td><td>-0.06 ± 0.15 (55)</td><td>...</td><td>0.11 ± 0.28 (166)</td><td>...</td><td>0.04 ± 0.24 (300)</td></tr><tr><td rowspan="3">Mn</td><td>DR13</td><td>0.03 ± 0.11 (78)</td><td>-0.21 ± 0.07 (36)</td><td>-0.05 ± 0.06 (23)</td><td>...</td><td>...</td><td>-0.03 ± 0.14 (137)</td></tr><tr><td>DR14</td><td>0.10 ± 0.11 (79)</td><td>-0.11 ± 0.10 (35)</td><td>0.03 ± 0.07 (25)</td><td>...</td><td>...</td><td>0.05 ± 0.14 (139)</td></tr><tr><td>Cannon</td><td>0.09 ± 0.11 (79)</td><td>-0.11 ± 0.10 (55)</td><td>0.01 ± 0.09 (27)</td><td>...</td><td>...</td><td>0.01 ± 0.15 (161)</td></tr><tr><td rowspan="3">Ni</td><td>DR13</td><td>-0.09 ± 0.10 (79)</td><td>-0.13 ± 0.05 (36)</td><td>-0.08 ± 0.06 (23)</td><td>0.01 ± 0.09 (71)</td><td>...</td><td>-0.07 ± 0.10 (209)</td></tr><tr><td>DR14</td><td>0.01 ± 0.10 (79)</td><td>-0.05 ± 0.04 (35)</td><td>-0.00 ± 0.05 (25)</td><td>0.09 ± 0.10 (104)</td><td>...</td><td>0.02 ± 0.10 (243)</td></tr><tr><td>Cannon</td><td>-0.01 ± 0.10 (79)</td><td>-0.05 ± 0.09 (55)</td><td>-0.02 ± 0.07 (27)</td><td>0.10 ± 0.15 (131)</td><td>...</td><td>0.03 ± 0.14 (292)</td></tr></tbody></table>

Note. These values encompass systematic and random uncertainties in as well the APOGEE analysis and the reference work. The number in parenthesis is the number of stars used in the comparison. None of the reference works has determined phosphorous abundances. For more information, see the relevant Sections 5.1–5.19.

dex, and the Cannon shows a shift of -0.03 dex and a spread of 0.22 dex.

However, as is obvious from the first and third rows of panels in Figure 5, there seem to be systematic differences among the high-resolution optical studies with, for example, the carbon abundances of Gaia-ESO being systematically lower compared to ASPCAP, and the carbon abundances of da Silva et al. (2015) being systematically higher. With the seemingly large systematic differences between the comparison samples, it is far from obvious which of the two ASPCAP-derived carbon abundances is more accurate; that derived from molecular lines or that from atomic lines. Nevertheless, looking at the individual references in Table 5 gives the impression that the molecular carbon abundance is closer to the reference values in two of the three cases (da Silva et al. 2015; Brewer

et al. 2016) which might indicate that the molecular carbon abundance reported by APOGEE is more accurate than the atomic carbon abundance. This is possibly corroborated by comparing the APOGEE molecular carbon trend with the atomic carbon trend in rows two and four of Figure 5, respectively, where the molecular trends are tighter, especially for the Cannon analysis. However, since the stars for which abundances are shown in this plot are all differently evolved giants, and hence are expected to have different amount of CN-processed material in their photospheres, it is far from certain that the trend is *expected* to be tight. Furthermore, we note that the number of lines available for “molecular” determination of carbon is much larger than for “atomic” carbon determination, and hence the abundance from molecular features is at least expected to be better for low-S/N spectra.
<table><caption>**Table 6**
Median and Standard Deviation for the Differences in Abundances between APOGEE and the References, in the Sense APOGEE – Reference, for Elements Showing Trends with Stellar Parameters When Comparing to the References That Warrant Further Investigation</caption><thead><tr><th></th><th></th><th>BACCHUS</th><th>Brewer+(2016)</th><th>da Silva+(2015)</th><th>Gaia-ESO DR3</th><th>Jönsson+(2017)</th><th>All</th></tr></thead><tbody><tr><td rowspan="3">N</td><td>DR13</td><td>...</td><td>0.09 ± 0.15 (33)</td><td>-0.20 ± 0.08 (22)</td><td>0.07 ± 0.17 (16)</td><td>...</td><td>-0.03 ± 0.18 (71)</td></tr><tr><td>DR14</td><td>...</td><td>0.19 ± 0.19 (34)</td><td>-0.11 ± 0.09 (23)</td><td>0.13 ± 0.15 (19)</td><td>...</td><td>0.08 ± 0.19 (76)</td></tr><tr><td>Cannon</td><td>...</td><td>0.03 ± 0.21 (54)</td><td>-0.13 ± 0.13 (25)</td><td>0.21 ± 0.23 (19)</td><td>...</td><td>-0.00 ± 0.22 (98)</td></tr><tr><td rowspan="3">O</td><td>DR13</td><td>-0.30 ± 0.16 (51)</td><td>-0.02 ± 0.13 (34)</td><td>-0.38 ± 0.07 (23)</td><td>-0.06 ± 0.16 (36)</td><td>-0.05 ± 0.12 (75)</td><td>-0.12 ± 0.18 (219)</td></tr><tr><td>DR14</td><td>-0.25 ± 0.16 (53)</td><td>0.04 ± 0.15 (34)</td><td>-0.33 ± 0.07 (24)</td><td>-0.00 ± 0.17 (38)</td><td>-0.01 ± 0.13 (84)</td><td>-0.07 ± 0.19 (233)</td></tr><tr><td>Cannon</td><td>-0.22 ± 0.17 (54)</td><td>-0.03 ± 0.18 (54)</td><td>-0.36 ± 0.09 (27)</td><td>-0.03 ± 0.24 (49)</td><td>-0.04 ± 0.15 (78)</td><td>-0.09 ± 0.21 (262)</td></tr><tr><td rowspan="3">K</td><td>DR13</td><td>-0.30 ± 0.14 (57)</td><td>...</td><td>...</td><td>...</td><td>...</td><td>-0.30 ± 0.14 (57)</td></tr><tr><td>DR14</td><td>-0.23 ± 0.15 (56)</td><td>...</td><td>...</td><td>...</td><td>...</td><td>-0.23 ± 0.15 (56)</td></tr><tr><td>Cannon</td><td>-0.18 ± 0.17 (57)</td><td>...</td><td>...</td><td>...</td><td>...</td><td>-0.18 ± 0.17 (57)</td></tr><tr><td rowspan="3">Ti I</td><td>DR13</td><td>-0.16 ± 0.13 (79)</td><td>-0.06 ± 0.15 (36)</td><td>-0.07 ± 0.10 (23)</td><td>0.03 ± 0.13 (109)</td><td>0.02 ± 0.12 (105)</td><td>-0.03 ± 0.15 (352)</td></tr><tr><td>DR14</td><td>-0.08 ± 0.13 (79)</td><td>0.02 ± 0.14 (35)</td><td>-0.01 ± 0.10 (25)</td><td>0.15 ± 0.13 (206)</td><td>0.11 ± 0.12 (120)</td><td>0.08 ± 0.15 (465)</td></tr><tr><td>Cannon</td><td>-0.06 ± 0.16 (79)</td><td>-0.06 ± 0.23 (55)</td><td>-0.01 ± 0.16 (27)</td><td>0.12 ± 0.22 (233)</td><td>0.07 ± 0.15 (108)</td><td>0.05 ± 0.21 (502)</td></tr><tr><td rowspan="3">V</td><td>DR13</td><td>-0.03 ± 0.16 (76)</td><td>-0.04 ± 0.11 (34)</td><td>-0.11 ± 0.11 (23)</td><td>0.03 ± 0.15 (34)</td><td>...</td><td>-0.04 ± 0.14 (167)</td></tr><tr><td>DR14</td><td>0.03 ± 0.15 (76)</td><td>0.08 ± 0.23 (34)</td><td>0.01 ± 0.12 (24)</td><td>0.13 ± 0.17 (34)</td><td>...</td><td>0.05 ± 0.18 (168)</td></tr><tr><td>Cannon</td><td>-0.08 ± 0.20 (79)</td><td>-0.04 ± 0.21 (54)</td><td>-0.01 ± 0.21 (27)</td><td>0.18 ± 0.31 (49)</td><td>...</td><td>-0.01 ± 0.25 (209)</td></tr><tr><td rowspan="3">Co</td><td>DR13</td><td>-0.06 ± 0.12 (78)</td><td>...</td><td>...</td><td>0.09 ± 0.14 (104)</td><td>...</td><td>0.03 ± 0.15 (182)</td></tr><tr><td>DR14</td><td>0.03 ± 0.14 (79)</td><td>...</td><td>...</td><td>0.22 ± 0.27 (196)</td><td>...</td><td>0.15 ± 0.24 (275)</td></tr><tr><td>Cannon</td><td>0.01 ± 0.17 (79)</td><td>...</td><td>...</td><td>0.19 ± 0.47 (222)</td><td>...</td><td>0.15 ± 0.42 (301)</td></tr></tbody></table>

Note. These values encompass systematic and random uncertainties in as well the APOGEE analysis and the reference work. The number in parenthesis is the number of stars used in the comparison. For more information, see the relevant Sections 5.1–5.19.

Regarding the cosmic origin of carbon, there are two isotopes of astrophysical interest with different origin: ¹²C and ¹³C. ¹²C is formed via the triple-alpha process in helium burning and, on a cosmic scale, about half of it is formed in massive stars and released into the interstellar medium (ISM) by type II supernovae (SNe II), and about half is formed in low-mass asymptotic giant branch (AGB) stars and is released into the ISM by stellar winds. ¹³C is formed in the CN cycle, and mainly by intermediate-mass AGB stars. The ¹²C/¹³C ratio in the photosphere of a star is expected to increase as the star ascends the giant branch, since material from deeper, hotter layers, where ¹²C can be turned into ¹³C via proton capture, is dredged up (Clayton 2003; Karakas & Lattanzio 2014). In future data releases, we hope to provide ¹²C/¹³C ratios.

## 5.2. Nitrogen, N

The nitrogen abundance is determined twice within ASPCAP, first from the entire spectra as one of the stellar parameters, and then once more during the abundance determinations, from wide regions of the spectra covering numerous CN molecular lines. Many of these lines are sensitive to the derived surface gravity. As with carbon, we use the second “non-parameter” nitrogen determination in the comparisons below. However, just as for carbon, the differences for the two nitrogen abundances are only of the order of a few hundredths of a dex for the stars considered here.

When comparing the references with nitrogen abundances determined to the nitrogen abundances of APOGEE (see the top row of panels in Figure 6), a very clear trend with [Fe/H] can be seen in all analyses, especially in the Cannon analysis. This trend could be a consequence of the trend of effective temperature with metallicity in DR13 and DR14. Since CN molecular lines influence much of the APOGEE spectra, the reason for this trend has to be tracked down: is it due to

systematics in the references or in ASPCAP? We will attend to this issue in future work.

In the bottom row of panels in Figure 6, a seemingly systematic difference between the nitrogen abundances derived in Brewer et al. (2016) and da Silva et al. (2015) can be seen. However, since the da Silva et al. stars are higher on the giant branch than the Brewer et al. stars, this difference might be the sign of convective motions dredging up nitrogen into the atmosphere of the more evolved giants.

Nitrogen is produced in the CN cycle, and on a cosmic scale, by intermediate-mass AGB stars and is released into the ISM via their stellar winds (Clayton 2003; Karakas & Lattanzio 2014).

## 5.3. Oxygen, O

In ASPCAP, the oxygen abundance is determined from 70 regions of the spectra covering numerous OH molecular lines. The OH lines, and thus the oxygen abundance derived from them, are very sensitive to the determined effective temperature.

Dobrovolskas et al. (2015) examined the 3D/NLTE-effects in H-band OH lines for extremely metal-poor stars ([Fe/H] ~ -3), and found negative corrections of about -0.2 dex. However, they did not specify any expected corrections for more metal-rich stars, like the bulk of the APOGEE sample. Asplund et al. (2004) investigated and compared different oxygen diagnostics—optical O1 lines, optical [O1] lines, OH vib-rot lines around 3 µm, OH rot-rot lines around 9–13 µm— and needed to use 3D/NLTE modeling to make them agree, something that might influence both the APOGEE and reference analyses, all performed in 1D LTE.

Oxygen is produced by massive stars through helium burning, and on a cosmic scale it is released to the ISM via SNe II (Clayton 2003), and is expected to show an alpha-
<figure><img src="image_7.png" /><figcaption>**Figure 5.** The first and third rows show differences in carbon abundance for the same stars in different analyses, and the second and fourth rows show [C/Fe] vs. [Fe/H] for the same stars in different analyses. In the upper two rows carbon abundances as derived from CN and CO molecules in ASPCAP are shown, while the lower two rows show the same thing, but with ASPCAP carbon abundances derived from atomic C I lines instead. The Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles.</figcaption></figure>

typical “knee”-like behavior in a [O/Fe] versus [Fe/H] plot. There is a strong trend between the ASPCAP-derived oxygen abundance and the references with [Fe/H] in the first row of panels in Figure 7. Just as for nitrogen, this trend might in fact be due to the temperature trend with metallicity affecting the determined oxygen abundance.

The bottom row of panels in Figure 7 shows the [O/Fe] versus [Fe/H] trends for the various analyses, and the trend of determined oxygen abundance with [Fe/H] from the top row of panels is reflected in that the APOGEE [O/Fe] versus [Fe/H] trends do not reach as high [O/Fe] for low metallicity stars as the reference works.

### 5.4. Sodium, Na

In ASPCAP, the sodium abundance is determined from two weak (in GK-giants) and possibly blended lines at 16373.9 and 16388.9 Å.

Souto et al. (2016) used the same lines, and derived sodium abundances about 0.2 dex higher than the calibrated DR13 abundances in their manual re-analysis of DR13 APOGEE spectra of 12 giants in NGC 2420 ([Fe/H] ~ -0.16). Cunha et al. (2015) discussed the NLTE effects for these lines for the 11 metal-rich giants from NGC 6791 ([Fe/H] ~ +0.3) and found them to be very small (maximum 0.04 dex). Lind et al. (2011), however, showed that NLTE corrections might be large for certain combinations of stellar parameters for several of the
<figure><img src="image_8.png" /><figcaption>**Figure 6.** The first row shows differences in nitrogen abundance for the same stars in different analyses, and the second row shows [N/Fe] vs. [Fe/H] for the same stars in different analyses. The Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>**Figure 7.** The first row shows differences in oxygen abundance for the same stars in different analyses, and the second row shows [O/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, the Jönsson et al. (2017) stars are marked with red dots, and the APOGEE results are marked with black filled circles. In the bottom row panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

often used optical spectral lines, something that possibly might influence the accuracy of some of the comparison works.

Compared to the references (see the top row of panels in Figure 8), the DR13 [Na/H] are systematically 0.18 dex lower and the abundance differences show a spread of 0.19 dex. In DR14, the systematic shift is −0.03 dex and the spread 0.16 dex. The sodium abundances from the Cannon show a systematic shift of −0.02 dex and a significantly larger spread of 0.24 dex as compared to the references. The da Silva et al.

(2015) stars in general have the highest sodium abundances; see the second row of panels in Figure 8. In general, the [Na/Fe] versus [Fe/H] trends for the giants follow the trend of the dwarf stars of Bensby et al. (2014).

Sodium is mainly produced by explosive carbon burning in SNe II, and is in this process deposited into the ISM. However, about a tenth of the sodium in the cosmos is instead produced in helium-burning shells of evolved lower-mass stars (especially in the more massive AGB stars), and later deposited into
<figure><img src="image_10.png" /><figcaption>**Figure 8.** The first row shows differences in sodium abundance for the same stars in different analyses, and the second row shows [Na/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>**Figure 9.** The first row shows differences in magnesium abundance for the same stars in different analyses, and the second row shows [Mg/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked using blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, the Jönsson et al. (2017) stars are marked with red dots, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

the ISM through stellar winds (Clayton 2003; Karakas & Lattanzio 2014).

### 5.5. Magnesium, Mg

In ASPCAP, the magnesium abundance is determined from 14 Mg I lines of different strengths, some of which are blended, and some which seem mostly unblended (in GK-giants). Three of the lines are strong in giants and have pressure-broadened

wings, meaning that the determined magnesium abundance is likely to be sensitive to the derived surface gravity.

Compared to the reference studies, the DR13 magnesium abundances are 0.09 dex lower with a spread of 0.10 dex, while for DR14, the systematic shift is −0.01 dex and the spread is 0.09 dex, making magnesium the alpha-element most accurately determined by APOGEE (as compared to the references).
<figure><img src="image_12.png" /><figcaption>**Figure 10.** The first row shows differences in aluminium abundance for the same stars in different analyses, and the second row shows [Al/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

There might however be a hint of a weak trend with [Fe/H]; see the first row of panels in Figure 9.

Zhang et al. (2017) evaluated the NLTE effects for eight H-band Mg I lines, finding relatively large negative corrections for the three strong lines at 15740.7, 15749.0, and 15765.8 Å of about −0.15 dex for GK-giants, and larger corrections for stars higher up the giant branch. If applicable to all of the H-band lines used in the ASPCAP analysis, this negative NLTE correction would mean that the magnesium abundance derived by APOGEE would be overestimated, but the opposite is suggested from the comparison with the references, especially for the metal-poor stars.

Magnesium is produced via carbon burning, and on a cosmic scale it is an alpha-element mainly returned to the ISM through SNe II (Clayton 2003). As such it is expected to show the typical “knee”-like behavior in a [Mg/Fe] versus [Fe/H] plot, which it does in all analyses; see the bottom row of panels of Figure 9. Also, the distinction between the thin and thick disk abundance patterns is clearly visible in all analyses.

## 5.6. Aluminium, Al

In ASPCAP, the aluminium abundance is determined mainly from three regions of the spectra covering the Al I lines around 16718.9, 16750.5, and 16763.4 Å. The lines are sensitive to the derived effective temperature and, since they are rather strong, they are also sensitive to the adopted surface gravity and microturbulence.

In Smith et al. (2013) and Souto et al. (2016), only the two Al I lines at 16718.9 and 16763.4 Å were used. Souto et al. (2016) derived aluminum abundances about 0.15 dex higher compared to calibrated DR13 abundances in their manual re-analysis of APOGEE spectra of 12 giants in NGC 2420 ([Fe/H] ~ −0.16).

Hawkins et al. (2016b) used only one line at 16763.4 Å, finding that the 16718.9 Å line is poorly fit in the core, and

suggested this is due to NLTE effects. This suspicion was corroborated by Nordlander & Lind (2017), showing that the NLTE effects for this line can be of the order of 0.2 dex, depending on the stellar parameters. Possibly for this reason, Hawkins et al. (2016b) found about 0.1 dex lower aluminium abundances for metal-poor stars compared to DR12.

The DR13 aluminium abundances are 0.12 dex lower than the references, with a spread of 0.15 dex. For DR14, the systematic shift is 0.01 dex and the spread is 0.14 dex, as shown in the first row of panels in Figure 10.

Aluminium is formed by carbon burning in massive stars, and on a cosmic scale it is released into the ISM by SNe II (Clayton 2003). Hence, as one would expect, the [Al/Fe] versus [Fe/H]-trend shows an alpha-like behavior in Bensby et al. (2014) and also in all APOGEE analyses in the second row of panels in Figure 10.

## 5.7. Silicon, Si

In ASPCAP, the silicon abundance is determined from 17 Si I lines of different strengths, some of which are blended, and some of which seem mostly unblended (in GK-giants).

Hawkins et al. (2016b) found systematically 0.2 dex lower silicon abundances compared to DR12, when using only the five lines at 15376.8, 15888.4, 16215.7, 16680.8, and 16828.2 Å.

In a global comparison with the references, the DR13 silicon abundances are 0.12 dex lower, with a spread of 0.10 dex, while for DR14, the systematic shift is −0.03 dex and the spread is 0.11 dex (see the first row of panels in Figure 11).

Zhang et al. (2016) predicted that NLTE-corrections for the H-band lines in GK-giants should be of the order of −0.2 dex for the two strong lines at 15888.4 and 16680.8 Å, but smaller for the other two lines investigated at 16380.2 and 16828.2 Å. However, they are also negative, meaning that the silicon abundance derived from those lines assuming LTE would be
<figure><img src="image_13.png" /><figcaption>**Figure 11.** The first row shows differences in silicon abundance for the same stars in different analyses, and the second row shows [Si/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

overestimated, at odds with what we are finding when comparing to the references. Zhang et al. (2016) also found that NLTE corrections should increase with decreasing surface gravity, making the situation more severe for the APOGEE targets on the top of the giant branch.

Silicon is mainly produced in oxygen burning, and on a cosmic scale it is deposited into the ISM by both SNe II and supernovae type Ia (SNe Ia) (Clayton 2003). It is expected to show an alpha-typical “knee”-like trend in a [Si/Fe] versus [Fe/H] plot. This shape is visible in all references and APOGEE analyses, but there are some systematic uncertainties affecting the silicon determination in the giant stars of Brewer et al. (2016), shifting the trend of the green diamonds downwards (second row of panels in Figure 11). The trends in all APOGEE analyses are tight, but do not clearly show the separation of the thin and thick disk type abundances. Surprisingly, there are two outliers in the otherwise very tight Cannon trend, which are not present in the DR14 analysis.

## 5.8. Phosphorus, P

In ASPCAP, the phosphorus abundance is determined from three P I lines that are all blended: 15711.5 Å, blended with a Fe I line, 16254.7 Å, blended with an OH molecular line, and 16482.9 Å, blended with a CO molecular line.

Hawkins et al. (2016b) used only the two lines at 15711.5 and 16482.9 Å to derive upper limits for the phosphorus abundances.

Phosphorus abundances have not been determined in any of the comparison works, and not much work has been done on the Galactic chemical evolution of phosphorus, mainly because there are no optical spectral lines. On a cosmic scale, phosphorus is believed to be formed by carbon and neon burning in massive stars (Clayton 2003), but current Galactic chemical evolution models encounter difficulties in fitting the

observations (Maas et al. 2017), leaving the actual origin of phosphorus relatively uncertain.

## 5.9. Sulfur, S

In ASPCAP, the sulfur abundance is determined from three S I lines, but for two of the lines, the number of usable data points in the reduced spectra—pixels in the APOGEE apStar files—is very low: from the line at 15403.8 Å, one pixel is used, and from the line at 16576.6 Å, two pixels are used. The last line at 15478.5 Å is blended with Fe I.

Hawkins et al. (2016b) derived lower sulfur abundances compared to DR12, especially for the most metal-rich stars, where they derived sulfur abundances up to about 0.2 dex lower. They dismissed a non-ASPCAP S I line at 15469.8 (blended with OH) on account of suspected NLTE/3D effects, and only used the ASPCAP S I line at 15478.5 Å (blended with Fe I).

Comparing with the references, the DR13 sulfur abundances are 0.06 dex lower, with a spread of 0.16 dex. For DR14, the systematic shift is -0.03 dex and the spread is 0.13 dex, as illustrated in the first row of panels in Figure 12.

Sulfur is produced via oxygen burning in massive stars, and then released into the ISM by SNe II (Clayton 2003), meaning that one would expect the alpha-typical “knee”-like trend in an [S/Fe] versus [Fe/H] plot. However, this is not visible in the trends in the second row of Figure 12 since there are no metal-poor stars in the overlapping sample; instead just the decreasing part of the trend is shown. No obvious separation between thin and thick disk abundance patterns can be seen.

## 5.10. Potassium, K

In ASPCAP, the potassium abundance is determined from two K I lines with suitable strengths (in GK-giants) at 15163.1 and 15168.4 Å that are slightly blended with CN-lines.
<figure><img src="image_14.png" /><figcaption>**Figure 12.** The first row shows differences in sulfur abundance for the same stars in different analyses, and the second row shows [S/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>**Figure 13.** The first row shows differences in potassium abundance for the same stars in different analyses, and the second row shows [K/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Reddy et al. (2003) are shown in the background as gray dots.</figcaption></figure>

On a cosmic scale, potassium is created in different amounts and ways in SNe II, depending on the mass of the progenitor, but is mainly the product of explosive oxygen burning (Clayton 2003).

The potassium abundances from the APOGEE spectra and the reference show trends with both $T_{\text{eff}}$ and metallicity, as shown in the top row of panels in Figure 13. The ASPCAP [K/Fe] versus [Fe/H] trends are much tighter and show a different behavior than the reference, while the results from the Cannon show a larger spread and a trend more resembling the optical (see the second row of panels in Figure 13). Since potassium is

believed to be produced by SNe II, an alpha-like trend as seen in the BACCHUS-analyzed ARCES stars seems more probable than the ASPCAP trends, even if the exact chemical evolution of potassium is unknown.

### 5.11. Calcium, Ca

In ASPCAP, the calcium abundance is determined from four Ca I lines at 16136.8, 16150.8, 16155.2, and 16157.4 Å. The lines all are of suitable strength and do not appear to be blended (in GK-giants).
<figure><img src="image_16.png" /><figcaption>**Figure 14.** The first row shows differences in calcium abundance for the same stars in different analyses, and the second row shows [Ca/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, the Jönsson et al. (2017) stars are marked with red dots, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

The DR13 calcium abundances are 0.04 dex lower than the references with a spread of 0.12 dex. For DR14, the systematic shift is +0.03 dex and the spread is 0.14 dex. There might also be a hint of a weak trend with $T_{\text{eff}}$; see the first row of panels in Figure 14.

Calcium is produced in massive stars through oxygen burning and silicon burning, and on a cosmic scale it is released into the ISM via SNe II (Clayton 2003). Calcium is therefore expected to show the alpha-typical “knee”-like behavior in a [Ca/Fe] versus [Fe/H] plot, and it does in all analyses; see the bottom row of panels of Figure 14. However, the distinction between the thin and thick disk abundance patterns is not nearly as obvious as for magnesium.

## 5.12. Titanium, Ti

ASPCAP derives both Ti I and Ti II abundances.

All nine Ti I lines used except the one at 15315.6 Å are very sensitive to the adopted effective temperature. However, this line has a very low weight in the ASPCAP windows used for determining Ti I, and therefore it does not influence the abundance very much. Therefore, the derived Ti I abundances in DR13/14 are expected to be very influenced by the trend of effective temperature with metallicity in the ASPCAP analysis. Indeed, the Ti I abundance is the only one which—according to our tests—most certainly would benefit from a change in methodology in ASPCAP, to instead use calibrated stellar parameters and not, as currently, uncalibrated parameters when determining abundances.

The Ti I lines at 15334.8 and 15715.6 Å are excluded in Hawkins et al. (2016b) due to them showing a strange [Ti/Fe] versus [Fe/H] trend (see their Figure 2). Their proposed explanation for this is NLTE and/or saturation effects, but since Hawkins et al. (2016b) used the DR10 effective temperatures in their analysis, these inaccurate trends are in

fact likely due to the trend of effective temperature with metallicity in the ASPCAP analysis.

The Ti II abundance is determined in APSCAP using a Ti II line at 15873.8 Å (Wood et al. 2014), and this is the only titanium line not discarded by Hawkins et al. (2016b) (see the first panel in their Figure 2). This line is not sensitive to the adopted effective temperature, but instead to the surface gravity.

Titanium is formed by explosive silicon burning and fusion of helium nuclei (to $^{48}$Cr that beta-decays to $^{48}$Ti). On a cosmic scale, titanium is mostly produced by SNe II (Clayton 2003), and as such it is expected to show the alpha-typical “knee”-like trend in a [Ti/Fe] versus [Fe/H] plot.

In the top row of panels in Figure 15, obvious trends of Ti I abundances versus [Fe/H] can be seen. Since the derived titanium abundance is sensitive to the adopted $T_{\text{eff}}$, this trend of titanium abundance difference with [Fe/H] could simply reflect the trend of effective temperature with [Fe/H]. In the second row of panels in Figure 15, the resulting inaccurate [Ti/Fe] versus [Fe/H] trends are shown.

In the third row of panels in Figure 15, the Ti abundances from the Ti II line are shown. Since they are derived from a single line that happens to fall close to the gap between two of the detectors, continuum placement is challenging and the abundances are very uncertain, which is reflected by the very large scatter. However, in the bottom row of panels, the [Ti/Fe] versus [Fe/H] plot from the Ti II line shows the expected “knee” but, since the trend has a lot of dispersion, no clear separation of thin and thick disk abundance trends can be seen.

## 5.13. Vanadium, V

In ASPCAP, the vanadium abundance is determined from five (hyperfine split, hfs) VI lines at 15924.8, 16031.1 Å (heavily blended with CN/CO),
<figure><img src="image_17.png" /><figcaption>**Figure 15.** The first row shows differences in Ti I abundance for the same stars in different analyses, and the second row shows [Ti I/Fe] vs. [Fe/H] for the same stars in different analyses. The third and fourth rows show the same, but for Ti II abundances. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, the Jönsson et al. (2017) stars are marked with red dots, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

16200.2 Å (blended with CO), 16406.1 Å (heavily blended with CO), and 16570.6 Å.

Hawkins et al. (2016b) and Souto et al. (2016) only used the 15924.8 Å line. Hawkins et al. (2016b) derived higher vanadium abundances as compared to DR12 for the metal-poor stars (almost 0.2 dex for [Fe/H] ~ -0.5). Souto et al. (2016) derived vanadium abundances about 0.1 dex higher, as compared to calibrated DR13 abundances in their manual re-analysis of APOGEE spectra of 12 giants in NGC 2420 ([Fe/H] ~ -0.16).

All lines used in ASPCAP are sensitive to the adopted effective temperature, and as such the derived vanadium abundances are expected to be influenced by the effective

temperature trend with metallicity in DR13/DR14. When comparing to the references in the upper row of panels in Figure 16, a positive trend with [Fe/H] indeed can be seen, especially in the Cannon analysis, which might be a product of the trend of effective temperature with metallicity.

Vanadium is produced in explosive oxygen and silicon burning, and on a cosmic scale, it is produced by both SNe II and SNe Ia in comparable amounts (Clayton 2003).

Regarding the [V/Fe] versus [Fe/H] trends in the lower row of panels in Figure 16, the only reference sample showing an obvious trend is that from Battistini & Bensby (2015). However, within the metallicity interval where comparison stars are available in any of the references (-0.5 ≲ [Fe/
<figure><img src="image_18.png" /><figcaption>**Figure 16.** The first row shows differences in vanadium abundance for the same stars in different analyses, and the second row shows [V/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Battistini & Bensby (2015) are shown in the background as gray dots.</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>**Figure 17.** The first row shows differences in chromium abundance for the same stars in different analyses, and the second row shows [Cr/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

H] ≲ 0.5), all trends are fairly flat. The BACCHUS-analyzed ARCES stars and those of Brewer et al. (2016) possibly show a slight negative trend of [V/Fe] for [Fe/H] ≳ 0, something that is not seen in the other samples. The trend found in Battistini & Bensby (2015) is very alpha-like, and indeed they draw the conclusion that vanadium likely is mainly produced in SNe II, which is the main producer of the alpha elements.

### 5.14. Chromium, Cr

In ASPCAP, the chromium abundance is determined from eight regions covering Cr I lines.

Compared with the references, the DR13 chromium abundances are 0.03 dex lower, with a spread of 0.11 dex. For DR14, the systematic shift is 0.04 dex and the spread is 0.12 dex, as illustrated in the first row of panels in Figure 17.

Chromium is produced in explosive silicon burning, and on a cosmic scale SNe Ia and SNe II contribute roughly equal parts
<figure><img src="image_20.png" /><figcaption>**Figure 18.** The first row shows differences in manganese abundance for the same stars in different analyses, and the second row shows [Mn/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the LTE values from Battistini & Bensby (2015) are shown in the background as light gray dots, and their NLTE-corrected values are shown as dark gray dots.</figcaption></figure>

of the total chromium budget (Clayton 2003). Its chemical evolution is expected to follow that of iron. Indeed, this is the case for all analyses in the [Cr/Fe] versus [Fe/H] trends in the bottom row of panels in Figure 17. Excluding some spurious outliers, the APOGEE trends seem tighter and more closely follow those of Bensby et al. (2014) than the trends of the BACCHUS-analyzed ARCES stars and the Gaia-ESO survey.

### 5.15. Manganese, Mn

In ASPCAP, the manganese abundance is determined from ten regions of the spectra covering (hfs) Mn I lines, but the three lines that are given the most weight in the manganese abundance determination are the (hfs) Mn I lines at 15159.2, 15217.8, and 15262.5 Å.

Compared with the references, the DR13 manganese abundances are 0.03 dex lower than the references with a spread of 0.14 dex, while for DR14, the systematic shift is 0.05 dex and the spread is 0.14 dex, see the first row of panels in Figure 18.

Manganese is produced by explosive silicon burning and in the alpha-rich freezeout in SNe II, and on a cosmic scale both SNe Ia and SNe II contribute significant parts of the manganese budget (Clayton 2003).

In the bottom row of Figure 18, the light gray dots mark the LTE results of Battistini & Bensby (2015), and the darker gray dots mark their NLTE-corrected abundances (M. Bergemann 2018, private communication). The APOGEE trends all appear very tight and closely follow the LTE trend of Battistini & Bensby (2015). This suggests that the manganese abundances of APOGEE might be very precise, but may need to be corrected for NLTE effects. We note that the [Mn/Fe] versus [Fe/H] trend from the BACCHUS-analyzed ARCES stars is much lower and more scattered than the APOGEE trend.

### 5.16. Iron, Fe

In ASPCAP, the iron abundance is determined from numerous windows in the spectra covering Fe I lines.

In this comparison work, [Fe/H] is considered one of the stellar parameters and hence discussed in Section 4.1.

### 5.17. Cobalt, Co

In ASPCAP, the cobalt abundance is determined from one (hfs) Co I line at 16757.6 Å, and three more regions with blended and weak lines (in GK-giants). The 16757.6 Å line is given the highest weight in the ASPCAP windows, and in practice drives the cobalt abundance determination.

Cobalt is produced by explosive silicon burning, in the alpha-rich freezeout in SNe II, and by the s-process. On a cosmic scale, both SNe Ia and SNe II contribute significant parts of the cobalt budget (Clayton 2003).

From the top row of panels in Figure 19, one can make out a trend of cobalt abundance difference between the APOGEE and reference values with metallicity, which is also seen as the upturn for the most metal-rich part of the [Co/Fe] versus [Fe/H] trends in the bottom row of panels. From the bottom row of plots, one can also note that the optical trends from the BACCHUS-analyzed ARCES stars and the Gaia-ESO survey do not follow that of Battistini & Bensby (2015).

As noted in Holtzman et al. (2018), the ASPCAP-derived cobalt abundances show strong effective temperature trends within clusters, and should be used with caution.

### 5.18. Nickel, Ni

In ASPCAP, the nickel abundance is determined from 30 regions of the spectra covering Ni I lines, several of which seem unblended and of suitable strength, while some are weak and blended (in GK-giants).
<figure><img src="image_21.png" /><figcaption>**Figure 19.** The first row shows differences in cobalt abundance for the same stars in different analyses, and the second row shows [Co/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the LTE values from Battistini & Bensby (2015) are shown in the background as light gray dots, and their NLTE-corrected values are shown as dark gray dots.</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>**Figure 20.** The first row shows differences in nickel abundance for the same stars in different analyses, and the second row shows [Ni/Fe] vs. [Fe/H] for the same stars in different analyses. The BACCHUS-analyzed ARCES stars are marked with blue squares, the Brewer et al. (2016) stars are marked with green diamonds, the da Silva et al. (2015) stars are marked with brown crosses, the values from Gaia-ESO DR3 are marked with black open circles, and the APOGEE results are marked with black filled circles. In the bottom row of panels, the values from Bensby et al. (2014) are shown in the background as gray dots.</figcaption></figure>

The DR13 nickel abundances are 0.07 dex lower than the references with a spread of 0.10 dex. For DR14, the systematic shift is +0.02 dex and the spread is 0.10 dex; see the first row of panels in Figure 20.

Nickel is produced by explosive silicon burning, in the alpha-rich freezeout in SNe II, and by the weak s-process. On a cosmic scale, both SNe Ia and SNe II contribute significant parts of the nickel budget (Clayton 2003).

The chemical evolution of nickel is expected to follow that of iron. Indeed, this is the case for all analyses in the [Ni/Fe] versus [Fe/H] trends in the bottom row of panels in Figure 20. All APOGEE analyses show very tight trends and nickel is the most accurate iron-peak element when compared to the references.
## 5.19. Neutron Capture Elements

The line list used for analysis in DR13 and DR14 did not include many transitions from neutron capture elements. All lines from such elements are weak and/or blended, and the current analysis methodology is likely to have significant challenges with these. As a result, DR13/DR14 abundances for these elements are not likely to be valid, and no calibrated abundances are given in DR14 (DR13 erroneously populated the calibrated arrays for Cu, Ge, Rb, and Y(Yb)); hence no comparisons are made here. Work is ongoing to improve the analysis. We describe the status for several elements below. Note that there are still unidentified lines in APOGEE spectra that may yield more possibilities.

### 5.19.1. Copper, Cu

In ASPCAP, a copper abundance determination is attempted from the heavily blended hfs Cu I line at 16005.8 Å and a very weak (in GK-giants) hfs Cu I line at 16639.0 Å.

Hawkins et al. (2016b) used the (hfs) Cu I lines at 16005.8 and 16006.6 Å, but only quoted upper limits on the copper abundance. Smith et al. (2013) used only the line at 16005.8 Å.

Further investigations will be made of these lines and their possible utility in APOGEE spectra before the next data release.

### 5.19.2. Germanium, Ge

A germanium abundance determination is attempted in ASPCAP from a very weak (in GK-giants) Ge I line at 16759.8 Å that is heavily blended with a Fe I line at the resolution of APOGEE. As noted in Holtzman et al. (2018), the ASPCAP-derived germanium abundances show strong effective temperature trends within clusters, and should be used with caution. Neither Smith et al. (2013) nor Hawkins et al. (2016b) attempted to determine the germanium abundance.

Further investigations will be made of this line and its possible utility in APOGEE spectra before the next data release.

### 5.19.3. Rubidium, Rb

In ASPCAP, a rubidium abundance determination is attempted from one weak (in GK-giants), heavily blended (with Fe I) Rb I line at 15289.5 Å.

Hawkins et al. (2016b) used the same line, but only quoted upper limits on the rubidium abundance, while Smith et al. (2013) did not measure rubidium at all.

Recent investigations of this line by the ASPCAP team have shown that it is too weak to be useful in the vast majority of the APOGEE spectra, and in fact the rubidium abundances have been removed in DR14.

### 5.19.4. Cerium, Ce

In upcoming versions of ASPCAP, cerium abundances will be determined from nine Ce II lines; see Cunha et al. (2017) for details. The lines are relatively unblended and of suitable strength (in GK-giants), and tests made by Cunha et al. suggest that cerium will be measurable in the bulk of APOGEE spectra through these lines. Cerium abundances are planned to be included in preparation for the next data release.

### 5.19.5. Neodymium, Nd

In ASPCAP, the neodymium abundances are currently derived using one very weak (in GK-giants), blended Nd I line at 16053.6 Å covering only two data points in the reduced spectrum. In upcoming versions, however, the 10 Nd I lines described in Hasselquist et al. (2016) will be added to the line list. Based on tests made by Hasselquist et al., these new lines will allow neodymium abundances to be reliably determined in about 18% of the APOGEE red giants.

### 5.19.6. Ytterbium, Yb

In ASPCAP, the quoted yttrium (Y) abundance in fact is the ytterbium (Yb) abundance, and it is derived from a weak (in GK-giants), CO-blended Yb II line 16498.4 Å. Hawkins et al. (2016b) used the same line to derive upper limits on the ytterbium abundance.

Further investigations will be made of this line and its possible use in APOGEE spectra in the next data release.

## 6. Conclusions

In this paper, we have compared the stellar parameters and abundances of independent optical studies to those presented by the APOGEE DR13 and DR14 (including the Cannon) on a star-by-star basis. We choose to only make the comparisons using stars with calibrated values supplied for all three “classic” spectroscopic stellar parameters in the APOGEE data releases, which leads us to restrict the comparison to subgiant and giant stars, and exclude dwarf stars. Since the giant stars are the main targets of the APOGEE survey, this approach leads to the best evaluation of the general performance of the analysis pipeline for the bulk of the surveyed stars.

For the stellar parameters, $log g$ and [Fe/H] match the optical works well, as is shown in Table 4. However, we have found that the effective temperatures in DR13, DR14, and in the DR14 analysis using the Cannon show trends with the metallicity of the star. The calibrated effective temperatures given in DR14 are better than those of DR13, and agree very well with the reference values for $-1.0 \lesssim [Fe/H] \lesssim -0.5$. For higher metallicities, the DR14 effective temperatures are too high by the order of 100 K, and the behavior is unclear for $[Fe/H] < -1.0$ since there are very few reference values for these low metallicities.

For most of the elements—C, Na, Mg, Al, Si, S, Ca, Cr, Mn, Ni—the DR14 ASPCAP analyses have systematic differences to the comparison samples of less than 0.05 dex (median), and random differences of less than 0.15 dex (standard deviation). Compared to the comparison samples, magnesium is the alpha element for which we find best consistency. The ASPCAP [Mg/Fe] versus [Fe/H] trend shows a clear thin/thick disk separation, is tight, and very similar to the reference works. When it comes to iron-peak elements, nickel is the most accurate element in APOGEE compared to the references (besides iron itself).

When it comes to elements formed by the r- and s-processes, work will be done to evaluate the possibilities of determining copper, germanium, and/or ytterbium abundances in upcoming data releases, and work is already ongoing on determining cerium and neodymium abundances for a majority of the observed stars, and they are planned to be included in the next data release.
The abundances of some elements—N, O, K, Ti, I, V, Co— show strong correlations with some determined stellar para- meters when comparing to the reference studies. Some of these trends might be due to the trend of determined effective temperature with metallicity in ASPCAP and the fact that these uncalibrated stellar parameters are used when subsequently determining the stellar abundances. Our tests have shown that this is the case for Ti I, but it is still uncertain whether the same can be said for the other elements. Regarding oxygen, for example, our tests have shown that the situation becomes worse if this change in methodology is invoked. This will be investigated further for the next data release, but for now we note that even if the accuracy of the ASPCAP-derived oxygen abundance might be in need of improvement, its precision is very high for stars of similar types (Bertran de Lis et al. 2016).

The best way to remove the impact of the effective temperature trend would be to identify and remove the source of the trend in the ASPCAP analysis; this might for example be accomplished by updating line lists and/or using specific windows in the spectra for the determination of the effective temperature (and/or the other stellar parameters). To better understand the origin and complete impact of the trend, more overlap between independently analyzed metal-poor stars is needed for the next data release. This could be accomplished by either targeting metal-poor giants from the work of Ruchti et al. (2011) with the APOGEE instrument, or by observing and analyzing already targeted metal-poor APOGEE stars using another spectrometer. Preferably both approaches should be used to make certain that any trend seen is not due to any possible systematics in the analysis of Ruchti et al. (2011).

Regarding the growing fraction of FGK-dwarf stars from DR13 to DR14 (from 26% to 34%), it would also be desirable to have more such stars in common between APOGEE DRs and independent analyses, to enable a evaluation of the performance of ASPCAP and possibly calibrate the results in this region of the HR diagram. There is a multitude of such studies available (Reddy et al. 2003, 2006; Bensby et al. 2014, etc.), some for which the overlap with APOGEE is already quite large (Brewer et al. 2016; Petigura et al. 2017). However, care must be taken to target a wide range of metallicities of these overlapping stars during coming observations with APOGEE so that any possible trend with metallicity can be traced.

H.J. acknowledges support from the Birgit and Hellmuth Hertz Foundation (via the Royal Physiographic Society of Lund), the Crafoord Foundation, and Stiftelsen Olle Engkvist Byggmästare. C.A.P. is grateful to the Spanish Ministry of Economy and Competitiveness (MINECO) for support through grant AYA2017-86389-P. D.K.F. acknowledges funds from the Alexander von Humboldt Foundation in the framework of the Sofja Kovalevskaja Award endowed by the Federal Ministry of Education and Research. K.H. is partially funded by the Simons Foundation Society of Fellows and the Flatiron Institute Center for Computational Astrophysics in New York City. S.M. has been supported by the Premium Postdoctoral Research Program of the Hungarian Academy of Sciences, and by the Hungarian NKFI Grants K-119517 of the Hungarian National Research, Development and Innovation Office. D.A. G.-H. and O.Z. acknowledge support provided by the Spanish Ministry of Economy and Competitiveness (MINECO) under

grant AYA-2017-88254-P. J.G.F.-T. is supported by FONDE-CYT No. 3180210.

Funding for the Sloan Digital Sky Survey IV has been provided by the Alfred P. Sloan Foundation, the U.S. Department of Energy Office of Science, and the Participating Institutions. SDSS acknowledges support and resources from the Center for High-Performance Computing at the University of Utah. The SDSS web site is www.sdss.org.

SDSS is managed by the Astrophysical Research Con- sortium for the Participating Institutions of the SDSS Collaboration including the Brazilian Participation Group, the Carnegie Institution for Science, Carnegie Mellon University, the Chilean Participation Group, the French Participation Group, Harvard-Smithsonian Center for Astrophysics, Instituto de Astrofísica de Canarias, The Johns Hopkins University, Kavli Institute for the Physics and Mathematics of the Universe (IPMU)/University of Tokyo, Lawrence Berkeley National Laboratory, Leibniz Institut für Astrophysik Potsdam (AIP), Max-Planck-Institut für Astronomie (MPIA Heidelberg), Max- Planck-Institut für Astrophysik (MPA Garching), Max-Planck- Institut für Extraterrestrische Physik (MPE), National Astro- nomical Observatories of China, New Mexico State University, New York University, University of Notre Dame, Observatório Nacional/MCTI, The Ohio State University, Pennsylvania State University, Shanghai Astronomical Observatory, United Kingdom Participation Group, Universidad Nacional Autón- oma de México, University of Arizona, University of Colorado Boulder, University of Oxford, University of Portsmouth, University of Utah, University of Virginia, University of Washington, University of Wisconsin, Vanderbilt University, and Yale University.

This publication made use of the SIMBAD database, operated at CDS, Strasbourg, France, and NASA's Astro- physics Data System.

Facilities: ARC (ARCES), Sloan (APOGEE).

Software: ASPCAP (García Pérez et al. 2016), ATLAS9 (Kurucz 1979, Castelli & Kurucz 2004), Brussels Automatic Code for Characterizing High AccUracy Spectra (BACCHUS) (Masseron et al. 2016), FERRE (Allende Prieto et al. 2006), MARCS (Gustafsson et al. 2008), Turbospectrum (Alvarez & Plez 1998; Plez 2012).

ORCID iDs

Henrik Jönsson https://orcid.org/0000-0002-4912-8609
Carlos Allende Prieto https://orcid.org/0000-0002- 0084-572X

Jon A. Holtzman https://orcid.org/0000-0002-9771-9622

Diane K. Feuillet https://orcid.org/0000-0002-3101-5921

Ricardo Carrera https://orcid.org/0000-0001-6143-8151

Steven R. Majewski https://orcid.org/0000-0003- 2025-3147

Marc H. Pinsonneault https://orcid.org/0000-0002- 7549-7766

Matthew Shetrone https://orcid.org/0000-0003-0509-2656

Diogo Souto https://orcid.org/0000-0002-7883-5425

Guy S. Stringfellow https://orcid.org/0000-0003-1479-3059

References

Abolfathi, B., Aguado, D. S., Aguilar, G., et al. 2018, ApJS, 235, 42
Adibekyan, V. Z., Sousa, S. G., Santos, N. C., et al. 2012, A&A, 545, A32
Albareti, F. D., Allende Prieto, C., Almeida, A., et al. 2017, ApJS, 233, 25
Allende Prieto, C., Beers, T. C., Wilhelm, R., et al. 2006, ApJ, 636, 804
Alvarez, R., & Plez, B. 1998, A&A, 330, 1109

Anders, F., Chiappini, C., Rodrigues, T. S., et al. 2017, A&A, 597, A30

Asplund, M., Grevesse, N., Sauval, A. J., Allende Prieto, C., & Kiselman, D. 2004, A&A, 417, 751

Bastien, F. A., Stassun, K. G., Basri, G., & Pepper, J. 2016, ApJ, 818, 43

Battistini, C., & Bensby, T. 2015, A&A, 577, A9

Battistini, C., & Bensby, T. 2016, A&A, 586, A49

Bensby, T., Feltzing, S., & Oey, M. S. 2014, A&A, 562, 71

Bertran de Lis, S., Allende Prieto, C., Majewski, S. R., et al. 2016, A&A, 590, A74

Blanco-Cuaresma, S., Soubiran, C., Heiter, U., & Jofré, P. 2014, A&A, 569, A111

Blanton, M. R., Bershady, M. A., Abolfathi, B., et al. 2017, AJ, 154, 28

Bressan, A., Marigo, P., Girardi, L., et al. 2012, MNRAS, 427, 127

Brewer, J. M., Fischer, D. A., Valenti, J. A., & Piskunov, N. 2016, ApJS, 225, 32

Carretta, E., Bragaglia, A., Gratton, R., D'Orazi, V., & Lucatello, S. 2009a, A&A, 508, 695

Carretta, E., Bragaglia, A., Gratton, R., & Lucatello, S. 2009b, A&A, 505, 139

Carretta, E., Bragaglia, A., Gratton, R. G., et al. 2009c, A&A, 505, 117

Casey, A. R., Hogg, D. W., Ness, M., et al. 2016, arXiv:1603.03040

Castelli, F., & Kurucz, R. L. 2004, arXiv:astro-ph/0405087

Cavallo, R. M., & Nagar, N. M. 2000, AJ, 120, 1364

Clayton, D. 2003, Handbook of Isotopes in the Cosmos, Hydrogen to Gallium (Cambridge: Cambridge Univ. Press)

Cohen, J. G., & Meléndez, J. 2005, AJ, 129, 303

Cunha, K., Smith, V. V., Hasselquist, S., et al. 2017, ApJ, 844, 145

Cunha, K., Smith, V. V., Johnson, J. A., et al. 2015, ApJL, 798, L41

da Silva, R., Milone, A. C., & Reddy, B. E. 2011, A&A, 526, A71

da Silva, R., Milone, A. D. C., & Rocha-Pinto, H. J. 2015, A&A, 580, A24

Delgado-Mena, E., Tsantaki, M., Adibekyan, V. Z., et al. 2017, A&A, 606, A94

Dobrovolskas, V., Kučínskas, A., Bonifacio, P., et al. 2015, A&A, 576, A128

Eisenstein, D. J., Weinberg, D. H., Agol, E., et al. 2011, AJ, 142, 72

Fernández-Trincado, J. G., Robin, A. C., Moreno, E., et al. 2016, ApJ, 833, 132

Fernández-Trincado, J. G., Zamora, O., García Hernández, D. A., et al. 2017, ApJL, 846, L2

Fuhrmann, K. 1998, A&A, 338, 161

Fuhrmann, K. 2004, AN, 325, 3

Fuhrmann, K. 2008, MNRAS, 384, 173

Fuhrmann, K. 2011, MNRAS, 414, 2893

Fuhrmann, K., & Chini, R. 2012, ApJS, 203, 30

Fuhrmann, K., & Chini, R. 2015, ApJ, 809, 107

Fuhrmann, K., Chini, R., Kaderhandt, L., & Chen, Z. 2017, MNRAS, 464, 2610

García Pérez, A. E., Allende Prieto, C., Holtzman, J. A., et al. 2016, AJ, 151, 144

Gilmore, G., Randich, S., Asplund, M., et al. 2012, Msngr, 147, 25

González Hernández, J. I., & Bonifacio, P. 2009, A&A, 497, 497

Grevesse, N., Asplund, M., & Sauval, A. J. 2007, SSRv, 130, 105

Gunn, J. E., Siegmund, W. A., Mannery, E. J., et al. 2006, AJ, 131, 2332

Gustafsson, B., Edvardsson, B., Eriksson, K., et al. 2008, A&A, 486, 951

Hasselquist, S., Shetrone, M., Cunha, K., et al. 2016, ApJ, 833, 81

Hawkins, K., Jofré, P., Heiter, U., et al. 2016a, A&A, 592, A70

Hawkins, K., Kordopatis, G., Gilmore, G., et al. 2015, MNRAS, 447, 2046

Hawkins, K., Masseron, T., Jofré, P., et al. 2015b, PhyS, 90, 054010

Heiter, U., & Eriksson, K. 2006, A&A, 452, 1039

Heiter, U., Jofré, P., Gustafsson, B., et al. 2015a, A&A, 582, A49

Heiter, U., Lind, K., Asplund, M., et al. 2015b, PhyS, 90, 054010

Hinkle, K., Wallace, L., & Livingston, W. C. 1995, PASP, 107, 1042

Holtzman, J. A., Hasselquist, S., Shetrone, M., et al. 2018, AJ, 156, 125

Holtzman, J. A., Shetrone, M., Johnson, J. A., et al. 2015, AJ, 150, 148

Ivans, I. I., Kraft, R. P., Sneden, C., et al. 2001, AJ, 122, 1438

Ivanyuk, O. M., Jenkins, J. S., Pavlenko, Y. V., Jones, H. R. A., & Pinfield, D. J. 2017, MNRAS, 468, 4151

Jofré, P., Heiter, U., Soubiran, C., et al. 2014, A&A, 564, A133

Jofré, P., Heiter, U., Soubiran, C., et al. 2015, A&A, 582, A81

Johnson, C. I., Kraft, R. P., Pilachowski, C. A., et al. 2005, PASP, 117, 1308

Johnson, C. I., & Pilachowski, C. A. 2012, ApJL, 754, L38

Jönsson, H., Ryde, N., Nordlander, T., et al. 2017, A&A, 598, A100

Karakas, A. I., & Lattanzio, J. C. 2014, PASA, 31, e030

Koch, A., & McWilliam, A. 2010, AJ, 139, 2289

Kraft, R. P., & Ivans, I. I. 2003, PASP, 115, 143

Kraft, R. P., Sneden, C., Langer, G. E., & Prosser, C. F. 1992, AJ, 104, 645

Kupka, F. G., Ryabchikova, T. A., Piskunov, N. E., Stempels, H. C., & Weiss, W. W. 2000, BaltA, 9, 590

Kurucz, R. L. 1979, ApJS, 40, 1

Lai, D. K., Smith, G. H., Bolte, M., et al. 2011, AJ, 141, 62

Lind, K., Asplund, M., Barklem, P. S., & Belyaev, A. K. 2011, A&A, 528, A103

Lind, K., Bergemann, M., & Asplund, M. 2012, MNRAS, 427, 50

Livingston, W., & Wallace, L. 1991, NSO Technical Report: An atlas of the solar spectrum in the infrared from 1850 to 9000 cm⁻¹ (1.1 to 5.4 micrometer) (Tucson, AZ: National Solar Observatory)

Maas, Z. G., Pilachowski, C. A., & Cescutti, G. 2017, ApJ, 841, 108

Majewski, S. R., Schiavon, R. P., Frinchaboy, P. M., et al. 2017, AJ, 154, 94

Martell, S. L., Sharma, S., Buder, S., et al. 2017, MNRAS, 465, 3203

Masseron, T., Merle, T., & Hawkins, K. 2016, BACCHUS: Brussels Automatic Code for Characterizing High accUracy Spectra, Astrophysics Source Code Library, ascl:1605.004

Mészáros, S., Allende Prieto, C., Edvardsson, B., et al. 2012, AJ, 144, 120

Mészáros, S., Martell, S. L., Shetrone, M., et al. 2015, AJ, 149, 153

Milone, A. P., Bedin, L. R., Piotto, G., et al. 2008, ApJ, 673, 241

Minniti, D., Peterson, R. C., Geisler, D., & Claria, J. J. 1996, ApJ, 470, 953

Ness, M., Hogg, D. W., Rix, H. W., Ho, A. Y. Q., & Zasowski, G. 2015, ApJ, 808, 16

Nidever, D. L., Holtzman, J. A., Allende Prieto, C., et al. 2015, AJ, 150, 173

Nordlander, T., & Lind, K. 2017, A&A, 607, A75

O'Connell, J. E., Johnson, C. I., Pilachowski, C. A., & Burks, G. 2011, PASP, 123, 1139

Pereira, C. B., Smith, V. V., Drake, N. A., et al. 2017, MNRAS, 469, 774

Petigura, E. A., Howard, A. W., Marcy, G. W., et al. 2017, AJ, 154, 107

Pinsonneault, M. H., Elsworth, Y., Epstein, C., et al. 2014, ApJS, 215, 19

Pinsonneault, M. H., Elsworth, Y. P., Tayar, J., et al. 2018, arXiv:1804.0998v1

Piotto, G., Bedin, L. R., Anderson, J., et al. 2007, ApJL, 661, L53

Piotto, G., Milone, A. P., Bedin, L. R., et al. 2015, AJ, 149, 91

Plez, B. 2012, Turbospectrum: Code for spectral synthesis, Astrophysics Source Code Library, ascl:1205.004

Ramirez, S. V., & Cohen, J. G. 2003, AJ, 125, 224

Reddy, B. E., Lambert, D. L., & Allende Prieto, C. 2006, MNRAS, 367, 1329

Reddy, B. E., Tomkin, J., Lambert, D. L., & Allende Prieto, C. 2003, MNRAS, 340, 304

Ruchti, G. R., Fulbright, J. P., Wyse, R. F. G., et al. 2011, ApJ, 737, 9

Ryabchikova, T., Piskunov, N., Kurucz, R. L., et al. 2015, PhyS, 90, 054005

Shetrone, M., Bizyaev, D., Lawler, J. E., et al. 2015, ApJS, 221, 24

Shetrone, M. D. 1996, AJ, 112, 1517

Smiljanic, R., Korn, A. J., Bergemann, M., et al. 2014, A&A, 570, A122

Smith, V. V., Cunha, K., Shetrone, M. D., et al. 2013, ApJ, 765, 16

Sneden, C., Kraft, R. P., Guhathakurta, P., Peterson, R. C., & Fulbright, J. P. 2004, AJ, 127, 2162

Sneden, C., Kraft, R. P., Prosser, C. F., & Langer, G. E. 1991, AJ, 102, 2001

Sneden, C., Kraft, R. P., Prosser, C. F., & Langer, G. E. 1992, AJ, 104, 2121

Sneden, C., Kraft, R. P., Shetrone, M. D., et al. 1997, AJ, 114, 1964

Sneden, C., Pilachowski, C. A., & Kraft, R. P. 2000, AJ, 120, 1351

Sousa, S. G., Santos, N. C., Israelian, G., et al. 2011a, A&A, 526, A99

Sousa, S. G., Santos, N. C., Israelian, G., Mayor, M., & Monteiro, M. J. P. F. G. 2007, A&A, 469, 783

Sousa, S. G., Santos, N. C., Israelian, G., Mayor, M., & Udry, S. 2011b, A&A, 533, A141

Sousa, S. G., Santos, N. C., Mayor, M., et al. 2008, A&A, 487, 373

Souto, D., Cunha, K., García Hernández, D. A., et al. 2017, ApJ, 835, 239

Souto, D., Cunha, K., Smith, V., et al. 2016, ApJ, 830, 35

Suárez-Andrés, L., Israelian, G., González Hernández, J. I., et al. 2016, A&A, 591, A69

Tsantaki, M., Sousa, S. G., Adibekyan, V. Z., et al. 2013, A&A, 555, A150

Valenti, J. A., & Fischer, D. A. 2005, ApJS, 159, 141

Valenti, J. A., & Piskunov, N. 1996, A&AS, 118, 595

Wilson, J. C., Hearty, F., Skrutskie, M. F., et al. 2012, Proc. SPIE, 8446, 84460H

Wilson, R. F., Teske, J., Majewski, S. R., et al. 2018, AJ, 155, 68

Wood, M. P., Lawler, J. E., & Shetrone, M. D. 2014, ApJL, 787, L16

Yong, D., Aoki, W., & Lambert, D. L. 2006, ApJ, 638, 1018

Yong, D., Meléndez, J., Cunha, K., et al. 2008, ApJ, 689, 1020

Zamora, O., García Hernández, D. A., Allende Prieto, C., et al. 2015, AJ, 149, 181

Zasowski, G., Cohen, R. E., Chojnowski, S. D., et al. 2017, AJ, 154, 198

Zasowski, G., Johnson, J. A., Frinchaboy, P. M., et al. 2013, AJ, 146, 81

Zhang, J., Shi, J., Pan, K., Allende Prieto, C., & Liu, C. 2016, ApJ, 833, 137

Zhang, J., Shi, J., Pan, K., Allende Prieto, C., & Liu, C. 2017, ApJ, 835, 90