Over 36% of US children have at least one chronic health condition. 1 Many of these children may benefit from the care of subspecialists, but families face substantial ongoing obstacles to accessing such care. The supply of pediatric subspecialists is limited and largely concentrated in urban areas, 2-4 creating significant geographic and socioeconomic barriers in access. [5] [6] [7] [8] [9] [10] [11] [12] Over 28% of children in need of subspecialty care have difficulty accessing this care, 1, 12 with children living in poverty, children identified as racial or ethnic minorities, and children living in rural communities disproportionally affected. [5] [6] [7] [8] 13 Telemedicine, broadly defined as remote medical care through telecommunication technology, 14 may expand access to pediatric subspecialty care and improve health outcomes. Live audiovisual telemedicine is one form of telemedicine which allows patients and physicians to communicate in real-time. Live interactive telemedicine has the potential to deliver subspecialty care feasibly and safely and can generate improved health outcomes. 15, 16 The American Academy of Pediatrics notes that telemedicine is an essential strategy to reduce healthcare disparities, 16 and many state Medicaid programs now cover some uses of live interactive telemedicine for pediatric care, with coverage rapidly expanded further in the context of the COVID-19 pandemic. 17 However, restrictions on telemedicine use remain, with state Medicaid regulations and state policy varying in details of use (e.g., facilities where patient is located; minimum distances between patient and subspecialist). 17 Many pediatric health systems offer some subspecialty telemedicine, but the volume and services offered vary. 18 Studies within systems and institutions that support telemedicine have found that use of telemedicine by individual clinicians and patients can still be highly variable, even when system-level infrastructure is in place. 19 Our objective, then, was to examine subspecialist and child-level factors associated with telemedicine utilization within state Medicaid programs with telemedicine coverage. During this phase of early adoption of telemedicine, we hypothesized that we would observe both potentially warranted variation in use of telemedicine (e.g., variation associated with geographic variables) and potentially unwarranted variation in use of telemedicine (e.g., variation associated with child race or ethnicity).
We examined 2014 Medicaid Analytic eXtract (MAX) data from 12 states, including all fee-for-service and managed care claims. These data represented the set of states with the most recent MAX data available from the Centers for Medicare and Medicaid Services at the time of our data acquisition in 2019. This study was determined to be exempt from human subjects review by the University of Pittsburgh Institutional Review Board.
Identifying Subspecialist Visits. Using the MAX "Other Services" record, which excludes inpatient stays and pharmacy records, we identified outpatient visits and consultations using common procedural terminology (CPT) evaluation and management codes (e.g., 99201-99205; 99211-99215; 99241-99245). We identified visits to medical or surgical subspecialists by linking billing National Provider Identifiers (NPIs) for each visit with National Plan and Provider Enumeration System (NPPES) data. Using the provider specialty taxonomy codes, we categorized a provider as a subspecialist if at least one listed taxonomy code was a medical subspecialty or surgical specialty. We then identified a small number of additional subspecialists (<1% of identified subspecialists) using Medicaid specialist codes. We excluded specialties with limited patient contact (e.g., radiology, pathology), and we excluded psychiatrists because telemedicine policy related to mental health care is quite different.
Identifying Telemedicine Visits and Telemedicine-Using Subspecialists. Among these visits, we identified telemedicine encounters by using billing modifiers "GT" or "GQ" or CPT code 99444, in keeping with prior studies. 20, 21 These modifiers, designed to indicate telemedicine encounters, have high sensitivity and specificity for outpatient telemedicine. 22 We examined the data for any use of telemedicine place of service codes as an alternative means of identifying telemedicine encounters, but found none of these codes. We labeled a subspecialist as "telemedicine using" if they had ≥1 identified telemedicine encounter. Subspecialists with no telemedicine encounters were labeled "telemedicine non-using." We identified all children who received care from telemedicine-using subspecialists, regardless of whether the child received care via telemedicine.
Subspecialist and Patient Characteristics. We used NPPES data to determine subspecialist gender and enumeration date. Most clinicians (85%) had enumeration dates consistent with the launch of the NPI system in 2007, and we could not further discern years in practice prior to this date for this group. Using taxonomy codes, we broadly categorized subspecialists as surgical versus medical subspecialists and as pediatric versus non-pediatric trained.
Child age, gender, Medicaid eligibility category (financial versus medical/disability), and months of enrollment in 2014 were obtained from the person summary file. Race and ethnicity were also obtained from the person summary file. Due to small cell sizes, we grouped race/ethnicity into 4 categories: non-Hispanic white; non-Hispanic Black; Hispanic or Latino/a/x; and "Other, Multiple, or Unknown." Among children categorized as "Other, Multiple, or Unknown" (15% of the overall sample), 20% were identified as Asian, 13% as American Indian or Pacific Islander, 7% as multiple races, and the rest had unknown race/ethnicity. We determined ZIP code median income using census data (categorized by federal poverty level for a family of four) and county rurality using rural-urban continuum codes. 23 Medicaid plan type (fee-for-service versus managed care organization) was encoded with individual visits, so was only available for children with at least one visit. For children with visits with a specific subspecialist, we determined the straight-line distance between the patient's ZIP code centroid and the subspecialist's primary practice location ZIP code centroid. Straight line distance provides a reasonable approximation to on-road travel time, 24 and was used because actual child street address was not available for more precise travel time.
Study Cohort. We included children 17 years of age and younger. We originally reviewed 17 states of MAX data, but 5 states had no telemedicine-using subspecialists. Because of our interest in subspecialist and child-level factors associated with telemedicine use within state Medicaid programs where telemedicine occurred, we excluded these 5 states from further analysis, leaving a sample of 12.8 million children across 12 states (Appendix Figure 1) . We further excluded children with <1 month of enrollment (2%) or missing geographic variables (2%).
Among included states, the percentage of people living in rural areas ranged from 5% to 51.3%, and the percentage of children covered by the Medicaid program ranged from 21.6% to 46.6% (Appendix Table 1 ). Included states had varied state Medicaid telemedicine policies in 2014, with prior analyses noting that eleven of these states had documented telemedicine policies or regulations; five included language specifically indicating that outpatient services were covered; two mandated payment parity for telemedicine services; and two specifically addressed geographic limitations of telemedicine use (Appendix Table 1 ). 17, 25 During this time period, the federal Medicare program limited telemedicine origin sites to health care facilities in rural communities.
Statistical Analysis. Our objective was to examine subspecialist and child-level telemedicine utilization, including subspecialist use of telemedicine, child receipt of telemedicine visits, and frequency of overall visits when cared for by subspecialists who use telemedicine. To contextualize these results, we also first described child-level use of subspecialty care.
Children with subspecialist visits. We described pediatric Medicaid beneficiaries with no subspecialty visits and with at least one subspecialist visit, with statistical differences tested using logistic regression with state-level cluster-robust standard errors.
Subspecialists with telemedicine use. We compared subspecialist characteristics associated with any vs. no telemedicine use through logistic regression with a penalized maximum likelihood method (Firth correction) to account for rarity of telemedicine use (n=146 clinicians use telemedicine); we selected this approach because it effectively addresses the potential for complete separation in samples as small as 30 as shown in simulation studies. 26, 27 Children with telemedicine visits, among those cared for by telemedicine-using subspecialists.
Next, we examined child and subspecialist characteristics associated with at least one telemedicine visit, limiting this analysis only to children receiving care from a telemedicineusing subspecialist. We used multi-level logistic regression including both child and subspecialist characteristics as independent variables and subspecialist-level random-effects. For children cared for by more than one telemedicine-using subspecialist, we sampled the childsubspecialist dyad with the most total visits. We examined the overall significance of each independent variable through separate Wald tests. We hypothesized that geographic barriers to in-person subspecialty care (i.e. increased travel time, rural county) would be associated with increased likelihood of a telemedicine visit, while non-geographic barriers to in-person subspecialty care (e.g., lack of continuous insurance) be associated with decreased likelihood of a telemedicine visit.
subspecialists. To examine visit patterns among patients seen by telemedicine-using versus nonusing subspecialists, we used descriptive statistics to describe the distribution of patients visit rates during 2014.
Next, to compare visit rates by patient characteristics among patients of telemedicine-using and non-using subspecialists, we used coarsened exact matching to match child-subspecialist dyads.
We matched on child age group, gender, race, Medicaid eligibility category and plan type, ZIP code median income level, county rurality, distance to care, months enrolled, subspecialist characteristics, and state. In coarsened-exact matching, 28 observations are matched many-tomany with weights then applied to each matched set to achieve covariate balance. The multivariate imbalance measure, when subtracted from 1, represents the percentage of the density of overlap between histograms of two samples. Our prematch L1 statistic of 0.862 improved to a post-match L1 statistic of 0.032, indicating an effective match. We then used negative binomial regression on the matched dataset to examine visit rates. We included child and subspecialist characteristics as independent variables, with model offset for the number of months of enrollment. We incorporated matching weights with robust standard errors. We tested the significance of interaction terms between each child characteristic and subspecialist telemedicine status (telemedicine-using vs. non-using), testing all interaction terms together and each interaction term separately. Because these interaction terms together yielded a significant Wald test, we subsequently ran stratified negative binomial models for children cared for by telemedicine-using versus non-using subspecialists to estimate incident risk ratios.
Using predictive margins, we then estimated adjusted visit rates, allowing comparison across models of adjusted annual visit rates by child characteristic. In this analysis, we focused on two child characteristics associated with specifically geographic barriers to care (distance to care, rural/urban status) and two characteristics associated with disparities in access not specifically due to geographic barriers (neighborhood income, child race). We hypothesized that compared to telemedicine non-using subspecialists, telemedicine-using subspecialists would have smaller differences in visit frequency across geographic variables, but persistent differences in visit frequency by neighborhood income and child race. Analyses were conducted in Stata 16 MP (StataCorp, College Station, Texas).
Children with subspecialist visits. Of were more likely to be identified as white, reside in small metropolitan counties, be eligible for Medicaid based on medical need rather than financial criteria, and be continuously enrolled.
Children identified as Hispanic and children residing in large metropolitan counties were underrepresented among children receiving subspecialty care.
Subspecialists with telemedicine use. Of 42,965 subspecialists identified in these claims, 146 used telemedicine in 2014 (0.3%). Odds of any telemedicine use by a subspecialist were lower for surgical subspecialists (OR 0.59, 95% CI 0.41-0.85 compared to medical subspecialist referent group) and non-pediatric trained subspecialists (OR 0.49, 95% CI 0.33-0.71 compared to pediatric trained referent group, Appendix Table 2 ). Among subspecialists who used telemedicine, telemedicine visit volume ranged from 1 to 829 telemedicine visits with pediatric Medicaid beneficiaries during the year (median: 4 telemedicine visits per telemedicine-using subspecialist, interquartile range: 1-13).
Of 23,583 children cared for by telemedicine-using subspecialists, telemedicine was used within 3,365 child-subspecialist dyads (14% Regarding subspecialist-level characteristics within dyads involving telemedicine-using subspecialists, odds of telemedicine use within a dyad were higher for dyads with subspecialists who were more recently trained (OR 19.0, 95% CI 4.5-80.2), who were in surgical fields (OR 4.39, 95% CI 2.19-8.81), and who were not specifically designated as pediatric-trained (OR 5.38, 95% CI 2.20-13.14; Table 2 ).
subspecialists. Both telemedicine-using and non-using subspecialists saw a similar percentage of their patients only once during the year (64% of telemedicine-using subspecialist patients vs 61%) and four or more times during the year (9%, both).
While this overall distribution of visit frequency was similar, visit frequency varied for children by sociodemographic and geographic characteristic ( Table 3) . Compared to children receiving care from telemedicine non-using subspecialists, matched children receiving care from telemedicine-using subspecialists had larger variation in incident rate ratios by distance to care, county rurality, ZIP code median income, and child race/ethnicity (p<0.001 for each interaction term).
We used predictive margins to estimate adjusted visit rates from these models across these four variables (Figure 1 ). Compared to children who received care from telemedicine non-using subspecialists, adjusted visit rates were higher for children living >90 miles from the subspecialist (+0.16 difference in annual visit rate) and children living in ZIP codes with the lowest median income (+0.36 difference in annual visit rates) who received care from telemedicine-using subspecialists. Compared to children who received care from telemedicine non-using subspecialists, adjusted visit rates were lower for children from large metropolitan areas (-0.27 difference in annual visit rate) and children identified as non-Hispanic Black (-0.20 difference in annual visit rate) who received care from telemedicine-using subspecialists ( Figure   1 ).
During a period of early adoption of telemedicine, our goal was to describe how telemedicine is being used for subspecialty care for children. We found that use of telemedicine for subspecialty care was uncommon in these 2014 data, with only 0.3% of subspecialists using telemedicine, and these subspecialists, in turn, completing telemedicine visits with only 14% of their patients.
Within the context of low use overall, however, we identified significant variation in use of telemedicine at the child and subspecialist level during early adoption within state Medicaid programs that may inform future strategies to guide appropriate telemedicine use and more equitable distribution of subspecialty care.
Focusing first on children characteristics associated with telemedicine use, we identified that likelihood of a child receiving a telemedicine visit varied by not only child geographic proximity to in-person care but also by child race/ethnicity. We also observed that the association between subspecialist visit rates and child geographic and non-geographic characteristics varied when cared for by telemedicine-using versus non-using subspecialists. Children in rural counties and children living at distance to care are often identified implicitly or explicitly as the target audience for telemedicine programs and policies. Our findings indicate that among subspecialists who use telemedicine, telemedicine is more likely to be used for visits with the children for whom current policies have directed its use, as expected. Adjusted visit rates were substantially higher for children living >90 minutes from the subspecialist cared for by a telemedicine-using subspecialist rather than a telemedicine non-using subspecialist. This finding indicates that telemedicine availability improves likelihood of ongoing care for children at greatest distance.
However, when adjusted for distance, rural designation did not have quite as straightforward of a relationship with visit frequency. Compared to telemedicine non-using subspecialists, matched patients of telemedicine-using subspecialists had slightly higher visit rates among children in smaller metropolitan communities, but minimal increase in visit rates among children in the most rural communities. One possible contributing factor to this finding may be that some telemedicine programs are directed primarily at specific telemedicine facilities due to insurer payment policies. Use of telemedicine facilities also allows may have clinical benefits by incorporating peripheral devices (e.g., tele-otoscope), nurse tele-presenters, and access to laboratory facilities. However, if telemedicine facilities are intentionally located with an eye to adequate volume, this could also result in the finding that the near-rural rather than the real-rural may see more benefit.
Because important differences in access and unmet need for subspecialty care also exist by child non-geographic variables, we also examined telemedicine use by child race/ethnicity and neighborhood median income. Among children cared for by telemedicine-using subspecialists, children identified as Hispanic were less likely to receive a telemedicine visit. Additionally, differences in visit rates were wider for Black versus white children cared for by telemedicineusing subspecialists compared to matched Black and white children cared for by telemedicine non-using subspecialists. Because telemedicine policy has been designed to reach rural populations, it has the potential to exacerbate racial or ethnic disparities in areas or states where minority populations are predominantly urban. Additionally, restricting telemedicine use to beneficiaries living at specific distances does not address transportation barriers facing many urban Medicaid beneficiaries, including lack of a personal vehicle, indirect bus lines, multiple family obligations, and time constraints which can make traveling even within urban areas a formidable task. As noted previously, when regional telemedicine is delivered primarily at specific telemedicine facilities, where these facilities are located could also have implications for access by race and ethnicity. Additionally, clinician decision-making may also contribute to racial/ethnic disparities. For example, subspecialists may be less likely to offer telemedicine if a caregiver has low English proficiency if interpreter services are not adequately integrated. It is worth noting, however, that with intentional, community-partnered design, telehealth services can achieve high uptake and satisfaction even among families facing language barriers. 29 Focusing on subspecialist characteristics, we observed first that overall use of telemedicine among subspecialists remains low, even among subspecialists caring for children in this subset of state Medicaid programs where at least some forms of payment for telemedicine services existed during the study period. Because few subspecialists used telemedicine, we were unable to compare use across specific subspecialties, and focused instead on broad categories (medical, surgical). We noted that surgical subspecialists and non-pediatric trained subspecialists were less likely to have ever used telemedicine, but that the likelihood of a child receiving a telemedicine visits, conditional on receiving care from a telemedicine-using subspecialist, was increased for more recently trained, surgical, and non-pediatric subspecialists. These results suggest that there is a larger group of pediatric medical subspecialists who have used telemedicine at least once, but a smaller group of non-pediatric or surgical specialists who are more likely to use telemedicine for a given clinical encounters. These results suggest that factors influencing likelihood of any use telemedicine do not necessarily translate into increased likelihood of using telemedicine for an individual child, and that supporting initial adoption by an individual clinician is not adequate to promote regular use with patients.
Overall, our finding show that as of 2014 telemedicine policy was achieving its intended goal (increasing utilization for children in specific geographic areas), but with important limitations (limited adoption overall and not increasing utilization for those in the most rural counties), and possible unintended consequences (perpetuating disparities for urban and minority patients).
These findings have implications for telemedicine policy, where state-specific Medicaid restrictions abounded prior to and even into the COVID-19 pandemic. As of 2018, 6 state Medicaid programs still placed geographic restrictions on which beneficiaries can receive telemedicine (e.g., greater than 60-minute travel time), 23 state Medicaid programs required that patients receiving telemedicine must be physically located at a designated facility, 16 allow schools to serve as the originating site for telemedicine visits and only 14 state Medicaid programs specifically allow telemedicine visits to occur with the patient at home. 17 Each of these restrictions limit opportunities for telemedicine use, and prioritize the transportation barriers experienced by rural populations over the transportation barriers experienced by urban populations. Reducing these restrictions, as is being done during the COVID-19 pandemic, could allow telemedicine to be used more broadly for families facing a range of transportation, logistic, and time barriers, potentially enhancing both overall use and equity in use. Of note, because some argue that removing geographic restrictions to telemedicine may result in over-use of services, ongoing evaluation of policy impact is necessary. Alignment of clinician payment and incentives with high-value telemedicine use (e.g., accountable care organization models as opposed to fee-for-service models) may be a way to promote judicious use among clinicians as geographic restrictions are removed. Additionally, given the low overall use, state Medicaid programs wishing to overcome access barriers may also wish to consider alternative types of telemedicine (e.g., remote patient monitoring or store-and-forward electronic consultations 30 ), which further reduce barriers by avoiding the need for a real-time visit, but are less frequently covered in state Medicaid programs. 17 Many state Medicaid programs have altered these restrictions during the COVID-19 pandemic. It will be important to assess the impact on equity of access and utilization as policies continue to evolve.
Key limitations of this analysis warrant comment. First, in this claims analysis, we lack clinical data to identify unmet need for subspecialty care. Instead we highlight variation in use of telemedicine among children receiving subspecialty care and among children receiving subspecialty care from a telemedicine-using subspecialist. We recognize that individual children with specific clinical needs may require different frequency of visits. Second, this claims analysis did not assess the quality of care delivered during telemedicine visits, focusing instead on questions of utilization. Third, our data were limited to Medicaid claims, such that we cannot assess subspecialist care patterns for children with commercial insurance. We may underestimate total telemedicine use as we do not capture grant-funded programs or telemedicine payments by other insurers. Fourth, our analysis is cross-sectional. Longitudinal analysis of the impact of telemedicine adoption on visit patterns would provide additional insight to the relationship between new adoption, access, and utilization. Fifth, we recognize that we included data from states with different regulatory environments. To account for variation in overall telemedicine use by state, we used state-level random-effects and state-level matching in our analyses; between-state variation in use of telemedicine for pediatric subspecialty care should be assessed in future work. Finally, we note that these data are from 2014, and telemedicine use and regulations continue to evolve. Thus these data do not reflect the current state of telemedicine use, but rather offer insight into patterns of early adoption of telemedicine for pediatric subspecialty care and opportunities to continue to center equity in future evaluations of telemedicine policy.
In conclusion, we found low use of telemedicine among subspecialists caring for pediatric Medicaid beneficiaries in 2014, but increased likelihood of telemedicine use among children in rural communities and at distance to subspecialty care. Within panels cared for by telemedicineusing subspecialists, children in smaller metropolitan counties and at distance to care had increased subspecialist visit rates, but geographic and sociodemographic variation in visit rates persisted. Evolving telemedicine policy, both during the COVID-19 pandemic and beyond, should be evaluated on its ability to improve equity in access and utilization for pediatric Medicaid beneficiaries in need of subspecialty care. Figure 1 : Adjusted annual visit rates among patients cared for by telemedicine-using versus telemedicine non-using subspecialists by patient characteristics Figure 1 Legend. Adjusted annual visit rates among matched patients cared for by telemedicine non-using subspecialists (gray) and telemedicine using subspecialists (black). Adjusted annual visit rates determined through predictive margins based on stratified negative binomial models with independent variables including listed variables (distance to subspecialist, county rurality, ZIP code median income, child race/ethnicity) as well as child age, child gender, insurance characteristics (eligibility category, Medicaid program type) and subspecialist characteristics (clinician years in practice, clinician gender, medical v surgical subspecialist, pediatric v non-pediatric subspecialist). Abbreviations, FPL, federal poverty level. Legend. We compared pediatric Medicaid beneficiaries with no subspecialty telemedicine visits and with at least one subspecialist visit, with testing for statistical differences using logistic regression with state-level cluster-robust standard errors. Abbreviations: FPL, federal poverty level. Legend. Incident risk ratios for children cared for by telemedicine-using and non-using subspecialists, determined through negative binomial regression on children matched through coarsened exact matching with child and subspecialist characteristics as independent variables, model offset for the number of months of child enrollment during 2014, and coarsened-exact matching weights with robust standard errors. In addition to listed characteristics, independent variables included subspecialist years in practice, gender, subspecialtist type (medical vs. surgical), and pediatric training (pediatric vs. non-pediatric). In a full model, we tested the significance of all interaction terms together (p<0.001) and each interaction term separately (provided in last column). Because all interaction terms together yielded a significant Wald test, final IRRs provided here were estimated through stratified negative binomial models. Abbreviations: IRR, incident risk ratio; CI, confidence interval; FPL, federal poverty level.