Introduction {#s1}
============

Given the scientific evidence of the benefits of male circumcision in reducing HIV incidence rates among men [@pone.0079524-Auvert1]--[@pone.0079524-Gray1], several sub-Saharan African countries have begun to scale-up voluntary medical male circumcision (VMMC) services for adult and adolescent males [@pone.0079524-Mwandi1]--[@pone.0079524-Edgil1].  High coverage of safe VMMC services could prevent up to 3.4 million new HIV infections [@pone.0079524-Njeuhmeli1].  However, the rapid expansion of VMMC programs has raised questions regarding the extent to which health systems can deliver VMMC according to minimum quality standards which are sustained at-scale [@pone.0079524-HermanRoloff1], [@pone.0079524-Mahler1].  Ensuring correct performance of VMMC procedures during scale-up is essential in realizing the potential health impact of averted HIV infections and in reducing health systems costs for HIV-related care [@pone.0079524-Musau1], [@pone.0079524-Strategies1]. 

In 2009, the World Health Organization (WHO) developed a quality assessment toolkit to assess a range of quality standards relating to facilities\' VMMC supply inventory, provider competence, surgical technique, infection prevention, and continuity of care [@pone.0079524-WHO1].  Preliminary findings suggest that VMMC surgical complications have resulted from inadequate provider competence and training, poor clinical infrastructure, and limited patient information and follow-up [@pone.0079524-Bailey1], [@pone.0079524-Dickson1].  Yet, there is a dearth of published data on other aspects of service quality among VMMC programs in resource-poor settings [@pone.0079524-Dickson1], [@pone.0079524-Ngo1], [@pone.0079524-Muula1].  Most quality assessments have focused on occurrence of adverse events [@pone.0079524-HermanRoloff1], [@pone.0079524-Muula1]--[@pone.0079524-Ford1], device malfunction rates [@pone.0079524-Musau1], procedure duration [@pone.0079524-Musau1], or patient acceptability and resumption of sexual activity [@pone.0079524-Ngo1].  Less attention has been given to the availability of minimum service components and provider performance of specific surgical tasks in contexts where countries are rapidly expanding the number of VMMC sites [@pone.0079524-Bailey1], [@pone.0079524-Ngo1], [@pone.0079524-Muula1].  To our knowledge, this study is the first published to-date to address this gap by examining quality of VMMC services among four African countries over two years of program scale-up. 

Methods {#s2}
=======

Study Design {#s2a}
------------

A comparative process evaluation was used to examine VMMC service provision over time.  Data were drawn from SYMMACS, the Systematic Monitoring of the Voluntary Medical Male Circumcision Scale-Up, which assessed the evolution of VMMC scale-up in Kenya, South Africa, Tanzania, and Zimbabwe.  SYMMACS conducted a two-stage multi-country evaluation with two cross-sectional data collection periods (one in 2011 and one in 2012) among health facilities providing VMMC and selected VMMC surgeries.  A detailed description of the SYMMACS evaluation and sampling methodology is provided in this supplement [@pone.0079524-Bertrand1]. 

Scale-up Setting {#s2b}
----------------

The scale-up model used to expand the number of VMMC sites varied by country.  Kenya and Tanzania adopted a focused implementation strategy using outreach sites and fixed clinical facilities supported by multiple implementing agencies within regions with the highest HIV prevalence and lowest male circumcision rates.  Kenya additionally provided services via mobile sites to deliver VMMC in underserved areas.  South Africa adopted a national approach using outreach and fixed sites with multiple partners, while Zimbabwe\'s national program was implemented by public sector service providers in outreach and fixed sites supported by a single implementing partner.  Kenya was the only country with a fully-scaled VMMC program in the evaluation\'s first and second years, while Tanzania, South Africa, and Zimbabwe substantially increased the number of VMMC sites from two- to five-fold over the evaluation period. 

Measurement {#s2c}
-----------

The comparative process evaluation measured two outcomes: VMMC facility preparedness and quality of VMMC surgeries. * VMMC facility preparedness* was assessed by direct observation using a pre-tested observation checklist based on the WHO VMMC quality assessment tool [@pone.0079524-WHO1].  The checklist included items grouped into five preparedness categories: "guidelines and protocols" comprised 3 standard operating protocols; "equipment and supplies" included 9 items relating to anesthesia, antibiotics, condoms, and other supplies; "basic life support" (BLS) consisted of 4 items: cardiopulmonary resuscitation bag and mask, oxygen, intravenous lines, and anaphylaxis medications; "hygienic and safe infrastructure" consisted of 4 items measuring adequacy of lighting, ventilation, cleanliness, and adverse event monitoring; and "patient services" included 5 items: circumcision, group education, individual counseling, HIV counseling and testing, and referral services. * Quality of VMMC surgeries* was evaluated by direct observation of randomly selected surgeries using a checklist of tasks across three surgical periods: pre-operative (1 item), operative (21 items), and post-operative (7 items).  Example tasks included pre-operative examination, use of sterile processes, correct surgical techniques, and provision of post-operative monitoring and counseling.  Operative period tasks were further separated into two categories: 'safe operative procedures' (8 items) relating to surgical safety and 'operative techniques' (13 items) relating to step-by-step circumcision procedures.  Trained observers scored items as "unsatisfactory" (scored = 0), "partially satisfactory" (scored = 1), and "satisfactory" (scored = 2). 

Statistical Analysis {#s2d}
--------------------

Data were analyzed using STATA (Version 12).  Two sample tests of proportions and t-tests were used to examine differences in facility preparedness and VMMC service quality by country between 2011 and 2012.  Composite measures of facility preparedness and service quality were calculated based on the percent of facilities with all requisite items per category and the mean number and mean percent of items scored as "satisfactory" per category.  Item-level differences between time points were also assessed using Pearson\'s chi-squared tests.  A significance level of *p*\<.05 was used in all analyses. 

Ethical Considerations {#s2e}
----------------------

Ethical approval was obtained from the Tulane University Institutional Review Board (United States), the Kenya Medical Research Institute, the University of Witwatersrand\'s Human Research Ethics Committee (South Africa), the Tanzanian National Institute for Medical Research, and the Medical Research Council of Zimbabwe. 

Results {#s3}
=======

VMMC Facility Preparedness {#s3a}
--------------------------

[Table 1](#pone-0079524-t001){ref-type="table"} presents changes in the percent of facilities having all preparedness items per category.  During the initial year, facilities in all four countries had an average of 1.3 to 2.0 guidelines on-site at the time of assessment.  Across countries, less than half of facilities (Kenya: 34.5%, South Africa: 33.3%, Tanzania: 7.7%, Zimbabwe: 7.1%) had all three guidelines.  In the second year, the percent of facilities having all three guidelines significantly improved in Zimbabwe (+34.6, 95% CI: 10.6, 58.4).  However, no significant changes occurred in the remaining countries: Kenya (+19.1, 95% CI: −6.2, 44.4), Tanzania (+19.9, 95% CI: −1.9, 41.7), South Africa (−20.8, 95% CI: −46.8, 5.1). 

10.1371/journal.pone.0079524.t001

###### Summary of Availability of VMMC Protocols, Equipment, Supplies, and Services from Initial Year (2011) to Expansion Year (2012). 

![]( pone.0079524.t001){#pone-0079524-t001-1}

                                                                                    Kenya     South Africa                                             Tanzania                                             Zimbabwe                                                                                                                                                                                                                                                             
  ------------------------------------------------------------------------------ ----------- -------------- ---------------------------------------------------------------------------------------------- ----------- ----------- ---------------------------------------------------------------------------------------------- ----------- ----------- --------------------------------------------------------------------------------------------- ----------- ------------ ----------------------------------------------------------------------------------------------
  Number of facilities                                                               30            29                                                                                                          15          40                                                                                                         14          29                                                                                                        14           24      
  **Guidelines**                                                                   n = 28        n = 29                                                                                                      n = 15      n = 40                                                                                                     n = 13      n = 29                                                                                                    n = 14       n = 24    
  Percent (%) facilities with all 3 guidelines                                      34.5          53.6                                                  +19.1                                                 33.3        12.5                                                 −20.8                                                  7.7        27.6                                                 +19.9                                                 7.1         41.7                              **+34.6** [\*](#nt102){ref-type="table-fn"}
  Mean number (%) guidelines[A](#nt101){ref-type="table-fn"}                      1.8 (60%)    2.0 (68%)                                              +0.2 (+8%)                                            2.0 (67%)   1.0 (35%)   **−1.0** [\*](#nt102){ref-type="table-fn"} **(−32%** [\*](#nt102){ref-type="table-fn"} **)**   1.6 (54%)   1.8 (62%)                                           +0.2 (+8%)                                            1.3 (43%)   2.1 (69%)    **+0.8** [\*](#nt102){ref-type="table-fn"} **(+26%** [\*](#nt102){ref-type="table-fn"} **)**
  **Equipment/Supplies**                                                           n = 28        n = 28                                                                                                      n = 15      n = 39                                                                                                     n = 13      n = 29                                                                                                    n = 14       n = 24    
  Percent (%) facilities with all 9 equipment and supplies                          25.0          10.7                                                  −14.3                                                 50.0         2.6                              **−47.4** [\*](#nt102){ref-type="table-fn"}                              23.1         6.9                                                 −16.2                                                28.6         45.8                                                 +17.2
  Mean number (%) equipment and supplies items[A](#nt101){ref-type="table-fn"}    7.8 (87%)    7.5 (83%)                                              −0.3 (−4%)                                            8.3 (92%)   6.6 (73%)      **−1.7** [\*](#nt102){ref-type="table-fn"} **(−19%** [\*](#nt102){ref-type="table-fn"}      8.2 (91%)   7.7 (86%)   **−0.5** [\*](#nt102){ref-type="table-fn"} **(−5%** [\*](#nt102){ref-type="table-fn"} **)**   7.9 (88%)   8.4 (94%)    **+0.5** [\*](#nt102){ref-type="table-fn"} **(+6%** [\*](#nt102){ref-type="table-fn"} **)**
  **Basic Life Support**                                                           n = 29        n = 29                                                                                                      n = 15      n = 40                                                                                                     n = 12      n = 29                                                                                                    n = 14       n = 24    
  Percent (%) facilities with all 4 basic life support items                          0           3.4                                                    +3.4                                                 64.3        27.5                              **−36.8** [\*](#nt102){ref-type="table-fn"}                               7.7        17.2                                                 +9.5                                                 35.7         58.3                                                 +22.6
  Mean number (%) basic life support items[A](#nt101){ref-type="table-fn"}        0.8 (20%)    1.5 (37%)     **+0.7** [\*](#nt102){ref-type="table-fn"} **(+17%** [\*](#nt102){ref-type="table-fn"} **)**   3.2 (80%)   2.3 (57%)   **−0.9** [\*](#nt102){ref-type="table-fn"} **(−23%** [\*](#nt102){ref-type="table-fn"} **)**   1.5 (37%)   2.1 (53%)                                           +0.6 (+16%)                                           2.3 (57%)   2.8 (70%)                                            +0.5 (+13%)
  **Hygiene and Safety**                                                           n = 29        n = 29                                                                                                      n = 15      n = 40                                                                                                     n = 12      n = 29                                                                                                    n = 14       n = 24    
  Percent (%) facilities with all 4 hygiene and safety items                        75.9          86.2                                                  +10.3                                                 53.3        12.5                              **−40.8** [\*](#nt102){ref-type="table-fn"}                              33.3        17.2                                                 −16.1                                                28.6         83.3                              **+54.7** [\*](#nt102){ref-type="table-fn"}
  Mean number (%) hygiene and safety items[A](#nt101){ref-type="table-fn"}        3.6 (91%)    3.8 (95%)                                              +0.2 (+4%)                                            3.2 (82%)   2.4 (61%)   **−0.8** [\*](#nt102){ref-type="table-fn"} **(−21%** [\*](#nt102){ref-type="table-fn"} **)**   3.0 (75%)   2.9 (73%)                                           −0.1 (−2%)                                            2.3 (57%)   3.6 (89%)    **+1.3** [\*](#nt102){ref-type="table-fn"} **(+32%** [\*](#nt102){ref-type="table-fn"} **)**
  **Service Package**                                                              n = 28        n = 29                                                                                                      n = 15      n = 40                                                                                                     n = 14      n = 29                                                                                                    n = 14       n = 24    
  Percent (%) facilities with all 5 package services                                32.1          58.6                               **+26.5** [\*](#nt102){ref-type="table-fn"}                              86.7        22.5                              **−64.2** [\*](#nt102){ref-type="table-fn"}                              85.7        58.6                                                 −27.1                                                85.7         100                                                  +14.3
  Mean number (%) package services[A](#nt101){ref-type="table-fn"}                4.3 (86%)    4.5 (90%)                                              +0.2 (+4%)                                            4.8 (96%)   3.5 (70%)   **−1.3** [\*](#nt102){ref-type="table-fn"} **(−26%** [\*](#nt102){ref-type="table-fn"} **)**   4.9 (98%)   4.6 (92%)                                           −0.3 (−6%)                                            4.9 (98%)   5.0 (100%)                                            +0.1 (+2%)

Includes only percent of facilities with total score ( = 1 point).  Items receiving a partial score ( = 0.5 point) at the facility were not counted as available. 

Denotes statistically significant at p\<0.05. 

Health facilities in all four countries reported having most of the 9 listed equipment and supplies during the initial year (Kenya: 7.8, South Africa 8.3, Tanzania 8.2, Zimbabwe 7.9).  With the exception of South Africa, roughly a quarter of facilities had all 9 items (Kenya 25%, South Africa 50%, Tanzania 23%, Zimbabwe 29%).  As compared to the expansion year, availability of supplies and equipment significantly declined in South Africa from 50.0% of facilities having all 9 items to 2.6% of facilities (−47.4, 95% CI: −74.1, −20.8).  No statistically significant changes were observed in the other countries, although the general trend was a decline in supply availability: Kenya (−14.3, 95% CI: −34.0, 5.4), Tanzania (−16.2, 95% CI: −40.9, 8.9), and Zimbabwe (+17.2, 95% CI: −13.7, 48.2).  Sites in Kenya and Tanzania scored poorly on available BLS equipment, ranging from an average of 0.8 to 1.5 of the 4 recommended items.  These levels increased slightly, but remained low during the expansion year, with 3.4% and 17.2% of facilities having all 4 BLS items.  Although more facilities in South Africa (27.5%) had all 4 BLS items, this represented a significant decline from 64.3% of facilities in its initial year (−36.8, 95% CI:−65.4, −8.1).  In Zimbabwe, approximately half (58.3%) of facilities had all 4 BLS items during the second year compared to 35.7% during the initial year (+22.6, 95% CI: −9.3, 54.5). 

Similar patterns were observed for hygienic and safe infrastructure.  Kenya maintained relatively high scores with no significant changes with 75.9% and 86.2% of facilities having all 4 items each year, respectively (+10.3, 95% CI: −9.7, 30.3).  Tanzania maintained relatively low scores with 33.3% and 17.2% of facilities (−16.1, 95% CI: −46.1, 13.9).  Higher scores in South Africa significantly declined from 53.3% to 12.5% of facilities (−40.8, 95% CI: −68.1, −13.6).  Lower scores in Zimbabwe significantly improved from 28.6% to 83.3% (+54.7, 95% CI: 26.8, 82.8).  Most facilities in Kenya, Tanzania, and Zimbabwe had high service availability, averaging 4.3 to 4.9 of the 5 package services in 2011, which were maintained during expansion.  Significant service erosion was observed in South Africa from 4.8 to 3.5 of the 5 package services. 

[Table 2](#pone-0079524-t002){ref-type="table"} shows item-level analyses of facility preparedness.  Protocol availability varied among countries.  Supplies most commonly unavailable were HIV post-exposure prophylaxis, followed by antibiotics and condoms.  Absence of oxygen and anaphylaxis medications appeared to drive low BLS scores, while absence of adverse event monitoring systems and lighting drove lower infrastructure scores.  With the exception of referral services, most VMMC facilities offered group education, individual counseling, and HIV counseling and testing. 

10.1371/journal.pone.0079524.t002

###### Item-Level Availability of VMMC Protocols, Equipment, Supplies, and Services from Initial Year (2011) to Expansion Year (2012). 

![]( pone.0079524.t002){#pone-0079524-t002-2}

  *Indicator:* Percent (%) of facilities with Total Score ( = 1 point) by Item[A](#nt103){ref-type="table-fn"}                                                                                                                                                                                                     
  -------------------------------------------------------------------------------------------------------------- ------ ------ --------------------------------------------- ------ ------ --------------------------------------------- ------ ------ --------------------------------------------- ------ ------ ---------------------------------------------
  Number of facilities                                                                                             30     29                                                   15     40                                                   14     29                                                   14     24   
  **Guidelines and Protocols** *(3 items)*                                                                                                                                                                                                                                                                         
  VMMC Standard Operating Procedures (SOP)                                                                        70.0   79.3                       \-                        40.0   42.5                       \-                        50.0   86.2   **+36.2** [\*](#nt104){ref-type="table-fn"}   21.4   45.8                       \-
  Post-Exposure (PE) Prophylaxis Guidelines                                                                       56.7   65.5                       \-                        86.7   32.5   **−54.2** [\*](#nt104){ref-type="table-fn"}   21.4   62.1   **+40.7** [\*](#nt104){ref-type="table-fn"}   50.0   70.8                       \-
  National Protocols for Syndromic Management and Treatment of Sexually Transmitted Infections (STI)              46.7   55.2                       \-                        73.3   30.0   **−43.3** [\*](#nt104){ref-type="table-fn"}   85.7   37.9   **−47.8** [\*](#nt104){ref-type="table-fn"}   57.1   91.7   **+34.6** [\*](#nt104){ref-type="table-fn"}
  **Equipment and Supplies** *(9 items)*                                                                                                                                                                                                                                                                           
  Sterilized VMMC Instruments                                                                                     100    100                        \-                        86.7   95.0                      **-**                      92.9   100                        \-                        92.9   100                        \-
  Local Anesthesia                                                                                                93.3   96.6                       \-                        100    90.0                      **-**                      92.9   100                        \-                        100    100                        \-
  Antibiotics                                                                                                     76.7   58.6                       \-                        80.0   60.0                      **-**                      78.6   89.7                       \-                        85.7   95.8                       \-
  Pain Medication                                                                                                 93.3   100                        \-                        100    75.0                      **-**                      92.9   100                        \-                        100    100                        \-
  Antiseptic Solution                                                                                             100    100                        \-                        100    87.5                      **-**                      92.9   100                        \-                        85.7   100                        \-
  Dressing Materials                                                                                              100    100                        \-                        100    77.5                      **-**                      92.9   100                        \-                        100    100                        \-
  HIV Post-Exposure Prophylaxis                                                                                   46.7   17.2   **−29.5** [\*](#nt104){ref-type="table-fn"}   60.0   7.5    **−52.5** [\*](#nt104){ref-type="table-fn"}   57.1   10.3   **−46.8** [\*](#nt104){ref-type="table-fn"}   57.1   50.0                       \-
  Sharps Container                                                                                                93.3   100                        \-                        100    95.0                       \-                        92.9   100                        \-                        100    100                        \-
  Distributable Male Condoms                                                                                      70.0   72.4                       \-                        86.7   70.0                       \-                        78.6   72.4                       \-                        71.4   100    **+28.6** [\*](#nt104){ref-type="table-fn"}
  **Basic Life Support** *(4 items)*                                                                                                                                                                                                                                                                               
  Cardiopulmonary Resuscitation (CPR) Bag/Mask                                                                    33.3   55.1                       \-                        66.7   47.5                       \-                        42.9   55.2                       \-                        42.9   66.7                       \-
  Oxygen Supply                                                                                                   6.7    6.9                        \-                        73.3   47.5                       \-                        7.1    17.2                       \-                        64.3   62.5                       \-
  IV Lines with Resuscitation Fluids                                                                              30.0   44.8                       \-                        73.3   77.5                       \-                        57.1   75.9                       \-                        64.3   79.2                       \-
  Anaphylaxis Medications (i.e., Antihistamine, Cortisone, and Adrenalin)                                         10.0   41.4   **+31.4** [\*](#nt104){ref-type="table-fn"}   86.7   55.0   **−31.7** [\*](#nt104){ref-type="table-fn"}   28.6   65.5   **+36.9** [\*](#nt104){ref-type="table-fn"}   57.1   70.8                       \-
  **Hygiene and Safety** *(4 items)*                                                                                                                                                                                                                                                                               
  Adequate Lighting in Surgical Area                                                                              80.0   89.7                       \-                        93.3   92.5                       \-                        57.1   31.0                       \-                        42.9   83.3   **+40.4** [\*](#nt104){ref-type="table-fn"}
  Adequate Ventilation in Surgical Area                                                                           90.0   93.1                       \-                        86.7   72.5                       \-                        64.3   93.1   **+28.8** [\*](#nt104){ref-type="table-fn"}   42.9   87.5   **+44.6** [\*](#nt104){ref-type="table-fn"}
  Clean and Hygienic Surgical Area                                                                                93.3   100                        \-                        86.7   55.0   **−31.7** [\*](#nt104){ref-type="table-fn"}   57.1   72.4                       \-                        42.9   91.7   **+48.8** [\*](#nt104){ref-type="table-fn"}
  Presence of Functional Adverse Events Monitoring System                                                         93.3   96.5                       \-                        60.0   22.5   **−37.5** [\*](#nt104){ref-type="table-fn"}   92.9   96.6                       \-                        100    95.8                       \-
  **Service Package** *(5 items)*                                                                                                                                                                                                                                                                                  
  Medical Male Circumcision                                                                                       100    100                        \-                        100    100                        \-                        100    100                        \-                        100    100                        \-
  VMMC Group Education                                                                                            93.3   100                        \-                        93.3   80.0                       \-                        92.9   100                        \-                        85.7   100                        \-
  VMMC Individual Counseling                                                                                      100    96.6                       \-                        100    62.5   **−37.5** [\*](#nt104){ref-type="table-fn"}   100    100                        \-                        100    100                        \-
  HIV Counseling and Testing                                                                                      96.7   96.6                       \-                        100    67.5   **−32.5** [\*](#nt104){ref-type="table-fn"}   100    100                        \-                        100    100                        \-
  Referral Services                                                                                               33.3   58.6                       \-                        86.7   42.5   **−44.2** [\*](#nt104){ref-type="table-fn"}   92.9   58.6   **−34.3** [\*](#nt104){ref-type="table-fn"}   100    100                        \-

Includes only percent of facilities with total score ( = 1 point).  Items receiving a partial score ( = 0.5 point) at the facility were not counted as available. 

Differences from T~0~ to T~1~ reported at p\<0.05 statistical significance. 

Quality of VMMC Surgeries {#s3b}
-------------------------

[Table 3](#pone-0079524-t003){ref-type="table"} summarizes the quality of observed VMMC surgeries.  Pre-operative tasks were correctly done in 79.5% and 97.1% of observed surgeries in Kenya and Zimbabwe, respectively, during the initial year and were maintained during the second year: (Kenya: −8.5, 95% CI: −17.6, 0.6 and Zimbabwe: +2.9, 95% CI: 0.6, 5.2).  Pre-operative tasks were omitted the most in Tanzania\'s observed surgeries with low scores during the initial (20.6%) and second (26.3%) year (+5.7, 95% CI: −3.4, 14.8).  Pre-operative tasks were correctly done in half (52.1%) of observed surgeries in South Africa, although this significantly declined over time (−39.8, 95% CI: −47.7, −32.0). 

10.1371/journal.pone.0079524.t003

###### Summary of Quality of VMMC Surgical Care from Initial Year (2011) to Expansion Year (2012). 

![]( pone.0079524.t003){#pone-0079524-t003-3}

                                                                                       Kenya      South Africa                                            Tanzania                                              Zimbabwe                                                                                                                                                                                                                                                                 
  --------------------------------------------------------------------------------- ------------ -------------- --------------------------------------------------------------------------------------------- ------------ ------------ ---------------------------------------------------------------------------------------------- ------------ ------------ --------------------------------------------------------------------------------------------- ------------ ------------ ----------------------------------------------------------------------------------------------
  Number of surgical observations                                                       154           218                                                    \-                                                   116          361                                                    \-                                                   133          251                                                   \-                                                   140          240                                                    \-
  **Pre-Operative Assessment**                                                                                                                                                                                                                                                                                                                                                                                                                                                           
  Percent (%) surgical observations with 1 pre-operative task done                      79.5          71.0                                                  −8.5                                                  52.1         12.3                              **−39.8** [\*](#nt107){ref-type="table-fn"}                               20.6         26.3                                                 +5.7                                                  97.1         100                                **+2.9** [\*](#nt107){ref-type="table-fn"}
  **Safe Operative Procedures**                                                                                                                                                                                                                                                                                                                                                                                                                                                          
  Percent (%) surgical observations with all 8 safe operative procedures done           4.1           4.7                                                   +0.6                                                  9.7          0.6                                **−9.1** [\*](#nt107){ref-type="table-fn"}                               9.8          31.0                              **+21.2** [\*](#nt107){ref-type="table-fn"}                               0           1.5                                                   +1.5
  Mean number (%) safe operative procedures done[A](#nt105){ref-type="table-fn"}     6.9 (86%)     6.8 (85%)                                             −0.1 (−1%)                                            6.6 (83%)    5.6 (70%)    **−1.0** [\*](#nt107){ref-type="table-fn"} **(−13%** [\*](#nt107){ref-type="table-fn"} **)**   6.3 (79%)    6.8 (85%)    **+0.5** [\*](#nt107){ref-type="table-fn"} **(+6%** [\*](#nt107){ref-type="table-fn"} **)**   6.5 (81%)    7.0 (87%)                        **+0.5** [\*](#nt107){ref-type="table-fn"} **(+6%)**
  **Operative Techniques**                                                                                                                                                                                                                                                                                                                                                                                                                                                               
  Percent (%) surgical observations with all 13 operative techniques done               54.0          50.2                                                  −3.8                                                  66.9         33.7                              **−33.2** [\*](#nt107){ref-type="table-fn"}                               66.4         36.7                              **−29.7** [\*](#nt107){ref-type="table-fn"}                              39.3         94.1                              **+54.8** [\*](#nt107){ref-type="table-fn"}
  Mean number (%) operative techniques done[A](#nt105){ref-type="table-fn"}          12.5 (96%)    12.4 (95%)                                            −0.1 (−1%)                                            12.4 (95%)   11.6 (89%)   **−0.8** [\*](#nt107){ref-type="table-fn"} **(−6%** [\*](#nt107){ref-type="table-fn"} **)**    12.5 (96%)   12.0 (92%)   **−0.5** [\*](#nt107){ref-type="table-fn"} **(−4%** [\*](#nt107){ref-type="table-fn"} **)**   11.5 (88%)   12.9 (99%)   **+1.4** [\*](#nt107){ref-type="table-fn"} **(+11%** [\*](#nt107){ref-type="table-fn"} **)**
  **Post-Operative Assessment**                                                                                                                                                                                                                                                                                                                                                                                                                                                          
  Percent (%) surgical observations with all 7 post-operative tasks done                16.2          18.9                                                  +2.7                                                  2.6          0.6                                                   −2.0                                                  16.5         8.5                               **−8.0** [\*](#nt107){ref-type="table-fn"}                                0           9.8                                **+9.8** [\*](#nt107){ref-type="table-fn"}
  Mean number (%) post-operative tasks done[A](#nt105){ref-type="table-fn"}          4.4 (62%)     3.8 (54%)     **−0.6** [\*](#nt107){ref-type="table-fn"} **(−8%** [\*](#nt107){ref-type="table-fn"} **)**   4.0 (58%)    2.4 (35%)    **−1.6** [\*](#nt107){ref-type="table-fn"} **(−23%** [\*](#nt107){ref-type="table-fn"} **)**   4.5 (64%)    3.9 (55%)    **−0.6** [\*](#nt107){ref-type="table-fn"} **(−9%** [\*](#nt107){ref-type="table-fn"} **)**   4.9 (70%)    6.1 (87%)    **+1.2** [\*](#nt107){ref-type="table-fn"} **(+17%** [\*](#nt107){ref-type="table-fn"} **)**
  **Composite Score**                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
  Percent (%) surgical observations with all 29 VMMC quality of care tasks done         1.4           1.4                                                    \-                                                    0            0                                                     \-                                                    0           1.7                                                  +1.7                                                   0            0                                                     \-
  Mean number (%) VMMC quality of care tasks done[A](#nt105){ref-type="table-fn"}    24.6 (85%)    23.8 (82%)    **−0.8** [\*](#nt107){ref-type="table-fn"} **(−3%** [\*](#nt107){ref-type="table-fn"} **)**   23.5 (81%)   19.7 (68%)   **−3.8** [\*](#nt107){ref-type="table-fn"} **(−13%** [\*](#nt107){ref-type="table-fn"} **)**   23.6 (81%)   22.9 (79%)   **−0.7** [\*](#nt107){ref-type="table-fn"} **(−2%** [\*](#nt107){ref-type="table-fn"} **)**   23.8 (82%)   27.0 (93%)   **+3.2** [\*](#nt107){ref-type="table-fn"} **(+11%** [\*](#nt107){ref-type="table-fn"} **)**
  **Quality Rating** [B](#nt106){ref-type="table-fn"}                                                                                                                                                                                                                                                                                                                                                                                                                                    
  Excellent (score 27+)                                                                 27.2          19.3                                                                                                        15.0         0.8                                                                                                         13.5         12.4                                                                                                       10.7         92.7     
  Good (score 24--26)                                                                   32.4          32.6                                                p = 0.11                                                30.8         7.8                                                p  = 0.00                                                39.1         25.5                                               p  = 0.05                                               52.9         4.4                                                 p = 0.00
  Fair (score 20--23)                                                                   33.1          39.0                                                                                                        29.2         33.8                                                                                                        36.1         45.0                                                                                                       29.3          0       
  Poor (19 or less)                                                                     1.3           5.1                                                                                                         10.0         39.1                                                                                                        8.3          10.8                                                                                                       6.4           0       

Includes only percent of surgical observations with total score ( = 1 point).  Items receiving a partial score ( = 0.5 point) during observation were not counted as correctly done. 

Includes only facilities with no missing items.  Thus, the sum of percentages does not equal zero. 

Denotes statistically significant at p\<0.05. 

Quality scores were highest during the operative period for surgical techniques.  On average, 11.5 to 12.5 of 13 tasks were correctly done across countries.  High scores were maintained during the follow-up year in Kenya (−0.1, 95% CI −0.3, 0.01) with significant improvements in Zimbabwe to 12.9 (+1.4, 95% CI 1.1, 1.7).  Scores significantly decreased in South Africa to 11.6 (−0.8, 95% CI: −1.1,−0.5) and in Tanzania to 12.0 (−0.5, 95% CI: −0.7, −0.3).  Scores for specific surgical tasks are shown in [Table 4](#pone-0079524-t004){ref-type="table"}.  Except for Zimbabwe, tying the surgical knot was consistently done incorrectly across countries, resulting in lower percentages of observed surgeries during expansion for which all 13 operative tasks were done correctly (Kenya 50.2%, South Africa 33.7%, Tanzania 36.7%, Zimbabwe 94.1%).  Other incorrectly performed operative tasks were identification of the skin to be excised, administration of local anesthesia, and securing mattress sutures. 

10.1371/journal.pone.0079524.t004

###### Item-Level Quality of VMMC Surgical Care from Initial Year (2011) to Expansion Year (2012). 

![]( pone.0079524.t004){#pone-0079524-t004-4}

  *Indicator:* Percent (%) of surgical observations with Total Score ( = 1 point) by Item[A](#nt108){ref-type="table-fn"}                                                                                                                                                                                                     
  ------------------------------------------------------------------------------------------------------------------------- ------ ------ --------------------------------------------- ------ ------ --------------------------------------------- ------ ------ --------------------------------------------- ------ ------ ---------------------------------------------
  Number of surgical observations                                                                                            154    218                        \-                        116    361                        \-                        133    251                        \-                        140    240                        \-
  **Pre-Operative Assessment** *(1 item)*                                                                                                                                                                                                                                                                                     
  Performs preoperative assessment including targeted history and physical exam                                              79.5   70.6                       \-                        51.7   12.2   **−39.5** [\*](#nt109){ref-type="table-fn"}   20.3   26.3                       \-                        97.1   100    **+2.9** [\*](#nt109){ref-type="table-fn"}
  **Safe Operative Procedures** *(8 items)*                                                                                                                                                                                                                                                                                   
  Uses sterile instruments and consumables                                                                                   100    98.2                       \-                        96.7   97.8                       \-                        98.5   99.2                       \-                        99.3   100                        \-
  Uses sterile gloves                                                                                                        99.3   99.5                       \-                        95.0   84.8   **−10.2** [\*](#nt109){ref-type="table-fn"}   100    99.6                       \-                        99.3   99.5                       \-
  Hand washes or disinfects between clients                                                                                  90.1   90.4                       \-                        90.0   58.2   **−31.8** [\*](#nt109){ref-type="table-fn"}   93.2   96.4                       \-                        87.1   100    **+12.9** [\*](#nt109){ref-type="table-fn"}
  Maintains adequate sterile surgical field                                                                                  100    95.4   **−4.6** [\*](#nt109){ref-type="table-fn"}    85.0   76.7                       \-                        57.1   79.3   **+22.2** [\*](#nt109){ref-type="table-fn"}   80.7   100    **+19.3** [\*](#nt109){ref-type="table-fn"}
  Ensures protective eyewear worn by all surgical staff                                                                      5.3    6.4                        \-                        10.8   5.3                        \-                        25.6   55.4   **+29.8** [\*](#nt109){ref-type="table-fn"}   0.7    2.5                        \-
  Safely stores and disposes of medical waste                                                                                98.0   100                        \-                        91.7   72.0   **−19.7** [\*](#nt109){ref-type="table-fn"}   100    99.2                       \-                        98.6   99.0                       \-
  Correctly and hygienically processes instrument(s)                                                                         98.0   96.3                       \-                        96.7   95.6                       \-                        92.5   87.7                       \-                        97.1   100    **+2.9** [\*](#nt109){ref-type="table-fn"}
  Disinfects surgical beds between patients                                                                                  98.7   97.3                       \-                        92.5   59.8   **−32.7** [\*](#nt109){ref-type="table-fn"}   66.9   64.5                       \-                        84.3   96.6   **+12.3** [\*](#nt109){ref-type="table-fn"}
  **Step-wise Operative Techniques** *(13 items)*                                                                                                                                                                                                                                                                             
  Cleans surgical area with Chlorhexidine-based solution or Povidine Iodine                                                  100    98.6                       \-                        99.2   89.2   **−10.0** [\*](#nt109){ref-type="table-fn"}   98.5   99.2                       \-                        68.6   100    **+31.4** [\*](#nt109){ref-type="table-fn"}
  Identifies skin to be excised                                                                                              100    95.9   **−4.1** [\*](#nt109){ref-type="table-fn"}    97.5   70.1   **−27.4** [\*](#nt109){ref-type="table-fn"}   93.2   84.1   **−9.1** [\*](#nt109){ref-type="table-fn"}    91.4   100    **+8.6** [\*](#nt109){ref-type="table-fn"}
  Ensures no other penile parts are at risk of injury                                                                        100    100                        \-                        96.7   96.7                       \-                        99.3   98.4                       \-                        95.7   99.5   **+3.8** [\*](#nt109){ref-type="table-fn"}
  Safely administers local anesthesia                                                                                        98.0   89.9   **−8.1** [\*](#nt109){ref-type="table-fn"}    89.2   70.4   **−18.8** [\*](#nt109){ref-type="table-fn"}   97.0   90.0   **−7.0** [\*](#nt109){ref-type="table-fn"}    89.3   100    **+10.7** [\*](#nt109){ref-type="table-fn"}
  Cautiously/gently removes foreskin                                                                                         100    99.1                       \-                        95.8   94.5                       \-                        100    95.2   **−4.8** [\*](#nt109){ref-type="table-fn"}    94.3   100    **+5.7** [\*](#nt109){ref-type="table-fn"}
  Uses electrocautery and/or ligating sutures                                                                                100    100                        \-                        97.5   95.3   **−2.2** [\*](#nt109){ref-type="table-fn"}    95.5   92.8                       \-                        92.1   99.5   **+7.4** [\*](#nt109){ref-type="table-fn"}
  Correctly ties of surgical knot(s)                                                                                         56.3   67.0                       \-                        73.3   61.8   **−11.5** [\*](#nt109){ref-type="table-fn"}   72.9   47.4   **−25.5** [\*](#nt109){ref-type="table-fn"}   83.6   99.5   **+15.9** [\*](#nt109){ref-type="table-fn"}
  Aligns frenulum and secures mattress suture                                                                                100    98.6                       \-                        98.3   87.5   **−10.8** [\*](#nt109){ref-type="table-fn"}   100    99.2                       \-                        90.0   100    **+10.0** [\*](#nt109){ref-type="table-fn"}
  Correctly aligns other quadrant sutures                                                                                    99.3   99.5                       \-                        94.2   96.4                       \-                        100    99.2                       \-                        92.1   100    **+7.9** [\*](#nt109){ref-type="table-fn"}
  Avoids placing deep sutures around the frenulum                                                                            100    100                        \-                        98.3   96.7                       \-                        100    99.2                       \-                        93.6   94.6                       \-
  Places interrupted sutures evenly                                                                                          98.7   90.8   **−7.9** [\*](#nt109){ref-type="table-fn"}    85.8   96.4   **+10.6** [\*](#nt109){ref-type="table-fn"}   98.5   98.0                       \-                        89.3   99.5   **+10.2** [\*](#nt109){ref-type="table-fn"}
  Ensures no significant bleeding present                                                                                    100    100                        \-                        97.5   94.2                       \-                        100    99.2                       \-                        85.0   99.5   **+14.5** [\*](#nt109){ref-type="table-fn"}
  Places secure dressing                                                                                                     98.7   98.7                       \-                        98.3   96.4                       \-                        98.5   99.6                       \-                        88.6   100    **+11.4** [\*](#nt109){ref-type="table-fn"}
  **Post-Operative Assessment** *(7 items)*                                                                                                                                                                                                                                                                                   
  Waits to determine abnormality or allergic reaction                                                                        56.3   56.0                       \-                        40.8   7.5    **−33.3** [\*](#nt109){ref-type="table-fn"}   64.7   73.7                       \-                        22.1   99.5   **+77.4** [\*](#nt109){ref-type="table-fn"}
  Reviews patient vital signs                                                                                                44.4   33.9   **−10.5** [\*](#nt109){ref-type="table-fn"}   66.7   58.2                       \-                        48.9   42.6                       \-                         0     9.8    **+9.8** [\*](#nt109){ref-type="table-fn"}
  Provides wound care and pain management advice                                                                             63.6   63.8                       \-                        79.2   45.2   **−34.0** [\*](#nt109){ref-type="table-fn"}   63.9   68.5                       \-                        68.6   100    **+31.4** [\*](#nt109){ref-type="table-fn"}
  Encourages at least one patient follow-up visit                                                                            63.6   59.6                       \-                        87.5   63.2   **−24.3** [\*](#nt109){ref-type="table-fn"}   91.7   89.6                       \-                        98.6   100                        \-
  Provides emergency contact details to patient                                                                              96.0   91.3                       \-                        66.7   35.2   **−31.5** [\*](#nt109){ref-type="table-fn"}   82.0   68.5   **−13.5** [\*](#nt109){ref-type="table-fn"}   98.6   99.5                       \-
  Provides post-operative patient counseling                                                                                 57.6   41.3   **−16.3** [\*](#nt109){ref-type="table-fn"}   10.0   4.4                        \-                        63.2   22.7   **−40.5** [\*](#nt109){ref-type="table-fn"}   98.6   100                        \-
  Reminds of 6-week post-operative abstinence                                                                                56.3   31.2   **−25.1** [\*](#nt109){ref-type="table-fn"}   52.5   28.3   **−24.2** [\*](#nt109){ref-type="table-fn"}   31.6   17.1   **−14.5** [\*](#nt109){ref-type="table-fn"}   100    100                        \-

Includes only percent of surgical observations with total score ( = 1 point).  Items receiving a partial score ( = 0.5 point) during observation were not counted as correctly done. 

Differences from T~0~ to T~1~ reported at p\<0.05 statistical significance. 

While on average the majority of safe operative procedures were correctly done during the initial year (6.3 to 6.9 of the 8 minimum tasks), protective eyewear was often omitted in all 4 countries resulting in few surgeries where all 8 safety tasks were completed (Kenya: 4.1%, South Africa, 9.7%, Tanzania 9.8%, Zimbabwe 0%).  Kenya maintained otherwise high safety scores (−0.1, 95% CI: −0.2, 0.04), with significant improvements in Tanzania (+0.5, 95% CI: 0.3, 0.7) and Zimbabwe (+0.5, 95% CI: 0.4, 0.6).  Significantly fewer safety tasks were done in South Africa\'s expansion year as compared to the initial year (−1.0, 95% CI: −1.3, −0.7).  Lower safety scores reflected lack of hand washing, not disinfecting hands between clients, or not maintaining sterile surgical fields. 

Quality scores were lowest during the post-operative surgical period with significant decreases in three countries: Kenya, 4.4 to 3.8 out of 7 items (−0.6, 95% CI: −1.0, −0.2); South Africa, 4.0 to 2.4 (−1.6, 95% CI: −2.0, −1.3); and Tanzania, 4.5 to 3.9 (−0.6, 95% CI: −1.0, −0.2).  Providers in Zimbabwe significantly increased the average number of post-operative tasks correctly done from 4.9 to 6.1 (+1.2, 95% CI: 1.1, 1.3).  In the second year, only a small proportion of surgeries had all 7 post-operative tasks performed (Kenya 18.9%, South Africa 0.6%, Tanzania 8.5%, Zimbabwe 9.8%).  Post-operative tasks usually omitted were monitoring patients for reactions and reviewing vital signs, as well as providing post-operative counseling and reminding patients of the post-operative abstinence period. 

Discussion {#s4}
==========

Scaling up VMMC programs is an important strategy in the prevention of HIV infection in sub-Saharan African countries.  Previous VMMC quality assessments have often focused on reporting of adverse events.  However, no research to-date has examined changes in facility preparedness and correct performance of VMMC procedures as countries scale-up the number of facilities.  Given the potentially significant contribution of VMMC in decreasing HIV transmission, monitoring quality is critical in identifying and eliminating gaps in care during introductory and scale-up periods. 

This study\'s comparative process evaluation demonstrated that high quality VMMC services can be provided during rapid scale-up in sub-Saharan African settings.  Nearly all VMMC operative tasks observed in the study were performed correctly, with the exception of tying surgical knots.  However, the evaluation revealed several areas of sub-optimal VMMC performance levels.  Lower scores in infection control during surgery and pre- and post-operative tasks highlighted the need to reinforce provider training and supervision in these areas.  Pre-operative examinations are necessary to assess eligibility for surgery and identify surgical contradictions which may lead to adverse events.  Post-operative tasks are essential in prevention, identification, and management of complications.  Reinforcement of counseling messages during the post-operative period on the importance of risk reduction and sexual abstinence during wound-healing is also crucial.  Lower scores among these items could influence the overall effectiveness of VMMC given that transmission rates are directly affected by these behaviors. 

When examining overall scores, there were considerable variations across the two years of program implementation. [ Results](#s3){ref-type="sec"} showed that high levels of service quality can be maintained at-scale and, in some cases, dramatically improved during the process of scale-up, as was the experience of Kenya and Zimbabwe, respectively.  However, analyses also underscored the challenge of replicating initial successes at-scale in the case of South Africa, and the challenge of managing concurrent performance improvement and deterioration within the VMMC program in Tanzania.  Some safety measures improved, while surgical technique and post-operative care quality declined.  During the second year in both countries, less than a third of sampled sites had all requisite protocols to equip staff with readily accessible information.  Equally as few sites had all necessary equipment and supplies for normal and emergency surgeries.  The implications for expanding VMMC services in South Africa are further detailed in this supplement [@pone.0079524-Rech1].  In all four countries, the absence of HIV post-exposure prophylaxis was more severe during the second year, and shortages at both time points of antibiotics and BLS equipment can hinder the management of severe adverse events. 

Several lessons learned from this evaluation may inform future efforts to scale-up VMMC services in similar settings.  One key finding is that while high quality VMMC programs can be achieved at-scale, the process of scaling-up does not guarantee compliance with all quality criteria.  This was evidenced by the variability over time in facility preparedness and surgical care within countries.  In some settings, quality measures declined over the two-year period, or the VMMC program was expanded in the second year with comparably low performance levels as observed during the first year.  This evaluation emphasizes the need to routinely measure aspects of quality, and ensure that VMMC program expansion is linked with quality improvement initiatives. 

Secondly, while the diversity of program experiences among countries precludes any definitive comparison, some insight may be drawn from the range of scale-up models which were used.  All countries partnered with implementing agencies with variations in the implementation of fixed, mobile, and outreach sites.  In addition, focused models provided the advantage of allocating resources to regions with greatest disparities in HIV prevalence and male circumcision, while national models benefited from population-wide implementation.  Zimbabwe additionally delivered its VMMC program through a single implementing organization.  There were also differences across countries in the volume of VMMC facilities added during the evaluation\'s two-year period.  A final lesson emerging from this analysis is the recognition that challenges in commodity procurement and provider capacity may have reflected broader health systems strengthening needs that were not unique to VMMC.  Careful attention will be needed to ensure that the urgency of scaling up VMMC does not bypass efforts to strengthen health systems or hinder quality of existing health services in facilities where VMMC is introduced. 

Limitations {#s4a}
-----------

The study\'s limitations deserve mention.  VMMC facility preparedness was measured at the time of the study visit and does not reflect cumulative availability or adequacy of supplies relative to client load.  Thus, use of point-in-time measures limited the evaluation\'s ability to examine seasonal trends in VMMC quality as the program expanded.  Data were also collected during low volume service periods which may have confounded the study\'s observations, and the small number of facilities may have contributed to the wide confidence intervals of some measures.  In addition, it is possible that in some instances the evaluation of VMMC surgeries was biased given that scores were based on the evaluator\'s interpretation.  To minimize this, all observers were trained using a standardized observation guide.  However, it was not possible to standardize ratings across countries which may have resulted in unequal scoring. 

The evaluation\'s summary measures should likewise be interpreted carefully.  Quality of VMMC services was examined from the perspective of the health system.  Client perspectives of quality such as comfort, privacy, or communication adequacy are not reflected, and other systems-level quality metrics not measured in this study may have been informative.  Furthermore, it can be argued that not all items were equally critical, and thus the analysis\' equal weighting of items unduly magnified (or diluted) the clinical relevance of certain tasks.  Although the development of differential weights was not feasible within the study\'s parameters, to reduce this possibility, we aimed to balance the interpretive weight of each score by including in the questionnaire only the minimally essential elements in each category.  Thus, the range of scores is designed to reflect only items necessary to perform safe and effective circumcision.  Finally, the results from this study are unique to the selected countries and may not be generalizable to other settings.  This analysis was not designed to identify factors associated with differences in observed quality measures.  Therefore, causal or comparative inferences cannot be drawn between countries or facilities. 

Conclusions {#s4b}
-----------

To our knowledge, this study is the first to examine facility preparedness and quality of VMMC services in contexts of scale-up in four African countries.  Lapses in high performance and, in some cases, expansion of considerably deficient services emphasize the need to better integrate VMMC scale-up and quality improvement processes.  Monitoring service quality, not just adverse event reporting, will be essential in realizing the full health impact of VMMC for HIV prevention in African settings. 

The authors wish to acknowledge the organizations that conducted the research in each country: FHI360/Kenya, CHAPS in South Africa, MCHIP Jhpiego/Tanzania, and PSI/Zimbabwe in collaboration with ZAPP-UZ.  Co-investigators included Dr. Nicholas Muraguri, Dr. Peter Cherutich, Dr. Kawango Agot, Dr. Walter Obiero and Dr. Jackson Kioko (Kenya); Dr. Dirk Taljaard Dr. James McIntyre (South Africa), Dr. Bennet Fimbo and Dr. Eleuter Samky (Tanzania), and Dr. Karin Hatzold and Mr. Christopher Samkange (Zimbabwe).  Participating in the data collection were Nicolas Pule, Mulashi Biola, Daniel Shabangu, Sindiswe Zwane, Sindiswe Maseko, Dr. Sifuni Koshuma, Milton Kabiligi, Dr. Tendai Mutwirah, Dr. Eric Nyazika, Dr. Kelvin Nemayire, Rosemary Owigar, and Dr. Violet Naanyu.  We thank the data managers: Flora Hezwa, Dr. Obadia Venance, Nyongole, Dudzai Mureyi.  Kate Macintyre assisted on sampling (Kenya) and Hally Mahler provided oversight (Tanzania).  Emmanuel Njeuhmeli, Delivette Castor, and Jason Reed were co-investigators on SYMMACS.  Dr. Arti Shankar at Tulane SPHTM provided statistical consultation.  Drs.  Deanna Kerrigan and Caitlin Kennedy of Johns Hopkins Bloomberg School of Public Health provided technical guidance over the life of the project.  Andrea Vazzano and Emily Hurley provided editorial assistance for the final manuscripts. 

[^1]: **Competing Interests:**The authors have declared that no competing interest exist. 

[^2]: Conceived and designed the experiments: DR JB EN CS BF PC DC .  Performed the experiments: DR JB LP DA.  Analyzed the data: LJ DA DR JB LP DC EN.  Contributed reagents/materials/analysis tools: LJ JB DR SH KH CS DA BF PC LP DC EN.  Wrote the paper: LJ JB DR SH KH CS DA BF PC LP DC EN.