Log in Access provided by KU Leuven ARTICLES| VOLUME 21, ISSUE 5, P668-676, MAY 01, 2021 Performance of a single-use, rapid, point-of-care PCR device for the detection of Neisseria gonorrhoeae, Chlamydia trachomatis, and Trichomonas vaginalis: a cross-sectional study Sheldon R Morris, MD Claire C Bristow, PhD Michael R Wierzbicki, PhD Mark Sarno, eJD Lenore Asbel, MD Audrey French, MD et al. Show all authors Published:November 23, 2020DOI:https://doi.org/10.1016/S1473-3099(20)30734-9 PlumX Metrics Summary Background Timely detection and treatment are important for the control of Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis. The objective of this study was to measure the performance of the Visby Medical Sexual Health Test, a single-use, point-of-care PCR device. Methods Women aged 14 years and older who presented consecutively to ten clinical sites across seven US states were enrolled for a cross-sectional, single-visit study. Patients who consented to participate, and who had not used any exclusionary products in the genital area in the previous 48 h, provided self-collected vaginal swabs for testing with the investigational device. Untrained operators received the specimens and ran the device using the guide provided. Specimens had to be run within 2 h of collection to be considered valid. For comparison, patient-infected status was derived by testing clinician-collected vaginal specimens with the Hologic Aptima Combo 2 Assay and Aptima Trichomonas vaginalis Assay, as well as the BD ProbeTec CT/GC Qx Amplified DNA Assay and BD ProbeTec Trichomonas vaginalis Qx Assay. If the results of those assays did not match, the BD MAX CT/GC/TV was used as a tiebreaker. The primary outcomes were the sensitivity and specificity of the investigational device for the detection of C trachomatis, N gonorrhoeae, and T vaginalis compared with patient-infected status. Findings Between Feb 25, 2019, and Jan 6, 2020, 1585 participants aged between 14 years and 80 years (mean 34·8 [SD 14·2]) were enrolled. 1555 participants had tests run with the investigational device, of whom 1532 (98·5%) had a valid result on either the first or repeat test. Among the patients with evaluable results (including a determinate patient-infected status), the device had a sensitivity of 97·6% (95% CI 93·2–99·2) and specificity of 98·3% (97·5–98·9) for C trachomatis (n=1457), sensitivity of 97·4% (86·5–99·5) and specificity of 99·4% (98·9–99·7) for N gonorrhoeae (n=1468), and sensitivity of 99·2% (95·5–99·9) and specificity of 96·9% (95·8–97·7) for T vaginalis (n=1449). Interpretation This innovative, rapid, easy-to-use, single-use, point-of-care device to detect C trachomatis, N gonorrhoeae, and T vaginalis infections showed excellent sensitivity and specificity, and could represent an important advance in the development of rapid diagnostics for sexually transmitted infections and other infectious diseases. Funding Division of Microbiology and Infectious Diseases, National Institute of Allergy and Infectious Diseases. Introduction Nucleic acid amplification tests (NAATs) are the standard for quality clinical services to detect Neisseria gonorrhoeae, Chlamydia trachomatis, and Trichomonas vaginalis infections in the genital tract.1 However, most NAATs require complex laboratory instrumentation, which limits the immediacy of diagnosis and treatment decisions. As a result, standard practices can err by treating some individuals who are not infected if antibiotics are given empirically for syndromic management, or by not treating infected individuals lost to follow-up when the results are available.2 , 3 A diagnostic test that could be done at the point of care for curable sexually transmitted infections (STIs) could alleviate inappropriate antibiotic use, treat everyone that presents with an infection, and reduce the anxiety of waiting for test results for patients. In fact, accurate, rapid, point-of-care tests have been identified to be one of the innovations needed to regain traction against increasing rates of STIs.4 , 5 , 6 The challenge in developing point-of-care STI tests for C trachomatis, N gonorrhoeae, and T vaginalis in women is that, until 2019, the most compact and rapid tests were immunoassay lateral-flow devices, which rarely achieved clinical sensitivity over 90%—with a few exceptions, such as the OSOM Trichomonas Rapid Test (Sekisui Diagnostics) and the aQcare Chlamydia TRF kit (Medisensor).7 The most robust advance in point-of-care STI testing in women was compact NAAT technology that can be used at the point of care, such as the Cepheid CT/NG Xpert rapid PCR test.8 These instruments can run one to many samples and give results in less than 90 min; however, that turnaround time is not within a typical clinical visit, and the tests are fairly expensive and require a stable electricity supply. Newer, table-top platforms retaining high sensitivity and specificity for N gonorrhoeae and C trachomatis (none are available for T vaginalis) have reduced the time to results to 30 min, closer to the 20 min that most patients find acceptable.9 , 10 Through the first quarter of 2020, no point-of-care tests that were close to receiving clearance by the US Food & Drug Administration (FDA) were able to detect all common STIs at the point of service with an easy-to-use, self-contained system that can deliver results in under 30 min.6 The Visby Medical Sexual Health Test (Visby Medical, San Jose, CA, USA) is an innovative, single-use, rapid, nucleic acid-based diagnostic test for the detection of N gonorrhoeae, C trachomatis, and T vaginalis infections. This test can be done at the point of care without complex instrumentation and give a result in less than 30 min. We aimed to assess the performance of the Visby Medical Sexual Health Test for the detection of N gonorrhoeae, C trachomatis, and T vaginalis from self-collected vaginal specimens. Methods Study design, setting, and participants We did a cross-sectional, single-visit study of women aged 14 years and older at ten clinics across seven US states. Sites included STI clinics, student health clinics, primary care clinics, academic clinical research centres, and private clinical research organisations located in the states of California, Florida, Illinois, Maryland, Mississippi, Nevada, and Pennsylvania. We offered enrolment to consecutively presenting women who were either symptomatic or asymptomatic. Participant inclusion criteria were as follows: willing and able to give voluntary written informed consent (or parent or legal guardian consent for minors); female at birth (including pregnant and breastfeeding women); age 14 years or older at the time of enrolment; able to read and understand the procedural information provided for the study; and able and willing to follow all study procedures, including self-collection of one vaginal swab and permitting a licensed health-care provider to collect three additional vaginal swabs. Exclusion criteria included the following: having a medical condition, serious intercurrent illness, or other circumstance that, in the investigator's judgment, could jeopardise the woman's safety or could interfere with study procedures; previous enrolment in this study; and use of antiperspirants, deodorants, or certain vaginal products (douches, washes, lubricants, wipes, moisturisers, or feminine hygiene sprays) in the genital area within 48 h before enrolment. This study included collaboration between the device manufacturer (Visby Medical), the National Institute of Allergy and Infectious Diseases of the US National Institutes of Health (NIH), and the STAR STI Clinical Trial Group. The study was approved by a combination of a central institutional review board and local institutional review boards as required by the sites. Investigational device The Visby Medical Sexual Health Test device (figure 1) is a single-use, disposable, fully integrated, rapid, compact instrument containing a PCR-based assay for the qualitative detection and differentiation of DNA from N gonorrhoeae, C trachomatis, and T vaginalis. The device is intended to detect N gonorrhoeae, C trachomatis, and T vaginalis DNA in a variety of environments without the need for complex off-board sample processing or instrumentation. Figure 1Visby Medical Sexual Health Test View Large Image Download Hi-res image Download (PPT) The device is meant to be used by any clinic personnel using the quick-start instructions: input sample, activate device, and visually read colorimetric results in less than 30 min. A patient vaginal swab self-collection kit with transport media is provided with the product. The patient-collected swab sample is placed into a vial containing collection medium. A health-care professional then uses a transfer pipette to transfer 650 μL of the sample-containing media into the device input port. The transfer pipette automatically meters the sample volume needed for the device. Buttons 1, 2, and 3 are pressed in succession. After button 3 is pressed and the device is plugged in, a white LED light turns on indicating the reaction is in progress. At this point, the remaining device operation is automatic, including pathogen lysis and DNA release, PCR amplification, and detection of amplified products. When the reaction is complete, the LED light turns green indicating to the user that the results are ready. The results should be read within 2 h. If the sample is positive, a colorimetric change (from white to purple) is visible in the appropriate spot for the pathogen. If the test run is valid, results for all three organisms are readable. The Sexual Health Test contains all the reagents and instrumentation required to run a single PCR-based test. The device stores buffers and reagents on board for release at the correct time. Printed circuit boards control temperature and time the movement of the motors and liquid flow. Sample processing occurs on the device using a combination of heat and chemical lysis to release pathogen DNA. The inactivated sample is mixed with lyophilised PCR reagents and then amplified by continuous-flow PCR using a serpentine-shaped, plastic-moulded, fluidic circuit that allows rapid heating and cooling. Biotin-labelled PCR primers specifically amplify N gonorrhoeae, C trachomatis, and T vaginalis genes. A detection flow cell has oligonucleotide capture probes that hybridise to the amplified pathogenic target (appendix p 1). A colorimetric signal is generated on the flow cell when horseradish peroxidase-linked streptavidin binds biotin-labelled amplicon and capture-probe pairs and catalyses the conversion of 3,3ʹ,5,5ʹ-tetramethylbenzidine, resulting in a purple precipitate. The presence of target pathogen in the sample thus leads to a white-to-purple colour change on the detection flow cell. A control non-pathogenic Neisseria species is lyophilised and present in the lysis chamber of the device and serves as the positive internal process control. Lyophilised PCR primers that amplify the Neisseria spp internal control are included with primers for N gonorrhoeae, C trachomatis, and T vaginalis. The test is thereby monitored for effective sample preparation, PCR amplification, and detection. If all elements in the Sexual Health Test function properly, the positive control spot will produce a purple colour. The test is considered valid if there is a purple signal in the control window and a green check mark is visible on the test that indicates the test ran without error. Study procedures Consecutive participants who met the enrolment criteria were asked to provide consent, assessed for clinical symptoms of infection, and asked about recent medication and topical vaginal product use. Participants were then provided with the manufacturer's self-collection instructions and urogenital specimen collection kit containing a swab and transport media, which they used to self-collect a vaginal specimen. An untrained operator received each specimen and ran the device using the provided operators' guide, without coaching or instruction. According to the protocol, the test needed to be run within 2 h of collection. If the result was invalid (no signal in control window) or an error occurred (no green check indicating that device operated correctly), a second test would be run, also within 2 h of collection. Meanwhile, a licensed health-care provider would collect three randomly ordered vaginal swabs (to account for the possible effect of collection order on the comparator assay performance). The study health-care provider recorded any signs of infection. Swabs were stored and transported per manufacturer guidelines for vaginal collection and testing, and specimen collection was standardised for all study sites. Three comparator systems were used for control procedures: the Aptima Combo 2 Assay11 and Aptima Trichomonas vaginalis Assay (Hologic, San Diego, CA, USA),12 the BD ProbeTec CT/GC Qx Amplified DNA Assay13 and BD ProbeTec Trichomonas vaginalis Qx Assay (BD Molecular Diagnostics, Franklin Lakes, NJ, USA),14 and the BD MAX CT/GC/TV (BD Molecular Diagnostics, Franklin Lakes, NJ, USA),15 assuring three distinct nucleic acid amplification targets for each organism. The Aptima Trichomonas vaginalis Assay is not FDA-cleared for self-collected vaginal specimens, so all patient-infected status comparators were clinician-collected for consistency. One reference laboratory (Molecular Testing Labs, Vancouver, WA, USA) processed and tested all of the comparator assays for the study. The laboratory conducted quality control and quality assurance procedures according to the manufacturers' recommendations and in compliance with the College of American Pathology. Trained laboratory staff processed swabs, tested the specimens, and interpreted the results according to each manufacturer's instructions on the respective FDA-cleared assays. For inclusion in the analysis, specimens were required to have the test initiated within the stability window stated by the manufacturer. For initial equivocal, invalid, or otherwise indeterminate results, tests were repeated once, per manufacturer recommendations, before being classified as positive, negative, equivocal, or invalid or unresolved. Laboratory testing staff were masked to clinical information and the investigational device results. To assess patient-infected status, all participants had their specimens tested by the Aptima and BD ProbeTec systems. If there was discordance between those two results then the BD MAX CT/GC/TV result was used as a tiebreaker. Possible outcomes for patient-infected status by organism included infected, not infected, and indeterminate, with definitions determined a priori in the study protocol. Agreement between two of the three comparator assay results was required to determine a positive or negative patient-infected status. If there was no match between at least two comparator results, the status was considered indeterminate. Participants were categorised as symptomatic if they responded affirmatively to whether they had any of the following symptoms on the clinical symptoms assessment case report form: unusual vaginal discharge, vaginal irritation (itching, burning, or soreness), lower abdominal or pelvic pain, painful urination, increased urinary frequency, abnormal bleeding or spotting, or pain or bleeding with sex or intercourse. Otherwise, the participant was classified as asymptomatic. Statistical methods The analytic population were those who were confirmed to have met the selection criteria and had specimens collected and run according to protocol. If results were missing from the Sexual Health Test, the data were not included in the analysis. If the patient-infected status was indeterminate or results were missing from the comparators that were required to determine a patient-infected status for an organism (ie, either the Aptima or Probetec result for any case, or the BD MAX CT/GC/TV result when a tiebreaker was needed), then the results for the organism without a valid positive or negative result for patient-infected status for that participant were excluded from the analysis. For each organism, we calculated the sensitivity, specificity, accuracy (total agreement), positive predictive value, and negative predictive value of the investigational device against the patient-infected status, with 95% Wilson Cls.16 The study was designed with a target sample size to achieve 120 positive cases of C trachomatis, 120 of T vaginalis, and 45 of N gonorrhoeae infections. The sample size was determined by the numbers required for a sensitivity of 95% with a lower confidence bound of 90% for C trachomatis and N gonorrhoeae and 85% for T vaginalis. The sample size was also to meet a specificity of at least 95% with a lower bound of 90%. In sub-analyses test characteristics were determined for those who were symptomatic and asymptomatic. Analyses and figure and table generation were done with SAS version 9.4 or above. This study is registered with ClinicalTrials.gov (NCT03852316). Role of the funding source The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication. Results Between Feb 25 2019, and Jan 6, 2020, 1585 women were enrolled in the study. Participants were aged between 14 years and 80 years (mean 34·8 [SD 14·2]), with broad demographic and geographical representation (table 1). Women who were pregnant or menstruating were included. Table 1Characteristics of enrolled study participants Participants (n=1585) Sex Female 1585 (100·0%) Ethnicity Not Hispanic or Latino 1168 (73·7%) Hispanic or Latino 310 (19·6%) Unknown 107 (6·8%) Race American Indian or Alaska Native 12 (0·8%) Asian 86 (5·4%) Native Hawaiian or other Pacific Islander 6 (0·4%) Black or African American 876 (55·3%) White 414 (26·1%) Multi-racial 70 (4·4%) Unknown 121 (7·6%) Age, years 14–17 9 (0·6%) 18–25 564 (35·6%) 26–35 366 (23·1%) 36–45 225 (14·2%) 46–55 225 (14·2%) >55 196 (12·4%) Mean (SD) 34·8 (14·2) Study site University of California San Diego Antiviral Research Center (San Diego, CA) 186 (11·7%) Stroger Hospital of Cook County—CORE (Chicago, IL) 243 (15·3%) San Francisco Department of Public Health (San Francisco, CA) 71 (4·5%) Philadelphia Department of Public Health (Philadelphia, PA) 281 (17·7%) University of Mississippi Medical Center (Jackson, MS) 62 (3·9%) Florida International University (Miami, FL) 279 (17·6%) AXIS Clinical Trials (Los Angeles, CA) 375 (23·7%) Johns Hopkins University Rangos (Baltimore, MD) 16 (1·0%) South Florida Clinical Trials (Hialeah, FL) 12 (0·8%) Impact Clinical Trial (Las Vegas, NV) 60 (3·8%) Data are n (%), unless otherwise specified. Open table in a new tab The Visby Medical Sexual Health Test was run for 1555 (98·1%) participants, of whom 1444 (92·9%) had a valid first test and a further 88 (5·7%) had valid results with a repeat test, for a total of 1532 (98·5% [95% CI 97·8–99·1]). Of those with a valid test result from the investigational device, 154 (10·1% [8·6–11·7]) were positive for C trachomatis, 50 (3·3% [2·5–4·3]) were positive for N gonorrhoeae, and 178 (11·6% [10·1–13·3]) were positive for T vaginalis (figure 2). In 1579 participants, the patient-infected status could be determined for at least one organism: 136 of 1579 participants were positive for C trachomatis (8·6% [7·3–10·1]), 44 of 1578 were positive for N gonorrhoeae (2·8% [2·0–3·6]), and 137 of 1579 were positive for T vaginalis (8·7% [7·4–10·2]). Subsequently, in addition to those with no valid result from the Visby Medical device, 75 patients were excluded from the C trachomatis analysis, 64 from the N gonorrhoeae analysis, and 83 from the T vaginalis analysis (figure 2), for reasons including the following: protocol deviations for study procedures (consenting error, incorrect collection of specimens, controls not done correctly before testing, testing outside the 2 h window, mislabelled specimen, non-completion of study visit), violation of study inclusion or exclusion criteria discovered after visit (one repeat participant and one participant with use of excluded product in the previous 48 h), no result available from investigational device (device not run or all runs within testing window were invalid or had an error), or no result for patient-infected status (lacking two comparators in agreement for any reason). In the final evaluable results, there were 1457 (93·7%) available for C trachomatis, 1468 (94·4%) for N gonorrhoeae, and 1449 (93·2%) for T vaginalis. Figure 2Study flow diagram Show full caption View Large Image Download Hi-res image Download (PPT) Table 2 shows the diagnostic performance measures for the investigational device by organism when compared with patient-infected status. The device had a sensitivity of 97·6% (95% CI 93·2–99·2) and a specificity of 98·3% (95% CI 97·5–98·9) for C trachomatis, sensitivity of 97·4% (86·5–99·5) and specificity of 99·4% (98·9–99·7) for N gonorrhoeae, and sensitivity of 99·2% (95·5–99·9) and specificity of 96·9% (95·8–97·7) for T vaginalis. Similar sensitivity and specificity were found for all organisms when the analysis was stratified by symptomatic and asymptomatic status (table 2). For all evaluable results, the Visby Medical Sexual Health Test had an overall accuracy (agreement with patient-infected status) in 98·3% (97·5–98·8) of results for C trachomatis, 99·4% (98·8–99·7) for N gonorrhoeae, and 97·1% (95% CI 96·1- 97·8) for T vaginalis. Table 2Test performance of Visby Medical Sexual Health Test for the detection of Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis, by organism and symptomatic status Patient-infected status designation, n Sensitivity* Specificity* Accuracy* Positive Negative Total Result in symptomatic and asymptomatic patients C trachomatis (n=1457) .. .. .. 97·6% (93·2–99·2) 98·3% (97·5–98·9) 98·3% (97·5–98·8) Positive 122 22 144 .. .. .. Negative 3 1310 1313 .. .. .. Total 125 1332 1457 .. .. .. N gonorrhoeae (n=1468) .. .. .. 97·4% (86·5–99·5) 99·4% (98·9–99·7) 99·4% (98·8–99·7) Positive 37 8 45 .. .. .. Negative 1 1422 1423 .. .. .. Total 38 1430 1468 .. .. .. T vaginalis (n=1449) .. .. .. 99·2% (95·5–99·9) 96·9% (95·8–97·7) 97·1% (96·1–97·8) Positive 121 41 162 .. .. .. Negative 1 1286 1287 .. .. .. Total 122 1327 1449 .. .. .. Result in symptomatic patients C trachomatis (n=735) .. .. .. 98·7% (93·0–99·8) 97·7% (96·3–98·6) 97·8% (96·5–98·7) Positive 76 15 91 .. .. .. Negative 1 643 644 .. .. .. Total 77 658 735 .. .. .. N gonorrhoeae (n=746) .. .. .. 100·0% (83·9–100·0) 99·4% (98·6–99·8) 99·5% (98·6–99·8) Positive 20 4 24 .. .. .. Negative 0 722 722 .. .. .. Total 20 726 746 .. .. .. T vaginalis (n=734) .. .. .. 98·6% (92·6–99·8) 95·8% (93·9–97·1) 96·0% (94·4–97·2) Positive 72 28 100 .. .. .. Negative 1 633 634 .. .. .. Total 73 661 734 .. .. .. Result in asymptomatic patients C trachomatis (n=722) .. .. .. 95·8% (86·0–98·8) 99·0% (97·9–99·5) 98·8% (97·6–99·3) Positive 46 7 53 .. .. .. Negative 2 667 669 .. .. .. Total 48 674 722 .. .. .. N gonorrhoeae (n=722) .. .. .. 94·4% (74·2–99·0) 99·4% (98·5–99·8) 99·3% (98·4–99·7) Positive 17 4 21 .. .. .. Negative 1 700 701 .. .. .. Total 18 704 722 .. .. .. T vaginalis (n=715) .. .. .. 100·0% (92·7–100·0) 98·0% (96·7–98·9) 98·2% (96·9–98·9) Positive 49 13 62 .. .. .. Negative 0 653 653 .. .. .. Total 49 666 715 .. .. .. The denominator for estimates is based on participants in the evaluable population for the specified organism. * As determined by patient-infected status designation; data are % (95% Wilson CI). Open table in a new tab Table 3 shows the agreement with each comparator. Positive agreement with the investigational device was highest for the Hologic Aptima and BD MAX assays for N gonorrhoeae and C trachomatis, and with the BD ProbeTec and BD MAX assays for T vaginalis. Positive predictive values and negative predictive values across a range of prespecified hypothetical prevalences of interest, adjusting the study specific values via Bayes' rule, are shown in table 4. Table 3Performance of the Visby Medical Sexual Health Test for the detection of Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis, by organism, against each comparator device Visby Medical Sexual Health Test result, n Positive agreement* Negative agreement* Accuracy* Positive Negative Total C trachomatis result Hologic Aptima .. .. .. 97·6% (93·1–99·2) 98·2% (97·3–98·8) 98·1% (97·3–98·7) Positive 120 3 123 .. .. .. Negative 24 1308 1332 .. .. .. Total 144 1332 1455 .. .. .. BD ProbeTec .. .. .. 92·3% (86·4–95·8) 98·2% (97·3–98·8) 97·7% (96·8–98·3) Positive 120 10 130 .. .. .. Negative 24 1303 1327 .. .. .. Total 144 1313 1457 .. .. .. BD MAX .. .. .. 98·2% (90·6–99·7) 98·0% (96·3–98·9) 98·0% (95·8–97·7) Positive 55 1 56 .. .. .. Negative 9 442 451 .. .. .. Total 64 443 507 .. .. .. N gonorrhoeae result Hologic Aptima .. .. .. 97·3% (86·2–99·5) 99·4% (98·8–99·7) 99·3% (98·7–99·6) Positive 36 1 37 .. .. .. Negative 9 1419 1428 .. .. .. Total 45 1420 1465 .. .. .. BD ProbeTec .. .. .. 83·3% (69·4–91·4) 99·3% (98·7–99·6) 98·8% (98·2–99·3) Positive 35 7 42 .. .. .. Negative 10 1413 1423 .. .. .. Total 45 1420 1465 .. .. .. BD MAX .. .. .. 100·0% (83·2–100·0) 99·2% (98·0–99·7) 99·2% (98·0–99·7) Positive 19 0 19 .. .. .. Negative 4 497 501 .. .. .. Total 23 497 520 .. .. .. T vaginalis result Hologic Aptima .. .. .. 82·0% (75·1–87·3) 97·1% (96·1–97·9) 95·6% (94·4–96·6) Positive 123 27 150 .. .. .. Negative 37 1259 1296 .. .. .. Total 160 1286 1446 .. .. .. BD ProbeTec .. .. .. 95·9% (90·8–98·2) 96·6% (95·5–97·5) 96·5% (95·5–97·4) Positive 117 5 122 .. .. .. Negative 45 1282 1327 .. .. .. Total 162 1287 1449 .. .. .. BD MAX .. .. .. 100·0% (91·0–100·0) 96·6% (94·5–97·9) 96·8% (94·9–98·0) Positive 39 0 39 .. .. .. Negative 16 448 464 .. .. .. Total 55 448 503 .. .. .. The denominator for estimates is based on participants in the evaluable population for the specified organism. BD Max was not run for every sample and was only used as a tiebreaker or, in some cases, was run if the sample was going to pass the testing window and the other comparator results were not completed. * Data are % (95% Wilson CI). Open table in a new tab Table 4Estimates of positive predictive value and negative predictive value across a range of hypothetical prevalences, by organism Positive predictive value Negative predictive value Chlamydia trachomatis prevalence 1% 37·4% >99·9% 2% 54·7% >99·9% 5% 75·7% 99·9% 10% 86·8% 99·7% 20% 93·7% 99·4% Neisseria gonorrhoeae prevalence 1% 63·7% >99·9% 2% 78·0% >99·9% 5% 90·2% 99·9% 10% 95·1% 99·7% 20% 97·8% 99·3% Trichomonas vaginalis prevalence 1% 24·5% >99·9% 2% 39·6% >99·9% 5% 62·8% >99·9% 10% 78·1% >99·9% 20% 88·9% 99·8% Open table in a new tab A second Sexual Health Test was done if the first was invalid and the specimen was still within the 2 h window. The first test was invalid in 111 (7·1%) of 1555 patients, and 23 (1·5%) patients had invalid tests on both the first and second runs (appendix p 2). Of 27 operators, four left the study before they completed a user survey. Of the 23 who completed the survey on usability, in which they were asked about ease of use (appendix p 3), all operators responded that they would agree or strongly agree that “It was easy to set up the device”, “The instructions for the device were easy to follow”, and “Overall, it was easy to run the device”. For the question “It was easy to see and understand the test results”, 21 (91%) operators responded that they agreed or strongly agreed, and two (9%) were neutral. Discussion The results of this study showed that a single-use rapid device to detect N gonorrhoeae, C trachomatis, and T vaginalis from a self-collected vaginal specimen in women was highly accurate when performed by clinic staff without laboratory training. The device also delivered a valid test result for 98·5% of the samples provided. Results from this study were used to support a regulatory submission and Clinical Laboratory Improvement Act (CLIA) waiver submission to the FDA for the first rapid, simple, molecular, point-of-care testing device for STIs. This represents a major step forward for the detection of STIs that could result in more timely and accurate patient care, improved control of these infections, and a reduction in STI complications.17 Widespread use of the rapid device would allow for treatment decisions in real time. Such use has been shown to improve appropriate treatment and reduce unnecessary antibiotic exposure.18 , 19 Given the rising burden of STIs, a simple point-of-care device with those characteristics is highly desirable.20 This device is the first to apply the advances in molecular and microfluidic technology towards true point-of-care, highly accurate, and rapid testing that can be applied to many prevalent infectious diseases globally.21 Compared with the Visby Medical Sexual Health Test, other point-of-care test systems—such as the binx io and GeneXpert—require samples to be placed in a desktop PCR instrument that once loaded will run for 30–45 min or longer.8 , 22 , 23 The Visby Medical device is compact, transportable, scalable and, because it is single-use, does not require maintenance. Samples for one or many patients can be run sequentially as patients are seen in real time. Although other technologies are in development,6 , 24 this device could be the first to be cleared for such use. The Visby Medical Sexual Health Test meets many of the desirable characteristics defined by WHO for a point-of-care device: sensitive, specific, user-friendly, rapid, robust, equipment-free, and deliverable to end users.17 Most importantly, we observed excellent sensitivity and specificity when compared with the patient-infected status reference for the detection of N gonorrhoeae, C trachomatis, and T vaginalis. Our findings are similar to published estimates of the performance for NAATs for STIs.12 , 14 , 15 , 25 , 26 The closest examples of true point-of-care PCR tests are the binx io and GenXpert, which have been investigated in studies of self-collected vaginal specimens, although neither covers all three of the organisms covered by the Visby Medical device. The binx io device had a sensitivity of 96·1% and specificity of 97·7% for the detection of C trachomatis (n=51).9 The GeneXpert had a sensitivity of 98·7% and a specificity of 99·4% for C trachomatis (n=79), and a sensitivity of 100·0% and a specificity of 99·9% for N gonorrhoeae (n=22).8 In the current study, operators of the Visby Medical device were untrained and used the enclosed quick-start instructions to operate the test. They found the test easy to use and were successful in operating the tests. These results suggest that the test could meet CLIA-waiver requirements. Strengths of this study include a diverse study population of participants from across different geographical and clinical sites. To minimise bias, all devices used to determine patient-infected status had different amplification targets and methodologies to avoid the results being systematically aligned. Weaknesses of the study include the moderate number of N gonorrhoeae cases, which might have affected the precision of the observed sensitivity. Finding a substantial number of N gonorrhoeae cases in women is a challenge, and we did attempt to go to higher prevalence areas and recall women who had tested positive for N gonorrhoeae in a clinic to come in for a study visit. Another weakness is that, because the patient-infected status composite reference method required at least two comparator assays to agree, limitations in the performance of the comparator assays had the potential to affect the observed device accuracy. The proportion of specimens that tested positive with the investigational device was higher than the proportion with a positive patient-infected status, and there were differences in agreement between devices, but not enough evidence to clearly support higher sensitivity of the investigational device. Only one false-negative result for N gonorrhoeae was recorded for the device when compared against patient-infected status, and in that case the stored device image revealed a probable misinterpretation by the operator when later reviewed by study staff. Additionally, because this study did not use self-collected specimens for the comparator assays but did for the test device, the positivity would also be biased in favour of the test device because self-collected specimens for women are more sensitive.27 To adopt the use of this or any point-of-care test for STIs, clinics will need to incorporate training and timing into their workflows. It will be important for clinicians to understand that positive predictive values of STI tests vary by organism, and that a positive value for an organism in a low prevalence setting (where positive predictive value is less than 90%) will require interpretation of the result within the context and possibly further confirmation by a second test. For usability, this technology will need to be validated with samples from all genders and anatomical sites. In summary, this study evaluated a new, first-in-class, diagnostic point-of-care device for the detection of STIs in women. The Visby Medical Sexual Health Test had excellent agreement with other FDA-approved laboratory-based assays for the detection of N gonorrhoeae, C trachomatis, and T vaginalis and should be suitable for a CLIA-waived test environment. This device has major potential for the rapid detection of STIs in clinical settings. Contributors SRM, LA, AF, CAG, LH, LM, PM, SP, SS, ands CB ran the clinical trial sites. SRM, CBB, MS, TW, MRW, and JDK wrote and led clinical trial protocol development and implementation. MRW and DS were involved in statistical analysis. SRM, CBB, and JDK drafted the manuscript. All authors approved the final submitted version of the manuscript. Declaration of interests SRM, CBB, DS, LH, CB, SS, CAG, TW, AF, SP, MS, LM, and JDK received funds via grants or contracts from the NIH. SP has received funding from University of California San Francisco. MS has received personal fees from Visby Medical. CAG has received funding support from Cepheid, SpeeDx, Binx Health, Becton Dickinson, and Hologic. LM has received grant funding from Roche Molecular, Evofem, Janssen Pharmaceutical, Prosoft Clinical, GlaxoSmithKlein, and SpeeDx. JDK has received support from Danaher, Cepheid, and Hologic. SRM held financial interests in Pfizer, Bristol Myers Squibb, Geron, and Forty Seven; and has received grant funding from Gilead, Merck, the California Institute of Regenerative Medicine, and the California HIV/AIDS Research Program. All other authors declare no competing interests. Acknowledgments Overall support for the STAR Sexually Transmitted Infections Clinical Trials Group and study sites was provided by the National Institute of Allergy and Infectious Diseases Division of Microbiology and Infectious Diseases (HHSN272201300014I and HHSN27200010). The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. We wish to also thank all collaborators on this project, especially Paula Walker and others at Visby Medical, and Chris Hemphill and Garrick Osterhold at the Molecular Testing Labs and Axis Clinical Trial sites. The authors acknowledge the scientific and programmatic guidance of Peter Wolff and Carolyn Deal of the NIH Division of Microbiology and Infectious Diseases. Supplementary Material Download .pdf (.1 MB) Help with pdf files Supplementary appendix References 1.Barrow RY Ahmed F Bolan GA Workowski KA Recommendations for providing quality sexually transmitted diseases clinical services, 2020. MMWR Recomm Rep. 2020; 68: 1-20 View in Article Scopus (24) PubMed Crossref Google Scholar 2.Huppert JS Reed JL Munafo JK et al. Improving notification of sexually transmitted infections: a quality improvement project and planned experiment. Pediatrics. 2012; 130: e415-e422 View in Article Scopus (21) PubMed Crossref Google Scholar 3.Filice GA Drekonja DM Thurn JR Hamann GM Masoud BT Johnson JR Diagnostic errors that lead to inappropriate antimicrobial use. Infect Control Hosp Epidemiol. 2015; 36: 949-956 View in Article Scopus (26) PubMed Crossref Google Scholar 4.Toskin I Peeling RW Mabey D et al. Point-of-care tests for STIs: the way forward. Sex Transm Infect. 2017; 93: S1-S2 View in Article Scopus (11) PubMed Crossref Google Scholar 5.Eisinger RW Erbelding E Fauci AS Refocusing research on sexually transmitted infections. J Infect Dis. 2019; 222: 1432-1434 View in Article Scopus (8) Crossref Google Scholar 6.Cristillo AD Bristow CC Peeling R et al. Point-of-care sexually transmitted infection diagnostics: proceedings of the STAR Sexually Transmitted Infection-Clinical Trial Group programmatic meeting. Sex Transm Dis. 2017; 44: 211-218 View in Article Scopus (27) PubMed Crossref Google Scholar 7.Herbst de Cortina S Bristow CC Joseph Davey D Klausner JD A systematic review of point of care testing for Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis. Infect Dis Obstet Gynecol. 2016; 20164386127 View in Article Scopus (66) PubMed Crossref Google Scholar 8.Gaydos CA Van Der Pol B Jett-Goheen M et al. Performance of the Cepheid CT/NG Xpert rapid PCR test for detection of Chlamydia trachomatis and Neisseria gonorrhoeae. J Clin Microbiol. 2013; 51: 1666-1672 View in Article Scopus (158) PubMed Crossref Google Scholar 9.Harding-Esch EM Cousins EC Chow SC et al. A 30-min nucleic acid amplification point-of-care test for genital Chlamydia trachomatis infection in women: a prospective, multi-center study of diagnostic accuracy. EBioMedicine. 2018; 28: 120-127 View in Article Scopus (20) PubMed Summary Full Text Full Text PDF Google Scholar 10.Gettinger J Van Wagoner N Daniels B Boutwell A Van Der Pol B Patients are willing to wait for rapid sexually transmitted infection results in a university student health clinic. Sex Transm Dis. 2020; 47: 67-69 View in Article Scopus (5) PubMed Crossref Google Scholar 11.Gaydos CA Quinn TC Willis D et al. Performance of the APTIMA Combo 2 assay for detection of Chlamydia trachomatis and Neisseria gonorrhoeae in female urine and endocervical swab specimens. J Clin Microbiol. 2003; 41: 304-309 View in Article Scopus (175) PubMed Crossref Google Scholar 12.Chapin K Andrea S APTIMA® Trichomonas vaginalis, a transcription-mediated amplification assay for detection of Trichomonas vaginalis in urogenital specimens. Expert Rev Mol Diagn. 2011; 11: 679-688 View in Article Scopus (29) PubMed Crossref Google Scholar 13.Van Der Pol B Ferrero DV Buck-Barrington L et al. Multicenter evaluation of the BDProbeTec ET System for detection of Chlamydia trachomatis and Neisseria gonorrhoeae in urine specimens, female endocervical swabs, and male urethral swabs. J Clin Microbiol. 2001; 39: 1008-1016 View in Article Scopus (243) PubMed Crossref Google Scholar 14.Lord E Newnham T Dorrell L et al. Detecting asymptomatic Trichomonas vaginalis in females using the BD ProbeTec™ Trichomonas vaginalis Qx nucleic acid amplification test. Int J STD AIDS. 2017; 28: 357-361 View in Article Scopus (5) PubMed Crossref Google Scholar 15.Kawa D Kostiha B Yu JH LeJeune M Elevating the standard of care for STIs: the BD MAX CT/GC/TV assay. http://moleculardiagnostics.bd.com/wp-content/uploads/2017/08/CT-GC-TV-Whitepaper.pdf Date: February, 2017 Date accessed: November 16, 2020 View in Article Google Scholar 16.US Food and Drug Adminstration Guidance for industry and FDA staff: statistical guidance on reporting results from studies evaluating diagnostic tests. https://www.fda.gov/media/71147/download Date: March 13, 2007 Date accessed: November 16, 2020 View in Article Google Scholar 17.Peeling RW Holmes KK Mabey D Ronald A Rapid tests for sexually transmitted infections (STIs): the way forward. Sex Transm Infect. 2006; 82: v1-v6 View in Article Scopus (368) PubMed Crossref Google Scholar 18.Fisk KM Derouin A Holm G Hicks L Getting it right: the impact of point-of-care testing for gonorrhea and chlamydia in the urgent care setting. J Nurse Pract. 2020; 16: 388-393 View in Article Scopus (2) Summary Full Text Full Text PDF Google Scholar 19.Turner KME Christensen H Adams EJ et al. Analysis of the potential for point-of-care test to enable individualised treatment of infections caused by antimicrobial-resistant and susceptible strains of Neisseria gonorrhoeae: a modelling study. BMJ Open. 2017; 7e015447 View in Article Scopus (28) PubMed Crossref Google Scholar 20.Kuehn BM A proactive approach needed to combat rising STIs. JAMA. 2019; 321: 330-332 View in Article Scopus (6) PubMed Crossref Google Scholar 21.Wang S Lifson MA Inci F Liang L-G Sheng Y-F Demirci U Advances in addressing technical challenges of point-of-care diagnostics in resource-limited settings. Expert Rev Mol Diagn. 2016; 16: 449-459 View in Article Scopus (61) PubMed Crossref Google Scholar 22.Jacobsson S Boiko I Golparian D et al. WHO laboratory validation of Xpert® CT/NG and Xpert® TV on the GeneXpert system verifies high performances. APMIS. 2018; 126: 907-912 View in Article Scopus (21) PubMed Crossref Google Scholar 23.Van Der Pol B Taylor SN Mena L et al. Evaluation of the performance of a point-of-care test for chlamydia and gonorrhea. JAMA Netw Open. 2020; 3e204819 View in Article Scopus (12) PubMed Crossref Google Scholar 24.Gaydos C Hardick J Point of care diagnostics for sexually transmitted infections: perspectives and advances. Expert Rev Anti Infect Ther. 2014; 12: 657-672 View in Article Scopus (85) PubMed Crossref Google Scholar 25.Van Der Pol B Williams JA Fuller D Taylor SN Hook 3rd, EW Combined testing for chlamydia, gonorrhea, and trichomonas by use of the BD Max CT/GC/TV assay with genitourinary specimen types. J Clin Microbiol. 2016; 55: 155-164 View in Article Scopus (26) PubMed Crossref Google Scholar 26.Schachter J Chernesky MA Willis DE et al. Vaginal swabs are the specimens of choice when screening for Chlamydia trachomatis and Neisseria gonorrhoeae: results from a multicenter evaluation of the APTIMA assays for both infections. Sex Transm Dis. 2005; 32: 725-728 View in Article Scopus (174) PubMed Crossref Google Scholar 27.Korownyk C Kraut RY Kolber MR Vaginal self-swabs for chlamydia and gonorrhea. Can Fam Physician. 2018; 64: 448 View in Article PubMed Google Scholar Article Info Publication History Published: November 23, 2020 Identification DOI: https://doi.org/10.1016/S1473-3099(20)30734-9 Copyright © 2020 Elsevier Ltd. All rights reserved. ScienceDirect Access this article on ScienceDirect Figures Figure 1Visby Medical Sexual Health Test Figure 2Study flow diagram Tables Table 1Characteristics of enrolled study participants Table 2Test performance of Visby Medical Sexual Health Test for the detection of Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis, by organism and symptomatic status Table 3Performance of the Visby Medical Sexual Health Test for the detection of Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis, by organism, against each comparator device Table 4Estimates of positive predictive value and negative predictive value across a range of hypothetical prevalences, by organism Linked Articles Accurate, rapid, point-of-care tests for sexually transmitted infections Full-Text PDF Related Specialty Collections This article can be found in the following collections: Diagnostics Sexually transmitted infections LANCET JOURNALS The Lancet The Lancet Child & Adolescent Health The Lancet Diabetes & Endocrinology The Lancet Digital Health The Lancet Gastroenterology & Hepatology The Lancet Global Health The Lancet Haematology The Lancet Healthy Longevity The Lancet HIV The Lancet Infectious Diseases The Lancet Microbe The Lancet Neurology The Lancet Oncology The Lancet Planetary Health The Lancet Psychiatry The Lancet Public Health The Lancet Regional Health – Americas The Lancet Regional Health – Europe The Lancet Regional Health – Western Pacific The Lancet Respiratory Medicine The Lancet Rheumatology EBioMedicine EClinicalMedicine CLINICAL INITIATIVES The Lancet Clinic Commissions Series Picture Quiz GLOBAL HEALTH INITIATIVES Global Health Hub Commissions Series Global Burden of Disease Climate Countdown MULTIMEDIA Infographics Podcasts Videos INFORMATION About us For authors For advertisers For press Publishing excellence Careers Conferences Community guidelines Peer review Preprints ACCESS Access our content Register an account My account Personal subscriptions Open Access Existing print subscribers Request institutional access CONNECT Lancet Alerts Contact us Customer service We use cookies to help provide and enhance our service and tailor content and ads. To update your cookie settings, please visit the Cookie Settings for this site. Copyright © 2021 Elsevier Inc. except certain content provided by third parties. The content on this site is intended for healthcare professionals. Privacy Policy Terms and Conditions We use cookies to analyse and improve our service, to improve and personalise content, advertising and your digital experience. We also share information about your use of our site with our social media, advertising and analytics partners. Cookie Policy Cookie Settings Accept all cookies