abstract: This blinded field safety study was conducted in Senegal to assess safety and immunogenicity of administration of the registered dose of Rift Valley fever virus (RVFV) Clone 13 vaccine (Onderstepoort Biological Products) to sheep and goats of West African breeds under natural conditions. A total of 267 small ruminants (220 sheep, 47 goats) were included; half received RVFV Clone 13 vaccine at the recommended dose and half received the diluent (as placebo) only. The study was performed on three commercial farms in the northern and eastern region of Senegal in accordance with veterinary good clinical practices. The animals were observed daily for 3 days after vaccination, and then weekly for 1 year. In both sheep and goats vaccinated against RVFV seroconversion rates above 70% were recorded. No seroconversion related to RVFV was observed in placebo-treated animals. No statistically significant differences were determined between placebo and vaccinated groups for mean rectal temperatures for the first 3 days after administration (*p* \> 0.05). No abnormal clinical signs related to treatment were noted, and only one slight injection site reaction was observed in one vaccinated animal for 2 days after vaccination. Out of 176 births assessed over 1 year (93 from the vaccinated group, 83 from the placebo group), 9 were abnormal in the placebo group and 3 in the vaccinated group (*p* \> 0.05). The frequency of adverse events was similar in the placebo and vaccinated groups. RVFV Clone 13 vaccine administered according to the manufacturer's instructions was safe and well tolerated in West African breeds of sheep and goats, including animals of approximately 6 months of age and pregnant females, under field conditions in Senegal. Antibody levels persisted up to 1 year after vaccination.
author: Modou M. Lo; Victor Mbao; Pascale Sierra; Yaya Thiongane; Mariame Diop; Meritxell Donadeu; Baptiste Dungu**Correspondence to:** Victor Mbao **Email:**<email@example.com>**Postal address:** Private Bag A 130, Lilongwe 03, Malawi[^1]
date: 2015-05-29
references:
title: Safety and immunogenicity of Onderstepoort Biological Products' Rift Valley fever Clone 13 vaccine in sheep and goats under field conditions in Senegal

# Introduction

Rift Valley fever (RVF) is an acute mosquito-borne disease affecting mainly ruminant animals and humans (Swanepoel & Coetzer 2004). It causes abortions and high mortality in young animals (Bird & Nichol 2012; Ikegami & Makino 2011). In humans it causes a severe influenza-like disease, with occasionally more serious central nervous system complications and death (Swanepoel & Coetzer 2004). RVF is caused by a single-stranded ribonucleic acid virus of the *Phlebovirus* genus in the family Bunyaviridae (Pepin *et al.* 2010). Major epidemics have occurred in many African countries, including Senegal, where the virus is believed to be endemic (Chevalier, Thiongane & Lancelot 2009).

The control of RVF has essentially relied on vaccination, using the live attenuated Smithburn vaccine or the inactivated vaccine (Smithburn 1949; Swanepoel & Coetzer 2004). Whilst these currently available RVF vaccines have been used extensively for the control of the disease, they have several shortcomings, such as poor immunogenicity, or toxicity when used in pregnant animals (Coetzer 1982; Dungu, Donadeu & Bouloy 2013). This has encouraged many research groups to develop new vaccine candidates that may address a large number of the current challenges and be suitable for use in currently RVF-free regions as well as being included in different contingency and emergency- preparedness strategies.

A number of research groups have been working on an alternative vaccine, the RVFV Clone 13 vaccine, which has been developed, evaluated (Dungu *et al.* 2010; Muller *et al.* 1995) and registered for use in cattle, sheep and goats in South Africa. The vaccine is based on a natural RVF virus mutant that carries a large in-frame deletion in the NSs gene. Evidence so far indicates that the vaccine is highly immunogenic and does not cause abortion or foetal teratogenicity in ewes and goats vaccinated during pregnancy. The risk of reversion to virulence is believed to be negligible given the substantial genetic deletion present in the vaccine virus (Muller *et al.* 1995).

Despite the occurrence of a number of RVF outbreaks in West Africa, more specifically Senegal and Mauritania (Caminade *et al.* 2014; Ndione *et al.* 2008), vaccination has never been used for the control of the disease in this region. This is mainly due to safety concerns associated with the RVF Smithburn vaccine and challenges with vaccine availability. The RVFV Clone 13 was considered to be a safer alternative for use in the region. In order to provide confidence to livestock owners and veterinary authorities, this study was designed and was aimed at evaluating the safety and efficacy (through the evaluation of immunogenicity) of Onderstepoort Biological Products' (OBP) Clone 13 RVFV vaccine, (Onderstepoort, Republic of South Africa) in Senegalese sheep and goat breeds under local conditions, and assessing the possible spread of virus from vaccinated to unvaccinated animals.

The OBP RVFV Clone 13 vaccine was tested in Senegal under field conditions, as it may potentially represent a safer and more effective option to protect livestock in West Africa than RVF vaccines currently being used elsewhere.

This trial was designed to demonstrate the safety and immunogenicity of OBP RVFV Clone 13 vaccine in sheep and goats when administered according to the manufacturer's recommendations, and to assess spread of virus from vaccinated to unvaccinated animals in local Senegalese goats and sheep under field conditions. The trial was conducted as a controlled, randomised and blinded study following the principles of good clinical practice as detailed by the International Cooperation on Harmonisation of Technical Requirements for Registration of Veterinary Medicinal Products guideline number nine.

# Research method and design

## Setting

The study was conducted from August 2011 to October 2012 on three commercial farms in the northern (Mpal, Thille Boubacar) and eastern (Diawara) regions of Senegal, using animals bred locally and kept under local farm conditions. The locations were favoured because of active surveillance implemented in these areas for more than 15 years that demonstrated endemicity of RVF (Thiongane *et al.* 1994).

## Animals

A total of 267 sheep (Touabir and Warale breeds) and goats (Sahelian breed) were included, as indicated in Table 1. They included 220 sheep (191 females and 29 males), and 47 goats (41 females and 6 males).

Experimental animals.

| Categories        | Diawara |         | Mpal  |         | Thille |         | Total   |
|-------------------|---------|---------|-------|---------|--------|---------|---------|
|                   | Males   | Females | Males | Females | Males  | Females |         |
| Goats, placebo    | 0       | 1       | 0     | 0       | 4      | 18      | 23      |
| Goats, vaccinated | 0       | 2       | 0     | 0       | 2      | 20      | 24      |
| Sheep, placebo    | 5       | 40      | 1     | 26      | 9      | 29      | 110     |
| Sheep, vaccinated | 4       | 40      | 0     | 28      | 10     | 28      | 110     |
| **Subtotal**      | **9**   | **83**  | **1** | **54**  | **25** | **95**  | **267** |

Age determination was based on incisors: the presence of lactating (milk) teeth (LT) only or the number of adult teeth (AT). LT: less than 14 months old – 68 females and 31 males; 2 AT: between 14 and approximately 20 months old (29 females and 3 males); 4 AT: between 20 and 28 months old (27 females, 0 males); 6 AT: between 28 and 36 months old (36 females, 0 males) and 8 AT: more than 36 months old (72 females, 1 male).

At the time of vaccination 123 animals were pregnant (99 sheep and 24 goats), and 53 animals (46 sheep and 7 goats) became pregnant during the study, as shown in Table 2.

Numbers of sheep and goats that gave birth in relation to time of vaccination using Onderstepoort Biological Products Rift Valley fever virus Clone 13 vaccine.

| Pregnancy status at vaccination | Goats |  | Sheep |  |
|----|----|----|----|----|
|  | Vaccinated | Placebo | Vaccinated | Placebo |
| Vaccinated during first trimester of gestation | 3 | 3 | 18 | 21 |
| Vaccinated during second trimester of gestation | 5 | 5 | 16 | 13 |
| Vaccinated during third trimester of gestation | 5 | 3 | 18 | 13 |
| Pregnant within 2 months of vaccination | 1 | 3 | 11 | 8 |
| Pregnant 2 months or more after vaccination | 1 | 2 | 15 | 12 |
| **Total** | **15** | **16** | **78** | **67** |

All animals were identified by ear tag. Appropriate treatments were given to animals in the 2 weeks preceding vaccination or placebo injection, including antiparasitic therapy with ivermectin and clorsulon.

All animals were seronegative to RVF at the start of the study, except for one goat and one sheep (both were positive at day 0 by enzyme-linked immunosorbent assay \[ELISA\] but not the viral neutralisation test \[VNT\]).

## Vaccination

The RVFV Clone 13 vaccine manufactured by OBP was used. The batch number was 13, with an expiry date of 01 January 2012. The placebo was the diluent of the same vaccine, batch number 8593, expiry date 01 April 2014. Storage conditions of the vaccine and diluent were appropriately monitored and complied with the manufacturer's instructions.

Each animal received either vaccine or placebo, at a dose of 1 mL subcutaneously. Vaccines were injected with sterile needles and syringes that were changed for each animal.

## Study personnel

All personnel were well experienced in conducting vaccine trials, and were formally trained to good clinical practice guidelines by an independent study monitor. The independent study monitor attended the crucial study time-points, including pre-study, vaccination, observation period and end of study, and maintained regular contact with Institut Sénégalais de Recherches Agricoles's (ISRA's) team throughout the observation period. Animals were examined at least weekly by the local veterinary representative and every 2 months by the investigator from ISRA Dakar.

## Serology and virology

All serology (VNT and ELISAs for IgG and IgM) and virology (virus isolation using Vero cells and conventional polymerase chain reaction) analyses were conducted at Laboratoire National de l'Elevage et de Recherches Vétérinaires (LNERV) at ISRA, Dakar.

Sera were taken from all animals before the study commenced and then from a selection of approximately 20 animals per group per site every 2 months. The samples were analysed by virus neutralisation (Davies, Jacobsen & Sylla 1988), RVF inhibition ELISA to detect IgM (National Institute for Communicable Diseases, Sandringham, South Africa; Paweska *et al.* 2005) for all samples up to day 135 and RVF recIgG ELISA to detect IgG (National Institute for Communicable Diseases, Sandringham, South Africa; Paweska, Jansen van Vuren & Swanepoel 2007) for all samples between days 195 and 365. The ELISAs used were commercial kits manufactured by Biological Diagnostic Supplies Limited, Ayrshire, Scotland.

## Study design

This study was a randomised, blinded and placebo-controlled trial conducted under field conditions. Vaccine safety was assessed by monitoring local (injection site) and general clinical reactions, including rectal temperature, once daily for the first 3 days then at least once weekly for 1 year following vaccination or placebo injection. All abnormal health observations, medical treatments and births or abortions were recorded for the year of observation.

Blood samples were taken on the day of vaccination and afterwards every 2 months (+/− 10 days) until day 365 to assess RVF antibody levels prior to vaccination and every 2 months afterwards.

Randomisation schedules were prepared for each species (sheep or goat) and each study site; half of the animals received vaccine and the other half received placebo based on order of catch.

To maintain blinding the person who administered treatments was never present at clinical examinations and the randomisation lists and treatment record forms were kept in a secure location which was not accessible to personnel conducting the health and safety observations.

## Statistical methods

Mean rectal temperatures were calculated daily for each treatment group and compared using two-factor analysis of variance (time and treatment). The frequencies of adverse events and abnormal births were compared between groups using the Chi-square test.

# Results

## Rectal temperatures

Mean and standard deviations for rectal temperatures from day 0 (prior to treatment) to day 3 (approximately 72 hours after treatment) are shown in Table 3 and Figure 1.

Mean and standard deviations for rectal temperature in °C from day 0 to day 3.

| Treatment            | Mean RT and SD | Study day |       |       |       |
|----------------------|----------------|-----------|-------|-------|-------|
|                      |                | Day 0     | Day 1 | Day 2 | Day 3 |
| Placebo              | Mean RT†       | 38.62     | 38.14 | 38.11 | 38.79 |
|                      | SD† RT         | 0.82      | 4.76  | 4.75  | 0.51  |
| RVF Clone 13 vaccine | Mean RT        | 38.62     | 37.52 | 38.03 | 38.41 |
|                      | SD RT          | 0.81      | 6.63  | 4.74  | 3.49  |

RVF, Rift Valley fever; RT, rectal temperature; SD, standard deviation.

†, rectal temperature in °C.

No statistically significant difference was noted between treatment groups for mean rectal temperature between days 0 and 3 (*p* = 0.27 or *p* \> 0.05) and also no significant time x treatment interaction (*p* = 0.79 or *p* \> 0.05) was recorded.

## Local tolerance: Injection site reactions

Of 267 animals that received either the placebo or the RVFV Clone 13 vaccine, only one sheep was observed to have slight swelling at the injection site on days 1 and 2. The signs disappeared after day 2 and no other abnormal signs were observed. None of the other animals presented any injection site reaction.

## General health observations

All animals were individually observed for 3 days after vaccination. During this period only one goat was recorded to have abnormal general health, diagnosed as foot rot, which was not considered to be related to the study.

In the year following vaccination or placebo injection all animals were observed weekly for abnormal health. Only those animals with health abnormalities were then individually examined. In total 45 health observations were recorded by the veterinarian (veterinary reports) in the placebo group and 50 in the vaccine group, as well as 12 potential adverse events (6 in each group). The frequency of veterinary reports was not significantly different between treatment groups (Chi-square test, *p* \> 0.05). No abnormal health observations were related to the trial.

## Births and abortions

A total of 176 births were observed during the 1-year study period, 31 in does and 145 in ewes. Ninety-three births occurred in the vaccinated group and 83 in the placebo group, and of these 12 were considered abnormal (3 in the vaccine group and 9 in the placebo group). Two abnormal births were noted less than a week after the start of the trial, one in each treatment group. All other abnormal births occurred at least one month (one animal) but generally more than 2 months after injection. The offspring from the ewe that aborted 5 days after vaccination was tested for RVF by polymerase chain reaction and isolation in Vero cells, and the results were negative. The abortion was considered not to be related to vaccination. The frequency of abnormal births was not significantly different between the vaccinated group and the placebo group (Chi-square test with Yate's correction, *p* \> 0.05).

## Seroconversion in the placebo group

No seroconversion to RVFV was observed in any animal from the placebo group, which supports the concept of no passage of the RVFV vaccine strain from the vaccinated animals to the non-vaccinated animals.

Natural infection did not occur, as demonstrated by lack of clinical signs and or seroconversion of the non-vaccinated animals, as well as lack of clinical signs in of neighbouring herds.

## Seroconversion after vaccination

On day 0 one goat and one sheep from the vaccinated group were positive on ELISA to IgM (RVF inhibition ELISA), and no animal was positive by VNT. In the placebo group the frequency of animals which were seropositive, both by ELISA (testing for IgM until day 135, and for IgG afterwards) and VNT stayed low throughout the observation period (as reflected in Tables 4 and 5). Amongst the animals that regularly had blood sampled, only one tested positive by VNT on consecutive samplings on days 195, 260 and 315, but remained negative on ELISA, except on day 195. This animal was then sold. Two other animals had antibody titres on VNT of 160 at one or more time-points, but became seronegative again at the following time-point. These animals were not positive on ELISA at these time-points. One animal was seropositive on day 365 only, on testing both by VNT and by ELISA.

Enzyme-linked immunosorbent assay and viral neutralisation test results in goats.

| Categories | Day 0 | Day 60 | Day 120 | Day 195 | Day 260 | Day 315 | Day 365 |
|----|----|----|----|----|----|----|----|
| Number of vaccinated goats sampled | 24 | 11 | 11 | 11 | 6 | 8 | 7 |
| Goats positive ELISA | 1 | 5 | 4 | 7 | 4 | 7 | 6 |
| % goats positive on ELISA | 4.17 | 45.45 | 36.36 | 63.64 | 66.67 | 87.50 | 85.71 |
| Goats positive on VNT 160 | 0 | 4 | 4 | 6 | 4 | 7 | 5 |
| Goats doubtful on VNT 80 | 0 | 2 | 4 | 2 | 1 | 1 | 2 |
| % Goats ≥ 80 VNT | 0.00 | 54.55 | 72.73 | 72.73 | 83.33 | 100.00 | 100.00 |
| **Number placebo goats sampled** | 22 | 6 | 4 | 5 | 5 | 6 | 7 |
| Goats positive ELISA | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
| % goats positive ELISA | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 14.29 |
| Goats positive VNT 160 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
| Goats doubtful VNT 80 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| % Goats ≥ 80 VNT | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 14.29 |

ELISA, enzyme-linked immunosorbent assay; VNT, viral neutralisation test.

Enzyme-linked immunosorbent assay and viral neutralisation test results in sheep.

| Categories | Day 0 | Day 60 | Day 120 | Day 195 | Day 260 | Day 315 | Day 365 |
|----|----|----|----|----|----|----|----|
| **Number vaccinated sheep sampled** | 110 | 64 | 61 | 58 | 58 | 58 | 55 |
| Sheep positive ELISA | 1 | 31 | 17 | 25 | 6 | 27 | 22 |
| % sheep positive ELISA | 0.91 | 48.44 | 27.87 | 43.10 | 10.34 | 46.55 | 40.00 |
| Sheep positive VNT 160 | 0 | 35 | 23 | 31 | 38 | 26 | 30 |
| Sheep doubtful VNT 80 | 0 | 12 | 10 | 10 | 3 | 6 | 9 |
| % sheep ≥ 80 VNT | 0.00 | 73.44 | 54.10 | 70.69 | 70.69 | 55.17 | 70.91 |
| **Number placebo sheep sampled** | 109 | 62 | 48 | 58 | 52 | 44 | 55 |
| Sheep positive ELISA | 0 | 1 | 1 | 5 | 1 | 0 | 0 |
| % sheep positive ELISA | 0.00 | 1.61 | 2.08 | 8.62 | 1.92 | 0.00 | 0.00 |
| Sheep positive VNT 160 | 0 | 1 | 1 | 1 | 0 | 0 | 0 |
| Sheep doubtful VNT 80 | 0 | 0 | 0 | 1 | 2 | 0 | 1 |
| % sheep ≥ 80 VNT | 0.00 | 1.61 | 2.08 | 3.45 | 3.85 | 0.00 | 1.82 |

ELISA, enzyme-linked immunosorbent assay; VNT, viral neutralisation test.

In the vaccine group seroconversion was observed by ELISA and VNT in more than 70% of the animals, starting on day 60. The seropositivity rates remained high throughout the observation period, including on the last study day, when more than 70% of the sheep and goats had neutralising antibody titres above the positivity threshold. Neutralising antibody titre results were generally more consistent over time for the same animal. Many animals in the vaccinated group regularly had an ELISA titre very close to but lower than the test threshold and were therefore not counted as positive. However, they definitely presented a significant rise in antibody titres over time.

The percentage of ELISA-positive animals at day 60 was as expected, based on the authors' previous experience.

# Ethical considerations

The study protocols were reviewed and approved by the scientific committees of the Global Alliance for Livestock Veterinary Medicines and the ISRA of Dakar. The study did not involve killing, administration of pain-causing substances or invasive procedures in animals. Only the registered vaccine was used and animals were routinely bled according to a standard operating procedure. Permission to conduct the study was granted by the Ministry of Livestock in Senegal.

# Discussion

Vaccination is the most effective tool to control RVF, accompanied by other measures such as effective surveillance, a good diagnostic strategy and reliable emergency-preparedness (Dungu *et al.* 2013). Preventing RVFV infection of livestock by vaccination is a key element in breaking the chain of human epidemics, and could lead to control of this significant public health threat (Bird & Nichol 2012). A human RVF vaccine is currently not commercially available.

The vaccine used in this trial, OBP RVFV Clone 13, has been registered and used in South Africa. It is based on an avirulent RVF virus isolated from a non-fatal case of RVF in the Central African Republic that had been passaged in mice and Vero cells and then plaque purified in order to study the homogeneity of virus subpopulations (Muller *et al.* 1995). A Clone designated 13 did not react with specific monoclonal antibodies against NSs, and when further investigated was found to be avirulent in mice, yet immunogenic (Muller *et al.* 1995; Vialat *et al.* 2000). This vaccine has been evaluated for safety and efficacy in sheep and cattle (Dungu *et al.* 2010; Von Teichman *et al.* 2011).

Unlike the Smithburn and MP-12 strains of RVFV vaccines, RVFV Clone 13 vaccine does not appear to induce abortions in pregnant ewes, making it one of the most promising live attenuated vaccine candidates thus far (Boshra *et al.* 2011). Results of experiments in sheep demonstrated that RVFV Clone 13 vaccine also has a better safety profile than other RVFV mutants, such as MP-12 and Smithburn (Dungu *et al.* 2010). The safety and efficacy of RVFV Clone 13 vaccine in cattle and sheep under the recommended conditions of use are also supported by the administration of more than 10 million doses during the 2009–2010 RVF outbreak in South Africa, where the vaccine played a key role in the control of the disease (Dungu *et al.* 2013). This vaccine has the advantage over inactivated vaccines as it is more cost-effective, requiring only one vaccination instead of two administrations, and can be produced in larger batches than the inactivated vaccine (Dungu *et al.* 2010).

The RVFV Clone 13 vaccine has not been used in West Africa, despite the presence of RVF in the region (Chevalier et  al. 2009; Clements *et al.* 2007). Prior to the field evaluation described here, this vaccine was tested in West African goats and sheep under laboratory conditions at ISRA Dakar (data not shown) as well as in field conditions in Kenya. In both studies the vaccine proved to be safe and induced a high level of seroconversion (B. Dungu, pers. comm., January 2012).

This is the first report on safety and immunogenicity data for the OBP RVFV Clone 13 vaccine in goats and sheep under field conditions in Senegal. The results are very encouraging, as the vaccine was confirmed to be safe in local breeds of both species in West Africa. Efficacy, evaluated as seroconversion after vaccination, was observed in more than 70% of the animals, which should ensure protective immunity at herd level. In experimental studies very good neutralising antibody responses measured by the World Organisation for Animal Health-recommended VNT were recorded in all RVFV OBP Clone 13 vaccinated sheep from day 7, and in some sheep up to day 60 post-vaccination (Dungu *et al.* 2010). In addition, in this field trial no signs of toxicity resulting from vaccination were detected when it was used in pregnant animals.

In conclusion, a total of 267 sheep and goats of local breeds were included in this field study to evaluate the safety and immunogenicity of the administration of the registered dose of RVFV Clone 13 vaccine. The animals were kept under local conditions and observed for 1 year after vaccination or placebo injection. More than 70% of the sheep and goats vaccinated with the OBP RVFV Clone 13 vaccine showed long-term seroconversion that persisted for the 1 year during which the trial was conducted. No signs of local or general intolerance and no signs of toxicity to vaccination of pregnant ewes and goats were noted. Vaccination with RVFV Clone 13 vaccine was safe and well tolerated in sheep and goats of Senegalese breeds, including pregnant females, under field conditions. These results suggest that vaccination of small ruminants with this vaccine could be effective in controlling RVF.

## Acknowledgements

The authors are grateful to all the technicians at LNERV, ISRA, Dakar as well as Ismaila Seck, field veterinary at Mpal, Maniang Diokh field veterinary at Thille Boubacar and Serigne Mbaye field veterinary at Diawara for their valuable contribution to the study. This work was funded by the Bill & Melinda Gates Foundation & the UK Government's Department for International Development (DFID).

### Competing interests

The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.

### Authors' contributions

P.S. (independent veterinary consultant) worked on the experimental design, data analysis and statistics, M.M.L. (ISRA) worked on the experimental design and reviewed the manuscript, Y.T. (ISRA) carried out data analysis, M.D. (ISRA) performed the ELISA for IgM and IgG, V.M. (Global Alliance for Livestock Veterinary Medicines) and M.D. (Global Alliance for Livestock Veterinary Medicines) conceived the study and edited the manuscript and B.D. (Global Alliance for Livestock Veterinary Medicines) was involved with conception of the study and manuscript preparation.

# References

[^1]: **How to cite this article:** Lo, M.M., Mbao, V., Sierra, P., Thiongane, Y., Diop, M., Donadeu, M. *et al*., 2015, 'Safety and immunogenicity of Onderstepoort Biological Products' Rift Valley fever Clone 13 vaccine in sheep and goats under field conditions in Senegal', *Onderstepoort Journal of Veterinary Research* 82(1), Art. \#857, 6 pages. <http://dx.doi.org/10.4102/ojvr.v82i1.857>


abstract: The number of published systematic reviews of studies of healthcare interventions has increased rapidly and these are used extensively for clinical and policy decisions. Systematic reviews are subject to a range of biases and increasingly include non-randomised studies of interventions. It is important that users can distinguish high quality reviews. Many instruments have been designed to evaluate different aspects of reviews, but there are few comprehensive critical appraisal instruments. AMSTAR was developed to evaluate systematic reviews of randomised trials. In this paper, we report on the updating of AMSTAR and its adaptation to enable more detailed assessment of systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. With moves to base more decisions on real world observational evidence we believe that AMSTAR 2 will assist decision makers in the identification of high quality systematic reviews, including those based on non-randomised studies of healthcare interventions*.*
author: Beverley J Shea; Barnaby C Reeves; George Wells; Micere Thuku; Candyce Hamel; Julian Moran; David Moher; Peter Tugwell; Vivian Welch; Elizabeth Kristjansson; David A HenryCorrespondence to: B J Shea <firstname.lastname@example.com>
date: 2017
references:
title: AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

# Summary points

1.  Systematic reviews of studies of healthcare interventions effects often include non-randomised studies

2.  AMSTAR is a popular instrument for critically appraising systematic reviews of randomised controlled clinical trials

3.  AMSTAR underwent further development to enable appraisal of systematic reviews of randomised and non-randomised studies of healthcare interventions

4.  The revised instrument (AMSTAR 2) retains 10 of the original domains, has 16 items in total (compared with 11 in the original), has simpler response categories than the original AMSTAR, includes a more comprehensive user guide, and has an overall rating based on weaknesses in critical domains

5.  AMSTAR 2 is not intended to generate an overall score

6.  With moves to base more decisions on real world observational evidence, AMSTAR 2 should assist in the identification of high quality systematic reviews

With the rapid increase in biomedical publishing, keeping up with primary research has become almost impossible for healthcare practitioners and policy makers.1 Consequently, healthcare decision makers rely on systematic reviews as one of the key tools for achieving evidence based healthcare.2 Systematic reviews provide an opportunity to base decisions on accurate, succinct, credible, and comprehensive summaries of the best available evidence on a topic.2

Uncritically accepting the results of a single systematic review has risks. One of us (DM) led efforts to improve standards for reporting of systematic reviews, which led to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement.3 The reporting guide for systematic reviews of observational (non-randomised) studies is MOOSE (Meta-analysis of Observational Studies in Epidemiology).4 The quality of reporting of a systematic review may, however, more accurately reflect authors' ability to write in a comprehensible manner rather than the way they conducted their review. This underscores the need for guidelines that evaluate the way in which reviews are planned and conducted.5 6

The Cochrane Collaboration Handbook provides a comprehensive guide for review authors, but it does not provide a concise critical appraisal instrument for completed reviews.5 Several instruments have been designed to evaluate individual studies that are being included in systematic reviews or how certain steps (eg, meta-analysis, testing for publication bias) should be conducted.7 8 9 10 11 12 13 14 15 But relatively few instruments assess all important steps in the conduct of a review.16 17 18 19 20 21

AMSTAR (A MeaSurement Tool to Assess systematic Reviews), published in 2007, is one of the most widely used instruments.22 23 24 AMSTAR was designed by us and our colleagues as a practical critical appraisal tool for use by health professionals and policy makers who do not necessarily have advanced training in epidemiology, to enable them to carry out rapid and reproducible assessments of the quality of conduct of systematic reviews of randomised controlled trials of interventions. Since publication, several critiques of the instrument have been published.25 26 27 28 29 30 31 These critiques plus feedback received at workshops and developments in the science of systematic reviews pointed to a need to revise and update the original AMSTAR instrument.

# Inclusion of non-randomised studies in systematic reviews

Almost half of published systematic reviews now include non-randomised studies of intervention effects.4 32 33 34 There are many concerns about the conduct and reporting of systematic reviews of non-randomised studies.32 35 36 To summarise, non-randomised studies of healthcare interventions (an important focus of this revision of AMSTAR) are subject to a range of biases that are either not present or are less noticeable in randomised controlled trials, thus requiring different risk of bias assessments. Observational studies are increasingly conducted within large population databases, sometimes with hundreds of thousands or even millions of recipients of healthcare interventions. These generate precise estimates of intervention effects, which may be inaccurate because of residual biases. If these estimates are combined with those from the (generally smaller) randomised controlled trials, the meta-estimates will be weighted towards the observational study estimates. The original AMSTAR instrument did not include an assessment of the risk of bias in non-randomised studies included in a review, which is a key issue given the diversity of designs that such studies may use and the biases that may affect them.

# Development of AMSTAR 2

The development and validation of the original AMSTAR instrument (published in 2007) has been described in detail elsewhere.22 23 24 Briefly, the original list of items was created from the results of a scoping review of the then available rating instruments. This review identified many over-lapping appraisal items, mainly from two extensively cited reports.16 17 The lists of items from these reports were combined and reduced by factor analysis. After pilot testing, items were reworded as needed and the reliability and usability of the tool was assessed. A modified version was validated externally and performed well against the global judgments of a panel of content experts.23 The publications describing the original AMSTAR instrument were widely cited and the instrument has been used and critiqued extensively.22 23 24 25 26 27 28 29 30 31

We convened an expert group, comprising authors of the original instrument, members with expertise in the conduct of non-randomised studies, development of appraisal instruments, biostatistics, and study designs. The expert group met for a day in Ottawa, Canada and members were presented with the results of updated literature reviews on relevant critical appraisal instruments, the results of surveys of AMSTAR users, recorded experience of participants in AMSTAR workshops at Cochrane Colloquiums in 2015 and 2016, feedback from the AMSTAR website ([www.amstar.ca](http://www.amstar.ca)), and published critiques of the original instrument.16 17 18 19 20 21 22 23 24 25 26 The perspective adopted by the expert group was to increase the value of AMSTAR as a broad critical appraisal instrument designed primarily for systematic reviews of studies of healthcare interventions. The expert group considered that revisions should address all aspects of the conduct of a systematic review, and the challenges of including non-randomised studies. They also thought the revised instrument should function as a teaching aid and as a concise checklist for those conducting reviews. The revisions were not intended to deal with the special requirements of diagnostic test reviews, individual patient data meta-analyses or network meta-analyses, scoping reviews, or realist reviews.37 38 39 40 41

We used a nominal group technique to propose and then prioritise specific changes to the instrument and to agree on the draft wording of items. Based on their experience of the instrument and the presentations made at the meeting, participants were asked to record their ideas independently and privately. The ideas were then enunciated in a round-robin format. One idea was collected from everyone, in turn, and presented to the group by the facilitator. This process was continued until all ideas had been listed. Individuals then privately recorded their judgments and rankings. These were aggregated statistically to derive the group judgments. The following changes were agreed on (these are not listed in order of priority as all were considered important enough to mandate modifications to the instrument):

-   Simplify the response categories

-   Align the definition of research questions with the PICO (population, intervention, control group, outcome) framework

-   Seek justification for the review authors' selection of different study designs (randomised and non-randomised) for inclusion in systematic reviews

-   Seek more details on reasons for exclusion of studies from the review

-   Determine whether the review authors had made a sufficiently detailed assessment of risk of bias for the included studies (whether randomised or non-randomised)

-   Determine whether risk of bias with included studies was considered adequately during statistical pooling of results (if this was performed)

-   Determine whether risk of bias with included studies was considered adequately when interpreting and discussing the review findings.

A description was formulated for each of the draft items. A small subgroup refined the wording of the items and assembled the draft instrument for testing. Initial pilot testing was performed by group members. Draft versions were presented at workshops held at the Cochrane Colloquiums in 2015 and 2016, where feedback directed further modifications and redrafting of the instrument. The version of the instrument presented here was subject to inter-rater reliability and usability testing.

# Comparison with the original instrument

The supplementary figure provides details of the new instrument (AMSTAR 2). Ten domains were retained from the original tool, with changes to the wording of items based on feedback about the original instrument and experience of testing drafts of the new instrument. Two domains were given more detailed coverage in AMSTAR 2 than in the original instrument: duplicate study selection and data extraction now have their own items (they were combined in the original tool). The possible influence of funding sources is now considered separately for individual studies included in the review and for the review itself. Previously they were combined in one item. We added more detailed and separate considerations of risk of bias for randomised and non-randomised studies. Both sub-items are based on content from the Cochrane risk of bias instruments for randomised and non-randomised (ROBINS-I) studies.42 43 One domain was removed—consideration of grey literature, previously a separate item, is now handled in the item on literature searching.

In total, four domains were added. Two of these came directly from the ROBINS-I tool—namely, elaboration of the PICO and the way in which risk of bias was handled during evidence synthesis.43 One of the other new domains—discussion of possible causes and significance of heterogeneity—is an elaboration of content in the original AMSTAR tool. Another new domain—justification of selection of study designs—was part of the adaptation of AMSTAR to deal with non-randomised designs.

The domain specific questions in AMSTAR 2 are framed so that a "Yes" answer denotes a positive result. We removed the "not applicable" and "cannot answer" options in the original AMSTAR instrument because we believe that all domains are relevant to contemporary systematic reviews of healthcare interventions. If no information is provided to rate an item, the review authors should not be given the benefit of doubt and the item should be rated as a "No." We have provided a "partial Yes" response in some instances where we considered it worthwhile to identify partial adherence to the standard.

# Rationale for selection of items

Here we summarise our thinking behind the items in AMSTAR 2, which are numbered as in the instrument (see supplementary figure). Supplementary appendix 1 provides a more complete user's guide.

*1. Did the research questions and inclusion criteria for the review include the components of PICO?*

It is common practice to use the PICO description (population, intervention, control group, and outcome) as a convenient and easily memorised framework for a study question. Sometimes a timeframe should be added if this is critical in determining the likelihood of a study capturing relevant clinical outcomes (eg, an effect of the intervention is only expected after several years).

2\. *Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant* *deviations from the protocol?*

Systematic reviews are a form of observational research, and the methods for the review should be agreed on before the review commences. Adherence to a well developed protocol reduces the risk of bias in the review. Authors should show that they worked with a written protocol with independent verification.

3\. *Did the review authors explain their selection of the study designs for inclusion in the review?*

For some questions, for instance the effects of policy changes, or for ethical reasons, non-randomised studies may be the only studies addressing the review question. With an expansion of AMSTAR 2 to appraise reviews that include randomised controlled trials or non-randomised studies, or both, it is important that authors justify the inclusion of different study designs in systematic reviews. The authors should indicate that they followed a strategy. When both randomised and non-randomised studies address the same question about the effects of an intervention, we believe that authors should consider whether a review that is restricted to randomised controlled trials will give an incomplete summary of the important effects of a treatment.

*4. Did the review authors use a comprehensive literature search strategy?*

The importance of adequate literature searching in systematic reviews is well established.5 This item was carried over with minimal changes to the wording from the original instrument. We have made the response options clearer in AMSTAR 2 and provide more detailed guidance on completion of the item, particularly in relation to the identification of non-randomised studies (see supplementary appendix 1).

*5. Did the review authors perform study selection in duplicate?*

Best practice requires two review authors to determine eligibility of studies for inclusion in systematic reviews.5 This involves checking the characteristics of a study against the elements of the research question. In the original AMSTAR, this item covered determining both study eligibility and data extraction. The expert group believed that they were sufficiently distinct processes to merit separate items in AMSTAR 2.

*6. Did the review authors perform data extraction in duplicate?*

The expert group recognised that data extraction might be more complex for non-randomised studies of healthcare interventions as it usually involves extraction of measures of treatment effects and other associations that have been adjusted for potential confounding, rather than raw outcome data from treated and control groups. A study report may present multiple treatment effects; judgment is therefore needed to select the one that conforms best to the PICO question and is at lowest risk from confounding.

*7. Did the review authors provide a list of excluded studies and justify the exclusions?*

In the revised instrument we consider excluded and included studies separately. Excluded studies should be accounted for fully by review authors, otherwise there is a risk that they remain invisible and the impact of their exclusion from the review is unknown.

*8. Did the review authors describe the included studies in adequate detail?*

The revised instrument requires review authors to provide detail about research designs, study populations, interventions, comparators, and outcomes. The detail should be sufficient for appraisers to make a judgment about the extent to which the studies were appropriately chosen (in relation to the PICO) and whether the study populations and interventions were relevant to their questions. This information is needed to determine the extent to which the results of different studies should be combined, help explain heterogeneity, and assist those applying the results.

*9. Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?*

Biases can be introduced at several stages in the design, planning, conduct, and analysis of a study. This item replaces a less detailed item on "scientific quality." The item specifies domains of bias for randomised and non-randomised studies that should have been considered by reviewers, based on the relevant Cochrane instruments.42 43 In AMSTAR 2 we ask whether the review authors made an adequate assessment of study level efforts to avoid, control, or adjust for baseline confounding, selection biases, bias in measurement of exposures and outcomes, and selective reporting of analyses or outcomes, or both. The guidance document (see supplementary appendix 1) and the ROBINS-I report provide more detail.43 We decided not to include assessment of time varying confounding, performance biases, and biases due to missing data, although they are currently included in ROBINS-I.43 This was because of the complex nature of techniques used to adjust for these potential sources of bias and the frequent lack of data (in contemporary primary studies) to enable assessment of these items. Version 2.0 of the Cochrane risk of bias instrument for randomised controlled trials is now available in draft form, and AMSTAR 2 will be aligned with this in the future.44

*10. Did the review authors report on the sources of funding for the studies included in the review?*

We added a consideration of funding sources in the light of evidence from several sources that the results of industry funded studies sometimes favoured sponsored products, and that industry funded studies were less likely to be published than those that were independently funded.45 46 47 Such influences may not be detected as flaws in design or methods (item 9).

*11. If meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results?*

This is a modified version of an item in the original instrument and is judged separately for randomised and non-randomised studies. Review authors should have stated explicitly in the review protocol the principles on which they based their decision to perform meta-analysis of data from the included studies. This includes the extent to which the studies are compatible (in terms of patients, controls, and interventions) and the value of a single pooled effect (for instance from several compatible but underpowered studies). Where reviewers consider it appropriate to conduct a meta-analysis, the inclusion of non-randomised studies increases the complexity of the analyses and may increase heterogeneity (see supplementary appendix 1).

*12. If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?*

This is a new item that requires reviewers to examine how results vary with inclusion or exclusion of primary studies judged to be at high risk of bias. In cases where review authors have chosen to include only high quality randomised controlled trials there may be little discussion of the potential impact of bias on the results. But where they have included randomised controlled trials of variable quality or non-randomised studies they should assess the impact of study level risk of bias on the results of the review.48

*13. Did the review authors account for RoB in primary studies when interpreting/discussing the results of the review?*

This is a modification of an item from the original instrument. With a greater emphasis on assessing risk of bias, the expectation is that reviewers will make explicit reference to the potential impacts of risk of bias when interpreting and discussing the results of their review and in drawing conclusions or making recommendations.

*14. Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?*

This item is carried over with modified wording from the original instrument. It is important that reviewers investigate possible causes of heterogeneity, including variation in those elements included in the PICO framework (see item 1) and those arising from design and methodological considerations (see item 9). With the inclusion of non-randomised studies, variations in design and analysis may contribute to heterogeneity.

*15. If they performed quantitative synthesis did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?*

This item is carried over from the original instrument but with modified wording. Publication bias is an important problem but it can be difficult for authors to resolve completely. Typically, statistical tests (several are available) or graphical displays are used and if the results are positive they indicate the presence of publication bias. Negative test results are not a guarantee of the absence of publication bias as they are insensitive. A minimum of 10 studies are required to show funnel plot asymmetry.5 The underlying tendency to selectively publish small positive studies may be compounded by the effects of lower methodological quality of small studies, a greater tendency to selectively report results, and increased clinical heterogeneity when conducted in patient subgroups.49

*16. Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?*

This item is carried over with modified wording from the original instrument and is now separate from consideration of funding of the primary studies included in the review (item 10). As with primary studies, review authors should report their funding sources.50 51

## Identification of critical domains

All steps in the conduct of a systematic review and meta-analysis are important, but we believe that seven domains can critically affect the validity of a review and its conclusions (box 1). Two of these concern risk of bias, whether it has been assessed adequately and how it can influence the results of a review. The prominence we give to risk of bias is because AMSTAR 2 is going to be used to appraise many systematic reviews that include non-randomised studies.

### Box 1 AMSTAR 2 critical domains

-   Protocol registered before commencement of the review (item 2)

-   Adequacy of the literature search (item 4)

-   Justification for excluding individual studies (item 7)

-   Risk of bias from individual studies being included in the review (item 9)

-   Appropriateness of meta-analytical methods (item 11)

-   Consideration of risk of bias when interpreting the results of the review (item 13)

-   Assessment of presence and likely impact of publication bias (item 15)

We recognise that the items listed in box 1 will not always be regarded as critical; for example, risk of bias related items may be considered less important when a review is confined to high quality randomised controlled trials. Other circumstances where the critical nature of items may be questioned are when a review team are using meta-analysis to summarise a known literature base (eg, the output from one or more established clinical trial collaborative groups). In this circumstance the adequacy of the literature search (item 4), listing of excluded studies (item 7), and possibility of publication bias (item 15) may not be considered critical. If a meta-analysis was not performed, the item covering the appropriateness of the meta-analytical methods (item 11) will not apply. However, it is important in this circumstance that appraisers are alert to the possible impact of risk of bias when review authors select individual studies to highlight in a narrative summary.

Flaws in the items that we have identified as critical may not be fatal if further information (eg, directly from the review authors) indicates that the original response option was wrong. This may provide reassurance about the review findings or enable an amendment of the review through additional analyses. We emphasise that our listing is a suggestion and appraisers may add or substitute other critical domains. For example, the failure to include non-randomised studies (item 3) in a review of adverse outcomes of treatment may be a critical flaw, as would the inability to explain large variations in treatment effects across a body of studies (item 14).

## Applying AMSTAR 2 to systematic reviews

If one or more systematic reviews will be the basis of important practice and policy decisions we recommend that the appraisal team agree on how the AMSTAR 2 items should be applied. This includes the practice or policy context and the questions that should be addressed, based on the relevant PICO components. For example, available systematic reviews may have included studies with different comparators or different follow-up times, and their relevance to the policy relevant questions needs to be established. The likely sources of bias should also be agreed on. For instance, in observational studies of intervention effects, confounding by indication (or disease severity) may be problematic when interventions are reserved for certain subgroups of patients.52 It is good practice to recruit new users of a technology or drug into studies to avoid prevalence bias.53 If the start of one intervention tends to be delayed the choice of comparator may introduce immortal time bias.54 Measurement errors can misclassify exposure and outcomes and may be unbalanced across comparison groups. Selective reporting among multiple analyses and outcomes may give an inaccurate measure of intervention effects.

Supplementary appendix 1 provides guidance on sections of AMSTAR 2. Some of the judgments (particularly whether review authors have adequately assessed risk of bias with individual non-randomised studies) are complex, and advice on both methodology and content may be needed. Content knowledge is sometimes necessary to determine if the review authors have made an adequate assessment of the relevant PICO elements (item 1), and to identify potential confounders.

We strongly recommend that individual item ratings are not combined to create an overall score.55 56 Rather, users should consider the potential impact of an inadequate rating for each item.

In box 2 we propose a scheme for interpreting weaknesses detected in critical and non-critical items. This is advisory and appraisers should decide which items are most important for the reviews under consideration.

### Box 2 Rating overall confidence in the results of the review

1.  **High**

2.  *No or one non-critical weakness*: the systematic review provides an accurate and comprehensive summary of the results of the available studies that address the question of interest

3.  **Moderate**

4.  *More than one non-critical weakness*\*: the systematic review has more than one weakness but no critical flaws. It may provide an accurate summary of the results of the available studies that were included in the review

5.  **Low**

6.  *One critical flaw with or without non-critical weaknesses*: the review has a critical flaw and may not provide an accurate and comprehensive summary of the available studies that address the question of interest

7.  **Critically low**

8.  *More than one critical flaw with or without non-critical weaknesses*: the review has more than one critical flaw and should not be relied on to provide an accurate and comprehensive summary of the available studies

## Inter-rater reliability of AMSTAR 2

We measured inter-rater agreement with three pairs of raters and three sets of systematic reviews (see supplementary appendix 2). The first pair of raters was involved in the development of AMSTAR 2 (coauthors MT and CH). They individually appraised 20 systematic reviews derived from a rapid search (conducted in 2015 on the terms "systematic review" and "meta-analysis" in the title) using Google Scholar. From the first 200 we selected 20 systematic reviews of any healthcare intervention. The other two pairs of raters were experienced in the appraisal of systematic reviews and were not involved in the development of AMSTAR or AMSTAR 2. They applied AMSTAR 2 during their routine work, performing appraisals of systematic reviews of two topics: interventions to reduce medication errors (14 reviews) and non-pharmacological therapies for Parkinson's disease (20 reviews) (see references in supplementary appendix 2). In both cases systematic reviews had been identified through comprehensive literature searches (details available on request). All raters had access to the user guide (see supplementary appendix 1), applied the instrument individually, and did not try to achieve consensus. In total, six raters applied the instrument to 54 systematic reviews, of which 20 included only randomised controlled trials, 18 included only non-randomised studies of interventions, and 16 included a mixture of both designs.

Supplementary appendix 2 provides summaries of the κ scores for agreement between the three pairs of raters across the three sets of reviews. The values varied substantially across items and between pairs of raters. Most values were in an acceptable range, with 46 of the 50 κ scores falling in the range of moderate or better agreement and 39 displaying good or better agreement. There were no large differences between raters, and those who had been involved in the development of AMSTAR 2 did not have higher levels of agreement than the rater who was not involved. Items 9, 12, and 13 are concerned with measurement of risk of bias and how this is handled during discussion of the meta-analysis and interpretation of the results. The ranges of κ scores for these items were similar to those seen with other items in the instrument (see supplementary appendix 2). For items 9 and 11 the κ values for risk of bias judgments for randomised controlled trials were similar to those for non-randomised studies.

## Usability of AMSTAR 2

The completion times for the 20 reviews used by reviewers 1 and 2 ranged from 15-32 minutes. These estimates do not include the time taken to read the reviews. This is almost twice the time taken to complete the original AMSTAR instrument (range 10-15 minutes), when it was applied to systematic reviews that were limited to randomised controlled trials.57 The comments from the reviewers included: that the removal of the "can't answer" and "not applicable" response options in the original instrument forced them to make judgments; that it takes longer to evaluate the non-randomised and mixed study reviews, but this requires the reviewer to confront important methodological issues; that it was common for review authors to mention the presence or absence of publication bias, but not provide any evidence; and that review authors would disclose their potential competing interests but not how they managed them.

# Discussion

AMSTAR 2 is a major revision of the original AMSTAR instrument, which was designed to appraise systematic reviews that included randomised controlled trials.22 23 24 The main modifications include simplified response categories; a more detailed consideration of risk of bias with included studies, and how this was handled by review authors in summarising and interpreting the results of their reviews; better alignment with the PICO framework for research questions; a more detailed justification of selection of study designs for inclusion in a review; and more information on studies that were excluded from reviews. In addition, we recommend defining critical domains before starting an appraisal of a systematic review. Identification of weaknesses in these domains should undermine confidence in the results of a systematic review.

We stress that responses to AMSTAR 2 items should not be used to derive an overall score.55 56 The original AMSTAR instrument was often used for this purpose and this was facilitated by the website ([www.amstar.ca](http://www.amstar.ca)). We accept that an overall score may disguise critical weaknesses that should diminish confidence in the results of a systematic review and we recommend that users adopt the rating process based on identification of critical domains (see box 2), or some variation based on these principles.56

We envisage that AMTAR 2, like its predecessor, may have a role as a convenient teaching aid and as a brief checklist for those conducting systematic reviews. However, we stress that the instrument does not explain in detail the logic and methods of conducting systematic reviews, and those looking for comprehensive advice should consult the Cochrane Handbook.5

The consideration of risk of bias in individual studies is equally important for randomised and non-randomised studies of healthcare interventions but is generally better understood with the former. Large non-randomised studies, often conducted in large administrative databases, are increasingly being used to assess the real world impact of a wide range of healthcare technologies and practices. Although such studies often use sophisticated methods, residual confounding or failure to deal with other sources of bias may lead to inaccurate estimates of effect. Inclusion of large observational studies in meta-analyses may generate precise but biased estimates of intervention effects.32

The items in AMSTAR 2 that deal with risk of bias identify domains specified in the Cochrane risk of bias instruments for randomised and non-randomised studies.42 43 These represent a consensus, in each case developed with input from more than 30 experts in methodology. However, AMSTAR 2 does not currently specify which risk of bias instruments review authors should have used to assess non-randomised studies included in a systematic review. The ROBINS-I instrument, which is the most comprehensive tool for non-randomised studies evaluating the effects of healthcare interventions, was released in 2016 and it is unrealistic to expect authors of reviews started before its release to have used it.43 Presently, AMSTAR 2 leaves it to the review authors and those appraising the review to satisfy themselves that the risk of bias instrument used by review authors has sufficient discriminatory ability for the specified risk of bias domains. A review by Sanderson and colleagues identified 86 tools for assessing quality of observational studies, without a clear preference among them.58 The authors pointed to the need to agree on critical elements for assessing susceptibility to bias in observational epidemiology. In part this review led to the development of ROBINS-I.43 Popular appraisal instruments for individual studies, such as the Newcastle Ottawa Scale and the Scottish Intercollegiate Guidelines Network (SIGN) checklist may not focus on validity alone.59 60 The Newcastle Ottawa Scale appears to lack sensitivity and is sometimes used to generate an overall score, something that is not recommended because it may disguise critical weaknesses in a review.56 61

AMSTAR 2, as a critical appraisal instrument for systematic reviews, joins several published instruments designed for this purpose.3 4 16 17 19 20 25 62 Two prominent examples are concerned with guidelines for reporting systematic reviews, rather than their conduct.3 4 Two highly cited instruments were the basis for the development of the original AMSTAR tool.16 17 22 Two published instruments are direct derivatives of the original AMSTAR.19 25 Another publication includes a checklist used to appraise systematic reviews that are being included in an umbrella review.20 Overlap between the content of this checklist and the original AMSTAR is considerable.22

AMSTAR 2 provides a broad assessment of quality, including flaws that may have arisen through poor conduct of the review (with uncertain impact on findings). In this respect it differs from another instrument, the Risk Of Bias In Systematic reviews (ROBIS).62 ROBIS is a sophisticated three phase instrument that focuses specifically on the risk of bias introduced by the conduct of the review. It covers most types of research question, including diagnosis, prognosis, and aetiology. In contrast, AMSTAR 2 is intended to be used for reviews of healthcare interventions. Inevitably there is overlap in the items considered by ROBIS and AMSTAR 2; indeed, two investigators (BCR, BJS) were involved in the development of both.

In developing AMSTAR 2 we sought to maintain its familiar and popular stepwise checklist approach and augmented this by the addition and modification of items. AMSTAR 2 will be familiar to users of the original instrument, although more demanding to use for reasons discussed previously. Because AMSTAR 2 is structured around the key sequential steps in the conduct of a systematic review, it may be used as a brief teaching aid or as a checklist by those conducting systematic reviews.

Unlike the original instrument, AMSTAR 2 identifies critical weaknesses (see box 1) that should reduce confidence in the findings of a review, and it asks users to prespecify how this list will vary for the review topic. We understand that there will be debate about membership of this list and propose that users may wish to prespecify a different set of critical items for a specific PICO research question or setting.

We did not perform an extensive validation of the revised AMSTAR 2 tool. In its development, 10 domains were retained from the original validated tool, albeit with some wording changes based on feedback and extensive experience of using it. Two domains were given more detailed coverage: duplicate study selection and data extraction now have their own items (they were combined in the original tool); we have added more detailed, and separate, considerations of risk of bias for randomised and non-randomised studies. The sub-items were derived from widely used Cochrane instruments. One domain was removed; consideration of grey literature, previously a separate item, is now handled in the item on literature searching. In total, four domains were added. Two of these come directly from the ROBINS-I tool—namely, elaboration of PICO in the review and the way in which risk of bias was handled during evidence synthesis.43 One of the other new domains, discussion of possible causes and importance of heterogeneity, is elaboration of content in the original AMSTAR tool.22 The final domain, justification of selection of study designs, is justified by adapting AMSTAR to deal with non-randomised designs. We do not think this needs validation because we believe it is obvious that authors of systematic reviews should justify why they have included study designs that are more susceptible to bias.

The levels of agreement achieved by the three pairs of raters varied across items, but they were moderate to substantial for most items. Notably, the agreement between two raters involved in the development of AMSTAR 2 was no higher than that achieved by experienced raters who had not been involved its development. We did not expect perfect agreement, and differences between raters reflect the demanding nature of some item level judgments and should prompt group discussion of their causes and importance, and, if needed, consultation with experts in subject matter and methods.

In developing AMSTAR 2 we relied heavily on the consensus of the expert panel, but we also received extensive feedback from users of the original instrument in the form of direct communications, website comments, and evaluations made at teaching workshops at Cochrane Colloquiums. In the later phases of development of AMSTAR 2 we had access to, and discussed, recently published critiques of AMSTAR.25 26 27 28 29 30 31

Our experience of releasing and using the original AMSTAR instrument is that judgments need to be made and users may sometimes decide to make modifications to the instrument.25 26 30 We encourage investigators to provide feedback, and, if they adapt the instrument for particular settings, to report their experience at [www.amstar.ca](http://www.amstar.ca).

We thank for their assistance in the development of the AMSTAR critical appraisal instruments: Lex Bouter, Maarten Boers, Alonso Carrasco-Labra, Jeremy Grimshaw, Ranjeeta Mallick, Jordi Pardo-Pardo, and Larissa Shamseer, and for conducting the reliability studies reported in this manuscript: Brian Hutton, Pauline Barbeau, Fatemeh Yazdi, Vesa Basha, and Roxanne Ward.

Extra material supplied by the authors

**Supplementary figure:** the AMSTAR 2 instrument

**Supplementary appendix 1:** AMSTAR 2 guidance document

**Supplementary appendix 2:** Inter-rater reliability of AMSTAR 2 items and references for studies included in the analyses


abstract: Interest in how the gut microbiome can influence the metabolic state of the host has recently heightened. One postulated link is bacterial fermentation of "indigestible" prebiotics to short-chain fatty acids (SCFAs), which in turn modulate the release of gut hormones controlling insulin release and appetite. We show here that SCFAs trigger secretion of the incretin hormone glucagon-like peptide (GLP)-1 from mixed colonic cultures in vitro. Quantitative PCR revealed enriched expression of the SCFA receptors *ffar2* (*grp43)* and *ffar3 (gpr41)* in GLP-1–secreting L cells, and consistent with the reported coupling of GPR43 to Gq signaling pathways, SCFAs raised cytosolic Ca^2+^ in L cells in primary culture. Mice lacking *ffar2* or *ffar3* exhibited reduced SCFA-triggered GLP-1 secretion in vitro and in vivo and a parallel impairment of glucose tolerance. These results highlight SCFAs and their receptors as potential targets for the treatment of diabetes.
author: Gwen Tolhurst; Helen Heffron; Yu Shan Lam; Helen E. Parker; Abdella M. Habib; Eleftheria Diakogiannaki; Jennifer Cameron; Johannes Grosse; Frank Reimann; Fiona M. GribbleCorresponding authors: Fiona M. Gribble, <email@example.com>, and Frank Reimann, <email@example.com>.
date: 2012-02
references:
title: Short-Chain Fatty Acids Stimulate Glucagon-Like Peptide-1 Secretion via the G-Protein–Coupled Receptor FFAR2

Targeting the release of anorectic and antidiabetic gut peptides is the focus of many ongoing drug development programs, as evidence is accumulating that enhanced secretion of glucagon-like peptide (GLP)-1 and peptide YY (PYY) from intestinal L cells may translate into beneficial effects in subjects with diabetes and obesity (1). Already, the diabetes field has witnessed the impact of therapeutic GLP-1 mimetics and dipeptidyl peptidase (DPP)4 inhibitors, which, respectively, mimic endogenous active GLP-1 and slow its enzymatic degradation in the circulation (2). The injectable GLP-1 mimetics, in particular, are associated not only with improved blood glucose control and reduced incidence of hypoglycemia but also with significant weight reduction (3). The observed correlation between elevated postprandial GLP-1 levels and improved glucose homeostasis in patients after bariatric surgery indicates that the body has excess capacity in the GLP-1 axis that can be recruited in obese diabetic patients, with downstream beneficial effects on food intake and diabetes control (4). Identifying and validating pharmaceutical strategies to enhance GLP-1 secretion are central to many ongoing L cell–targeting research programs.

L cells are a component of the enteroendocrine system, diffusely located along the length of the intestinal epithelium (5). They make contact with the gut lumen via apical processes and are believed to respond directly to luminal signals. Among the best characterized triggers of GLP-1 secretion are sugars, amino acids, and long-chain fatty acids, which stimulate L cells by a variety of pathways including transporter-associated uptake, metabolism, and G-protein–coupled receptor activation (5,6). L cells are, however, found in highest density in the colonic epithelium, where these nutrients are unlikely to reach significant concentrations (7). Short-chain fatty acids (SCFAs), derived from bacterial fermentation of macrofibrous material reaching the distal gut, by contrast are known to reach high concentrations under physiological conditions in the colons of healthy subjects. Intraluminal concentrations beyond 100 mmol/L, comprising \~60% acetate (C2), 25% propionate (C3), and 15% butyrate (C4), have been reported (8). Plasma levels of SCFA are also dominated by acetate but are generally below ∼200 μmol/L unless elevated by ethanol metabolism (9,10). Nondigestible and fermentable dietary fiber, as well as SCFAs themselves, has been shown to increase GLP-1 secretion in humans (11,12) and rodents (13,14), and enhanced PYY release has been proposed as a link between luminal SCFAs and altered gut motility (15,16).

SCFAs act as a local nutrient source but can also trigger cell-specific signaling cascades by activation of the G-protein–coupled free fatty acid receptor (FFAR)2 (GPR43) and FFAR3 (GPR41) (17,18). These two receptors share \~40% amino acid sequence similarity and are conserved across several mammalian species (17–19). Both receptors respond to SCFAs containing two to five carbons, although a preference of FFAR2 for C2 and C3 fatty acids and of FFAR3 for C3–C5 carbon chain lengths has been reported (17,18,20). The receptors differ in their intracellular signaling capabilities, with FFAR2 reportedly coupling to either G~q~ or G~i/o~ and FFAR3 exclusively activating G~i/o~ pathways (17,18,20). The finding that both receptors are located in colonic L cells by immunostaining (21–23) suggests that SCFAs may use this pathway to modulate L-cell function. Experimental data in support of this idea are, however, still lacking.

The aim of the current study was to establish and explore the link between SCFAs and GLP-1 secretion, making use of our transgenic mouse model (GLU-Venus) in which L cells are identifiable by their expression of a yellow fluorescent protein derivative, Venus (24).

# RESEARCH DESIGN AND METHODS

All animal procedures were approved by the local ethics review committees and conformed to Home Office regulations. Experiments were performed using mice on a C57B/6 background, except in the case of the *ffar2*^−/−^ and *ffar3*^−/−^ mice and littermate controls, which were on a 129/SvEv background.

## Generation of knockout mice.

For generation of the targeting vectors, homology arms (*ffar3*, 1,250 base pairs \[bp\] 5′ and 3,914 bp 3′; *ffar2*, 1,761 bp 5′ and 4,199 bp 3′) were PCR amplified from genomic DNA and cloned into a vector containing internal ribosomal entry site *β-gal* reporter gene, neomycin selection marker, and two thymidine kinase negative selection markers ([Supplementary Figs. 1 and 2](http://diabetes.diabetesjournals.org/lookup/suppl/doi:10.2337/db11-1019/-/DC1)). The linearized vector was electroporated into embryonic stem cells (129/SvEv), and homologous recombination was detected by screening PCR and Southern blot. Chimeras were generated by blastocyst injection, bred with 129/SvEv animals, and maintained inbred in this background.

## Glucose-stimulated GLP-1 secretion in vivo.

Experiments were performed on 3- to 4-month-old *ffar2*^−/−^ and *ffar3*^−/−^ mice or wild-type littermate control 129/SvEv mice. No significant differences in body weight were observed between the wild-type and knockout groups. Mice were fasted for 4 h and dosed per os with 20 mg/kg DPP4 inhibitor (cat. no. KR-62436; Sigma). Thirty minutes later, mice were dosed by gavage with 1.5 g/kg glucose solution (Fisher Scientific). For initial GLP-1 measurement, 150 μL blood was collected from awake mice via tail bleed into EDTA-coated capillary tubes (Bilbate) before glucose dosing. Thirty minutes after glucose dosing, mice were killed by CO~2~ inhalation and blood was collected via cardiac puncture into tubes containing aprotinin (0.6 trypsin inhibiting units/mL). Blood samples were centrifuged immediately, and plasma was frozen on dry ice before assay in duplicate for active GLP-1 (MesoScale, Gaithersburg, Maryland).

## Oral glucose tolerance test.

Three- to four-month-old *ffar2*^−/−^, *ffar3*^−/−^, and wild-type littermate control 129/SvEv mice were fasted for 14 h and then dosed by gavage with 1.5 g/kg glucose. Blood glucose was measured using a handheld glucometer (OneTouch Ultra) via tail bleed. Samples for insulin measurements were taken via tail bleed from awake mice into heparinized capillary tubes (Bilbate) and assayed by ELISA (Crystalchem Ultra sensitive mouse insulin ELISA). Mice undergoing the procedure were of similar body weight: *ffar2^−/−^*, 22.5 ± 0.4 g (*n* = 11), wild type, 21.5 ± 0.5 g (*n* = 8); *ffar3^−/−^*, 22.7 ± 1.5 g (*n* = 6), and wild type, 20.8 ± 0.6 g (*n* = 7).

## Insulin tolerance test.

Three- to four-month-old *ffar2^−/−^*, *ffar3^−/−^*, and wild-type littermate control 129/SvEv mice were fasted for 4 h and then dosed with 0.75 units/kg insulin i.p. (Actrapid insulin, supplied by our Named Veterinary Surgeon). Blood glucose was measured using a handheld glucometer (OneTouch Ultra) via tail bleed from awake mice.

## Colonic tissue and cell preparation.

Mice aged 6–26 weeks were killed by cervical dislocation, and colons were collected in ice-cold Leibovitz-15 medium. Cultures for secretion and calcium imaging experiments were prepared as previously described (24). Colonic tissue used for RNA and protein extraction was washed in PBS and placed in RNA later or a protein lysis buffer, respectively, and frozen until processed.

GLUTag cells were cultured in Dulbecco's modified Eagle's medium (5.5 mmol/L glucose) supplemented with 10% FBS, 2 mmol/L L-glutamine, 100 units/mL penicillin, and 0.1 mg/mL streptomycin.

## RNA extraction and quantitative PCR.

Total RNA from cells sorted with fluorescence-activated cell sorter (FACS) prepared from GLU-Venus transgenic mice (24) was isolated using a microscale RNA isolation kit (Ambion). Total RNA from GLUTag cells and murine colonic tissue was prepared following the Tri-Reagent protocol (Sigma). All samples were reverse transcribed according to standard protocols. Quantitative RT-PCR was performed with a 7900 HT Fast Real-Time PCR system (Applied Biosystems) using Taqman probes for β-actin*, gcg, pyy*, *ffar2*, and *ffar3* from Applied Biosystems. Expression was compared with that of β-actin measured in parallel on the same sample, giving a ΔCt for β-actin minus the test gene. If the test gene was undetectable, it was assigned a Ct value of 40. Means ± SE were calculated and statistics were performed for the ΔCt data and only converted to relative expression levels (2^ΔCt^) for presentation in the Figures.

## Colonic protein analysis.

Tissue was mechanically homogenized in lysis buffer. Active GLP-1 was assessed by ELISA (Millipore, Watford, U.K.) and expressed relative to total protein content, measured using a Bradford assay (Sigma).

## Secretion from primary mixed colonic culture.

Secretion studies were performed 24–36 h after culture preparation. Where applicable, cultures were preincubated with 0.2 μg/mL pertussis toxin for 18 h. Cultures were washed with standard saline and incubated with test substances for 2 h at 37°C. Secreted and cellular GLP-1 was extracted as previously described (24), and active GLP-1 was quantified by ELISA (Millipore). GLP-1 secretion was expressed as a fraction of the total hormone (secreted plus extracted) and normalized to basal secretion measured in the same set of experiments.

## Ca^2+^ imaging.

Experiments were performed 4–7 days postplating, using colonic tissue from GLU-Venus transgenic mice with L cell–specific Venus expression (24). Cells were loaded with 7 μmol/L Fura2-AM (Invitrogen, Paisley, U.K.), 0.01% pluronic F127, and 300 μmol/L eserine in standard saline solution for 30 min at 37°C. Single-cell imaging was performed using an inverted fluorescence microscope (Olympus IX71; Southall, U.K.) with a 40× oil-immersion objective. Fura2 was excited at 340/6 and 380/4 nm and Venus at 475/10 nm, using a 75-W xenon arc lamp and a monochromator (Cairn Research, Faversham, U.K.) controlled by MetaFluor software (Molecular Devices, Wokingham, U.K.). Emission was recorded with an Orca ER CCD camera (Hamamatsu, Welwyn Garden City, U.K.) using a dichroic mirror and a 510-nm long-pass filter. Test substances were added to the bath solution and perfused at ∼1 mL/min. Fura2 fluorescence measurements were taken every 2 s, background corrected, and expressed as the 340:380 nm ratio. Average fluorescence ratios from individual cells were determined over 20-s periods before addition and during perfusion of a test agent. Peak responses to a test agent were expressed as the mean ratio of the test agent divided by the averaged ratios measured prior to drug application.

## Solutions.

Standard in vitro saline solution contained (in millimoles per liter) 4.5 KCl, 138 NaCl, 4.2 NaHCO~3~, 1.2 NaH~2~PO~4~, 2.6 CaCl~2~, 1.2 MgCl~2~, 10 glucose, and 10 HEPES, pH 7.4 (NaOH). Essentially fatty acid–free BSA (0.1%) was added to solutions used for static secretion experiments. SCFAs were dissolved directly in saline solution and pH corrected if necessary. All other drugs for in vitro experiments were prepared as 1,000× stocks. The protein and secretion lysis buffer contained 50 mmol/L Tris-HCl, 150 mmol/L NaCl, 1% IGEPAL-CA 630, 0.5% deoxycholic acid, and one tablet of complete EDTA-free protease inhibitor cocktail (Roche). All reagents were supplied by Sigma (Poole, U.K.) unless otherwise indicated. The synthetic GPR43 agonist (S)-2-(4-chlorophenyl)-*N*-(5-fluorothiazol-2-yl)-3-methylbutanamide (CFMB) (25) was synthesized, its identity was confirmed by nuclear magnetic resonance analysis, and it was dissolved in DMSO as a 1,000× stock.

## Data analysis.

Comparisons between conditions were made using Student *t* test (Microsoft Excel) or by one- or two-way ANOVA with post hoc Bonferroni correction or Dunnett test (Prism5; GraphPad), as indicated, with a threshold for significance of *P* \< 0.05. All data are expressed as means ± SEM.

# RESULTS

## SCFAs enhance GLP-1 release.

In primary murine colonic cultures, acetate and propionate (1 mmol/L) significantly stimulated GLP-1 secretion over a 2-h incubation period (Fig. 1A). Secretion was further enhanced in the presence of 100 μmol/L phosphodiesterase inhibitor isobutyl methyl xanthine (IBMX), and under these conditions a significant stimulation by 1 mmol/L butyrate was also observed. To mimic the higher SCFA content of the colonic lumen, we also examined the effect of a 140 mmol/L mixture containing 80 mmol/L acetate, 20 mmol/L butyrate, and 40 mmol/L propionate, which was added to the standard bath solution and compared with 140 mmol/L additional NaCl as an osmotic control. The effect of the high-SCFA mixture was similar to that triggered by the 1 mmol/L concentrations, enhancing GLP-1 secretion ∼1.3-fold compared with the osmotic control (*n* = 5–6, *P* \< 0.01) (Fig. 1B). By contrast, GLUTag cells were not found to secrete GLP-1 in response to 1 or 10 mmol/L SCFA (data not shown).

## SCFA-induced calcium responses in primary mouse colonic L cells.

L cells in mixed cultures were identified by their expression of Venus (Fig. 2B), and the calcium concentration, \[Ca^2+^\]~i~, was monitored in both L cells and their nonfluorescent neighbors after loading of the monolayer with Fura2-AM (Fig. 2A). \[Ca^2+^\]~i~ is represented in the figures by the 340:380 fluorescence ratio. Propionate (1 mmol/L) triggered a transient Ca^2+^ response (Fig. 2A, C, and D), with a mean 1.5 ± 0.1-fold increase (*n* = 17, *P* \< 0.01) above baseline. Acetate (1 mmol/L) triggered a comparable \[Ca^2+^\]~i~ response of 1.3 ± 0.1-fold (*n* = 6, *P* \< 0.05) (Fig. 2D). The SCFA-induced Ca^2+^ mobilization appeared specific to L cells, as intracellular \[Ca^2+^\]~i~ in the non–L-cell population was unaffected by acetate and only marginally elevated by propionate (Fig. 2D).

## *ffar2* and *ffar3* expression is enriched in primary L cells.

Expression of *ffar2* and *ffar3* was investigated by quantitative RT-PCR using mRNA from FACS-sorted intestinal L cells from transgenic GLU-Venus mice (24). Nonfluorescent cells collected in parallel were used to represent the mixed non–L-cell population. As shown in Fig. 3, *ffar2* and *ffar3* were expressed in small intestinal and colonic L cells and significantly enriched in L-cell compared with non–L-cell populations (*P* \< 0.05). GLUTag cells expressed *ffar3* but had barely detectable levels of *ffar2*.

## Acute SCFA-induced GLP-1 secretion is not mediated via a G~i~-signaling pathway.

The findings that SCFAs trigger Ca^2+^ elevation in L cells and enhance GLP-1 secretion are consistent with the involvement of a G~q~-mediated pathway, potentially, therefore, implicating FFAR2. Hence, we tested whether a synthetic phenylacetamide agonist for FFAR2, CFMB (25), could also mobilize Ca^2+^ and found that 30 μmol/L CFMB indeed elevated intracellular Ca^2+^ in L cells (Fig. 2D). A smaller but significant increase was also observed in the non–L-cell population. To evaluate any opposing contribution from G~i~ signaling, we examined GLP-1 release in the presence of the G~i~ inhibitor pertussis toxin. Whereas SCFAs were stimulatory in the presence of IBMX (Fig. 1B), somatostatin (100 nmol/L), used as a positive control for G~i~-coupled pathways, abolished GLP-1 secretion under these conditions (*n* = 3, *P* \< 0.001) (Fig. 4A), an effect that was partially reversed by preincubation with 0.2 μg/mL pertussis toxin (*n* = 3, *P* \< 0.001). By contrast, pertussis toxin did not have a significant effect on the response to 1 mmol/L propionate (Fig. 4B), suggesting that there is little signaling by SCFA through G~i~-coupled pathways during 2-h secretion studies in vitro.

## FFAR2 mediates SCFA-induced GLP-1 release.

To evaluate the relative contributions of FFAR2 and FFAR3 to SCFA-triggered secretion, we examined the effects of propionate and acetate on primary colonic cultures from mice with knockout of either *ffar2* (*ffar2*^−/−^) or *ffar3* (*ffar3*^−/−^). While GLP-1 secretion was triggered by 1 mmol/L propionate and acetate in cultures from wild-type littermate controls (Fig. 5A), the response to propionate was reduced by 70% (*P* \< 0.001) and the response to acetate was abolished (*P* \< 0.001) in the *ffar2* knockout tissue. Knockout of *ffar3* also impaired secretory responses to both acetate and propionate (*P* \< 0.01) (Fig. 5A), although to a lesser extent than *ffar2* knockout. Similar, but more pronounced, effects were observed in the presence of 100 μmol/L IBMX (Fig. 5A). The higher concentration of mixed SCFA (140 mmol/L, as described above) also failed to enhance GLP-1 secretion significantly in both the *ffar2* and *ffar3* knockout mouse models (Fig. 5B).

Analysis of mRNA expression in whole colonic tissue extracts from wild-type and knockout mice revealed that, as expected, *ffar2*^−/−^ mice lacked *ffar2* mRNA and *ffar3*^−/−^ mice lacked *ffar3* (Fig. 5C and D). *ffar3* expression was also significantly decreased in *ffar2*^−/−^ mice, but, although we also observed a trend toward a reduced *ffar2* expression in *ffar3*^−/−^ mice, this did not reach statistical significance.

## *ffar2* knockout decreases GLP-1 content.

To examine whether the receptors have additional effects on L cells operating over a longer time scale, we examined whether colonic tissue from mice lacking *ffar2* or *ffar3* had altered levels of mRNA for glucagon (*gcg*) (which includes the coding sequence for GLP-1), *pyy*, or active GLP-1 peptide. Knockout of *ffar3* did not have a significant effect on colonic *gcg* mRNA or active GLP-1. There was a trend toward reduced colonic *gcg* and *pyy* mRNA expression in the *ffar2* receptor knockout model that did not reach statistical significance (Fig. 5E and F). *ffar2*-deficient mice, however, had significantly reduced colonic GLP-1 protein content (*n* = 6, *P* \< 0.05) (Fig. 5G).

## FFAR2 and FFAR3 affect GLP-1 release in vivo.

To evaluate whether the reduced in vitro GLP-1 secretory capacity of *ffar2*- and *ffar3*-deficient mice translates into impaired hormone secretion in vivo, we examined plasma GLP-1–level responses to oral glucose administration. Basal levels of active GLP-1 were reduced by ∼40% in *ffar2*-deficient mice (*P* \< 0.01) (Fig. 6A) and were also lower, although not significantly, in *ffar3*-deficient mice compared with those in wild-type littermate controls. Following an oral glucose load, more pronounced impairments of plasma GLP-1 responses were observed in both *ffar2* and *ffar3* knockout models (*n* = 5, *P* \< 0.05) (Fig. 6A). Coinciding with the reduced circulating GLP-1 concentrations, *ffar2*- and *ffar3*-deficient mice also exhibited impaired glucose tolerance when tested with gavage administration of 1.5 g/kg glucose (Fig. 6B and C). This correlated with reduced plasma insulin levels, although again this reached significance in mice lacking *ffar2* but not *ffar3* when compared with their respective littermate controls (Fig. 6D and E). To test for possible differences in insulin sensitivity, *ffar2*^−/−^ and *ffar3*^−/−^ mice and littermate controls were injected with 0.75 units/kg insulin after a 4-h fast. No significant differences were found between the different genotypes (Fig. 6F and G).

# DISCUSSION

Our results demonstrate a direct link between SCFA activation of FFAR2, elevation of intracellular Ca^2+^ in L cells, and enhanced GLP-1 secretion from primary colonic cultures. A stimulatory role for FFAR2 in vivo is supported by the finding that knockout of *ffar2* lowers both basal and glucose-stimulated GLP-1 concentrations.

By quantitative PCR, we could demonstrate that expression of mRNA for *ffar2* and *ffar3* is enriched in L cells, consistent with the detection of these receptors in PYY- and GLP-1–positive cells in rat and human colon by immunohistochemistry (21–23). Interestingly, *ffar2* expression was very low in the GLP-1–secreting cell line GLUTag, perhaps explaining the poor responsiveness of GLUTag cells to SCFAs (data not shown) and emphasizing the importance of studying primary L cells in parallel with cell line models.

FFAR2 reportedly couples to G~q~- or G~i/o~-signaling pathways and FFAR3 exclusively to G~i~. The finding that GLP-1 secretion is enhanced, rather than inhibited, by SCFAs suggests that G~q~-coupled pathways predominate over any G~i~ activation, and indeed, we did not observe any evidence that G~i~ coupling blunts SCFA-triggered GLP-1 secretion either in secretion studies with added pertussis toxin or after knockout of *ffar3*. Somatostatin, by contrast, inhibited IBMX-triggered GLP-1 release in a pertussis toxin–sensitive manner, indicating that downstream G~i~-coupled pathways are globally functional in L cells.

The idea that FFAR2 is coupled to G~q~-signaling pathways in L cells is suggested by the Ca^2+^ imaging data, which showed that acetate and propionate triggered intracellular Ca^2+^ responses in the L-cell population. In support of this idea, knockout of *ffar2* abolished SCFA-triggered GLP-1 secretion in vitro. The explanation for the blunted GLP-1 secretory response to SCFA in cultures from *ffar3*^−/−^ mice is less clear, as FFAR3 has not been reported to couple to stimulatory G~q~- or G~s~-signaling pathways. One contributing factor could be the reduced *ffar2* expression that was observed in *ffar3*^−/−^ mice, although this did not reach statistical significance in our samples. Reduced expression of *ffar2* has, however, previously been reported in adipose tissue of *ffar3* knockout mice (26). Alternatively, there may be a previously unidentified component to FFAR3 signaling in L cells that also contributes to SCFA-triggered GLP-1 secretion. This is not, however, supported by the observation that stimulation of GLP-1 release was not observed in GLUTag cells, which express *ffar3* but very little *ffar2*. A dominant role of FFAR2 over FFAR3 in SCFA-triggered L-cell activation is further suggested by the acute Ca^2+^-elevations seen in primary L cells in response to the FFAR2-specific agonist CFMB, which reportedly does not have activity against FFAR3 (25).

It is not known whether FFAR2 and FFAR3 reside on the apical or basolateral membrane of L cells or whether they primarily detect luminal or plasma SCFA. SCFA levels in the colonic lumen are in the area of 100 mmol/L. Although this is considerably above the half maximal effective concentrations (EC~50~s) of FFAR2 and FFAR3 for SCFA (0.5–1 mmol/L) (17,18,20), SCFA concentrations may be lower in the immediate vicinity of the L cells as a result of the diffusional barrier provided by the mucous layer and active uptake by neighboring enterocytes. Circulating plasma SCFA concentrations are in the range of 100–200 μmol/L (10), which is closer to the working range of the receptors, and while most studies have examined how luminally applied SCFAs affect GLP-1 release (14,27,28) it has also been shown that systemically infused acetate can enhance GLP-1 secretion (28). As luminal SCFA concentrations are not predicted to change markedly in response to acute food ingestion, it is possible that SCFAs produced by colonic fermentation provide a chronic stimulatory tone on L cells via apical or basolaterally located SCFA receptors. This could account for the circulating levels of active GLP-1 that are detectable even in the fasting state and amplify the responses to ingested nutrients, mediated, for example, via neurohormonal signals triggered by nutrient arrival higher up the gastrointestinal tract.

Consistent with the impaired responsiveness of *ffar2*^−/−^ and *ffar3*^−/−^ colonic cultures to SCFAs in vitro, we also observed lower GLP-1 levels in the knockout mice in vivo. As *ffar2*^−/−^ mice exhibited both reduced colonic GLP-1 content and impaired SCFA-triggered secretion in vitro, we cannot conclude from our data which of these components underlies the reduced basal and glucose-triggered GLP-1 secretory responses in vivo*.* In *ffar3*^−/−^ mice, however, the colonic GLP-1 content was not reduced, suggesting that the lower plasma GLP-1 levels reflect the impaired SCFA-triggered secretory response.

In addition to having low circulating GLP-1 levels, *ffar2*^−/−^ and *ffar3*^−/−^ mice exhibited impaired glucose tolerance. The reduced GLP-1 concentrations are likely to contribute to the impaired glucose homeostasis but may not be the only cause because *ffar2* and *ffar3* are not exclusively expressed in intestinal cells. In fact, they are better characterized as mediators of SCFA effects on immune cells and adipocytes (17,18,20), where they have been implicated in modulating neutrophil activation (18,20), reducing inflammation by inhibition of cytokine and chemokine expression, enhancing leptin production (29), stimulating adipogenesis, and inhibiting lipolysis (30,31). They are also expressed in pancreatic β and α cells (G. Tolhurst, H. Parker, A. Habib, F. Reimann, and F. Gribble, unpublished observations). Global knockout of these receptors may therefore affect adipocyte function, inflammation, or pancreatic β-cell function, which would themselves impact on glucose tolerance. The fact that we observed reduced insulin levels in the *ffar2*^−/−^ mice during an oral glucose tolerance test but did not observe differences in insulin tolerance indicates that the observed impaired glucose tolerance in our model reflects an impairment of insulin secretion, possibly in part due to an impaired incretin axis. Another *ffar2*^−/−^ mouse model was recently reported to exhibit increased food intake but had lower body weight and insulin levels compared with wild-type controls after a prolonged period on a high-fat diet (32). No explanation was found for the increased food intake, but our data suggest that it may be related to reduced secretion of L-cell peptides like GLP-1 and PYY. Unlike the findings of our study, those of Bjursell et al. (32) did not show a significant difference in glucose tolerance between chow-fed *ffar2*^−/−^ and control mice. The apparent differences between the two knockout models may reflect the fact that the mice were maintained on different genetic backgrounds (C57B/6JOlaHsd vs. 129/SvEv) or that they were studied at different ages.

Interest in the gut microbiome has heightened recently, with the recognition that our complement of colonic bacteria may not simply reflect factors such as diet and illness but can also in return influence food intake and metabolism. Probiotics, prebiotics, and high-fiber diets are promoted for their potential beneficial effects on obesity, diabetes, inflammation, and immunity, as well as local effects on the health and integrity of the gut. One potential link between colonic microflora and systemic activity are SCFAs, which are produced by bacterial fermentation of fiber and which not only act as a local nutrient source but also trigger the release of anorectic hormones like GLP-1 and PYY. Our data suggest that SCFAs produced by bacterial fermentation in the gut can directly influence L cells to enhance the release of peptides such as GLP-1 and PYY. Whether this interrelationship is altered by metabolic conditions such as diabetes and obesity remains to be determined. FFAR2 should perhaps be added to the list of target receptors that may be exploitable for the pharmacological stimulation of the enteroendocrine system.

## ACKNOWLEDGMENTS

This work was funded by Wellcome Trust grants to F.M.G. and F.R. (WT088357 and WT084210, respectively) and the European Union Seventh Framework Programme (FP7/2007-2013, grant agreement no. 266408).

The in vivo experiments on the knockout and control mice were performed at Takeda Cambridge Ltd, and the knockout mice used were originally created for Takeda Cambridge Ltd. The ffar2 selective agonist CFMB used in the study was synthesized for Takeda Cambridge Ltd. No other potential conflicts of interest relevant to this article were reported.

G.T., H.H., Y.S.L., H.E.P., A.M.H., E.D., and J.C. researched data. J.G. designed the study. F.R. and F.M.G. designed the study and wrote the manuscript.

The authors thank Keith Burling and the Medical Research Council Centre for Obesity and Related Disorders (Cambridge, U.K.) for performing plasma GLP-1 assays. Daniel Drucker (Samuel Lunenfeld Research Institute, Toronto, Canada) kindly provided the GLUTag cell line.

# REFERENCES