(2021) 10:32
Drepper et al. Environmental Evidence
https://doi.org/10.1186/s13750-021-00247-7

Environmental EvidenceOpen Access

SYSTEMATIC REVIEW PROTOCOL

Strategies for managing spring frost risks
in orchards: effectiveness and conditionality—A
systematic review protocol
Bianca Drepper1* , Brecht Bamps1, Anne Gobin1,2 and Jos Van Orshoven1

Abstract
Background: Spring frosts pose an important threat to orchard productivity in temperate zones and predictions of
future occurrences do not exclude damaging events. However, there is no up-to-date and systematic comparison of
the effectiveness and conditionality of the existing passive and active damage prevention strategies. This review seeks
to answer the questions “How do the performances of spring frost damage reduction strategies in temperate fruitorchards compare?” and “How do environmental conditions affect frost damage reduction strategies in temperate
fruit orchards?”.
Methods and output: The review covers on-site frost damage prevention mechanisms and interventions for themost important temperate pome, stone and citrus fruit trees as well as grapevine. Searches include the core collection and regional databases on the Web of Science platform, Scopus, as well as specialized libraries like Agris, Agricola,
CAB Abstracts, Groenekennis and selected institutional websites. Included studies report on the effectiveness of atleast one intervention in reducing spring frost damage or increasing temperatures in the field. The validity of thestudies will be evaluated based on their risk of general research bias and on topic-specific, stakeholder co-designed
validity criteria. Data will be extracted regarding the study setup (study design, location, characteristics of frosts) andthe resulting temperature increase, crop development alteration and/or damage reductions. The effectiveness ofinterventions will be evaluated in terms of yields, damage to flowers, ambient temperature and/or crop development.
An Evidence Atlas will identify general and geographic research gaps, as well as research trends. Meta-regression ofeffectiveness on environmental conditions will be attempted.
Keywords: Fruit trees, Damage, Prevention, Protection, Bloom
Background
Problem statement

Spring frosts are among the most important causes ofyield loss in fruit orchards in temperate regions [1]. Frostdamage occurs when the ambient temperature drops
below a crop and development stage-specific criticalthreshold [2]. A single frost night on 19th April 2017 led*Correspondence: bianca.drepper@kuleuven.be1
Division Forest Nature and Landscape, University of Leuven,3001 Leuven, Belgium
Full list of author information is available at the end of the article

to apple yield losses of up to 78% in Europe compared tothe preceding seven years (Fig. 1) [3].
Since the 1950s several fruit research stations acrossEurope report an earlier start of flowering [4] which could
potentially lead to higher probabilities for frosts during
the flowering period. However, in a warming climate,also the last frost day is occurring earlier in the year andregional studies arrive at contrasting conclusions overthe evolution of the frost hazard during flowering. Whilesome studies report on unchanged or increased frequencies [5–12] for certain regions, decreased risks werereported for other regions or by other studies [8–15].

© The Author(s) 2021. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to theoriginal author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images orother third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of thislicence, visit http://​creat​iveco​mmons.​org/​licen​ses/​by/4.​0/. The Creative Commons Public Domain Dedication waiver (http://​creat​iveco​
mmons.​org/​publi​cdoma​in/​zero/1.​0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Drepper et al. Environmental Evidence(2021) 10:32

Page 2 of 13

Fig. 1 Relative apple and pear yields in European countries for 2017 in comparison to the mean yield of 2009–2016, as reported by the WorldApple and Pear Association [3]Although studies differ substantially in the employedphenological models, climate models and assessment
methodologies, most of them confirm that frost during
flowering will remain a major risk factor for fruit production in warming temperate regions.
Research and interests

Similar reviews were conducted in the past, whereby
the most prominent reference is a handbook for frost
protection, published by the FAO and edited by Snyder and de Melo-Abreu [1]. With the majority of thereferenced studies dating prior to 1995, an updated
and systematic review seems needed to summarise
new research findings and identify gaps. Specialised
reviews have recently been published on protection
against frost by means of wind turbines [16] and on
innovative sensor networks for early frost detection[17]. Another recent and comprehensive summary on
frost prevention techniques was published by ‘Les instituts techniques agricoles’ [18], which is geared towards
practitioners and lacks academic referencing. In 2018and 2019 the EIP-AGRI Focus Group collected knowledge of stakeholders and experts from across Europe on
fruit protection against frost [19]. Besides a characterisation of strengths and weaknesses of common passive
and active interventions, a list of research needs wascompiled. First and foremost it mentioned the need for“Studying and comparing the effectiveness of methods
under different conditions” (p. 35) followed by “Morebiology to the models” and “Establishing a database on
potential yields for different species/varieties and critical temperatures on species/variety level”.
The interests of stakeholders in this topic converge in a better assessment and understanding of theeffectiveness of techniques. The targeted end-usersof the review are the fruit producers and their advisors. Other major stakeholders are insurance businesses and governmental agricultural services at theregional, national, and international level as well as
academic, national or regional fruit research centres. The Belgian ‘Proefcentrum fruitteelt vzw’, 3800
Sint-Truiden, Kerkom, (PCFruit vzw), a major experimental and applied research station for fruit production, was directly involved in the design of the present
review protocol, regarding the selection of databases,
search strings, population definition, data extraction
and validity assessment criteria. An ultimate group of
stakeholders are the consumers of fresh fruit who prefer fruits with low prices and high quality, implying nodamage suffered from frost or frost damage remediation treatments.

Drepper et al. Environmental Evidence(2021) 10:32

Page 3 of 13

Objective of the review
Farmers can choose between or combine different
kinds of frost risk management strategies: financial(insurances, income diversification) or physical (windbreaks, foliar applications, sprinklers, wind machines,…). This review will focus on the latter kind, provided,that they affect either ambient temperatures or cropdevelopment.
This review seeks to answer two research questions. The primary one is “How do the performancesof spring frost damage reduction strategies in temperate fruit orchards compare?” while the second is “’How
do environmental conditions affect the effectivenessof frost damage reduction strategies in temperate fruitorchards?”.
Thereby the main objective is to compare the effectiveness of the different measures that target at least
partially the reduction of yield losses due to spring
frost. The identified environmental preconditions shallbe used for a classification of the techniques as a basis
for informed decision making in the fruit sector. Only
tested interventions described in the published literature are considered, thereby excluding patents.
The review will aim to produce a comprehensive
inventory of the available evidence of frost protection
measures for commercially important temperate fruit
tree orchards and vineyards. The studied fruits wereselected from a list of the major temperate fruits [20]making use of expert knowledge (Table 1). The review
will include all geographical locations that are under atemperate climate according to the Köppen-Geiger climate classification system [21], corresponding to themain climate type ‘C’, see Fig. 2.
We will limit the review to preventing frost damage
during early development stages (after bud break inspring) and not during the harvest period (autumn) nor
storage. We consider damages during dormancy as wellas the temporal evolution or spatial distribution of frost

events and associated losses beyond the scope of thisreview.

Methods
The reporting of this protocol follows the RepOrting
standards for Systematic Review Syntheses (ROSES)
for systematic review protocols [23]. The completed form
can be found in the Additional file 2.
Searches
Bibliographic databases

Several kinds of sources will be drawn upon for thisreview: academic literature from peer-reviewed journals as well as ‘practitioners’, ‘specialist’ or ‘grey’ literature comprising theses, institutional and governmental
reports, which will be identified via dedicated searchengines. The selection of relevant databases and journals
was based on consultations with librarians of the University of Leuven and fruit research experts, and on the recommendations of the Collaboration for Environmental
Evidence (CEE) [24].
Peer-reviewed articles will be searched for based on
Title, Abstract and Keywords (Author based and generated) using the advanced search facility of the databases“Scopus” and the following collections in “Web of Science” (WoS): Core collection and regional collections
covering publications from Korea (KJD), Russia (RSCI),
China (CSCD) and Latin America (SCIELO), as detailedin Table 2, where the temporal coverage of every index
is given. No period, document type or language will be
excluded at this stage.
Libraries and databases specialised in agriculture
including Agricola (United States Department of Agriculture), CAB Abstracts (Centre for Agriculture and Bioscience International), FAO Agris (Food and Agriculture
Organisation), Groenekennis (University of Wageningen) will be searched for with time and index settings asdetailed in Table 2.

Table 1 Elements of the systematic review question
Term

Explanation

Population

Commercially important temperate fruits:Apple, Pear, Sweet cherry, Peach,
Nectarine, Plum, Apricot, Avocado, Lemon, Orange, Grapefruit, Mandarine,
Pomelo, GrapevineIntervention

Passive or active frost damage reduction tools

Comparator

No intervention
Interventions characterised by parameters (intensity, concentration, timing…)

Outcome

Measures of ambient temperature
Measures of damage on flowers, buds, or fruitlets
Measures of production (yield)
Advance/delay of phenological stages

Drepper et al. Environmental Evidence(2021) 10:32

Page 4 of 13

Fig. 2 Fruit production by botanical fruit group and region, based on FAOSTAT production data for 2019. Yellow shades are Temperate zones after
Köppen—Geiger (type ’C’). Regional grouping follows the definitions and standards used in FAOSTAT [22]. Fruit groupings were defined based on
both botanical and geographical criteria and listed in Additional file 1

For an article to be returned, at least one term referring to the review population (common name, Latinname or type of fruit) is needed and either a combination of highly specific terms like “frost” and a term ofcontext or action—or a predefined association including generic terms like “cold” or “low temperature”. A
successful combination could thus be “apple” AND“protection” AND “frost”. The search string was initially developed for the Web of Science (Table 3) andwas then adjusted to the specific syntax of the otherdatabases/search engines (Additional file 3). The
Groene Kennis database includes (Dutch) classification tags, although only until 2015, including “fruitteelt” (Fruit production) and “beschadigingen door
vorst” (frost damage), which will be searched for separately. By default, the search widget of the Groene
Kennis database searches full text, leading to significant noise. With the URLs given in Additional file 3,
only the Abstract, Title and Keywords are considered(Rob Genderen, Wageningen University, personalcommunication on 9/2/2021). Also the CAB Abstracts
are searched using the descriptors "frost injury" and"frost protection" while the search in AGRICOLA isamended by searching by the more generic Code 4520
for “Miscellaneous plant diseases, injuries and control”.
Since the Groene Kennis and FAO Agris search
results cannot be exported to a .RIS file at once, thebibliometric data will be scraped and written into .RIS
files. Entries from FAO Agris will be filtered by the eligible languages at this step.
Estimations of the comprehensiveness of the searches

It became evident during preliminary scoping exercisesthat an important share of studies was published before
1990. This is confirmed by a recent review on wind-based
protection against frost damage [16]. Therefore, no datefilters will be applied. The earliest records listed in theemployed databases range from 1903 (Agricola) to 1955
(Web of Science databases) (Table 2).
Prior to the protocol definition, 149 references to articles dealing with frost protection at large were extracted
from past reviews [1, 16–18]. Based on the title only,around two thirds potentially satisfy the eligibility criteria
for the review (listed in Table 5). They served as a reference for the formulation of the search strings. A substantial part was retrieved through the Web of Science andthe remainder -mostly grey literature- was traced backto their sources or libraries. These were added to thelists of additional databases and websites to be searched(Table 4).
Automated alerts for the defined searches will be activated in Scopus and Web of Science, in order to include
papers published during the review period. The otherdatabases will be queried again, including a filter for theDrepper et al. Environmental Evidence(2021) 10:32

Page 5 of 13

Table 2 Overview of libraries to be queried with the temporal coverage and number of hits for the defined search strings
Platform/publisher

Database/library

Web of science

Core collection

Index/specification

Years covered

SCI-EXPANDED

1955–2021

SSCI

1956–2021

A&HCI

1975–2021

CPCI-S

1990–2021

CPCI-SSH

1990–2021

BKCI-S

2005–2021

BKCI-SSH

2005–2021

ESCI
CSCD

Hits on
15/06/2021

2005–2021

2475

1989–2021

415

KJD

1980–2021

83

RSCI

2005–2021

50

2002–2021

58

ScopusSCIELO‘Documents’

1788–2021

2518

FAO

AGRIS

‘Publications’

1954–2021

USDA National Agricultural
Library Agricola

NAL Article Citation Database, NAL
Cataloging Database

Elsevier

1510
1123

CAB Direct

CAB Abstracts

1910–2021

WUR Groenekennis

All content

1543–2021
1981–2015 (tags)

63 (EN) 44(DE) 238(NL) 78(tags)

Table 3 Complete search strings by database and language
Search string
Web of Science (collections as in Table 1)

TS = (("orchard$" OR "fruit tree$" OR "pome fruit$" OR "stone fruit$" OR "hesperidium" OR "hesperidia" ORmalus* OR pyrus* OR prunus* OR persea* OR citrus* OR "vitis vinifera" OR "apple$" OR "pear" OR "pears" OR"cherry" OR "cherries" OR "peach" OR "peaches" OR "nectarine$" OR "plum" OR "plums" OR "apricot$" OR"avocado$" OR "lemon$" OR "orange$" OR "grapefruit$" OR "mandarine$" OR "pomelo$" OR "grape$" OR"vine$" OR "vineyard$")AND(((prevent* OR protect* OR "manage" OR "management" OR damag* OR injur* OR “flowering" OR "flowers"
OR "bloom" OR "blooming" OR "blossom*")AND("frost" OR "frosts" OR freez* OR "cold weather")) OR ((("cold” OR "low* temperature*" $) NEAR (damag* OR
injur*)) OR "Freeze avoidance" OR "antifreeze" OR "anti freeze")))Table 4 Selected websites for specialist searchInstitutional websites

Simplified keywords

https://​op.​europa.​eu/​en/​web/​gener​al-​publi​catio​ns/​envir​onment

Frost, freeze, gélée, heladas, vorst

https://​docum​ents.​world​bank.​org/​en/​publi​cation/​docum​ents-​repor​ts/​docum​entli​st
https://​www.​cgiar.​org/​resea​rch/​publi​catio​ns/
https://​www.​fibl.​org/​en/
https://​www.​opena​grar.​de
https://​www.​agros​cope.​admin.​ch/​agros​cope/​en/​home/​publi​catio​ns/​publi​cation-​search.​html
https://​aureli.​inrae.​fr/("gélées" OR “gel”) AND ("protection", OR "proteger")Drepper et al. Environmental Evidence(2021) 10:32

Page 6 of 13

Table 5 Primary inclusion and exclusion criteria based on title and abstract
Criteria

Inclusion

Exclusion

Population

All temperate perennial fruit trees of commercial interest (apple,
pear, sweet cherry, peach, nectarine, plum, apricot, avocado,
lemon, orange, grapefruit, mandarine, pomelo, grapevine)

Low-height shrubs
Berry plants
Other non-fruit horticulture
Other non-temperate horticulture

Interventions

Physical on-site treatments or devices that can be applied inManagement choices like income or crop diversification
anticipation, during or just after a frost event, like wind machines, Crop breeding
sprinklers, foliar applications of chemicals, coverages, modified
Genetic modifications
pruning
Financial insurances

Outcome

Measures of ambient temperature
Measures of damage on flowers, buds or fruitlets

Evolution of frost hazards or vulnerability through time andspace

Measures of production

Change in cold hardiness during dormancy

Evidence on post-harvest conditions

Advance/delay of phenological stages
Comparators

Investigations of effectiveness against a control population orvariants of the same treatment/device

Lack of comparators or other measures of successClimate zones

Temperate

TropicalMediterranean

SubtropicalLanguage

English, French, German, Dutch, Spanish

Any other language

Type of publicationPeer-reviewed journal

Web page/blog

Organisational report

Unpublished communication

Model vs observation

Cold climates

Professional journal article
ThesisType of study

Field experiment

Literature review

Greenhouse experiment

Mathematical model

Laboratory experiment

Risk assessment studies
Micro meteorological studies to explore the potential for anintervention to function without observations of the intervention itself (i.e. Inversion strength characterisations)publication year 2021 if possible, at the end of the data
extraction phase and before the analysis phase.
In the international databases, searches are exclusively
conducted in English, since we found that searching withGerman or French translations of the search strings did
not have an added value, due to abstracts translated in
English. Publications with English abstracts and full texts
in Dutch, German, French and Spanish are thus considered as well. Including five languages in the review process reduces the language bias and the risk of missingout from key literature [25]. While the most prominent
publications stem from northern America and Europe,
within the available resources we will include publications from fruit growing regions in temperate Asia andLatin America [22], considering their importance forfruit production (Fig. 2).

Specialist searches

Google Scholar and Baidu Scholar are queried in English(see Table 1 in the Additional file 1), as a complementarysource, but are not used as a primary source due to thelack of transparency of the sorting algorithm [26]. The
results will be restricted to the first 200 results.
National fruit research institutes in Germany (INRES),France (CTIFL), Italy (Laimburg Research Centre) andSpain (IRTA) were contacted to inquire about their relevant publications. The provided papers will undergo aneligibility check and will be listed accordingly as additional material.
Further, selected relevant institutional websites aresearched for using simplified keywords in English, German, French, Dutch and Spanish (Table 4). This list may
increase during the review process.

Drepper et al. Environmental Evidence(2021) 10:32

Article screening process

The screening process will be conducted by the authorsof the protocol. In accordance with the CEE guidelines[24], several screening steps can be distinguished and thenumber of eliminated studies at every step of the screening process will be reported in a ROSES flow diagram
using a dedicated tool [27].
Firstly, all search results will be collected and duplicates
removed using Endnote Desktop software [28] following
the methodology by Bramer et al. [29].
Secondly, the title and abstract of every remaining
article will be read. Articles will be removed if they do
not fulfil one of the eligibility criteria (see Table 5). The
online tool ‘Rayyan’ (https://​rayyan.​qcri.​org/) is preferred
for this step because it enables multiple users to assess
papers and conflict solving. Every abstract will be readindividually, but papers can be filtered by potential exclusion keywords (e.g. “mango”) which supports an efficient
inclusion or exclusion of articles while recording the reason for exclusion, e.g. “wrong population” [30].
For consistency checking, a random selection of 10% of
the articles will be reviewed by more than one reviewer
and a Kappa score [31] will be computed. If a lower
agreement than 0.6 is reached, the criteria definition andexamples in Table 5 will be rediscussed and adjusted. Incase no consensus can be found, a third reviewer will be
consulted for mediation. The resulting list of excluded
articles and conflict resolution documentation will bemade accessible online and added as supplementarymaterial to the review.
Thirdly, the included references will be gathered bymeans of a reference manager software [28] and full texts
will be retrieved. Corresponding authors will be asked fortheir copy if there is no institutional access (incl. interlibrary loan). Articles without full text access will be listed
separately and attempts made to retrieve the full text will
be documented.
Fourthly, publications will be read at full text level. In
case its abstract led to the inclusion of a publication, but
the full text reveals ineligibility, the exclusion and reason
thereof, will be documented in a separate table as additional material. At this stage 10% of the full texts will be
reviewed by more than one author and conflicts solved
with the mediation of a third reviewer. The results will
equally be reported.
Eligibility criteria

Articles will be retained or excluded based on the criteria listed in Table 5. Studies will be retained when they
compare the effect of a given measure on the local temperature, the plant development (e.g. budburst delay),
the condition/count of buds or flowers or yields against a

Page 7 of 13

control population. Articles that fail to meet one or morecriteria will be excluded.
Study validity assessment

Studies in the environmental sciences rarely satisfy gold
standards of internal and external validity simultaneously. Studies that do not satisfy certain criteria (e.g. min.
sample size, random selection of experimental units) areconsidered of low validity for this study. This does notnecessarily mean that they generally lack quality. Hence,existing critical appraisal tools for systematic reviews
need to be adjusted for the context of environmental sciences [32]. Studies not reporting the necessary details
have an uncertain validity which in this review will betreated as low validity.
To avoid a false sense of hierarchy, no ranking of study
set-ups will be undertaken, unlike systematic reviewsin the medical field, where pre-defined rankings exist[32]. Instead, the design of each included study will beassessed based on predefined quality criteria (Table 6).
The first set of criteria is based on Bilotta et al. [32] where
typical biases are listed regarding general aspects of thestudy design.
The second list of specific criteria was added in consultation with experts. Distinctive sections were designed
for the judgement of sample sizes. In this review, thespatial application scale (experimental unit from which
random sampling is possible) of an intervention can varyfrom an entire field (e.g. wind turbines) to a row (e.g.
water sprinklers) or individual trees (e.g. foliar application), of which then trees or branches are sampled foranalysis (observations). Considering the outcome measures, temporal replication at daily/nightly units within afield is possible if the outcomes are temperature measurements during or following an intervention, while inthe case of analysing flower damage or yields repetitionsof several days cannot be considered as independent
treatments. Several crop cycles (years) are considered to
be independent replicates. Considering the scale of theinterventions, we differentiate between the application
and assessment level, since not always all manipulated
trees are assessed for damage.
All criteria formulations result from an interactive
testing process with a subset of 20 papers which wereindependently assessed by two authors. This subset consisted of papers that were identified as likely eligible anddirectly accessible in the Web of Science Core Collection
during the search string finetuning process. Ambiguous
criteria were discussed and refined iteratively until there
was a full consensus. The final criteria have been presented to the fruit research experts (PCFruit) and wereapproved with minor edits.

Adapted criteria

Study protocol is available and pre-specified outcomes arereported on; study protocol not available but all expected
outcomes are reported on

Resulting data points lower than expected from the methodology; no reason given for missing data related to outcome;bad imputation
Spatial replication (experimental units)

Attrition bias (incomplete results (observations))

Number of spatial or temporal replications(experimental unit) and observations [33]> 2 rows> 2 trees

Or 2 rows (row-based installation)
Or 2 trees (targeted application)< 4 trees (widespread or row-based)

Closest meteorological station > 10 km away; no detail given
Mean effects per experiment only
Cold chamber experiment

Location of temperature sensor given

Reporting method

External validity

Field experiment

Raw data, measure of variation

Local sensors on field and beside flowers at several tree heights(lower, middle and upper position in tree)

At least one of the following differs or is not described:
All of the following factors are comparable: management
management (pruning), soil type, topographic position, differ- (pruning), soil type, topographic position, differences in cultivar
ences in cultivar and/or rootstock, orchard age
and/or rootstock, orchard age≥ 4 trees

Baseline of test and control groups compared< 4 branches (targeted application)≥ 4 branches> 1 year

1 year (temporal replication yield/damage)Observations (assessment level) per experimental unit

> 1 frost night

1 frost night (temporal replication temperature change)

Temporal replicationOR> 2 fields

2 fields (widespread application)

Reported data points correspond to methodology; data omission or imputation well documented and justified

Blinding done

Not all pre-specified outcomes reported; measurements/
methods for some outcomes were not pre-specified; incomplete reporting of one or more outcomes; not reporting on
an expected outcome

Randomisation (site/tree selection based on random number
generator/tossing a coin)

Low

Reporting bias (incomplete results (variables))No randomisation (site/tree selection based on availability,
expert judgement or results)

High

Performance and detection bias (researcher bias) No blinding/blinding broken/insufficient information

General bias based Selection bias (Inadequate randomisation)
on Bilotta et al. [32]Risk of bias

Table 6 Study bias assessment categories

Drepper et al. Environmental Evidence(2021) 10:32
Page 8 of 13

Drepper et al. Environmental Evidence(2021) 10:32

The validity assessment will be done on full texts ofthe eligible selection in a public project on the SysRev
platform (https://​sysrev.​com). This software is preferred
due to its efficient data extraction capabilities and is also
suited for transparent conflict solving between multiple
users [30]. For copyright reasons, full texts will not be
uploaded.
Based on the criteria in Table 6, a final judgement of thestudy validity as low, middle or high will be made, without subjectively judging some criteria more important
than others. If the study reports on methods and results
in a rigorous way, at most one criteria may be classified ashaving a high risk of bias. These studies will be classified
as overall low risk of bias. On the other hand, overall highrisk of bias is assigned if the study has major shortcomings in conducting and/or presenting the research. Thisis defined as having a risk for bias for at least 3 criteria.
All other studies will be classified as overall medium riskof bias. A detailed assessment of individual criteria will
be reported in the supplementary materials of the review.
Data coding and extraction strategy

Within the project environment on sysrev.com, a set ofpre-coded fields, as well as several open fields, will beestablished to extract details on the study in a systematicmanner. Four categories of data will be extracted. The
first collects metadata on the study including the settingof the experiment. Secondly, details on the populationand the intervention will be recorded. Thirdly, the conditions for effectiveness will be retained when they areprovided. Lastly, the results will be recorded with any
given measure of variance. Table 7 shows the pre-coded
answers or specifications for the open text fields. If applicable, several options can be selected at once, or written
as free text (separated by a semicolon).
The definition of codes results from an iterative process. The repeatability of this process was tested before
the finalisation of the protocol. Two authors extracted
the data for the subset of 20 papers used for testing thevalidity assessment. When there was substantial disagreement, codes were adjusted until a Kappa score > 0.6
was reached. Furthermore, the codes were reviewed byexperts from the research centre PCFruit.
During the data extraction phase, a random selectionof studies (at least 10%) will be cross-checked by anotherauthor, with special attention to the outcome variables
and human errors. The extracted data (tidy extraction
sheets) will be available as additional material.
No Data/ Not applicable/Other is always an optional
answer. Missing information will be reported during thesynthesis stage. The extracted data records will be added
as supplementary information to the review.

Page 9 of 13

Potential effect modifiers/reasons for heterogeneity

To our best knowledge, no protocol exists for studies ofthe effectiveness of frost protection measures. Every fruit
type has a specific sensitivity to temperatures which varies throughout the development stages [2].
Further, the variation in environmental conditions, thescale of the fields, the distance to the operated technology, the resolution, accuracy and position of the temperature measurement devices and setup in open fields
or controlled environments (e.g. greenhouse or lab)
make the comparison between studies difficult. Table 8
provides a list of the most important factors that may
influence the effects. The list has been compiled basedon experiences of the Frost Inno project by the Belgian
research centre PCFruit [34].
As additional information to the review, the dominantsoil textural class will be identified based on a global soil
database (http://​www.​soilg​rids.​org) using the soilDB
package in R [35]. When no data is available (urban
areas), the nearest possible gridpoint will be retrieved.
Data from the global Shuttle Radar Topography Mission[36] will be used for filling gaps regarding elevation.
Data synthesis and presentation

The research outputs will primarily be descriptive statistics of the identified publication and secondly a comparison of the performance of spring frost damage reduction
strategies in temperate fruit orchards. The conditions fortheir effectiveness will be explained, to the extent thatthey are reported on in the selected literature.
A ROSES flow diagram will expose the body of literature found with the described search strings. The
retained studies will be presented in an ‘evidence atlas’, as
an interactive world map with datapoints comprising thecollected metadata per study [37]. This ‘atlas’ will indicate
the number of included studies, the studied fruit types
over time etc. These findings will be contrasted withmaps of global fruit production based on FAO statistics.
Timelines on the level of detail in reporting and bar-plotsof studies by set-up or environmental conditions willcomplement the ‘atlas’.
The narrative synthesis will be supported by limited
quantitative analyses of the identified intervention effectiveness by outcome category. Accordingly, effect sizes
will be computed separately for measures of (i) temperature, (ii) crop yields, (iii) damage to flowers or buds or (iv)delay/advance in phenology.
Based on extensive scoping exercises we anticipate very
low levels of detail regarding the reported variability of
the results and foresee working with Raw Mean Differences (RMD) for temperature, flower damage and budding delays.

Drepper et al. Environmental Evidence(2021) 10:32

Page 10 of 13

Table 7 Data extraction fields and codes filled with example answers to be pre-coded

Metadata of the study

Set-upCategory

Pre-coded options

Type of publication

Peerreviewed journal, Organisational
report, Professional journal, Conference
paper, Thesis

Study setup

Field experiment, Greenhouse experiment, Cold chamber, otherStudy design

Blocked randomised design, randomised design (blocked), blocked
design, randomised design (splitplot),randomised design, factorial design,
Latin square, split-plot design, stripplot design, field-scale comparison(no replication within a field), paired
design, purposive, other, NA, ND

Variability measureNone, Unclear, Overall 95% CI, Overall
CV, Overall LSD, Overall p values, Overall SE, Raw data, Overall SED, Individual
CI, Individual 95% CI, Individual SD,
Individual LSD, Individual SE, Individual
p values, Individual SED

Free text

Sample size

[number of spatial replicates pertreatment]

Number of observations per replication[number of trees/ branches]

Funding

Governmental, university, private sector, NGOs, foundations

Country

List of temperate countries

Longitude/latitude[longitude and latitude retrieved fromgiven address]

Height above sea level[m]

ReliefDescription

Land use type (surroundings)

Description

Dominant soil texture

Description

Intervention(s)Description of intervention

Intervention class

Water, wind, foliar application,mechanic intervention, covering,heating

Fruit

Apple, Pear, Cherry, Peach, Nectarine, Plum, Apricot, Avocado, Lemon,
Orange, Grapefruit, Mandarine,
Pomelo, Grapevine

Cultivar

Variety nameRootstock

Rootstock name

Development stage[BBCH code or description]

Pruning details

Any information given

Distance between rows and trees

Meter x meter

Field size

[m2]

Planting year

Year(s)Measurement year

Year(s)Groundcover

Description

Orientation of rows

North: South, East: West, North-East:
South-West, North-West: South-East

Temperature sensors position

On trees ≥ 2 m, On trees < 2 m,
between trees ≥ 2 m, between
trees < 2 m, off-field, in flowers, ingreenhouse

Drepper et al. Environmental Evidence(2021) 10:32

Page 11 of 13

Table 7 (continued)

Conditions of effective-ness

Category

Pre-coded options

Frost type—tested

Radiative, advective

Frost type—effective

Radiative, advective

Free text

Details if applicable
Timing, concentration, strength of
inversion, discharge
Wind speed (max)In m/sWind direction

Results

Relative humidity

In %Min. temp. recorded[°C]

Min. temp. effective[°C]

Length of frost period[h]Identified limitations

Description

Costs per ha (€)[€ or volume of resources]

Yield

[kg]

Intervention
Control
Difference
Flower/ bud damage[%]

Intervention
Control
Difference
Bud break[days]

Difference in days
Max. recorded temperature increase(intervention–control)
Effective measure[°C]
Yes, noTable 8 Effect modifiers
Effect modifier

ValuesFrost typeRadiative or advective (or both)Field scale

Size of field, density of trees and row
orientation

Temperature sensor

Brand, accuracy, precision, number andlocation relative to trees

Environmental conditions

Relief
Surrounding land use types
Climate: measured temperatureWind direction and speed
Height above sea level

Management practices

Pruning schemes
Groundcover between the tree rows
Age of the orchard

The convention of the RMD is defined as X
­ intervention− ­Xcontrol, with X being the mean outcome measures
per experiment. A positive RMD is an indication of aneffective intervention in the case of temperature andbud break dates and of an ineffective/counterproductive
one for the damages to flowers and buds. Comparisons
between studies will be possible without standardisation
since in all cases the scales are inherently meaningful(temperature in °C, and flower survival in %).
Studies reporting on final harvests (yields) as outcome measures (kg per tree or unit area) need to be preprocessed. First, all results will be converted to kg/tree,using the reported planting density or row distances.
Still, these yields per tree cannot be compared directly
between studies on different fruit types, given their various normal yield values. The Response Ratio (RR) will becomputed instead as a relative measure of change independent of the order of magnitude [38]. The RR is computed as the ­Xintervention/Xcontrol, with X being the meanoutcome measure per experiment. Values between 0 and
1 indicate a negative impact of the intervention on yields,
a 1 indicates no effect and values > 1 indicate a positiveeffect of the intervention. Due to the asymmetry of theDrepper et al. Environmental Evidence(2021) 10:32

RR, a natural logarithmic transformation is used to makethe scale symmetric [39].
Studies that fail to provide mean differences between
populations in any of the outcome categories will be
excluded from quantitative data synthesis.
We will further conduct linear mixed model regression of the effectiveness against reported characteristicsof the fields and frost events. Since information on the
precision is not available, this regression is an exploratory
analysis only. Hypothesized contributing fixed factors
include continuous variables like the recorded minimum
temperature during the intervention, the elevation andlatitude of the test sites and soil texture (given in % sand,clay or silt). Random factors include the study identifier,the fruit type and the development stage during which
frost occurred.
Analysis of effectiveness and reported costs will be
attempted, as this information is of high relevance forfruit farmers.
Sensitivity analysis to the study validity will be conducted, leaving out low validity studies, in order to test
if low validity studies report higher or lower effect sizes.
All described analyses will be conducted in R language.
The script will be provided as additional material.
Publication bias

As in all scientific domains, there is a higher likelihood
to publish positive results than negative ones and studies
with statistically significant findings are also more likely
to be cited and by consequence to be found. Commonly
funnel plots are used as means to express the publication
bias. Without an estimation of variances only the sample
sizes can be used for constructing these plots, acknowledging the limitations of this approach [40] .

Supplementary Information
The online version contains supplementary material available at https://​doi.​
org/​10.​1186/​s13750-​021-​00247-7.
Additional file 1. Details of fruit classification in Fig. 2.
Additional file 2. ROSES for Systematic Review Protocols.
Additional file 3. Complete search strings by database and language.
Acknowledgements
The authors wish to thank Thomas Vandendriessche, Sylvette Vanderstraeten
and Krizia Tuand, the biomedical reference librarians of the KU Leuven
Libraries–2Bergen (Leuven, Belgium), for their help in setting up and conducting the systematic literature search; as well as Wim Van den Noortgate, Facultyof Psychology and Educational Sciences, KU Leuven and Koenraad Van Meerbeek, Department Earth and Environmental Sciences, KU Leuven, for advice
regarding the data synthesis design.
The protocol benefitted substantially from input by the Research Centre
for Fruit (PCFruit vzw), 3800 Sint-Truiden (Kerkom), Belgium.

Page 12 of 13

Authors’ contributions
BD, JVO and AG acquired funding and designed the study setup. BD wrote
and tested the manuscript. BB was the major contributor to preliminary
analysis and the development of search strings, inclusion and validity criteria.
All authors read and approved the final manuscript.
Funding
BD was funded by Fonds Wetenschappelijk Onderzoek Vlaanderen (Belgium),
grant number 1S34920N. BB was funded by a predoctoral scholarship KU
Leuven with internal reference number 3E200801.
Availability of data and materials
This article will be published open access, under the terms of the Creative
Commons Attribution 4.0 International License http://​creat​iveco​mmons.​
org/​licen​ses/​by/4.​0/). Unrestricted use, distribution, and reproduction in any
medium are permitted provided you give appropriate credit to the original
author(s) and the source, provide a link to the Creative Commons license, andindicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://​creat​iveco​mmons.​org/​publi​cdoma​in/​zero/1.​0/) applies
to the data made available in this article, unless otherwise stated.

Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests. Authors of
research studies included in this review will not be involved in any decisions
regarding their own work. Procedural independence is guaranteed as none of
the authors has (co-)authored studies which could be included in this review.
Author details
1
Division Forest Nature and Landscape, University of Leuven, 3001 Leuven,
Belgium. 2 Vlaamse Instelling voor Technologisch Onderzoek NV, 2400 Mol,
Belgium.
Received: 29 March 2021 Accepted: 12 November 2021

References
1. Snyder RL, de Melo-Abreu JP. Frost protection: fundamentals, practiceand economics vol. 1. Rome: Food and Agriculture Organization of theUnited Nations; 2005 (Environment and natural resources series).
2. Miranda C, Bilavcik A, Chaloupka R, Dreisiebner-Lanz S, Gastol M,
Luedeling E, et al. Phenology and critical temperatures. 2019. (EIP-AGRI
Focus Group Protecting fruit production from frost damage). Report No.:
5. https://​ec.​europa.​eu/​eip/​agric​ulture/​sites/​agri-​eip/​files/​fg30_​mp5_​
pheno​logy_​criti​cal_​tempe​ratur​es.​pdf. Accessed 15 Feb 2021.
3. WAPA. European apple and pear crop forecast. Brussels: World Appel andPear Organisation; 2018.
4. Legave J-M, Guédon Y, Malagi G, El Yaacoubi A, Bonhomme M. Differentiated responses of apple tree floral phenology to global warming incontrasting climatic regions. Front Plant Sci. 2015;6:1054. https://​doi.​org/​
10.​3389/​fpls.​2015.​01054/​abstr​act.
5. Kunz A, Blanke MM. Effects of climate change on fruit tree physiology—based on 55 years of meteorological and phenological data at
Klein-Altendorf. Acta Hortic. 2016;1130:49–54.
6. Unterberger C, Brunner L, Nabernegg S, Steininger KW, Steiner AK, Stabentheiner E, et al. Spring frost risk for regional apple production under awarmer climate. PLoS ONE. 2018;13(7):e0200201.
7. Vitasse Y, Schneider L, Rixen C, Christen D, Rebetez M. Increase in the riskof exposure of forest and fruit trees to spring frosts at higher elevations in
Switzerland over the last four decades. Agric For Meteorol. 2018;248:60–9.

Drepper et al. Environmental Evidence

8.
9.
10.
11.
12.
13.
14.
15.

16.
17.
18.

19.
20.

21.
22.
23.

24.

25.

26.

27.
28.
29.
30.
31.

(2021) 10:32

Chmielewski F, Müller A, Küchler W. Climate changes and frost hazard for
fruit trees. Ann Meteorol. 2005;41(2):488–91.
Pfleiderer P, Menke I, Schleussner C-F. Increasing risks of apple tree frost
damage under climate change. Clim Change. 2019;157(3):515–25.
Sgubin G, Swingedouw D, Dayon G, García de Cortázar-Atauri I, Ollat N,
Pagé C, et al. The risk of tardive frost damage in French vineyards in achanging climate. Agric For Meteorol. 2018;250–251:226–42.
Chmielewski FM, Götz K-P, Weber KC, Moryson S. Climate change andspring frost damages for sweet cherries in Germany. Int J Biometeorol.
2018;62(2):217–28.
Meier M, Fuhrer J, Holzkämper A. Changing risk of spring frost damage in
grapevines due to climate change? A case study in the Swiss Rhone Valley. Int J Biometeorol. 2018. https://​doi.​org/​10.​1007/​s00484-​018-​1501-y.
Eccel E, Rea R, Caffarra A, Crisci A. Risk of spring frost to apple production
under future climate scenarios: the role of phenological acclimation. Int J
Biometeorol. 2009;53(3):273–86.
Hoffmann H, Rath T. Future bloom and blossom frost risk for malus
domestica considering climate model and impact model uncertainties.
PLoS ONE. 2013;8(10):e75033.
Molitor D, Caffarra A, Sinigoj P, Pertot I, Hoffmann L, Junk J. Late frost
damage risk for viticulture under future climate conditions: a case study
for the Luxembourgish winegrowing region. Aust J Grape Wine Res.
2014;20(1):160–8.
Hu Y, Asante EA, Lu Y, Mahmood A, Buttar NA, Yuan S. A review of air
disturbance technology for plant frost protection. Int J Agric Biol Eng.
2018;11(3):21–8.
Diedrichs A, Bromberg F, Dujovne D, Brun K, Watteyne T. Prediction of
frost events using machine learning and IoT sensing devices. IEEE Internet Things J. 2018;5(6):4589–97.
Hirschy M. Gel et grêle en viticulture et arboriculture - état des lieux desdispositifs de protection contre les aléas climatiques. Paris: Acta – lesinstituts techniques agricoles; 2020. p. 56. https://​hal.​archi​ves-​ouver​tes.​fr/​
hal-​02769​435/​docum​ent. Accessed 18 Aug 2020.
EIP-Agri Focus Group. Protecting fruit production from frost damage.
2019.
Smith LG, Somerset SM. Fruits of temperate climates|Commercial andDietary Importance. In: Caballero B, editor. Encyclopedia of food sciences and nutrition. 2nd ed. Oxford: Academic Press; 2003. p. 2753–61.
http://​www.​scien​cedir​ect.​com/​scien​ce/​artic​le/​pii/​B0122​27055​X0053​07.
Accessed 20 Nov 2020.
Kottek M, Grieser J, Beck C, Rudolf B, Rubel F. World Map of the KöppenGeiger climate classification updated. Meteorol Z. 2006;15(3):259–63.
FAO. FAOSTAT. 2021. http://​www.​fao.​org/​faost​at. Accessed 16 June 2021.
Haddaway NR, Macura B, Whaley P, Pullin AS. ROSES RepOrting standards
for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic
reviews and systematic maps. Environ Evid. 2018;7(1):7.
Collaboration for Environmental Evidence. Guidelines and standards forevidence synthesis in environmental management. In: Pullin AS, Frampton GK, Livoreil B, Petrokofsky G, editors. 2018. (Version 5.0). https://​envir​
onmen​talev​idence.​org/​infor​mation-​for-​autho​rs/. Accessed 4 June 2021.
Neimann Rasmussen L, Montgomery P. The prevalence of and factors
associated with inclusion of non-English language studies in Campbell
systematic reviews: a survey and meta-epidemiological study. Syst Rev.
2018;7(1):129.
Gusenbauer M, Haddaway NR. Which academic search systems are
suitable for systematic reviews or meta-analyses? Evaluating retrieval
qualities of Google Scholar, PubMed, and 26 other resources. Res Synth
Methods. 2020;11(2):181–217.
Haddaway NR. ROSES_flowchart: R package and Shiny app for generating
systematic review and map flow diagrams. Zenodo; 2020. https://​zenodo.​
org/​record/​42948​10. Accessed 11 Mar 2021.
The EndNote Team. EndNote. Philadelphia: Clarivate; 2013.
Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplicationof database search results for systematic reviews in EndNote. J Med Libr
Assoc JMLA. 2016;104(3):240–3.
van der Mierden S, Tsaioun K, Bleich A, Leenaars CHC. Software tools forliterature screening in systematic reviews in biomedical research. ALTEX
Altern Anim Exp. 2019;36(3):508–17.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.

Page 13 of 13

32. Bilotta GS, Milner AM, Boyd IL. Quality assessment tools for evidence fromenvironmental science. Environ Evid. 2014;3(1):14.
33. OECD. Test No. 509: Crop Field Trial. OECD; 2009 (OECD Guidelines for
the Testing of Chemicals, Section 5). https://​www.​oecd-​ilibr​ary.​org/​envir​
onment/​test-​no-​509-​crop-​field-​trial_​97892​64076​457-​en. Accessed 13
Feb 2021.
34. Vanhoutte B, Remy S. FROSTinno: Innovative and energy efficient frost
control in fruit growing. pcfruit. https://​www.​pcfru​it.​be/​en/​frost​inno-​
innov​ative-​and-​energy-​effic​ient-​frost-​contr​ol-​fruit-​growi​ng. Accessed 15
Feb 2021.
35. Skovlin JM, Beaudette DE, Roecker SM, Brown AG. Soil database interface.
2021.
36. Earth Resources Observation and Science (EROS) Center. Shuttle radar
topography mission (SRTM) 1 arc-second global. U.S. Geological Survey;
2017. https://​www.​usgs.​gov/​cente​rs/​eros/​scien​ce/​usgs-​eros-​archi​ve-​
digit​al-​eleva​tion-​shutt​le-​radar-​topog​raphy-​missi​on-​srtm-1-​arc?​qt-​scien​
ce_​center_​objec​ts=0#​qt-​scien​ce_​center_​objec​ts. Accessed 14 June
2021.
37. Haddaway NR, Feierman A, Grainger MJ, Gray CT, Tanriver-Ayder E,
Dhaubanjar S, et al. EviAtlas: a tool for visualising evidence synthesis
databases. Environ Evid. 2019;8(1):22.
38. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction tometa-analysis. Chichester: Wiley; 2009. https://​doi.​org/​10.​1002/​97804​
70743​386.
39. Higgins JPT, Tianjing L, Deeks JJ, editors. Chapter 6: Choosing effect measures and computing estimates of effect. In: version 6.2 (updated February2021). Cochrane; 2021. (Higgins JPT, Thomas J, Chandler J, Cumpston M,
Li T, Page M, et al., editors. Cochrane Handbook for Systematic Reviews ofInterventions version 6.2 (updated February 2021)). https://​train​ing.​cochr​
ane.​org/​handb​ook/​curre​nt/​chapt​er-​06. Accessed 14 Aug 2021.
40. Sterne JA, Egger M. Funnel plots for detecting bias in meta-analysis:
guidelines on choice of axis. J Clin Epidemiol. 2001;54(10):1046–55.

Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Ready to submit your research ? Choose BMC and benefit from:• fast, convenient online submission• thorough peer review by experienced researchers in your field• rapid publication on acceptance• support for research data, including large and complex data types• gold Open Access which fosters wider collaboration and increased citations• maximum visibility for your research: over 100M website views per year
At BMC, research is always in progress.
Learn more biomedcentral.com/submissions