Input: How are the relationships between catch per set and fishing behavior variables different for different measures of catch per unit effort (CPUE)?

Context:
Overfishing is a major threat to the survival of shark species, primarily driven by international trade in high-value fins, as well as meat, liver oil, skin and cartilage. The Convention on the International Trade in Endangered Species of Wild Fauna and Flora (CITES) aims to ensure that commercial trade does not threaten wild species, and several shark species have recently been listed on CITES as part of international efforts to ensure that trade does not threaten their survival. However, as international trade regulations alone will be insufficient to reduce overexploitation of sharks, they must be accompanied by practical fisheries management measures to reduce fishing mortality. To examine which management measures might be practical in the context of a targeted shark fishery, we collected data from 52 vessels across 595 fishing trips from January 2014 to December 2015 at Tanjung Luar fishing port in East Lombok, Indonesia. We recorded 11,920 landed individuals across 42 species, a high proportion of which were threatened and regulated species. Catch per unit effort depended primarily on the number of hooks and type of fishing gear used, and to a lesser degree on month, boat engine power, number of sets and fishing ground. The most significant factors influencing the likelihood of catching threatened and regulated species were month, fishing ground, engine power and hook number. We observed significant negative relationships between standardised catch per unit effort and several indicators of fishing effort, suggesting diminishing returns above relatively low levels of fishing effort. Our results suggest that management measures focusing on fishing effort controls, gear restrictions and modifications and spatiotemporal closures could have significant benefits for the conservation of shark species, and may help to improve the overall sustainability of the Tanjung Luar shark fishery. These management measures may also be applicable to shark fisheries in other parts of Indonesia and beyond, as sharks increasingly become the focus of global conservation efforts.
Copyright: © 2018 Yulianto et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: Data collection of this study was funded by the Darwin Initiative (grant number 2805). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Overfishing is the greatest global threat to marine fish stocks [1–5]. Several shark species (Selachimorpha) are particularly vulnerable to overexploitation due to their conservative life history strategies, large body sizes and the high economic value of their preserved body parts [6–8]. With increasing fishing pressure in recent decades, primarily driven by international demand for a range of consumer goods (including fins, liver oil, skin, cartilage and meat), it is estimated that annual fishing mortality now exceeds the intrinsic rebound potential of most commercially exploited species [5, 9, 10]. This fishing pressure is taking its toll, with an estimated one in four Chondrichthyan species now threatened with extinction, making sharks amongst the most threatened species groups in the world .
It is also increasingly acknowledged that sharks play a critical role in maintaining functional and productive ocean ecosystems , as well as providing an important source of food and income for many coastal communities . Recognising both the plight and importance of shark populations, there is growing professional and public interest to improve shark conservation, and the management of shark fisheries and trade . This is reflected in several recent policy decisions to afford new international regulations for 12 species of sharks across seven genera under the Convention on the International Trade of Endangered Species of Wild Fauna and Flora (CITES). This is a promising step for shark conservation; however, in order to create tangible outcomes for species conservation CITES must be implemented through domestic measures that are adapted to national and local contexts.
Indonesia is the world’s largest shark fishing nation [9, 14], and a global priority for shark conservation . Until recently Indonesia’s shark fishery has largely functioned as de facto open-access [12, 16]. However, in the past five years the Indonesian government has demonstrated a clear commitment to shark conservation and resource management, with domestic measures put in place to implement international obligations under CITES . Exploitation of all CITES-listed species is now regulated, either through full species protection or export controls (these species are hereafter referred to as ‘regulated’ species). However, CITES only affords protection to a small number of Indonesia’s 112 known shark species , of which 83 are threatened with extinction according to the IUCN Red List of Threatened Species (i.e. Vulnerable (VU), Endangered (EN) or Critically Endangered (CR) , these species are hereafter referred to as ‘threatened’ species), many of which continue to be landed throughout the country . Further, these policy measures predominantly regulate trade at the point of export, but do not necessarily influence fisher behaviour or local demand at the point of catch, such that the ‘trickle-down’ impacts on species mortality are unknown. In addition, effectively implementing species-specific shark mortality controls remains challenging due to the non-selectivity of fishing gears, and practical and cultural barriers to changing fisher preferences for certain gear-types and fishing methods. As such, existing regulations alone (e.g. Indonesian Law on Fisheries 31/2004 and its derivative regulations) will likely be insufficient to curb mortality of threatened and regulated species, as fishers must be both willing and able to change their fishing behaviour . Moreover, most of Indonesia’s shark fisheries are small-scale, and in relatively poor coastal communities where there are often no legal, sustainable marine-based alternatives to shark fishing that offer similar financial returns [22, 23]. It is therefore imperative to consider the ethical and socioeconomic impacts of shark trade controls. Most shark species listed under CITES are listed on Appendix II, which is designed for sustainable use. International trade is permitted for CITES Appendix II species provided it is non-detrimental to wild populations of the species, as proven through a scientific non-detriment finding (NDF) report and implemented through a system of export permits. However, in Indonesia there is currently a lack of species-specific trade data for conducting NDFs and setting sustainable export quotas, such that the Indonesian government has to introduce trade bans for these species in order to meet CITES obligations. With new CITES-listings for thresher sharks (Alopias spp.) and silky shark (Carcharhinus falciformis) recently coming into force, this is likely to have huge implications for Indonesia’s economically important shark industry, and the coastal communities depending on it. In order to balance conservation and socioeconomic objectives, robust management systems must be put in place that ensure and allow sustainable fishing and trade. This necessitates the identification of practical management measures that can reduce mortality of threatened and regulated species at the point of catch, and provide realistic options for fishers to effectively and measurably improve the sustainability of their fishing practices.
This study analyses two years of qualitative and quantitative data from one of Indonesia’s targeted shark fisheries in Tanjung Luar, West Nusa Tenggara Province. We outline the key characteristics of the fishery, including fishing behaviour and overall catch volumes and composition. We analyse the impacts of different fishing techniques, and present factors influencing overall catch per unit effort (CPUE) of individual shark fishing trips, as well as factors influencing the likelihood of catching threatened and regulated species. Finally, we discuss the implications of our findings, and provide practical recommendations for fisheries management measures, which can support CITES implementation for sharks and reduce the catch of threatened and regulated species, in Indonesia and beyond.
This work was conducted under a Memorandum of Understanding (MoU) and Technical Cooperation Agreement (TCA) between the Wildlife Conservation Society (WCS) and the Ministry of Environment and Forestry (MoEF), Ministry Marine Affairs and Fisheries (MMAF) and the Marine and Fisheries Agency (MFA) of West Nusa Tenggara Province. These documents were approved and signed by Sonny Partono (Director General of Conservation of Natural Resources and Ecosystem MoEF), Sjarief Widjaja (Secretary General MMAF), and Djoko Suprianto (Acting Head of MFA of West Nusa Tenggara Province). Due to this MoU and TCA no specific research permit was required. We collected data by measuring sharks that were already caught, dead, and landed by fishers in Tanjung Luar, with no incentives, compensation or specific requests for killing sharks for this study. WCS participates in the Conservation Initiative on Human Rights and the rules and guidelines of our Internal Review Board ensures that any research protects the rights of human subjects. We did not apply for an IRB permit for this study because our study design focused on collecting fish and fisheries data as opposed to personal socio-economic data. The FDGs and interviews were conducted to obtain early scoping information about fishing practices, and to establish protocols for more detailed fisheries data collection (as used in this study), and socio-economic data collection (as used in a later study (Lestari et al ), which underwent further ethical review due to the specific focus on human subjects).
Tanjung Luar, located in East Lombok, West Nusa Tenggara Province, Indonesia (Fig 1), is a landing site for one of Indonesia’s most well-known targeted shark fisheries. Tanjung Luar serves at least 1,000 vessels, and the majority of these are less than 10 gross tonnes (GT) in size . A group of specialised fishers operating from Tanjung Luar village and a neighbouring island, Gili Maringkik, specifically target sharks. Shark catch is landed in a dedicated auction facility at the Tanjung Luar port. The shark industry is well established in Tanjung Luar, with product processing facilities and trade connections to local, national and international markets. Research by Lestari et al.  indicates that the shark industry is significantly more profitable than non-shark fisheries in Tanjung Luar, particularly for boat owners. Strong patron-client relationships exist between boat owners and fishers, with shark fishers exhibiting high dependency on shark fishing, limited occupational diversity and low adaptive capacity for shifting into other fisheries .
Fig 1. Sharks landing monitoring site and fishing grounds of shark fishers that land at Tanjung Luar.
In January 2014 we conducted preliminary scoping research to better understand the operational and socioeconomic characteristics of Tanjung Luar’s shark fishery. During a three-week scoping visit a team of four trained Indonesian enumerators conducted semi-structured interviews and focus group discussions (FDGs) with fishers, boat owners and traders, alongside naturalistic observation in the field. Respondents were selected through purposive sampling, since the research was exploratory in nature and a priori sampling decisions were not possible . We conducted a total of 34 semi-structured interviews (S1 File) and four FDGs, which were attended by a total of 30 individuals. All interviews and discussions took place in Indonesian, with the help of a local enumerator who was fluent in the Tanjung Luar local dialect. Interviews took approximately 30 minutes, with no remuneration for participating. All respondents gave their full prior and informed consent before contributing to the research. During the interviews and FDGs we gathered information on number of boats, fishing gears used, fishing grounds, fishery operational characteristics, and shark supply chain, including estimated volumes and value of shark catch relative to other fisheries. We improved the accuracy of information on shark fishery characteristics and fishing behaviour through informal daily interactions and discussions with 131 shark fishers during our daily landings data collection and community engagement activities. More detailed socioeconomic data were collected in a full household survey in 2016, as outlined in Lestari et al. .
Shark landings data were collected by three experienced enumerators, who were trained in species identification and data collection methods during a two-day workshop and three weeks of field mentoring to ensure the accuracy of the data collected. Landings were recorded every morning at the Tanjung Luar shark auction facility where shark fishers usually landed dead sharks, from 5am to 10am from January 2014 to December 2015. The enumerators recorded data on catch composition and fishing behaviour (Table 1) from 52 different vessels across a total of 595 fishing trips. The enumerators also measured the weight of selected sharks to calculate biomass and length-weight relationship.
Table 1. Types of data collected on fishing behaviour and catch composition during daily landings data collection at Tanjung Luar.
From fishing behaviour and catch data we calculated the overall species composition of catch. We calculated catch per unit effort (CPUE) by number of individuals using both catch per set (hereafter CPUE per set) and catch per 100 hooks per set (hereafter standardised CPUE) [25,26]. This was deemed necessary since different vessels and gear-types systematically deploy different numbers of hooks, and standardised CPUE allows for a more meaningful comparison.
To understand factors influencing overall CPUE we log transformed CPUE per trip to fit a normal distribution, and fitted linear models (LMs) of CPUE per trip to fishing behaviour variables (Table 1). We considered all variables and used minimum AIC values with stepwise analysis of variance to identify the best fit and most significant influencing variables.
To inform the development of practical fisheries management measures (e.g. gear restrictions), we also specifically analysed differences in CPUE for surface and bottom longline gears employed in the fishery, using two-way ANOVAs.
Factors affecting catch of threatened and regulated species.
To identify variables influencing the catch of threatened and regulated species we conducted a two-step process. In the first step, we identified factors influencing the likelihood of catching any threatened/regulated species during a given fishing trip, by creating binary response variables for whether a threatened species had been caught during a trip (yes = 1, no = 0), and separately for whether a regulated species had been caught during a trip (yes = 1, no = 0). We then fitted generalised linear models (GLMs) with binomial errors to the binary response variables, separately for catch of threatened species and catch of regulated species. In the second step we identified variables that significantly influenced the CPUE of threatened species and the CPUE of regulated species, given that any were caught. We removed all records in which no threatened or regulated species were caught, log transformed standardised CPUE of threatened and regulated species, and fitted linear models (LMs) of standardised CPUE of threatened species and standardised CPUE to regulated species to fishing behaviour variables. Again, we considered all meaningful models and used minimum AIC values with stepwise analysis of variance to identify the best fit  and most significant influencing variables. This approach was necessary since catch of threatened and regulated species is zero-inflated, and creating binary response variables with a binomial error structure allowed for a simpler and more powerful statistical analysis. Note that we conducted two separate analyses, one for threatened species only and one for regulated species only, but used the same methods and process, as outlined above, for each analysis. We did not group threatened and protected species together, since although some species are both threatened and protected, this is not the case for all shark species landed in Tanjung Luar.
A total of 52 shark fishing vessels operate from Tanjung Luar, all of which are classified as small-scale according to the Indonesian Ministry of Marine Affairs and Fisheries (MMAF) vessel categorisation system, with <7GT capacity. These vessels are operated by approximately 150 highly-specialised shark fishers, from Tanjung Luar village and Gili Maringkik, who make up roughly 5% of the local fisher population. The shark industry is more profitable than non-shark fisheries, and shark fishers report high household dependency on shark resources, low occupational diversity, and limited capacity and aspirations to move into other fisheries or industries.
Surface and bottom longlines are used as the primary fishing gears to target sharks, with pelagic fish (e.g. Euthynnus spp., Rastrellinger spp.) used as bait. Surface and bottom longlines systematically vary in length, depth deployed, number of sets, number of hooks used, and soak times (Table 2). Gear types are typically associated with certain vessel types, and fishers–captain and crew—tend to exhibit preferences for specific gear types. Shark fishers also use gillnets and troll lines as secondary gears, to catch bait and opportunistically target other species, such as grouper, snapper, skipjack and mackerel tuna.
Table 2. Characteristics of surface and bottom longlines.
The shark fishing vessels can be divided into two broad categories according to fishing behaviour: larger vessels (≥14 m) with higher horsepower (HP) engines spend more time at sea than smaller vessels (≤12m) (p<0.001), and reach fishing grounds outside of West Nusa Tenggara. These vessels primarily fish in southern Sumbawa and Sumba Islands, however, they also reach as far as eastern Flores, Timor Island, and the Java Sea (Fig 1). Larger, higher HP vessels also tend to employ surface longlines (p<0.001), and since they spend more time at sea, have a higher number of sets per trip than smaller vessels (p<0.001). Smaller vessels (≤12 m) with smaller engines tend to remain in waters around West Nusa Tenggara only, carrying out shorter fishing trips using bottom longlines (Table 3).
Table 3. Characterisation of the different fishing vessels used to target sharks in Tanjung Luar.
During the study period we recorded shark catch from a total of 595 fishing trips. We recorded 11,678 individual sharks, with an average total catch of 963 individuals per month (SD ± 434) and 19.7 individuals per trip (SD ± 15.6). Standardised CPUE (per 100 hooks per set) ranged from 0.05 to 22.13 individuals, with an average of 0.96 and a mode of 0.20. Catch consisted of 42 different species from 18 families (Table 4). 22% of all landings were classified as threatened species (i.e. VU, EN, CR) according to the IUCN Red List of Threatened Species, and 73% were near threatened. Almost half (46.3%) of landings were regulated (i.e. CITES-listed) species. The most commonly caught species were silky shark (Carcharhinus falciformis), black tip shark (Carcharhinus limbatus) and scalloped hammerhead (Sphyrna lewini).
Table 4. Sharks species landed in Tanjung Luar from January 2014 –December 2015 (VU = Vulnerable, EN = Endangered, NT = Near Threatened, LC = Least Concern, NE = Not Evaluated (VU and EN classified as ‘threatened’ in this study); II = CITES Appendix II, N = Not CITES-listed (II species classified as ‘regulated’ in this study)).
Measures of CPUE for the Tanjung Luar shark fishery vary spatially and temporally, and with several aspects of fishing effort including gear type, hook number, engine power and number of sets. An initial comparison of average catch per trip and catch per set of the two major gear types, surface longline and bottom longline, indicates that CPUE of surface longlines was significantly higher than that of bottom longlines (ANOVA, p<0.001). CPUE (individuals per set) was also positively associated with number of hooks, engine power, and number of sets (Fig 2). However, these relationships are for unstandardised CPUE i.e. without controlling for number of hooks.
Plots of CPUE: Number of individuals per set (A) and number of individuals per 100 hooks per set (standardised CPUE) (B) by gear type (1), number of hooks (2), number of sets (3) and engine horsepower (4).
When controlling for hook number using standardised CPUE (individuals per 100 hooks per set) the relationships were reversed, with standardised CPUE of bottom longlines significantly higher than that of surface longlines (ANOVA, p<0.001; Fig 2). A similar pattern was observed when comparing relationships between CPUE (individuals per set) and standardised CPUE for other measures of fishing effort, including numbers of hooks, engine power and number of sets (Fig 2). There was a positive relationship between unstandardised CPUE (individuals per set) and number of hooks, number of sets and engine power, but a negative relationship between CPUE and these fishing behaviour variables when CPUE was standardised by hook number (individuals per 100 hooks per set).
The best fit LM of standardised CPUE indicated that the most significant factors influencing standardised CPUE were fishing gear and number of hooks (p<0.001). Month, engine power, number of sets and fishing ground were also identified as significant variables (Table 5), although there was considerable covariance between these factors. Standardised CPUE was significantly lower in January, and decreased with higher numbers of hooks, despite a higher total catch per trip and set (Fig 2).
Table 5. Analysis of variance for linear model of standardised CPUE (individuals per 100 hooks per set) data from Tanjung Luar; significant values (p<0.05) are given in bold.
Best fit GLMs indicated that the most significant factors influencing the likelihood of catching threatened species were month (January and November were significantly lower: p<0.001 and p<0.05, respectively) and fishing ground (Other (i.e. fishing grounds outside of WNTP and ENTP) was significantly higher: p<0.01). Significant factors associated with standardised CPUE of threatened species were number of hooks (p<0.001), fishing ground (other: p<0.001, ENTP p<0.05), engine power (p<0.001) and trip length (p<0.001) (Table 6 and Fig 3).
Plots of most significant factors affecting standardised CPUE (number of individuals per 100 hooks per set) of threatened species: a) hook number, b) fishing ground, c) engine power and d) trip length.
Analysis of variance for the best fit models of factors affecting: a) the likelihood of catching and the standardised CPUE of threatened species b) the likelihood of catching and the standardised CPUE of regulated species.
The most significant factors influencing the likelihood of catching regulated species were month (January was significantly lower: p<0.001), number of hooks (p<0.001) and engine power (<0.01). Significant factors associated with standardised CPUE of regulated species were number of hooks (p<0.001), fishing gear (<0.001), number of sets (p<0.001), engine power (p<0.01) and month (November and January: p<0.05) (Table 5 and Fig 4).
Plots of most significant factors affecting standardised CPUE (number of individuals per 100 hooks per set) of regulated species: a) hook number, b) gear type, c) number of sets.
Although Tanjung Luar’s targeted shark fishery is small in scale, considerable numbers of shark are landed, including a large proportion of threatened and regulated species. A key finding is that measures of CPUE, for all sharks and for threatened and regulated species, vary spatially and temporally, and with several aspects of fishing effort including gear type, hook number, engine power and number of sets. Moreover, the relationships between CPUE and fishing behaviour variables are different for different measures of CPUE (CPUE per trip, CPUE per set, CPUE per 100 hooks per set). This highlights the importance of using appropriate standardisation for meaningful comparisons of CPUE across different gears and vessel types, and has important implications for fisheries management.
Unstandardised CPUE (individuals per set) was significantly lower in January. This is during the west monsoon season, which is characterised by high rainfall and adverse conditions at sea for fishing. Unstandardised CPUE was also significantly lower in West Nusa Tenggara Province (WNTP) than East Nusa Tenggara Province (ENTP) and other provinces, suggesting a lower abundance of sharks in this area. Engine power had a significant positive influence on unstandardised CPUE, and was also associated with longer trips and more sets, which was likely due to the ability of vessels with larger engines to travel longer distances, over longer time periods, and with higher numbers of sets, to favoured fishing grounds. Unstandardised CPUE was also significantly higher for surface longlines than bottom longlines. However, when standardising CPUE for the number of hooks (i.e. individuals per 100 hooks per set) this relationship was reversed. Bottom longlines exhibit a higher standardised CPUE, with negative relationships between catch per 100 hooks per set and number of hooks and frequency of sets. Vessels with moderate engine horsepower (50-59hp) also had the highest standardised CPUE. Since surface longlines systematically employ significantly more hooks than bottom longlines (400–600 vs 25–200 hooks), and tend to be associated with larger boats, longer trips and more sets, these findings suggest that although increasing fishing effort increased total catch for these gears and trips, there were diminishing returns of this increased effort above low to moderate levels.
A large proportion of Tanjung Luar’s shark catch consisted of threatened (22%) and regulated species (46%). Month is a significant factor in explaining standardised CPUE of both threatened and regulated species, which could indicate seasonal variation in the abundance of these species in the Tanjung Luar fishing grounds, or seasonal impacts on CPUE due to poor weather conditions. Fishing ground was a significant factor in explaining the catch of threatened species but not the catch in regulated species. This may be due to differences in range, distribution and relative abundance of species within these groups. Threatened species make up a relatively small proportion of Tanjung Luar’s catch in comparison to regulated species, which make up almost half of the catch (46%). As such, regulated species may generally be more abundant and spatially diffuse than threatened species, and therefore caught more uniformly across fishing grounds. For example, regulated species catch is dominated by silky sharks (Carcharhinus falciformis), which are circum-tropical and coastal-pelagic, and exhibit limited site-fidelity or aggregation behaviour, while threatened species catch is dominated by scalloped hammerheads (Sphyrna lewini), which are known to aggregate in schools. These schools of scalloped hammerheads may be more restricted to specific aggregation sites outside of WNTP and ENTP waters, while silky sharks are found in uniform abundance throughout fishing grounds.
As with CPUE of all catch, there was a positive relationship between unstandardised CPUE (catch per set) of threatened and regulated species and number of hooks, but a significant negative relationship between standardised CPUE (catch per 100 hooks per set). This was likely due to diminishing returns of adding additional hooks, and indicates that the effort for threatened and regulated species was exceeding maximum sustainable yield effort, such that increases in effort (e.g. hook number) were leading to decreases in catch [28–30].
Due to the profitability of the shark industry in Tanjung Luar, and limited adaptive capacity and willingness of shark fishers to move into other industries, it is necessary to identify practical and ethical management interventions that can improve the sustainability of the fishery whilst also mitigating the negative socio-economic consequences for coastal communities. Our findings indicate that spatiotemporal closures and restrictions on fishing effort could improve the overall catch per unit effort and sustainability of the Tanjung Luar shark fishery, and lead to positive conservation outcomes for priority species.
Since the location of shark fishing grounds plays a significant role in determining the likelihood of catching threatened species and their associated CPUE, improved marine spatial planning, with the identification of marine protected areas (MPAs) that protect critical shark habitat and shark populations, could reduce catch of species of conservation concern [31–33] and increase abundance of sharks [34, 35]. Provincial governments in West Papua and West Nusa Tenggara have already established ‘shark sanctuary’ MPAs, which protect critical shark habitat and ban shark fishing within their boundaries [16, 36], and monitoring data indicates positive impacts of shark-specific closures on shark abundance [37, 38]. Strengthening Indonesia’s existing MPA network for shark conservation, such as making all MPAs no-take zones for sharks and expanding spatial protection to critical shark habitat, including aggregation sites or pupping and nursery grounds for species of conservation concern, could have considerable conservation benefits. It should be noted, however, that MPAs may only be effective for certain species, such as those with small ranges or site-fidelity . More research is required to identify critical shark habitat and life history stages. For Tanjung Luar these efforts could focus on better understanding scalloped hammerhead (Sphyrna lewini) aggregation sites. Well-targeted spatial closures for this species could significantly reduce catch of threatened species in this fishery.
The relationships between gear type, several aspects of fishing effort (i.e. hook number, engine power, number of sets, trip length), standardised CPUE of all shark species and standardised CPUE of threatened and regulated species suggest that there is an optimal effort that could increase overall CPUE of the fishery and significantly reduce fishing mortality of species of conservation concern. For example, our data suggest that CPUE peaks with low to intermediate trip lengths and gear sets, intermediate engine power and hook numbers of less than 75 per set longline. Although standardised CPUE of threatened and regulated species is also higher when fewer hooks are deployed, the catch per set and overall mortality is significantly lower. Regulations that control the number of hooks in combination with incentives for shark fishers to tightly manage the number of hooks they deploy could significantly reduce mortality of threatened and endangered species, maximise the overall CPUE of the fishery, and reduce operational costs for fishers, making shark fishing in Tanjung Luar more sustainable and more cost effective [39–41].
Acknowledging that almost half of Tanjung Luar’s shark catch consists of CITES-listed species, developing measures that ensure both the sustainability of the fishery, and full traceability and control of onward trade, will be crucial for implementing CITES . The Indonesian government has demonstrated a strong commitment to regulating shark trade and implementing CITES [17–18], as demonstrated through several policy decisions to confer full and partial protection to CITES-listed shark and ray species (Marine Affairs and Fisheries Ministerial Decree No 4./KEPMEN-KP/2014, Regulation No. 48/PERMEN-KP/2016). This includes zero quotas/export bans for hammerhead and oceanic whitetip sharks. However, these export bans should be considered intermediate policy measures as monitoring systems and data availability are improved, and sustainable quotas are established. This will be challenging, as shark products are often traded in large volumes of fresh and/or preserved body parts, with high morphological similarity between products from regulated species and non-regulated species. To guarantee that trade is not detrimental to the survival of species, sustainable fisheries management will need to be complemented with species-specific trade quotas. This will require catch documentation systems which trace shark products from point of catch to point of export and rapid, low-cost species identification methods.
As baseline data on shark population health are limited, and there is no standardised, fisheries-independent system for monitoring long-term changes in shark populations, indirect bio-indicators (e.g. endo- and ectoparasites, [43–45]) could help to elucidate the impact of management measures on fisheries and populations of wild species. In the future, shark conservation and fisheries management could benefit from long-term monitoring of agreed indices of population abundance and health status.
These lessons may also apply to shark fisheries in other parts of the world. As sharks increasingly become the focus of global conservation efforts it should be acknowledged that species protection alone will not be enough to reduce mortality of priority species. More needs to be done to identify practical fisheries management measures that can reduce pressure on the most vulnerable species and populations, but also support sustainable use of species that are less susceptible to overfishing. Shark fishing forms an integral part of the livelihood strategies of many coastal communities [22, 23], and prohibiting catches will not necessarily lead to positive conservation outcomes [21, 46]. Management interventions must take into account local context and the motivations and well-being of fisher communities in order to be ethical, feasible and impactful.
S1 Dataset. Data of landed sharks at Tanjung Luar auction that had been used for this study.
S1 File. Questionnaires have been used to interview shark fishers, collector, traders, and processors.
We wish to acknowledge the support provided by fishers in Tanjung Luar for their great cooperation during fieldwork. We also thank I Made Dharma Aryawan, Muhsin, Abdul Kohar, and Abdurrafik for their assistance during field research, Benaya M Simeon, Peni Lestari, and Siska Agustina for helping with data processing, Ken Kassem for carefully reading the manuscript and providing useful inputs, and the anonymous reviewers for their constructive comments.
3. Hutchings JA, Reynolds JD. Marine fish population collapses: consequences for recovery and extinction risk. AIBS Bulletin. 2004 Apr;54(4):297–309.
4. Costello C, Ovando D, Clavelle T, Strauss CK, Hilborn R, Melnychuk MC, et al. Global fishery prospects under contrasting management regimes. Proceedings of the national academy of sciences. 2016 May 3;113(18):5125–9.
5. Davidson LN, Krawchuk MA, Dulvy NK. Why have global shark and ray landings declined: improved management or overfishing?. Fish and Fisheries. 2016 Jun 1;17(2):438–58.
6. Stevens JD, Bonfil R, Dulvy NK, Walker PA. The effects of fishing on sharks, rays, and chimaeras (chondrichthyans), and the implications for marine ecosystems. ICES Journal of Marine Science. 2000 Jun 1;57(3):476–94.
9. Dent F, Clarke S. State of the global market for shark products. FAO Fisheries and Aquaculture Technical Paper (FAO) eng no. 590. 2015.
12. Christensen J, Tull M, editors. Historical perspectives of fisheries exploitation in the Indo-Pacific. Springer Science & Business Media; 2014 Apr 1.
13. Simpfendorfer CA, Heupel MR, White WT, Dulvy NK. The importance of research and public opinion to conservation management of sharks and rays: a synthesis. Marine and Freshwater Research. 2011 Jul 21;62(6):518–27.
14. Lack M, Sant G. The future of sharks: a review of action and inaction. TRAFFIC International and the Pew Environment Group. 2011 Jan:44.
15. Bräutigam A, Callow M, Campbell IR, Camhi MD, Cornish AS, Dulvy NK, et al. Global priorities for conserving sharks and rays: A 2015–2025 strategy. The Global Sharks and Rays Initiative; 2015. 27p.
16. Satria A, Matsuda Y. Decentralization of fisheries management in Indonesia. Marine Policy. 2004 Sep 30;28(5):437–50.
17. Dharmadi , Fahmi , Satria F. Fisheries management and conservation of sharks in Indonesia. African journal of marine science. 2015 Apr 3;37(2):249–58.
20. Sembiring A, Pertiwi NP, Mahardini A, Wulandari R, Kurniasih EM, Kuncoro AW, Cahyani ND, Anggoro AW, Ulfa M, Madduppa H, Carpenter KE. DNA barcoding reveals targeted fisheries for endangered sharks in Indonesia. Fisheries Research. 2015 Apr 30;164:130–4.
21. Clarke S. Re-examining the shark trade as a tool for conservation. SPC Fisheries Newsletter. 2014:49–56.
22. Jaiteh VF, Loneragan NR, Warren C. The end of shark finning? Impacts of declining catches and fin demand on coastal community livelihoods. Marine Policy. 2017 Mar 24.
24. Cohen D, Crabtree B. Qualitative research guidelines project. Robert Wood Johnson Foundation, Princeton. 2006 Available from: http://www.qualres.org/index.html Cited in August 2016.
25. Skud BE. Manipulation of fixed gear and the effect on catch-per-unit effort. FAO Fisheries Report (FAO). 1984.
26. Damalas D, Megalofonou P, Apostolopoulou M. Environmental, spatial, temporal and operational effects on swordfish (Xiphias gladius) catch rates of eastern Mediterranean Sea longline fisheries. Fisheries Research. 2007 Apr 30;84(2):233–46.
27. Burnham KP, Anderson DR. Model selection and multimodel inference: a practical information-theoretic approach. Springer Science & Business Media; 2003 Dec 4.
28. Schaefer MB. Some aspects of the dynamics of populations important to the management of the commercial marine fisheries. Inter-American Tropical Tuna Commission Bulletin. 1954;1(2):23–56.
29. Fox WW Jr. An exponential surplus-yield model for optimizing exploited fish populations. Transactions of the American Fisheries Society. 1970 Jan 1;99(1):80–8.
30. Purwanto P, Nugroho D, Suwarso S. Potential production of the five predominant small pelagic fish species groups in the Java Sea. Indonesian Fisheries Research Journal. 2014 Dec 1;20(2):59–67.
31. Barker MJ, Schluessel V. Managing global shark fisheries: suggestions for prioritizing management strategies. Aquatic Conservation: Marine and Freshwater Ecosystems. 2005 Jul 1;15(4):325–47.
34. Ward-Paige CA, Worm B. Global evaluation of shark sanctuaries. Global Environmental Change. 2017 Nov 30;47:174–89.
35. Speed CW, Cappo M, Meekan MG. Evidence for rapid recovery of shark populations within a coral reef marine protected area. Biological Conservation. 2018 Apr 30;220:308–19.
36. West Nusa Tenggara Provincial Government. 2017. [Management and zoning plan of Lunyuk Marine Protected Area]. Mataram: West Nusa Tenggara Provincial Government; 2017. Indonesian.
37. Jaiteh VF, Lindfield SJ, Mangubhai S, Warren C, Fitzpatrick B, Loneragan NR. Higher abundance of marine predators and changes in fishers' behavior following spatial protection within the world's biggest shark fishery. Frontiers in Marine Science. 2016 Apr 7;3:43.
39. Kumoru L. The shark longline fishery in Papua New Guinea. InReport prepared for Billfish and bycatch research group, at the 176th meeting of the standing committee on Tuna and Billfish, Mooloolaba, Australia, 9th-16th July 2003 2003 Jul.
40. Cartamil D, Santana-Morales O, Escobedo-Olvera M, Kacev D, Castillo-Geniz L, Graham JB, Rubin RD, Sosa-Nishizaki O. The artisanal elasmobranch fishery of the Pacific coast of Baja California, Mexico. Fisheries Research. 2011 Mar 31;108(2):393–403.
42. Vincent AC, Sadovy de Mitcheson YJ, Fowler SL, Lieberman S. The role of CITES in the conservation of marine fishes subject to international trade. Fish and Fisheries. 2014 Dec 1;15(4):563–92.
43. Palm HW. Fish parasites as biological indicators in a changing world: can we monitor environmental impact and climate change?. InProgress in Parasitology 2011 (pp. 223–250). Springer Berlin Heidelberg.
44. Palm HW, Yulianto I, Piatkowski U. Trypanorhynch Assemblages Indicate Ecological and Phylogenetical Attributes of Their Elasmobranch Final Hosts. Fishes. 2017 Jun 17;2(2):8.
46. Booth H. Using the case of illegal manta ray trade in Indonesia to evaluate the impact of wildlife trade policy (Master Thesis, Imperial College London).
47. White WT, Last PR, Stevens JD, Yearsley GK. Economically important sharks & rays of Indonesia. Austr-alian Centre for International Agricultural Research (ACIAR); 2006.

Answer: The relationships between catch per set and fishing behavior variables differ when comparing unstandardized CPUE and standardized CPUE.


Input: How does the transition probability of the environment affect the learning rate in the static agent?

Context:
Paper Info

Title: Environmental variability and network structure determine the optimal plasticity mechanisms in embodied agents
Publish Date: Unkown
Author List: Sina Khajehabdollahi (from Department of Computer Science, University of Tübingen)

Figure

Figure2: An outline of the network controlling the foraging agent.The sensor layer receives inputs at each time step (the ingredients of the nearest food), which are processed by the plastic layer in the same way as the static sensory network, Fig.1.The output of that network is given as input to the motor network, along with the distance d and angle α to the nearest food, the current velocity v, and energy E of the agent.These signals are processed through two hidden layers to the final output of motor commands as the linear and angular acceleration of the agent
Figure4: The evolved parameters θ = (θ 1 , . . ., θ 8 ) of the plasticity rule for the reward prediction (a.) and the decision (b.) tasks, for a variety of parameters (p tr = 0.01, d e ∈ 0, 0.1, . . ., 1, and σ ∈ 0, 0.1, . . ., 1 in all 100 combinations).Despite the relatively small difference between the tasks, the evolved learning rules differ considerably.For visual guidance, the lines connect θs from the same run.
Figure5: a.The trajectory of an agent (blue line) in the 2D environment.A well-trained agent will approach and consume food with positive values (green dots) and avoid negative food (red dots).b.The learning rate of the plastic sensory network eta p grows with the distance between environments d e c. and decreases with the frequency of environmental change.d.The fitness of an agent (measured as the total food consumed over its lifetime) increases over generations of the EA for both the scalar and binary readouts in the sensory network.e.The Pearson correlation coefficient of an evolved agent's weights with the ingredient value vector of the current environment (E 1 -blue, E 2 -red).In this example, the agent's weights are anti-correlated with its environment, which is not an issue for performance since the motor network can interpret the inverted signs of food.

abstract

The evolutionary balance between innate and learned behaviors is highly intricate, and different organisms have found different solutions to this problem. We hypothesize that the emergence and exact form of learning behaviors is naturally connected with the statistics of environmental fluctuations and tasks an organism needs to solve.
Here, we study how different aspects of simulated environments shape an evolved synaptic plasticity rule in static and moving artificial agents. We demonstrate that environmental fluctuation and uncertainty control the reliance of artificial organisms on plasticity. Interestingly, the form of the emerging plasticity rule is additionally determined by the details of the task the artificial organisms are aiming to solve.
Moreover, we show that coevolution between static connectivity and interacting plasticity mechanisms in distinct sub-networks changes the function and form of the emerging plasticity rules in embodied agents performing a foraging task. One of the defining features of living organisms is their ability to adapt to their environment and incorporate new information to modify their behavior.
It is unclear how the ability to learn first evolved , but its utility appears evident. Natural environments are too complex for all the necessary information to be hardcoded genetically and more importantly, they keep changing during an organism's lifetime in ways that cannot be anticipated ; . The link between learning and environmental uncertainty and fluctuation has been extensively demonstrated in both natural ; , and artificial environments .
Nevertheless, the ability to learn does not come without costs. For the capacity to learn to be beneficial in evolutionary terms, a costly nurturing period is often required, a phenomenon observed in both biological , and artificial organisms . Additionally, it has been shown that in some complex environments, hardcoded behaviors may be superior to learned ones given limits in the agent's lifetime and envi-ronmental uncertainty ; ; .
The theoretical investigation of the optimal balance between learned and innate behaviors in natural and artificial systems goes back several decades. However, it has recently found also a wide range of applications in applied AI systems ; . Most AI systems are trained for specific tasks, and have no need for modification after their training has been completed.
Still, technological advances and the necessity to solve broad families of tasks make discussions about life-like AI systems relevant to a wide range of potential application areas. Thus the idea of open-ended AI agents that can continually interact with and adapt to changing environments has become particularly appealing.
Many different approaches for introducing lifelong learning in artificial agents have been proposed. Some of them draw direct inspiration from actual biological systems ; . Among them, the most biologically plausible solution is to equip artificial neural networks with some local neural plasticity , similar to the large variety of synaptic plasticity mechanisms ; ; that performs the bulk of the learning in the brains of living organisms .
The artificial plasticity mechanisms can be optimized to modify the connectivity of the artificial neural networks toward solving a particular task. The optimization can use a variety of approaches, most commonly evolutionary computation. The idea of meta-learning or optimizing synaptic plasticity rules to perform specific functions has been recently established as an engineering tool that can compete with stateof-the-art machine learning algorithms on various complex tasks ; ; Pedersen and Risi (2021); .
Additionally, it can be used to reverse engineer actual plasticity mechanisms found in biological neural networks and uncover their functions ; . Here, we study the effect that different factors (environ-arXiv:2303.06734v1 [q-bio.NC] 12 Mar 2023 mental fluctuation and reliability, task complexity) have on the form of evolved functional reward-modulated plasticity rules.
We investigate the evolution of plasticity rules in static, single-layer simple networks. Then we increase the complexity by switching to moving agents performing a complex foraging task. In both cases, we study the impact of different environmental parameters on the form of the evolved plasticity mechanisms and the interaction of learned and static network connectivity.
Interestingly, we find that different environmental conditions and different combinations of static and plastic connectivity have a very large impact on the resulting plasticity rules. We imagine an agent who must forage to survive in an environment presenting various types of complex food particles. Each food particle is composed of various amounts and combinations of N ingredients that can have positive (food) or negative (poison) values.
The value of a food particle is a weighted sum of its ingredients. To predict the reward value of a given resource, the agent must learn the values of these ingredients by interacting with the environment. The priors could be generated by genetic memory, but the exact values are subject to change. To introduce environmental variability, we stochastically change the values of the ingredients.
More precisely, we define two ingredient-value distributions E 1 and E 2 and switch between them, with probability p tr for every time step. We control how (dis)similar the environments are by parametrically setting E 2 = (1 − 2d e )E 1 , with d e ∈ [0, 1] serving as a distance proxy for the environments; when d e = 0, the environment remains unchanged, and when d e = 1 the value of each ingredient fully reverses when the environmental transition happens.
For simplicity, we take values of the ingredients in E 1 equally spaced between -1 and 1 (for the visualization, see Fig. ). The static agent receives passively presented food as a vector of ingredients and can assess its compound value using the linear summation of its sensors with the (learned or evolved) weights, see Fig. .
The network consists of N sensory neurons that are projecting to a single post-synaptic neuron. At each time step, an input X t = (x 1 , . . . , x N ) is presented, were the value x i , i ∈ {1, . . . , N } represents the quantity of the ingredient i. We draw x i independently form a uniform distribution on the [0, 1] interval (x i ∼ U (0, 1)).
The value of each ingredient w c i is determined by the environment (E 1 or E 2 ). The postsynaptic neuron outputs a prediction of the food X t value as y t = g(W X T t ). Throughout the paper, g will be either the identity function, in which case the prediction neuron is linear, or a step-function; however, it could be any other nonlinearity, such as a sigmoid or ReLU.
After outputting the prediction, the neuron receives feedback in the form of the real value of the input R t . The real value is computed as R t = W c X T t + ξ, where W c = (w c 1 , . . . , w c N ) is the actual value of the ingredients, and ξ is a term summarizing the noise of reward and sensing system ξ ∼ N (0, σ).
Figure : An outline of the static agent's network. The sensor layer receives inputs representing the quantity of each ingredient of a given food at each time step. The agent computes the prediction of the food's value y t and is then given the true value R t ; it finally uses this information in the plasticity rule to update the weight matrix.
For the evolutionary adjustment of the agent's parameters, the loss of the static agent is the sum of the mean squared errors (MSE) between its prediction y t and the reward R t over the lifetime of the agent. The agent's initial weights are set to the average of the two ingredient value distributions, which is the optimal initial value for the case of symmetric switching of environments that we consider here.
As a next step, we incorporate the sensory network of static agents into embodied agents that can move around in an environment scattered with food. To this end, we merge the static agent's network with a second, non-plastic motor network that is responsible for controlling the motion of the agent in the environment.
Specifically, the original plastic network now provides the agent with information about the value of the nearest food. The embodied agent has additional sensors for the distance from the nearest food, the angle between the current velocity and the nearest food direction, its own velocity, and its own energy level (sum of consumed food values).
These inputs are processed by two hidden layers (of 30 and 15 neurons) with tanh activation. The network's outputs are angular and linear acceleration, Fig. . The embodied agents spawn in a 2D space with periodic boundary conditions along with a number of food particles that are selected such that the mean of the food value distribution is ∼ 0. An agent can eat food by approaching it sufficiently closely, and each time a food particle is eaten, it is The sensor layer receives inputs at each time step (the ingredients of the nearest food), which are processed by the plastic layer in the same way as the static sensory network, Fig. .
The output of that network is given as input to the motor network, along with the distance d and angle α to the nearest food, the current velocity v, and energy E of the agent. These signals are processed through two hidden layers to the final output of motor commands as the linear and angular acceleration of the agent re-spawned with the same value somewhere randomly on the grid (following the setup of ).
After 5000 time steps, the cumulative reward of the agent (the sum of the values of all the food it consumed) is taken as its fitness. During the evolutionary optimization, the parameters for both the motor network (connections) and plastic network (learning rule parameters) are co-evolved, and so agents must simultaneously learn to move and discriminate good/bad food.
Reward-modulated plasticity is one of the most promising explanations for biological credit assignment . In our network, the plasticity rule that updates the weights of the linear sensor network is a rewardmodulated rule which is parameterized as a linear combination of the input, the output, and the reward at each time step:
Additionally, after each plasticity step, the weights are normalized by mean subtraction, an important step for the stabilization of Hebbian-like plasticity rules . We use a genetic algorithm to optimize the learning rate η p and amplitudes of different terms θ = (θ 1 , . . . , θ 8 ). The successful plasticity rule after many food presentations must converge to a weight vector that predicts the correct food values (or allows the agent to correctly decide whether to eat a food or avoid it).
To have comparable results, we divide θ = (θ 1 , . . . , θ 8 ) by We then multiply the learning rate η p with θ max to maintain the rule's evolved form unchanged, η norm p = η p • θ max . In the following, we always use normalized η p and θ, omitting norm . To evolve the plasticity rule and the moving agents' motor networks, we use a simple genetic algorithm with elitism .
The agents' parameters are initialized at random (drawn from a Gaussian distribution), then the sensory network is trained by the plasticity rule and finally, the agents are evaluated. After each generation, the bestperforming agents (top 10 % of the population size) are selected and copied into the next generation.
The remaining 90 % of the generation is repopulated with mutated copies of the best-performing agents. We mutate agents by adding independent Gaussian noise (σ = 0.1) to its parameters. To start with, we consider a static agent whose goal is to identify the value of presented food correctly. The static reward-prediction network quickly evolves the parameters of the learning rule, successfully solving the prediction task.
We first look at the evolved learning rate η p , which determines how fast (if at all) the network's weight vector is updated during the lifetime of the agents. We identify three factors that control the learning rate parameter the EA converges to: the distance between the environments, the noisiness of the reward, and the rate of environmental transition.
The first natural factor is the distance d e between the two environments, with a larger distance requiring a higher learning rate, Fig. . This is an expected result since the convergence time to the "correct" weights is highly dependent on the initial conditions. If an agent is born at a point very close to optimality, which naturally happens if the environments are similar, the distance it needs to traverse on the fitness landscape is small.
Therefore it can afford to have a small learning rate, which leads to a more stable convergence and is not affected by noise. A second parameter that impacts the learning rate is the variance of the rewards. The reward an agent receives for the plasticity step contains a noise term ξ that is drawn from a zero mean Gaussian distribution with standard deviation σ.
This parameter controls the unreliability of the agent's sensory system, i.e., higher σ means that the information the agent gets about the value of the foods it consumes cannot be fully trusted to reflect the actual value of the foods. As σ increases, the learning rate η p decreases, which means that the more unreliable an environment becomes, the less an agent relies on plasticity to update its weights, Fig. .
Indeed for some combinations of relatively small distance d e and high reward variance σ, the EA converges to a learning rate of η p ≈ 0. This means that the agent opts to have no adaptation during its lifetime and remain at the mean of the two environments. It is an optimal solution when the expected loss due to ignoring the environmental transitions is, on average, lower than the loss the plastic network will incur by learning via the (often misleading because of the high σ) environmental cues.
A final factor that affects the learning rate the EA will converge to is the frequency of environmental change during an agent's lifetime. Since the environmental change is modeled as a simple, two-state Markov process (Fig. ), the control parameter is the transition probability p tr . When keeping everything else the same, the learning rate rapidly rises as we increase the transition probability from 0, and after reaching a peak, it begins to decline slowly, eventually reaching zero (Fig. ).
This means that when environmental transition is very rare, agents opt for a very low learning rate, allowing a slow and stable convergence to an environment-appropriate weight vector that leads to very low losses while the agent remains in that environment. As the rate of environmental transition increases, faster learning is required to speed up convergence in order to exploit the (comparatively shorter) stays in each environment.
Finally, as the environmental transition becomes too fast, the agents opt for slower or even no learning, which keeps them ) and the decision (b.) tasks, for a variety of parameters (p tr = 0.01, d e ∈ 0, 0.1, . . . , 1, and σ ∈ 0, 0.1, . . . , 1 in all 100 combinations). Despite the relatively small difference between the tasks, the evolved learning rules differ considerably.
For visual guidance, the lines connect θs from the same run. near the middle of the two environments, ensuring that the average loss of the two environments is minimal (Fig. ). The form of the evolved learning rule depends on the task: Decision vs. Prediction The plasticity parameters θ = (θ 1 , . . . , θ 8 ) for the rewardprediction task converge on approximately the same point, regardless of the environmental parameters (Fig. ).
In particular, θ 3 → 1, θ 5 → −1, θ i → 0 for all other i, and thus the learning rule converges to: Since by definition y t = g(W t X T t ) = W t X T t (g(x) = x in this experiment) and R t = W c X T t + ξ we get: Thus the distribution of ∆W t converges to a distribution with mean 0 and variance depending on η p and σ and W converges to W c .
So this learning rule will match the agent's weight vector with the vector of ingredient values in the environment. We examine the robustness of the learning rule the EA discovers by considering a slight modification of our task. Instead of predicting the expected food value, the agent now needs to decide whether to eat the presented food or not.
This is done by introducing a step-function nonlinearity (g(x) = 1 if x ≥ 1 and 0 otherwise). Then the output y(t) is computed as: Instead of the MSE loss between prediction and actual value, the fitness of the agent is now defined as the sum of the food values it chose to consume (by giving y t = 1). Besides these two changes, the setup of the experiments remains exactly the same.
The qualitative relation between η p and parameters of environment d e , σ and p tr is preserved in the changed experiment. However, the resulting learning rule is significantly different (Fig. ). The evolution converges to the following learning rule: In both cases, the rule has the form ∆W t = η p X t [α y R t + β y ].
Thus, the ∆W t is positive or negative depending on whether the reward R t is above or below a threshold (γ = −β y /α y ) that depends on the output decision of the network (y t = 0 or 1). Both learning rules (for the reward-prediction and decision tasks) have a clear Hebbian form (coordination of preand post-synaptic activity) and use the incoming reward signal as a threshold.
These similarities indicate some common organizing principles of reward-modulated learning rules, but their significant differences highlight the sensitivity of the optimization process to task details. We now turn to the moving embodied agents in the 2D environment. To optimize these agents, both the motor network's connections and the sensory network's plasticity parameters evolve simultaneously.
Since the motor network is initially random and the agent has to move to find food, the number of interactions an agent experiences in its lifetime can be small, slowing down the learning. However, having the larger motor network also has benefits for evolution because it allows the output of the plastic network to be read out and transformed in different ways, resulting in a broad set of solutions.
The fitness of an agent (measured as the total food consumed over its lifetime) increases over generations of the EA for both the scalar and binary readouts in the sensory network. e. The Pearson correlation coefficient of an evolved agent's weights with the ingredient value vector of the current environment (E 1 -blue, E 2 -red).
In this example, the agent's weights are anti-correlated with its environment, which is not an issue for performance since the motor network can interpret the inverted signs of food. The agents can solve the task effectively by evolving a functional motor network and a plasticity rule that converges to interpretable weights (Fig. ).
After ∼ 100 evolutionary steps (Fig. ), the agents can learn the ingredient value distribution using the plastic network and reliably move towards foods with positive values while avoiding the ones with negative values. We compare the dependence of the moving and the static agents on the parameters of the environment: d e and the state transition probability p tr .
At first, in order to simplify the experiment, we set the transition probability to 0, but fixed the initial weights to be the average of E 1 and E 2 , while the real state is E 2 . In this experiment, the distance between states d e indicates twice the distance between the agent's initial weights and the optimal weights (the environment's ingredient values) since the agent is initialized at the mean of the two environment distributions.
Same as for the static agent, the learning rate increases with the distance d e (Fig. ). Then, we examine the effect of the environmental transition probability p tr on the evolved learning rate η p . In order for an agent to get sufficient exposure to each environment, we scale down the probability p tr from the equivalent experiment for the static agents.
We find that as the probability of transition increases, the evolved learning rate η p decreases (Fig. ). This fits with the larger trend for the static agent, although there is a clear difference when it comes to the increase for very small transition probabil-ities that were clearly identifiable in the static but not the moving agents.
This could be due to much sparser data and possibly the insufficiently long lifetime of the moving agent (the necessity of scaling makes direct comparisons difficult). Nevertheless, overall we see that the associations observed in the static agents between environmental distance d e and transition probability p tr and the evolved learning rate η p are largely maintained in the moving agents.
Still, more data would be needed to make any conclusive assertions about the exact effect of these environmental parameters on the emerging plasticity mechanisms. A crucial difference between the static and the moving agents is the function the plasticity has to perform. While in the static agents, the plasticity has to effectively identify the exact value distribution of the environment in order to produce accurate predictions, in the embodied agents, the plasticity has to merely produce a representation of the environment that the motor network can evolve to interpret adequately enough to make decisions about which food to consume.
To illustrate the difference, we plot the Pearson correlation coefficient between an agent's weights and the ingredient values of the environment it is moving in (Fig. ). We use the correlation instead of the MSE loss (which we used for the static agents in Fig. ) because the amplitude of the vector varies a lot for different agents and meaningful The evolved parameters of moving agents' plasticity rule for the g(s) = x, identity (a.) and the step function (Eq.
4) (b.) sensory networks (the environmental parameters here are d e ∈ [0, 1], σ = 0 and p tr = 0.001). The step function (binary output) network evolved a more structured plasticity rule (e.g., θ 3 > 0 for all realizations) than the linear network. Moreover, the learned weights for the identity network (c.) have higher variance and correlate significantly less with the environment's ingredient distribution compared to the learned weights for the thresholded network (d.)
conclusions cannot be drawn from the MSE loss. For many agents, the learned weights are consistently anti-correlated with the actual ingredient values (an example of such an agent is shown in Fig. ). This means that the output of the sensory network will have the opposite sign from the actual food value.
While in the static network, this would lead to very bad predictions and high loss, in the foraging task, these agents perform exactly as well as the ones where the weights and ingredients values are positively correlated, since the motor network can simply learn to move towards food for which it gets a negative instead of a positive sensory input.
This additional step of the output of the plastic network going through the motor network before producing any behavior has a strong effect on the plasticity rules that the embodied agents evolve. Specifically, if we look at the emerging rules the top performing agents have evolved (Fig. ), it becomes clear that, unlike the very well-structured rules of the static agents (Fig. ), there is now virtually no discernible pattern or structure.
The difference becomes even clearer if we look at the learned weights (at the end of a simulation) of the best-performing agents (Fig. ). While there is some correlation with the environment's ingredient value distribution, the variance is very large, and they do not seem to converge on the "correct" values in any way.
This is to some extent expected since, unlike the static agents where the network's output has to be exactly correct, driving the evolution of rules that converge to the precise environmental distribution, in the embodied networks, the bulk of the processing is done by the motor network which can evolve to interpret the scalar value of the sensory network's output in a variety of ways.
Thus, as long as the sensory network's plasticity rule co-evolves with the motor network, any plasticity rule that learns to produce consistent information about the value of encountered food can potentially be selected. To further test this assumption, we introduce a bottleneck of information propagation between the sensory and motor networks by using a step-function nonlinearity on the output of the sensory network (Eq.
4). Similarly to the decision task of the static network, the output of the sensory network now becomes binary. This effectively reduces the flow of information from the sensory to the motor network, forcing the sensory network to consistently decide whether food should be consumed (with the caveat that the motor network can still interpret the binary sign in either of two ways, either consuming food marked with 1 or the ones marked with 0 by the sensory network).
The agents perform equally well in this variation of the task as before (Fig. ), but now, the evolved plasticity rules seem to be more structured (Fig. ). Moreover, the variance of the learned weights in the bestperforming agents is significantly reduced (Fig. ), which indicates that the bottleneck in the sensory network is in-creasing selection pressure for rules that learn the environment's food distribution accurately.
We find that different sources of variability have a strong impact on the extent to which evolving agents will develop neuronal plasticity mechanisms for adapting to their environment. A diverse environment, a reliable sensory system, and a rate of environmental change that is neither too large nor too small are necessary conditions for an agent to be able to effectively adapt via synaptic plasticity.
Additionally, we find that minor variations of the task an agent has to solve or the parametrization of the network can give rise to significantly different plasticity rules. Our results partially extend to embodied artificial agents performing a foraging task. We show that environmental variability also pushes the development of plasticity in such agents.
Still, in contrast to the static agents, we find that the interaction of a static motor network with a plastic sensory network gives rise to a much greater variety of wellfunctioning learning rules. We propose a potential cause of this degeneracy; as the relatively complex motor network is allowed to read out and process the outputs from the plastic network, any consistent information coming out of these outputs can be potentially interpreted in a behaviorally useful way.
Reducing the information the motor network can extract from the sensory system significantly limits learning rule variability. Our findings on the effect of environmental variability concur with the findings of previous studies that have identified the constraints that environmental variability places on the evolutionary viability of learning behaviors.
We extend these findings in a mechanistic model which uses a biologically plausible learning mechanism (synaptic plasticity). We show how a simple evolutionary algorithm can optimize the different parameters of a simple reward-modulated plasticity rule for solving simple prediction and decision tasks.
Reward-modulated plasticity has been extensively studied as a plausible mechanism for credit assignment in the brain ; ; and has found several applications in artificial intelligence and robotics tasks ; . Here, we demonstrate how such rules can be very well-tuned to take into account different environmental parameters and produce optimal behavior in simple systems.
Additionally, we demonstrate how the co-evolution of plasticity and static functional connectivity in different subnetworks fundamentally changes the evolutionary pressures on the resulting plasticity rules, allowing for greater diversity in the form of the learning rule and the resulting learned connectivity.
Several studies have demonstrated how, in biological networks, synaptic plasticity heavily interacts with and is driven by network topology . Moreover, it has been recently demonstrated that biological plasticity mechanisms are highly redundant in the sense that any observed neural connectivity or recorded activity can be achieved with a variety of distinct, unrelated learning rules .
This observed redundancy of learning rules in biological settings complements our results and suggests that the function of plasticity rules cannot be studied independently of the connectivity and topology of the networks they are acting on. The optimization of functional plasticity in neural networks is a promising research direction both as a means to understand biological learning processes and as a tool for building more autonomous artificial systems.
Our results suggest that reward-modulated plasticity is highly adaptable to different environments and can be incorporated into larger systems that solve complex tasks. This work studies a simplified toy model of neural network learning in stochastic environments. Future work could be built on this basic framework to examine more complex reward distributions and sources of environmental variability.
Moreover, a greater degree of biological realism could be added by studying more plausible network architectures (multiple plastic layers, recurrent and feedback connections) and more sophisticated plasticity rule parametrizations. Additionally, our foraging simulations were constrained by limited computational resources and were far from exhaustive.
Further experiments can investigate environments with different constraints, food distributions, multiple seasons, more complex motor control systems and interactions of those systems with different sensory networks as well as the inclusion of plasticity on the motor parts of the artificial organisms.

Answer: As the transition probability increases, the learning rate initially rises and then declines.