prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: How are the relationships between catch per set and fishing behavior variables different for different measures of catch per unit effort (CPUE)?

Read the following text and answer briefly.

Context 1: Overfishing is a major threat to the survival of shark species, primarily driven by international trade in high-value fins, as well as meat, liver oil, skin and cartilage  The Convention on the International Trade in Endangered Species of Wild Fauna and Flora (CITES) aims to ensure that commercial trade does not threaten wild species, and several shark species have recently been listed on CITES as part of international efforts to ensure that trade does not threaten their survival  However, as international trade regulations alone will be insufficient to reduce overexploitation of sharks, they must be accompanied by practical fisheries management measures to reduce fishing mortality  To examine which management measures might be practical in the context of a targeted shark fishery, we collected data from 52 vessels across 595 fishing trips from January 2014 to December 2015 at Tanjung Luar fishing port in East Lombok, Indonesia  We recorded 11,920 landed individuals across 42 species, a high proportion of which were threatened and regulated species  Catch per unit effort depended primarily on the number of hooks and type of fishing gear used, and to a lesser degree on month, boat engine power, number of sets and fishing ground  The most significant factors influencing the likelihood of catching threatened and regulated species were month, fishing ground, engine power and hook number  We observed significant negative relationships between standardised catch per unit effort and several indicators of fishing effort, suggesting diminishing returns above relatively low levels of fishing effort  Our results suggest that management measures focusing on fishing effort controls, gear restrictions and modifications and spatiotemporal closures could have significant benefits for the conservation of shark species, and may help to improve the overall sustainability of the Tanjung Luar shark fishery  These management measures may also be applicable to shark fisheries in other parts of Indonesia and beyond, as sharks increasingly become the focus of global conservation efforts 
Copyright: © 2018 Yulianto et al  This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited 
Funding: Data collection of this study was funded by the Darwin Initiative (grant number 2805)  The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript 
Overfishing is the greatest global threat to marine fish stocks [1–5]  Several shark species (Selachimorpha) are particularly vulnerable to overexploitation due to their conservative life history strategies, large body sizes and the high economic value of their preserved body parts [6–8]  With increasing fishing pressure in recent decades, primarily driven by international demand for a range of consumer goods (including fins, liver oil, skin, cartilage and meat), it is estimated that annual fishing mortality now exceeds the intrinsic rebound potential of most commercially exploited species [5, 9, 10]  This fishing pressure is taking its toll, with an estimated one in four Chondrichthyan species now threatened with extinction, making sharks amongst the most threatened species groups in the world  
It is also increasingly acknowledged that sharks play a critical role in maintaining functional and productive ocean ecosystems , as well as providing an important source of food and income for many coastal communities   Recognising both the plight and importance of shark populations, there is growing professional and public interest to improve shark conservation, and the management of shark fisheries and trade   This is reflected in several recent policy decisions to afford new international regulations for 12 species of sharks across seven genera under the Convention on the International Trade of Endangered Species of Wild Fauna and Flora (CITES)  This is a promising step for shark conservation; however, in order to create tangible outcomes for species conservation CITES must be implemented through domestic measures that are adapted to national and local contexts 
Indonesia is the world’s largest shark fishing nation [9, 14], and a global priority for shark conservation   Until recently Indonesia’s shark fishery has largely functioned as de facto open-access [12, 16]  However, in the past five years the Indonesian government has demonstrated a clear commitment to shark conservation and resource management, with domestic measures put in place to implement international obligations under CITES   Exploitation of all CITES-listed species is now regulated, either through full species protection or export controls (these species are hereafter referred to as ‘regulated’ species)  However, CITES only affords protection to a small number of Indonesia’s 112 known shark species , of which 83 are threatened with extinction according to the IUCN Red List of Threatened Species (i e  Vulnerable (VU), Endangered (EN) or Critically Endangered (CR) , these species are hereafter referred to as ‘threatened’ species), many of which continue to be landed throughout the country   Further, these policy measures predominantly regulate trade at the point of export, but do not necessarily influence fisher behaviour or local demand at the point of catch, such that the ‘trickle-down’ impacts on species mortality are unknown  In addition, effectively implementing species-specific shark mortality controls remains challenging due to the non-selectivity of fishing gears, and practical and cultural barriers to changing fisher preferences for certain gear-types and fishing methods  As such, existing regulations alone (e g  Indonesian Law on Fisheries 31/2004 and its derivative regulations) will likely be insufficient to curb mortality of threatened and regulated species, as fishers must be both willing and able to change their fishing behaviour   Moreover, most of Indonesia’s shark fisheries are small-scale, and in relatively poor coastal communities where there are often no legal, sustainable marine-based alternatives to shark fishing that offer similar financial returns [22, 23]  It is therefore imperative to consider the ethical and socioeconomic impacts of shark trade controls  Most shark species listed under CITES are listed on Appendix II, which is designed for sustainable use  International trade is permitted for CITES Appendix II species provided it is non-detrimental to wild populations of the species, as proven through a scientific non-detriment finding (NDF) report and implemented through a system of export permits  However, in Indonesia there is currently a lack of species-specific trade data for conducting NDFs and setting sustainable export quotas, such that the Indonesian government has to introduce trade bans for these species in order to meet CITES obligations  With new CITES-listings for thresher sharks (Alopias spp ) and silky shark (Carcharhinus falciformis) recently coming into force, this is likely to have huge implications for Indonesia’s economically important shark industry, and the coastal communities depending on it  In order to balance conservation and socioeconomic objectives, robust management systems must be put in place that ensure and allow sustainable fishing and trade  This necessitates the identification of practical management measures that can reduce mortality of threatened and regulated species at the point of catch, and provide realistic options for fishers to effectively and measurably improve the sustainability of their fishing practices 
This study analyses two years of qualitative and quantitative data from one of Indonesia’s targeted shark fisheries in Tanjung Luar, West Nusa Tenggara Province  We outline the key characteristics of the fishery, including fishing behaviour and overall catch volumes and composition  We analyse the impacts of different fishing techniques, and present factors influencing overall catch per unit effort (CPUE) of individual shark fishing trips, as well as factors influencing the likelihood of catching threatened and regulated species  Finally, we discuss the implications of our findings, and provide practical recommendations for fisheries management measures, which can support CITES implementation for sharks and reduce the catch of threatened and regulated species, in Indonesia and beyond 
This work was conducted under a Memorandum of Understanding (MoU) and Technical Cooperation Agreement (TCA) between the Wildlife Conservation Society (WCS) and the Ministry of Environment and Forestry (MoEF), Ministry Marine Affairs and Fisheries (MMAF) and the Marine and Fisheries Agency (MFA) of West Nusa Tenggara Province  These documents were approved and signed by Sonny Partono (Director General of Conservation of Natural Resources and Ecosystem MoEF), Sjarief Widjaja (Secretary General MMAF), and Djoko Suprianto (Acting Head of MFA of West Nusa Tenggara Province)  Due to this MoU and TCA no specific research permit was required  We collected data by measuring sharks that were already caught, dead, and landed by fishers in Tanjung Luar, with no incentives, compensation or specific requests for killing sharks for this study  WCS participates in the Conservation Initiative on Human Rights and the rules and guidelines of our Internal Review Board ensures that any research protects the rights of human subjects  We did not apply for an IRB permit for this study because our study design focused on collecting fish and fisheries data as opposed to personal socio-economic data  The FDGs and interviews were conducted to obtain early scoping information about fishing practices, and to establish protocols for more detailed fisheries data collection (as used in this study), and socio-economic data collection (as used in a later study (Lestari et al ), which underwent further ethical review due to the specific focus on human subjects) 
Tanjung Luar, located in East Lombok, West Nusa Tenggara Province, Indonesia (Fig 1), is a landing site for one of Indonesia’s most well-known targeted shark fisheries  Tanjung Luar serves at least 1,000 vessels, and the majority of these are less than 10 gross tonnes (GT) in size   A group of specialised fishers operating from Tanjung Luar village and a neighbouring island, Gili Maringkik, specifically target sharks  Shark catch is landed in a dedicated auction facility at the Tanjung Luar port  The shark industry is well established in Tanjung Luar, with product processing facilities and trade connections to local, national and international markets  Research by Lestari et al   indicates that the shark industry is significantly more profitable than non-shark fisheries in Tanjung Luar, particularly for boat owners  Strong patron-client relationships exist between boat owners and fishers, with shark fishers exhibiting high dependency on shark fishing, limited occupational diversity and low adaptive capacity for shifting into other fisheries  
Fig 1  Sharks landing monitoring site and fishing grounds of shark fishers that land at Tanjung Luar 
In January 2014 we conducted preliminary scoping research to better understand the operational and socioeconomic characteristics of Tanjung Luar’s shark fishery  During a three-week scoping visit a team of four trained Indonesian enumerators conducted semi-structured interviews and focus group discussions (FDGs) with fishers, boat owners and traders, alongside naturalistic observation in the field  Respondents were selected through purposive sampling, since the research was exploratory in nature and a priori sampling decisions were not possible   We conducted a total of 34 semi-structured interviews (S1 File) and four FDGs, which were attended by a total of 30 individuals  All interviews and discussions took place in Indonesian, with the help of a local enumerator who was fluent in the Tanjung Luar local dialect  Interviews took approximately 30 minutes, with no remuneration for participating  All respondents gave their full prior and informed consent before contributing to the research  During the interviews and FDGs we gathered information on number of boats, fishing gears used, fishing grounds, fishery operational characteristics, and shark supply chain, including estimated volumes and value of shark catch relative to other fisheries  We improved the accuracy of information on shark fishery characteristics and fishing behaviour through informal daily interactions and discussions with 131 shark fishers during our daily landings data collection and community engagement activities  More detailed socioeconomic data were collected in a full household survey in 2016, as outlined in Lestari et al   
Shark landings data were collected by three experienced enumerators, who were trained in species identification and data collection methods during a two-day workshop and three weeks of field mentoring to ensure the accuracy of the data collected  Landings were recorded every morning at the Tanjung Luar shark auction facility where shark fishers usually landed dead sharks, from 5am to 10am from January 2014 to December 2015  The enumerators recorded data on catch composition and fishing behaviour (Table 1) from 52 different vessels across a total of 595 fishing trips  The enumerators also measured the weight of selected sharks to calculate biomass and length-weight relationship 
Table 1  Types of data collected on fishing behaviour and catch composition during daily landings data collection at Tanjung Luar 
From fishing behaviour and catch data we calculated the overall species composition of catch  We calculated catch per unit effort (CPUE) by number of individuals using both catch per set (hereafter CPUE per set) and catch per 100 hooks per set (hereafter standardised CPUE) [25,26]  This was deemed necessary since different vessels and gear-types systematically deploy different numbers of hooks, and standardised CPUE allows for a more meaningful comparison 
To understand factors influencing overall CPUE we log transformed CPUE per trip to fit a normal distribution, and fitted linear models (LMs) of CPUE per trip to fishing behaviour variables (Table 1)  We considered all variables and used minimum AIC values with stepwise analysis of variance to identify the best fit and most significant influencing variables 
To inform the development of practical fisheries management measures (e g  gear restrictions), we also specifically analysed differences in CPUE for surface and bottom longline gears employed in the fishery, using two-way ANOVAs 
Factors affecting catch of threatened and regulated species 
To identify variables influencing the catch of threatened and regulated species we conducted a two-step process  In the first step, we identified factors influencing the likelihood of catching any threatened/regulated species during a given fishing trip, by creating binary response variables for whether a threatened species had been caught during a trip (yes = 1, no = 0), and separately for whether a regulated species had been caught during a trip (yes = 1, no = 0)  We then fitted generalised linear models (GLMs) with binomial errors to the binary response variables, separately for catch of threatened species and catch of regulated species  In the second step we identified variables that significantly influenced the CPUE of threatened species and the CPUE of regulated species, given that any were caught  We removed all records in which no threatened or regulated species were caught, log transformed standardised CPUE of threatened and regulated species, and fitted linear models (LMs) of standardised CPUE of threatened species and standardised CPUE to regulated species to fishing behaviour variables  Again, we considered all meaningful models and used minimum AIC values with stepwise analysis of variance to identify the best fit  and most significant influencing variables  This approach was necessary since catch of threatened and regulated species is zero-inflated, and creating binary response variables with a binomial error structure allowed for a simpler and more powerful statistical analysis  Note that we conducted two separate analyses, one for threatened species only and one for regulated species only, but used the same methods and process, as outlined above, for each analysis  We did not group threatened and protected species together, since although some species are both threatened and protected, this is not the case for all shark species landed in Tanjung Luar 
A total of 52 shark fishing vessels operate from Tanjung Luar, all of which are classified as small-scale according to the Indonesian Ministry of Marine Affairs and Fisheries (MMAF) vessel categorisation system, with <7GT capacity  These vessels are operated by approximately 150 highly-specialised shark fishers, from Tanjung Luar village and Gili Maringkik, who make up roughly 5% of the local fisher population  The shark industry is more profitable than non-shark fisheries, and shark fishers report high household dependency on shark resources, low occupational diversity, and limited capacity and aspirations to move into other fisheries or industries 
Surface and bottom longlines are used as the primary fishing gears to target sharks, with pelagic fish (e g  Euthynnus spp , Rastrellinger spp ) used as bait  Surface and bottom longlines systematically vary in length, depth deployed, number of sets, number of hooks used, and soak times (Table 2)  Gear types are typically associated with certain vessel types, and fishers–captain and crew—tend to exhibit preferences for specific gear types  Shark fishers also use gillnets and troll lines as secondary gears, to catch bait and opportunistically target other species, such as grouper, snapper, skipjack and mackerel tuna 
Table 2  Characteristics of surface and bottom longlines 
The shark fishing vessels can be divided into two broad categories according to fishing behaviour: larger vessels (≥14 m) with higher horsepower (HP) engines spend more time at sea than smaller vessels (≤12m) (p<0 001), and reach fishing grounds outside of West Nusa Tenggara  These vessels primarily fish in southern Sumbawa and Sumba Islands, however, they also reach as far as eastern Flores, Timor Island, and the Java Sea (Fig 1)  Larger, higher HP vessels also tend to employ surface longlines (p<0 001), and since they spend more time at sea, have a higher number of sets per trip than smaller vessels (p<0 001)  Smaller vessels (≤12 m) with smaller engines tend to remain in waters around West Nusa Tenggara only, carrying out shorter fishing trips using bottom longlines (Table 3) 
Table 3  Characterisation of the different fishing vessels used to target sharks in Tanjung Luar 
During the study period we recorded shark catch from a total of 595 fishing trips  We recorded 11,678 individual sharks, with an average total catch of 963 individuals per month (SD ± 434) and 19 7 individuals per trip (SD ± 15 6)  Standardised CPUE (per 100 hooks per set) ranged from 0 05 to 22 13 individuals, with an average of 0 96 and a mode of 0 20  Catch consisted of 42 different species from 18 families (Table 4)  22% of all landings were classified as threatened species (i e  VU, EN, CR) according to the IUCN Red List of Threatened Species, and 73% were near threatened  Almost half (46 3%) of landings were regulated (i e  CITES-listed) species  The most commonly caught species were silky shark (Carcharhinus falciformis), black tip shark (Carcharhinus limbatus) and scalloped hammerhead (Sphyrna lewini) 
Table 4  Sharks species landed in Tanjung Luar from January 2014 –December 2015 (VU = Vulnerable, EN = Endangered, NT = Near Threatened, LC = Least Concern, NE = Not Evaluated (VU and EN classified as ‘threatened’ in this study); II = CITES Appendix II, N = Not CITES-listed (II species classified as ‘regulated’ in this study)) 
Measures of CPUE for the Tanjung Luar shark fishery vary spatially and temporally, and with several aspects of fishing effort including gear type, hook number, engine power and number of sets  An initial comparison of average catch per trip and catch per set of the two major gear types, surface longline and bottom longline, indicates that CPUE of surface longlines was significantly higher than that of bottom longlines (ANOVA, p<0 001)  CPUE (individuals per set) was also positively associated with number of hooks, engine power, and number of sets (Fig 2)  However, these relationships are for unstandardised CPUE i e  without controlling for number of hooks 
Plots of CPUE: Number of individuals per set (A) and number of individuals per 100 hooks per set (standardised CPUE) (B) by gear type (1), number of hooks (2), number of sets (3) and engine horsepower (4) 
When controlling for hook number using standardised CPUE (individuals per 100 hooks per set) the relationships were reversed, with standardised CPUE of bottom longlines significantly higher than that of surface longlines (ANOVA, p<0 001; Fig 2)  A similar pattern was observed when comparing relationships between CPUE (individuals per set) and standardised CPUE for other measures of fishing effort, including numbers of hooks, engine power and number of sets (Fig 2)  There was a positive relationship between unstandardised CPUE (individuals per set) and number of hooks, number of sets and engine power, but a negative relationship between CPUE and these fishing behaviour variables when CPUE was standardised by hook number (individuals per 100 hooks per set) 
The best fit LM of standardised CPUE indicated that the most significant factors influencing standardised CPUE were fishing gear and number of hooks (p<0 001)  Month, engine power, number of sets and fishing ground were also identified as significant variables (Table 5), although there was considerable covariance between these factors  Standardised CPUE was significantly lower in January, and decreased with higher numbers of hooks, despite a higher total catch per trip and set (Fig 2) 
Table 5  Analysis of variance for linear model of standardised CPUE (individuals per 100 hooks per set) data from Tanjung Luar; significant values (p<0 05) are given in bold 
Best fit GLMs indicated that the most significant factors influencing the likelihood of catching threatened species were month (January and November were significantly lower: p<0 001 and p<0 05, respectively) and fishing ground (Other (i e  fishing grounds outside of WNTP and ENTP) was significantly higher: p<0 01)  Significant factors associated with standardised CPUE of threatened species were number of hooks (p<0 001), fishing ground (other: p<0 001, ENTP p<0 05), engine power (p<0 001) and trip length (p<0 001) (Table 6 and Fig 3) 
Plots of most significant factors affecting standardised CPUE (number of individuals per 100 hooks per set) of threatened species: a) hook number, b) fishing ground, c) engine power and d) trip length 
Analysis of variance for the best fit models of factors affecting: a) the likelihood of catching and the standardised CPUE of threatened species b) the likelihood of catching and the standardised CPUE of regulated species 
The most significant factors influencing the likelihood of catching regulated species were month (January was significantly lower: p<0 001), number of hooks (p<0 001) and engine power (<0 01)  Significant factors associated with standardised CPUE of regulated species were number of hooks (p<0 001), fishing gear (<0 001), number of sets (p<0 001), engine power (p<0 01) and month (November and January: p<0 05) (Table 5 and Fig 4) 
Plots of most significant factors affecting standardised CPUE (number of individuals per 100 hooks per set) of regulated species: a) hook number, b) gear type, c) number of sets 
Although Tanjung Luar’s targeted shark fishery is small in scale, considerable numbers of shark are landed, including a large proportion of threatened and regulated species  A key finding is that measures of CPUE, for all sharks and for threatened and regulated species, vary spatially and temporally, and with several aspects of fishing effort including gear type, hook number, engine power and number of sets  Moreover, the relationships between CPUE and fishing behaviour variables are different for different measures of CPUE (CPUE per trip, CPUE per set, CPUE per 100 hooks per set)  This highlights the importance of using appropriate standardisation for meaningful comparisons of CPUE across different gears and vessel types, and has important implications for fisheries management 
Unstandardised CPUE (individuals per set) was significantly lower in January  This is during the west monsoon season, which is characterised by high rainfall and adverse conditions at sea for fishing  Unstandardised CPUE was also significantly lower in West Nusa Tenggara Province (WNTP) than East Nusa Tenggara Province (ENTP) and other provinces, suggesting a lower abundance of sharks in this area  Engine power had a significant positive influence on unstandardised CPUE, and was also associated with longer trips and more sets, which was likely due to the ability of vessels with larger engines to travel longer distances, over longer time periods, and with higher numbers of sets, to favoured fishing grounds  Unstandardised CPUE was also significantly higher for surface longlines than bottom longlines  However, when standardising CPUE for the number of hooks (i e  individuals per 100 hooks per set) this relationship was reversed  Bottom longlines exhibit a higher standardised CPUE, with negative relationships between catch per 100 hooks per set and number of hooks and frequency of sets  Vessels with moderate engine horsepower (50-59hp) also had the highest standardised CPUE  Since surface longlines systematically employ significantly more hooks than bottom longlines (400–600 vs 25–200 hooks), and tend to be associated with larger boats, longer trips and more sets, these findings suggest that although increasing fishing effort increased total catch for these gears and trips, there were diminishing returns of this increased effort above low to moderate levels 
A large proportion of Tanjung Luar’s shark catch consisted of threatened (22%) and regulated species (46%)  Month is a significant factor in explaining standardised CPUE of both threatened and regulated species, which could indicate seasonal variation in the abundance of these species in the Tanjung Luar fishing grounds, or seasonal impacts on CPUE due to poor weather conditions  Fishing ground was a significant factor in explaining the catch of threatened species but not the catch in regulated species  This may be due to differences in range, distribution and relative abundance of species within these groups  Threatened species make up a relatively small proportion of Tanjung Luar’s catch in comparison to regulated species, which make up almost half of the catch (46%)  As such, regulated species may generally be more abundant and spatially diffuse than threatened species, and therefore caught more uniformly across fishing grounds  For example, regulated species catch is dominated by silky sharks (Carcharhinus falciformis), which are circum-tropical and coastal-pelagic, and exhibit limited site-fidelity or aggregation behaviour, while threatened species catch is dominated by scalloped hammerheads (Sphyrna lewini), which are known to aggregate in schools  These schools of scalloped hammerheads may be more restricted to specific aggregation sites outside of WNTP and ENTP waters, while silky sharks are found in uniform abundance throughout fishing grounds 
As with CPUE of all catch, there was a positive relationship between unstandardised CPUE (catch per set) of threatened and regulated species and number of hooks, but a significant negative relationship between standardised CPUE (catch per 100 hooks per set)  This was likely due to diminishing returns of adding additional hooks, and indicates that the effort for threatened and regulated species was exceeding maximum sustainable yield effort, such that increases in effort (e g  hook number) were leading to decreases in catch [28–30] 
Due to the profitability of the shark industry in Tanjung Luar, and limited adaptive capacity and willingness of shark fishers to move into other industries, it is necessary to identify practical and ethical management interventions that can improve the sustainability of the fishery whilst also mitigating the negative socio-economic consequences for coastal communities  Our findings indicate that spatiotemporal closures and restrictions on fishing effort could improve the overall catch per unit effort and sustainability of the Tanjung Luar shark fishery, and lead to positive conservation outcomes for priority species 
Since the location of shark fishing grounds plays a significant role in determining the likelihood of catching threatened species and their associated CPUE, improved marine spatial planning, with the identification of marine protected areas (MPAs) that protect critical shark habitat and shark populations, could reduce catch of species of conservation concern [31–33] and increase abundance of sharks [34, 35]  Provincial governments in West Papua and West Nusa Tenggara have already established ‘shark sanctuary’ MPAs, which protect critical shark habitat and ban shark fishing within their boundaries [16, 36], and monitoring data indicates positive impacts of shark-specific closures on shark abundance [37, 38]  Strengthening Indonesia’s existing MPA network for shark conservation, such as making all MPAs no-take zones for sharks and expanding spatial protection to critical shark habitat, including aggregation sites or pupping and nursery grounds for species of conservation concern, could have considerable conservation benefits  It should be noted, however, that MPAs may only be effective for certain species, such as those with small ranges or site-fidelity   More research is required to identify critical shark habitat and life history stages  For Tanjung Luar these efforts could focus on better understanding scalloped hammerhead (Sphyrna lewini) aggregation sites  Well-targeted spatial closures for this species could significantly reduce catch of threatened species in this fishery 
The relationships between gear type, several aspects of fishing effort (i e  hook number, engine power, number of sets, trip length), standardised CPUE of all shark species and standardised CPUE of threatened and regulated species suggest that there is an optimal effort that could increase overall CPUE of the fishery and significantly reduce fishing mortality of species of conservation concern  For example, our data suggest that CPUE peaks with low to intermediate trip lengths and gear sets, intermediate engine power and hook numbers of less than 75 per set longline  Although standardised CPUE of threatened and regulated species is also higher when fewer hooks are deployed, the catch per set and overall mortality is significantly lower  Regulations that control the number of hooks in combination with incentives for shark fishers to tightly manage the number of hooks they deploy could significantly reduce mortality of threatened and endangered species, maximise the overall CPUE of the fishery, and reduce operational costs for fishers, making shark fishing in Tanjung Luar more sustainable and more cost effective [39–41] 
Acknowledging that almost half of Tanjung Luar’s shark catch consists of CITES-listed species, developing measures that ensure both the sustainability of the fishery, and full traceability and control of onward trade, will be crucial for implementing CITES   The Indonesian government has demonstrated a strong commitment to regulating shark trade and implementing CITES [17–18], as demonstrated through several policy decisions to confer full and partial protection to CITES-listed shark and ray species (Marine Affairs and Fisheries Ministerial Decree No 4 /KEPMEN-KP/2014, Regulation No  48/PERMEN-KP/2016)  This includes zero quotas/export bans for hammerhead and oceanic whitetip sharks  However, these export bans should be considered intermediate policy measures as monitoring systems and data availability are improved, and sustainable quotas are established  This will be challenging, as shark products are often traded in large volumes of fresh and/or preserved body parts, with high morphological similarity between products from regulated species and non-regulated species  To guarantee that trade is not detrimental to the survival of species, sustainable fisheries management will need to be complemented with species-specific trade quotas  This will require catch documentation systems which trace shark products from point of catch to point of export and rapid, low-cost species identification methods 
As baseline data on shark population health are limited, and there is no standardised, fisheries-independent system for monitoring long-term changes in shark populations, indirect bio-indicators (e g  endo- and ectoparasites, [43–45]) could help to elucidate the impact of management measures on fisheries and populations of wild species  In the future, shark conservation and fisheries management could benefit from long-term monitoring of agreed indices of population abundance and health status 
These lessons may also apply to shark fisheries in other parts of the world  As sharks increasingly become the focus of global conservation efforts it should be acknowledged that species protection alone will not be enough to reduce mortality of priority species  More needs to be done to identify practical fisheries management measures that can reduce pressure on the most vulnerable species and populations, but also support sustainable use of species that are less susceptible to overfishing  Shark fishing forms an integral part of the livelihood strategies of many coastal communities [22, 23], and prohibiting catches will not necessarily lead to positive conservation outcomes [21, 46]  Management interventions must take into account local context and the motivations and well-being of fisher communities in order to be ethical, feasible and impactful 
S1 Dataset  Data of landed sharks at Tanjung Luar auction that had been used for this study 
S1 File  Questionnaires have been used to interview shark fishers, collector, traders, and processors 
We wish to acknowledge the support provided by fishers in Tanjung Luar for their great cooperation during fieldwork  We also thank I Made Dharma Aryawan, Muhsin, Abdul Kohar, and Abdurrafik for their assistance during field research, Benaya M Simeon, Peni Lestari, and Siska Agustina for helping with data processing, Ken Kassem for carefully reading the manuscript and providing useful inputs, and the anonymous reviewers for their constructive comments 
3  Hutchings JA, Reynolds JD  Marine fish population collapses: consequences for recovery and extinction risk  AIBS Bulletin  2004 Apr;54(4):297–309 
4  Costello C, Ovando D, Clavelle T, Strauss CK, Hilborn R, Melnychuk MC, et al  Global fishery prospects under contrasting management regimes  Proceedings of the national academy of sciences  2016 May 3;113(18):5125–9 
5  Davidson LN, Krawchuk MA, Dulvy NK  Why have global shark and ray landings declined: improved management or overfishing?  Fish and Fisheries  2016 Jun 1;17(2):438–58 
6  Stevens JD, Bonfil R, Dulvy NK, Walker PA  The effects of fishing on sharks, rays, and chimaeras (chondrichthyans), and the implications for marine ecosystems  ICES Journal of Marine Science  2000 Jun 1;57(3):476–94 
9  Dent F, Clarke S  State of the global market for shark products  FAO Fisheries and Aquaculture Technical Paper (FAO) eng no  590  2015 
12  Christensen J, Tull M, editors  Historical perspectives of fisheries exploitation in the Indo-Pacific  Springer Science & Business Media; 2014 Apr 1 
13  Simpfendorfer CA, Heupel MR, White WT, Dulvy NK  The importance of research and public opinion to conservation management of sharks and rays: a synthesis  Marine and Freshwater Research  2011 Jul 21;62(6):518–27 
14  Lack M, Sant G  The future of sharks: a review of action and inaction  TRAFFIC International and the Pew Environment Group  2011 Jan:44 
15  Bräutigam A, Callow M, Campbell IR, Camhi MD, Cornish AS, Dulvy NK, et al  Global priorities for conserving sharks and rays: A 2015–2025 strategy  The Global Sharks and Rays Initiative; 2015  27p 
16  Satria A, Matsuda Y  Decentralization of fisheries management in Indonesia  Marine Policy  2004 Sep 30;28(5):437–50 
17  Dharmadi , Fahmi , Satria F  Fisheries management and conservation of sharks in Indonesia  African journal of marine science  2015 Apr 3;37(2):249–58 
20  Sembiring A, Pertiwi NP, Mahardini A, Wulandari R, Kurniasih EM, Kuncoro AW, Cahyani ND, Anggoro AW, Ulfa M, Madduppa H, Carpenter KE  DNA barcoding reveals targeted fisheries for endangered sharks in Indonesia  Fisheries Research  2015 Apr 30;164:130–4 
21  Clarke S  Re-examining the shark trade as a tool for conservation  SPC Fisheries Newsletter  2014:49–56 
22  Jaiteh VF, Loneragan NR, Warren C  The end of shark finning? Impacts of declining catches and fin demand on coastal community livelihoods  Marine Policy  2017 Mar 24 
24  Cohen D, Crabtree B  Qualitative research guidelines project  Robert Wood Johnson Foundation, Princeton  2006 Available from: http://www qualres org/index html Cited in August 2016 
25  Skud BE  Manipulation of fixed gear and the effect on catch-per-unit effort  FAO Fisheries Report (FAO)  1984 
26  Damalas D, Megalofonou P, Apostolopoulou M  Environmental, spatial, temporal and operational effects on swordfish (Xiphias gladius) catch rates of eastern Mediterranean Sea longline fisheries  Fisheries Research  2007 Apr 30;84(2):233–46 
27  Burnham KP, Anderson DR  Model selection and multimodel inference: a practical information-theoretic approach  Springer Science & Business Media; 2003 Dec 4 
28  Schaefer MB  Some aspects of the dynamics of populations important to the management of the commercial marine fisheries  Inter-American Tropical Tuna Commission Bulletin  1954;1(2):23–56 
29  Fox WW Jr  An exponential surplus-yield model for optimizing exploited fish populations  Transactions of the American Fisheries Society  1970 Jan 1;99(1):80–8 
30  Purwanto P, Nugroho D, Suwarso S  Potential production of the five predominant small pelagic fish species groups in the Java Sea  Indonesian Fisheries Research Journal  2014 Dec 1;20(2):59–67 
31  Barker MJ, Schluessel V  Managing global shark fisheries: suggestions for prioritizing management strategies  Aquatic Conservation: Marine and Freshwater Ecosystems  2005 Jul 1;15(4):325–47 
34  Ward-Paige CA, Worm B  Global evaluation of shark sanctuaries  Global Environmental Change  2017 Nov 30;47:174–89 
35  Speed CW, Cappo M, Meekan MG  Evidence for rapid recovery of shark populations within a coral reef marine protected area  Biological Conservation  2018 Apr 30;220:308–19 
36  West Nusa Tenggara Provincial Government  2017  [Management and zoning plan of Lunyuk Marine Protected Area]  Mataram: West Nusa Tenggara Provincial Government; 2017  Indonesian 
37  Jaiteh VF, Lindfield SJ, Mangubhai S, Warren C, Fitzpatrick B, Loneragan NR  Higher abundance of marine predators and changes in fishers' behavior following spatial protection within the world's biggest shark fishery  Frontiers in Marine Science  2016 Apr 7;3:43 
39  Kumoru L  The shark longline fishery in Papua New Guinea  InReport prepared for Billfish and bycatch research group, at the 176th meeting of the standing committee on Tuna and Billfish, Mooloolaba, Australia, 9th-16th July 2003 2003 Jul 
40  Cartamil D, Santana-Morales O, Escobedo-Olvera M, Kacev D, Castillo-Geniz L, Graham JB, Rubin RD, Sosa-Nishizaki O  The artisanal elasmobranch fishery of the Pacific coast of Baja California, Mexico  Fisheries Research  2011 Mar 31;108(2):393–403 
42  Vincent AC, Sadovy de Mitcheson YJ, Fowler SL, Lieberman S  The role of CITES in the conservation of marine fishes subject to international trade  Fish and Fisheries  2014 Dec 1;15(4):563–92 
43  Palm HW  Fish parasites as biological indicators in a changing world: can we monitor environmental impact and climate change?  InProgress in Parasitology 2011 (pp  223–250)  Springer Berlin Heidelberg 
44  Palm HW, Yulianto I, Piatkowski U  Trypanorhynch Assemblages Indicate Ecological and Phylogenetical Attributes of Their Elasmobranch Final Hosts  Fishes  2017 Jun 17;2(2):8 
46  Booth H  Using the case of illegal manta ray trade in Indonesia to evaluate the impact of wildlife trade policy (Master Thesis, Imperial College London) 
47  White WT, Last PR, Stevens JD, Yearsley GK  Economically important sharks & rays of Indonesia  Austr-alian Centre for International Agricultural Research (ACIAR); 2006 
Context 2: Paper Info

Title: Efficient nonparametric estimation of Toeplitz covariance matrices
Publish Date: March 20, 2023
Author List: Karolina Klockmann (from Department of Statistics and Operations Research, Universität Wien), Tatyana Krivobokova (from Department of Statistics and Operations Research, Universität Wien)

Figure

Figure 1: Spectral density functions (first row) and autocovariance functions (second row) for examples 1, 2, 3 
Figure 2: Distance between the first atom and the first center of mass of aquaporin (left) and the opening diameter y t over time t (right) 
black line in the left plot) confirms that the covariance matrix estimated with our VST-DCT method almost completely decorrelates the channel diameter Y on the training data set Next, we estimated the regression coefficients β with the usual PLS algorithm, ignoring the dependence in the data Finally, we estimated β with PLS that takes into account dependence using our covariance estimator Σ Based on these regression coefficient estimators, the prediction on the test set was calculated The plot on the right side of Figure 2 shows the Pearson correlation between the true channel diameter on the test set and the prediction on the same test set based on raw (grey) and decorrelated data (black) 
Figure 3: On the left, the auto-correlation function of Y (grey) and of Σ−1/2 Y (black), where Σ is estimated with the VST-DCT method; On the right, correlation between the true values on the test data set and prediction based on partial least squares (in grey) and corrected partial least squares (black) 
Uniform distributionThe observations follow a uniform distribution with covariance matrices Σ 1 , Σ 2 , Σ 3 of examples 1, 2, 3, i e , Y i = Σ 1/2 j X i , j = 1, 3, with X 1 ,    the parameter innov of the R function arima sim is used to pass the innovations X 1 ,    , X n i i d Table4, 5 and 6 show respectively the results for (A) p = 5000, n = 1, (B) p = 1000, n = 50 and (C) p = 5000, n = 10 
(A) p = 5000, n = 1: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral and L 2 norm, respectively, as well as the average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 
(C) p = 5000, n = 10: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral and L 2 norm, respectively, as well as the average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 
(A) p = 5000, n = 1: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral norm and the L 2 norm, respectively Average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 
(B) p = 1000, n = 50: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral norm and the L 2 norm, respectively Average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 
(C) p = 5000, n = 10: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral norm and the L 2 norm, respectively Average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 

abstract

A new nonparametric estimator for Toeplitz covariance matrices is proposed  This estimator is based on a data transformation that translates the problem of Toeplitz covariance matrix estimation to the problem of mean estimation in an approximate Gaussian regression  The resulting Toeplitz covariance matrix estimator is positive definite by construction, fully data-driven and computationally very fast 
Moreover, this estimator is shown to be minimax optimal under the spectral norm for a large class of Toeplitz matrices  These results are readily extended to estimation of inverses of Toeplitz covariance matrices  Also, an alternative version of the Whittle likelihood for the spectral density based on the Discrete Cosine Transform (DCT) is proposed 
The method is implemented in the R package vstdct that accompanies the paper 

Introduction

Estimation of covariance and precision matrices is a fundamental problem in statistical data analysis with countless applications in the natural and social sciences  Covariance matrices with a Toeplitz structure arise in the study of stationary stochastic n = 1, to the best of our knowledge, there is no fully data-driven approach for selecting the banding/tapering/thresholding parameter 
suggested first to split the time series into non-overlapping subseries and then apply the cross-validation criterion of   However, it turns out that the right choice of the subseries length is crucial for this approach, but there is no data-based method available for this  In this work, an alternative way to estimate a Toeplitz covariance matrix and its inverse is chosen 
Our approach exploits the one-to-one correspondence between Toeplitz covariance matrices and their spectral densities  First, the given data are transformed into approximate Gaussian random variables whose mean equals to the logarithm of the spectral density  Then, the log-spectral density is estimated by a periodic smoothing spline with a data-driven smoothing parameter 
Finally, the resulting spectral density estimator is transformed into an estimator for Σ or its inverse  It is shown that this procedure leads to an estimator that is fully data-driven, automatically positive definite and achieves the minimax optimal convergence rate under the spectral norm over a large class of Toeplitz covariance matrices 
In particular, this class includes Toeplitz covariance matrices that correspond to long-memory processes with bounded spectral densities  Moreover, the computation is very efficient, does not require iterative or resampling schemes and allows to apply any inference and adaptive estimation procedures developed in the context of nonparametric Gaussian regression 
Estimation of the spectral density from a stationary time series is a research topic with a long history  Earlier nonparametric methods are based on smoothing of the (log-)periodogram, which itself is not a consistent estimator   Another line of nonparametric methods for estimating the spectral density is based on the Whittle likelihood, which is an ap-proximation to the exact likelihood of the time series in the frequency domain 
For example, estimated the spectral density from a penalized Whittle likelihood, while used polynomial splines to estimate the log-spectral density function maximizing the Whittle likelihood  Recently, Bayesian methods for spectral density estimation have been proposed (see , but these may become very computationally intensive in large samples due to posterior sampling 
The minimax optimal convergence rate for nonparametric estimators of Hölder continuous spectral densities from Gaussian stationary time series was obtained by under the L p , 1 ≤ p ≤ ∞, norm  Only few works on spectral density estimation show the optimality of the corresponding estimators  In particular, and derived convergence rates of their estimators for the log-spectral density under the L 2 norm, while neglecting the Whittle likelihood approximation error 
In general, most works on spectral density estimation do not exploit further the close connection to the corresponding Toeplitz covariance matrix estimation  In particular, an upper bound for the L ∞ risk of a spectral density estimator automatically provides an upper bound for the risk of the corresponding Toeplitz covariance matrix estimator under the spectral norm 
This fact is used to establish the minimax optimality of our nonparametric estimator for the Toeplitz covariance matrices  The main contribution of this work is to show that our proposed spectral density estimator is not only numerically very efficient, but also achieves the minimax optimal rate in the L ∞ norm, which in turn ensures the minimax optimality of the corresponding Toeplitz covariance matrix estimator 
The paper is structured as follows  In Section 2, the model is introduced and ap-proximate diagonalization of Toeplitz covariance matrices with the discrete cosine transform is discussed  Moreover, an alternative version of the Whittle's likelihood is proposed  In Section 3, new estimators for the Toeplitz covariance matrix and the precision matrix are derived, while in Section 4 their theoretical properties are presented 
Section 5 contains simulation results, Section 6 presents a real data example, and Section 7 closes the paper with a discussion  The proofs are given in the appendix to the paper 

Set up and diagonalization of Toeplitz matrices

Let Y 1 ,       , Y n i i d  ∼ N p (0 p , Σ), where Σ is a (p × p)-dimensional positive definite covariance matrix with a Toeplitz structure, that is, Σ = {σ |i−j| } p i,j=1 0  The sample size n may tend to infinity or to be a constant  The case n = 1 corresponds to a single observation of a stationary time series, and in this case the data are simply denoted by Y ∼N p (0 p , Σ) 
The dimension p is assumed to grow  The spectral density function f , corresponding to a Toeplitz covariance matrix Σ, is given by so that for f ∈ L 2 (−π, π) the inverse Fourier transform implies Hence, Σ is completely characterized by f , and the non-negativity of the spectral density function implies the positive definiteness of the covariance matrix 
Moreover, the decay of the autocovariance σ k is directly connected to the smoothness of f   Finally, the convergence rate of a Toeplitz covariance estimator and that of the corresponding spectral density estimator are directly related via Σ ≤ f ∞ := sup x∈ |f (x)|, where • denotes the spectral norm (see  
As in , we introduce a class of positive definite Toeplitz covariance matrices with Hölder continuous spectral densities  For β = γ + α > 0, where The optimal convergence rate for estimating Toeplitz covariance matrices over P β (M 0 , M 1 ) depends crucially on β  It is well known that the k-th Fourier coefficient of a function whose γ-th derivative is α-Hölder continuous decays at least with order O(k −β ) (see  
Hence, β determines the decay rate of the autocovariances σ k , which are the Fourier coefficients of the spectral density f , as k → ∞  In particular, this implies that for β ∈ (0, 1], the class P β (M 0 , M 1 ) includes Toeplitz covariance matrices corresponding to long-memory processes with bounded spectral densities, since the sequence of corresponding autocovariances is not summable 
A connection between Toeplitz covariance matrices and their spectral densities is further exploited in the following lemma  Lemma 1  Let Σ ∈ P β (M 0 , M 1 ) and let x j = (j − 1)/(p − 1), j = 1,    , p, then where δ i,j is the Kroneker delta, O(•) terms are uniform over i, j = 1,       , p and divided by √ 2 when i, j ∈ {1, p} is the Discrete Cosine Transform I (DCT-I) matrix 
The proof can be found in Appendix A 1  This result shows that the DCT-I matrix approximately diagonalizes Toeplitz covariance matrices and that the diagonalization error depends to some extent on the smoothness of the corresponding spectral density  In the spectral density literature the discrete Fourier transform (DFT) matrix
, where i is the imaginary unit, is typically employed to approximately diagonalize Toeplitz covariance matrices  Using the fact that introduced an approximation for the likelihood of a single Gaussian stationary time series (case n = 1), the so-called Whittle likelihood (1) The quantity , where F j denotes the j-th column of F , is known as the periodogram at the j-th Fourier frequency 
Note that due to periodogram symmetry, only p/2 data points I 1 ,    , I p/2 are available for estimating the mean f (2πj/p), j = 1,       , p/2 , where x denotes the largest integer strictly smaller than x  The Whittle likelihood has become a popular tool for parameter estimation of stationary time series, e g , for nonparametric and parametric spectral density estimation or for estimation of the Hurst exponent, see e g , ;  
Lemma 1 yields the following alternative version of the Whittle likelihood where W j = (D t j Y ) 2   Note that this likelihood approximation is based on twice as many data points W j as the standard Whittle likelihood  Thus, it allows for a more efficient use of the data Y to estimate the parameter of interest, such as the spectral density or the Hurst parameter 
Equations ( ) or (2) invite for the estimation of f by maximizing the (penalized) likelihood over certain linear spaces (e g , spline spaces), as suggested e g , in or   However, such an approach requires well-designed numerical methods to solve the corresponding optimization problem, since the spectral density in the second term of (1) or ( ) is in the denominator, which does not allow to obtain a closed-form expression for the estimator and often leads to numerical instabilities 
Also, the choice of the smoothing parameter becomes challenging  Therefore, we suggest an alternative approach that allows the spectral density to be estimated as a mean in an approximate Gaussian regression  Such estimators have a closed-form expression, do not require an iterative optimization algorithm and a smoothing parameter can be easily obtained with any conventional criterion 
First Hence, for W j = (D t j Y ) 2 , j = 1,       , p it follows with Lemma 1 that where Γ(a, b) denotes a gamma distribution with a shape parameter a and a scale parameter b  Note that the random variables W 1 ,       , W p are only asymptotically independent  Obviously, E(W j ) = f (πx j ) + O(1), j = 1,      
, p  To estimate f from W 1 ,       , W p , one could use a generalized nonparametric regression framework with a gamma distributed response, see e g , the classical monograph by   However, this approach requires an iterative procedure for estimation, e g , a Newton-Raphson algorithm, with a suitable choice for the smoothing parameter at each iteration step 
Deriving the L ∞ rate for the resulting estimator is also not a trivial task  Instead, we suggest to employ a variance stabilizing transform of that converts the gamma regression into an approximate Gaussian regression  In the next section we present the methodology in more detail for a general setting with n ≥ 1 

Methodology

For Y i ∼ N p (0 p , Σ), i = 1,       , n, it was shown in the previous section that with Lemma 1 the data can be transformed into gamma distributed random variables     , n, j = 1,       , p, where for each fixed i the random variable W i,j has the same distribution as W j given in (3)  Now the approach of Cai et al  ( ) is adapted to the setting n ≥ 1 
First, the transformed data points W i,j are binned, that is, fewer new variables     , T   Note that the number of observations in a bin is m = np/T   In Theorem 1 in Section 4, we show that setting T = p υ for any υ ∈ ((4 − 2 min{β, 1})/3, 1) leads to the minimax optimal rate for the spectral density estimator 
To simplify the notation, m is handled as an integer (otherwise, one can discard several observations in the last bin)  Next, applying the variance stabilizing transform (VST) ∼ where H(y) = {φ(m/2) + log (2y/m)} / √ 2 and φ is the digamma function (see   Now, the scaled and shifted log-spectral density H(f ) can be estimated with a periodic smoothing spline
where h > 0 denotes a smoothing parameter, q ∈ N is the penalty order and S per (2q − 1) a space of periodic splines of degree 2q − 1  The smoothing parameter h can be chosen either with generalized cross-validation (GCV) as derived in or with the restricted maximum likelihood, see   Once an estimator H(f ) is obtained, application of the inverse transform function H −1 (y) = m exp √ 2y − φ (m/2) /2 yields the spectral density estimator f = H −1 H(f )  
Finally, using the inverse Fourier transform leads to the fol- The precision matrix Ω is estimated by the inverse Fourier transform of the reciprocal of the spectral density estimator, i e , Ω = (ω |i−j| ) p i,j=1 with ωk = The estimation procedure for Σ and Ω can be summarised as follows  1  Data Transformation:
where D is the (p × p)-dimensional DCT-I matrix as given in Lemma 1 and D j is its j-th column  2  Binning: Set T = p υ for any υ ∈ ((4 − 2 min{β, 1})/3, 1) and calculate W i,j , k = 1,       , T 

VST:

where k are asymptotically i i d  Gaussian variables  Inverse VST: Estimate the spectral density f with f = H −1 H(f ) , where Note that Σ and Ω are positive definite matrices by construction, since their spectral density functions f and f −1 are non-negative, respectively  Unlike the banding and tapering estimators, the autocovariance estimators σk are controlled by a single smoothing parameter h, which can be estimated fully data-driven with several available automatic methods, which are numerically efficient and well-studied 
In addition, one can also use methods for adaptive mean estimation, see e g , , which in turn leads to adaptive Toeplitz covariance matrix estimation  All inferential procedures developed in the Gaussian regression context can also be adopted accordingly 

Theoretical Properties

In this section, we study the asymptotic properties of the estimators f , Σ and Ω  The results are established under the asymptotic scenario where p → ∞ and p/n → c ∈ (0, ∞], that is, the dimension p grows, while the sample size n either remains fixed or also grows but not faster than p  This corresponds to the asymptotic scenario when the sample covariance matrix is inconsistent 
Let f be the spectral density estimator defined in Section 3, i e , f = m exp{ √ 2 H(f ) − φ(m/2)}/2, where H(f ) is given in (4), m = np/T and φ is the digamma function  Furthermore, let Σ be the Toeplitz covariance matrix estimator and Ω the corresponding precision matrix defined in equations ( ) and (6), respectively 
The following theorem shows that both Σ and Ω attain the minimax optimal rate of convergence over the class and hT → ∞, then with T = p υ for any υ ∈ ((4 − 2 min{β, 1})/3, 1) and q = max{1, γ}, the spectral density estimator f , the corresponding covariance matrix estimator Σ and the precision matrix estimator Ω satisfy sup
For h {log(np)/(np)}   The proof of Theorem 1 can be found in the Appendix A 3 and is the main result of our work  The most important part of this proof is the derivation of the convergence rate for the spectral density estimator f under the L ∞ norm  In the original work, established an L 2 rate for a wavelet nonparametric mean estimator in a gamma regression where the data are assumed to be independent 
In our work, the spectral density estimator f is based on the gamma distributed data W i,1 ,       , W i,p , which are only asymptotically independent  Moreover, the mean of these data is not exactly f (πx 1 ),       , f (πx p ), but is corrupted by the diagonalization error given in Lemma 1  This error adds to the error that arises via binning and VST and that describes the deviation from a Gaussian distribution, as derived in  
Finally, we need to obtain an L ∞ rather than an L 2 rate for our spectral density estimator  Overall, the proof requires different tools than those used in   To get the L ∞ rate for f , we first derive that for the periodic smoothing spline estimator H(f ) of the log-spectral density  To do so, we use a closed-form expression of its effective kernel obtained in , thereby carefully treating various (dependent) errors that describe deviations from a Gaussian nonparametric regression with independent errors and mean f (πx i ) 
Note also that although the periodic smoothing spline estimator is obtained on T binned points, the rate is given in terms of the vector dimension p  Then, using the Cauchy-Schwarz inequality and a mean value argument, this rate is translated into the L ∞ rate for the spectral density estimator f   To obtain the rate for the Toeplitz covariance matrix estimator is enough to note that

Simulation Study

In this section, we compare the performance of the proposed Toeplitz covariance estimator, denoted as VST-DCT, with the tapering estimator of and with the sample covariance matrix  We consider Gaussian vectors Y 1 ,    , (3) such that the corresponding spectral density is Lipschitz continuous but not differentiable: f (x) = 1 44{| sin(x + 0 5π)| 1 7 + 0 45} 
In particular, var(Y i ) = 1 44 in all three examples  Figure shows the spectral densities and the corresponding autocorrelation functions for the three examples  A Monte Carlo simulation with 100 iterations is performed using R (version 4 1 2, seed 42)  For our VST-DCT estimator, we use a cubic periodic spline, i e , q = 2 is set in (4) 
The binning parameters are set to T = 500 bins with m = 10 points for (A) and T = 500 bins with m = 100 points for both (B) and (C)  To select the regularisation parameter for our estimator, we implemented the restricted maximum likelihood (ML) method, generalized cross validation (GCV) and the corresponding oracle versions, i e , as if Σ were known The tapering parameter
where T ap k (Σ ν 2 ) is the tapering estimator of with parameter k  If n = 1, that is, under scenario (A), suggest to split the time series Y into l non-overlapping subseries of length p/l and then proceed as before to select the tuning parameter k  To the best of our knowledge, there is no data-driven method for selecting this parameter l 
Using the true covariance matrix Σ, we selected l = 30 subseries for the example 1 and l = 15 subseries for the exam-ples 2 and 3  The parameter k can then be chosen by cross-validation as above  We employ this approach under scenario (A) instead of an unavailable fully data-driven criterion and name it semi-oracle 
Finally, for all three scenarios (A), (B) and (C), the oracle tapering parameter is computed using grid search for each Monte Carlo sample as kor = arg min k=2,3,   ,p/2 T ap k ( Σ) − Σ , where Σ is the sample covariance matrix  To speed up the computation, one can replace the spectral norm by the 1 norm, as suggested by  
In Tables , the errors of the Toeplitz covariance estimators with respect to the spectral norm and the computation time for one Monte Carlo iteration are given for scenarios (A), (B) and (C), respectively  To illustrate the goodness-of-fit of the spectral density, the L 2 norm f − f 2 is also computed 
The results show that the tapering and VST-DCT estimator perform overall similar in terms of the spectral norm risk  This is not surprising as both estimators are proved to be rate-optimal  Moreover, both the tapering and VST-DCT estimators are clearly superior to the inconsistent sample Toeplitz covariance matrix 
A closer look at the numbers shows that the VST-DCT method has better constants, i e , VST-DCT estimators have somewhat smaller errors in the spectral norm than the tapering estimators across all examples, but especially under scenario (C)  The oracle estimators show similar behaviour, but are slightly less variable compared to the data-driven estimators 
In general, both the tapering and VST-DCT estimators perform best for example 1, second best for example 3 and worst for example 2, which traces back to functions complexity  In terms of computational time, both methods are similarly fast for scenarios (A) and (B)  For scenario (C), the tapering method is much slower due to the multiple high-dimensional matrix multiplications in the cross-validation method 
It is expected that for larger p the tapering estimator is much more computationally intensive than the corresponding VST-DCT estimator  (1) polynomial σ ( (  To test how robust our approach is to deviations from the Gaussian assumption, we simulated the data from gamma and uniform distributions and conducted a simulation study for the same scenarios and examples 
The results are very similar to those of the Gaussian distribution, see supplementary materials for the details 

Application to Protein Dynamics

We revisit the data analysis of protein dynamics performed in Krivobokova et al  (2012) and 
Context 3: Paper Info

Title: Two-stage Pipeline for Multilingual Dialect Detection
Publish Date: Unkown
Author List: Ankit Vaidya (from Pune Institute of Computer Technology), Aditya Kane (from Pune Institute of Computer Technology)

Figure

Figure 1: Class distribution of dialects
Figure 2: System diagram for dialect classification The LID classifies the input into one of 3 languages The sample is then further classified into dialects by language specific models 
Figure 3: Confusion matrix of 9-way classification Note that rows are normalized according to the number of samples is that class 
Our complete results for Track-1 using the two-stage dialect detection pipeline Model-* denotes the language of the models used for the experiments 
Performance on Track-1 validation dataset of individual models used in the two-stage pipeline "Lg" stands for language of the model used 
Comparative results of two-way classification using the finetuned (F T ) predictions and predictions adapted from three-way classification models 

abstract

Dialect Identification is a crucial task for localizing various Large Language Models  This paper outlines our approach to the VarDial 2023 DSL-TL shared task  Here we have to identify three or two dialects from three languages each which results in a 9-way classification for Track-1 and 6-way classification for Track-2 respectively 
Our proposed approach consists of a two-stage system and outperforms other participants' systems and previous works in this domain  We achieve a score of 58 54% for Track-1 and 85 61% for Track-2  Our codebase is available publicly 1  

Introduction

Language has been the primary mode of communication for humans since the pre-historic ages  Studies have explored the evolution of language and outlined mathematical models that govern the intricacies of natural language   Inevitably, as humans established civilization in various parts of the world, this language was modified by, and for the group of people occupied by that particular geographical region 
This gave rise to multiple national dialects of the same language  The VarDial workshop (colocated with EACL 2023) explores various dialects and variations of the same language  We participated in the Discriminating Between Similar Languages -True Labels (DSL-TL) shared task  In this task, the participants were provided with data from three languages, with each language having three varieties 
This shared task consisted of two tracks -Track-1 featuring nine-way classification and Track-2 featuring six-way classification  The second track included two particular national dialects of each language (eg  American English and British English), and the first track had one general  We ranked 1 st in both of the tracks 
Moreover, we beat the next best submission by a margin of 4 5% in the first task and 5 6% in the second task We were the only team to surpass the organizer baseline scores  We present our winning solution in this paper  We used an end-to-end deep learning pipeline which consisted of a language identification model and three language-specific models, one for each language 
We converged upon the best combination by doing an elaborate analysis of various models available  Furthermore, in this work we also analyze the performance of the pipeline as a whole and also provide an ablation study  Lastly, we provide some future directions in this area of research 

Related Work

The present literature encompasses various aspects of dialect identification  We study this from three perspectives: large language models, language identification and dialect classification problems 

Large Language Models

The success of transformers and BERT based models was inevitable since the initial boom of the transformer  2017) model  In recent years, many other architectures like RoBERTa and ELECTRA have further pushed the state-of-the-art in this domain  Moreover, autoregressive models like GPT and GPT-2 have also shown their prowess 
Multilingual versions of RoBERTA, namely XLM-RoBERTa are also available  Lastly, language specific models like Spanish BERT (la Rosa y Eduardo G  Ponferrada y Manu Romero y Paulo Villegas y Pablo González de Prado Salas y María Grandury, 2022) and Portuguese BERT are available as well  Our winning solution makes use of these large language models trained on specific languages 

Language Identification Models

Many multilingual language identification models have been developed in order to classify the language of the input sentence beforehand  Even though the initial works used n-gram models and generative mixture models or even conditional random fields and other classical machine learning methods like naive bayes , modern methods have shifted to the use of deep learning for language identification 



ground_truth: The relationships between catch per set and fishing behavior variables differ when comparing unstandardized CPUE and standardized CPUE.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What is the advantage of decorrelating the data before running the PLS algorithm?

Read the following text and answer briefly.

Context 2:  We consider data generated by the molecular dynamics (MD) simulations for the yeast aquaporin (Aqy1) -the gated water channel of the yeast Pichi pastoris  MD simulations are an established tool for studying biological systems at the atomic level on timescales of nano-to microseconds 
The data are given as Euclidean coordinates of all 783 atoms of Aqy1 observed in a 100 nanosecond time frame, split into 20 000 equidistant observations  Additionally, the diameter of the channel y t at time t is given, measured by the distance between two centers of mass of certain residues of the protein 
The aim of the analysis is to identify the collective motions of the atoms responsible for the channel opening  In order to model the response variable y t , which is a distance, based on the motions of the protein atoms, we chose to represent the protein structure by distances between atoms and certain fixed base points instead of Euclidean coordinates 
That is, we calculated where A t,i ∈ R 3 , i = 1,       , 783 denotes the i-th atom of the protein at time t, B j ∈ R 3 , j = 1, 2, 3, 4, is the j-th base point and d(•, •) is the Euclidean distance  Figure shows the diameter y t and the distance between the first atom and the first center of mass  It can therefore be concluded that a linear model Y = Xβ + holds, where
  This linear model has two specific features which are intrinsic to the problem: first, the observations are not independent over time and second, X t is high-dimensional at each t and only few columns of X are relevant for Y   have shown that the partial least squares (PLS) algorithm performs exceptionally well on this type of data, leading to a small-dimensional and robust representation of proteins, which is able to identify the atomic dynamics relevant for Y  
Singer et al  ( ) studied the convergence rates of the PLS algorithm for dependent observations and showed that decorrelating the data before running the PLS algorithm improves its performance  Since Y is a linear combination of columns of X, it can be assumed that Y and all columns of X have the same correlation structure 
Hence, it is sufficient to estimate Σ = cov(Y ) to decorrelate the data for the PLS algorithm, i e , Σ −1/2 Y = Σ −1/2 Xβ + Σ −1/2 results in a standard linear regression with independent errors  Our goal now is to estimate Σ and compare the performance of the PLS algorithm on original and decorrelated data 
For this purpose, we divided the data set into a training and a test set (each with p = 10 000 observations)  First, we tested whether the data are stationary  The augmented Dickey-Fuller test confirmed stationarity for Y with a p-value< 0 01  The Hurst exponent of Y is 0 85, indicating moderate long-range dependence supported by a rather slow decay of the sample autocovariances (see grey line in the left plot of Figure ) 
Therefore, we set q = 1 for the VST-DCT estimator to match the low smoothness of the corresponding spectral density  Moreover, the smoothing parameter is selected with the restricted maximum likelihood method and T = 550 bins are used  Obviously, the performance of the PLS algorithm on the decorrelated data is significantly better for the small number of components 
In particular, with just one PLS component, the correlation between the true opening diameter on the test set and its prediction that takes into account the dependence in the data is already 0 54, while it is close to zero for PLS that ignores the dependence in the data  showed that the estimator of β based on one PLS component is exactly the ensemble-weighted maximally correlated mode (ewMCM), which is defined as the collective mode of atoms that has the highest probability to achieve a specific alteration of the response Y  
Therefore, an accurate estimator of this quantity is crucial for the interpretation of the results and can only be achieved if the dependence in the data is taken into account  Estimating Σ with a tapered covariance estimator has two practical problems  First, since we only have a single realization of a time series Y (n = 1), there is no datadriven method for selecting the tapering parameter 
Second, the tapering estimator turned out not to be positive definite for the data at hand  To solve the second problem, we truncated the corresponding spectral density estimator ftap to a small positive value, i e , f + tap = max{ ftap , 1/ log(p)} (see   To select the tapering parameter with cross-validation, we experimented with different subseries lengths and found that the tapering estimator is very sensitive to this choice 
For example, estimating the tapered covariance matrix based on subseries of length 8/15/30 yields a correlation of 0 42/0 53/0 34 between the true diameter and the first PLS component, respectively  Altogether, our proposed estimator is fully data-driven, fast even for large sample sizes, automatically positive definite and can handle certain long-memory processes 
In contrast, the tapering estimator is not data-driven and must be manipulated to become positive definite  Our method is implemented in the R package vstdct 

Discussion

In this paper, we proposed a simple, fast, fully data-driven, automatically positive definite and minimax optimal estimator of Toeplitz covariance matrices from a large class that also includes covariance matrices of certain long-memory processes  Our estimator is derived under the assumption that the data are Gaussian 
However, simulations show that the suggested approach yields robust estimators even when the data are not normally distributed  In the context of spectral density estima- , for mixing processes (see Theorem 5 3 of Rosenblatt, 2012), as well as for non-linear processes (see   Since DFT and DCT matrices are closely related, we expect that equation (3) also holds asymptotically for these non-Gaussian time series, but consider a rigorous analysis to be beyond the scope of this paper 
In fact, our numerical experiments have even shown that if the spectral density is estimated from W j = f (πx j ) + j , that is, as if W j were Gaussian instead of gamma distributed, then the resulting spectral density estimator has almost the same L ∞ risk (and hence the corresponding covariance matrix has almost the same spectral norm) 
Of course, such an estimator would lead to a wrong inference about f (πx j ), since the growing variance of W j would be ignored  Since our approach translates Toeplitz covariance matrix estimation into a mean estimation in an approximate Gaussian nonparametric regression, all approaches developed in the context of Gaussian nonparametric regression, such as (locally)
adaptive estimation, as well as the corresponding (simultaneous) inference, can be directly applied  Bayesian tools for adaptive estimation and inference in Gaussian nonparametric regression as proposed in can also be employed 

Appendix

Throughout the appendix, we denote by c, c 1 , C, C 1 ,       etc  generic constants, that are independent of n and p  To simplify the notation, the constants are sometimes skipped and we write for less than or equal to up to constants  We embed the p-dimensional Toeplitz matrix Σ = toep(σ 0 ,       , σ p−1 ) in a (2p − 2)dimensional circulant matrix Σ = toep(σ 0 ,      
, σ p−1 , σ p−2 ,       , σ 1 )  Then, Σ = with the conjugate transpose U * , and Λ is a diagonal matrix with the k-th diagonal value for k = 1,    , p given by Furthermore, Σ = V * ΛV , where V ∈ C (2p−2)×p contains the first p columns of U   In particular, b(j, r) = b(j, 2p−r) and c(j, r) = −c(j, 2p−r) for r = p+1,      
, 2p−2  Together, we have (A 1) Some calculations show that for r = 1,       p Using the Taylor expansion of cot(x) for 0 < |x| < π one obtains for r = 1,       p where the O term does not depend on j and the hidden constant does not depend on r, p  If i = j, equations (A 1) -(A 3) imply where the O terms do not depend on j 
Since the complex exponential function is Lipschitz continuous with constant L = 1, it holds λ r = λ j + L r,j |r − j|p −1 where −1 ≤ L r,j ≤ 1 is a constant depending on r, j  Then, , it is sufficient to consider j = 1,    , p − 1  We begin with first sum  For a shorter notation, we use k := r − 1 and l := j − 1 in the following 
Then, summing the squares of the first term in (A 4) for l = 0,    , p−2 on sums of reciprocal powers  If p is even, then the residual terms are given by where φ and φ (1) denote the digamma function and its derivative  If p is odd, similar remainder terms can be derived  To see that R i (l, p) = O(p −1 ) for i = 1, 2, 3 and uniformly in l we use that asymptotically φ(x)∼ log(x)−1/(2x) and
The mixed term are both of the order p −1   Furthermore, since the harmonic sum diverges at a rate of log(p)  Finally, λ j = f (x j )+O{log(p)p −β } by the uniform approximation properties of the discrete Fourier series for Hölder continuous functions (see   All together, we have shown that (DΣD) j,j = where the O terms are uniform over j = 1,    , p 
Case i = j and |i − j| is even In this case, (DΣD) i,j = a i a j uniformly in i, j  To show that a i a j 2p−2 r=1 λ r c(i, r)c(j, r) = O(p −1 ), we proceed similarly as before  Setting k=r−1, l=j−1, m=i−1 and using that l =m and |l−m| is even, one obtains where for even p the residual terms are given by If p is odd, analogous residual terms can be derived 
Using similar techniques as before, one can show that the two residual terms and the remaining mixed and square terms vanish at a rate of the order O(p −1 ) and uniformly in i, j  Case i = j and |i − j| is odd |r − i| and |r − j| are either odd and even, or even and odd  Without loss of generality, assume that |r − i| is even 
Then, (DΣD) i,j = a i a j 2p−2 r=1 λ r b(i, r)c(j, r)  Since b(i, •) is an even function, c(j, •) is an odd function and λ r = λ 2p−r , it follows (DΣD) i,j = 0  The structure of the proof is as follows  First, we derive the L ∞ rate of the periodic smoothing spline estimator H(f )  Then, using the Cauchy-Schwarz inequality and a mean value argument, the convergence rate of the spectral density estimator f is
∞ the first claim of the theorem follows  Finally, we prove the second statement on the precision matrices  For the sake of clarity, some technical lemmas used in the proof are listed separately in A 4  hT → ∞, then with T = p υ for any υ ∈ ((4 − 2 min{1, β})/3, 1), the estimator H(f ) described in Section 3 with q = max{1, γ} satisfies
Proof : Application of the triangle inequality yields a bias-variance decomposition Set T = 2T − 2 and x k = (k − 1)/ T for k = 1,    , T   Using Lemma 4, we can write where Mirroring and renumerating ζ k , η k , k is similar as for Y * k , k = 1,    , T   Using the above representation, one can write First we reduce the supremum to a maximum over a finite number of points 
If q > 1, then W (•, x k ) is Lipschitz continuous with constant L > 0  In this case, it holds almost surely that sup ) is a piecewise linear function with knots at x j = j/ T   The factor (ζ k + ξ k ) can be considered as stochastic weights that do not affect the piecewise linear property  Thus, the supremum is attained at one of the knots x j = j/ T , j = 1,    , T , and (A 7) is also valid for q = 1 
Again with (a + b) 2 ≤ 2a 2 + 2b 2 we obtain We start with bounding   This requires a bound on 1 • ψ 2 denotes the sub-Gaussian norm  In case of a Gaussian random variable the norm equals to the variance  Thus with Lemma 2 and Lemma 4, we obtain Lemma 1 6 of ) then yields Recall that T = p υ for some fixed υ ∈ ((4 − 2 min{1, β})/3, 1) 
Using the inequality log(x) ≤ x a /a one can find constants x υ , C υ > 0 depending on υ but not on n, p such that log(2 T ) log(p) Next, we derive a bound for the second term The exponential decay property of the kernel K stated in Lemma 2 yields The first term in (A 9) can be bounded again with Lemma 1 6 of  
We use the fact that for not necessarily independent random variables X 1 ,    , X N R and R > 0 are constants  This is a consequence of Lemma 1 of which yields , it follows that N i=1 a i X i has a subGaussain distribution and the subGaussian norm is bounded by 2R( N i=1 a 2 i ) 1/2   See for further details on the subgaussian distribution 
T h   For the second inequality Lemma 2(ii) is used  Applying Lemma 1 6 of then yields To bound the second term in (A 9), we use the moment bounds for ξ k derived in Lemma 4  Then, for all integers > 1 Combining the error bounds (A 10) and (A 11) and choosing R=m −1/2 gives By assumption T = p υ and m = np (1−υ) for some fixed υ ∈ ((4 − 2 min{1, β})/3, 1) 
If is an integer such that ≥ 1/(1 − υ), then where we used log(x) ≤ x a /a with a = 1/(4 )  Consider 1/2 < β ≤ 1 and let 0 < χ < 1 be a constant  Applying log(x) ≤ x a /a twice with a = χ/(2 ) yields For any fixed υ∈((4 − 2 min{1, β})/3, 1) one can find an integer which is independent of n, p such that the right side of (A 12) holds 
Since p/n → c ∈ (0, ∞] and thus n/p = O(1) and p −1 = O(n −1 ), it follows for satisfying (A 12) that In total, choosing an integer Using the representation in Lemma 4 once more gives for each x ∈ [0, 1] The bounds on k in Lemma 4 imply Consider the case that β ≥ 1  In particular, q = γ and f (q) is α-Hölder continuous 
Since f is a periodic function with f (x) ∈ [δ, M 0 ] and H(y) ∝ φ(m/2)+ log (2y/m), it follows that {H(f )} (q) is also α-Hölder continuous  Extending g := H(f ) to the entire real line, we get Expanding g(t) in a Taylor series around x and using that h −1 K h is a kernel of order 2q, see Lemma 2(iii), it follows that for any x ∈ [0, 1]
where ξ x,t is a point between x and t  Using the fact that the kernel K h decays exponentially and that g (q) is α-Hölder continuous on [δ, M 0 ] with some constant L, the logarithm is Lipschitz continuous on a compact interval, it follows g = H(f ) is β-Hölder continuous  Expanding g to the entire line and using Lemma 2(iii) with
In a similar way as before, one obtains Note that T −β =o(h β ) as β > 1/2, T h → ∞ and h → 0 by assumption  Since the derived bounds are uniform for x ∈ [0, 1] it holds Putting the bounds A 13 and A 14 together gives If h > 0 such that h → 0 and hT → ∞, then with T = p υ for any υ ∈ ((4 − 2 min{1, β})/3, 1), the estimator f described in Section 3 with q = max{1, γ} satisfies
Proof : By the mean value theorem, it holds for some function g between H(f ) and To show that the second term on the right hand side of (A 15) is negligible we use the moment generating function of H(f ) ∞   In the next paragraph, we derive the asymptotic order of E[exp{λ H(f ) ∞ }] for n, p → ∞, where λ > 0 may depend on n, p or not 
By the exponential decay property of the kernel K stated in Lemma 2 holds First, H(f ) ∞ is bounded with the maximum over a finite number of points  Calculating the derivative of s : Since δ δx s(x) > 0 almost surely for x = x k , the extrema occur at x k , k = 1,    , T   Thus, for λ > 0 the moment generating function of H(f ) ∞ is bounded by
Let M j = ( T h) −1 T k=1 γ h (x j , x k ), which by Lemma 2 is bounded uniformly in j by some global constant M > 0  By the convexity of the exponential function we obtain √ 2 and by assumption 0 ≤ δ ≤ f ≤ M 0   Using Lemma 3, Q k can be written as a sum of m = np/T independent gamma random variables, i e 
The moment generating function of | log(X)| when X follows a Γ(a, b)-distribution is given by where Γ(a) is the gamma function and γ(a, b) is the lower incomplete gamma function  In particular, To derive the asymptotic order of E[exp{λ H(f ) ∞ }] for n, p → ∞ we first establish the asymptotic order of the ratio Γ(a + t)/Γ(a) for a → ∞ 
We distinguish the two cases where t is independent of a and where t linearly depends on a  Thus, for 0 < t < a and t independent of a, equation (A 17) implies for a → ∞ that Γ(a + t)/Γ(a) = O(a t )  Similarly, it can be seen that Γ(a − t)/Γ(a) = O(a −t )  If 0 < t < a and t linearly depends on a, i e  t = ca for some constant c ∈ (0, 1), then we get Γ(a ± t)/Γ(a) = O(a ±t exp{a}) for a → ∞ 
Hence, for a fixed λ not depending on n, p and such that 0 < λ < m/( √ 2M j ) we get for sufficiently large n, p If λ = cm such that 0 < λ < m/( √ 2M j ), then for sufficiently large n, p b∈{cδ/m,cM 0 /m} (bm/2) for some constant L > 1  Set K = min j=1,   , T 1/( √ 2M j ) which is a constant independent of n, p  Altogether, we showed that for 0 < λ < Km and n, p → ∞
Bounding the right hand side of (A 15) for some constants c 0 , c 1 > 0 and n, p → ∞ Since g lies between H(f ) and H(f ), and f almost surely pointwise  Thus, for C > f ∞ = M 0 it holds where c 1 := H(C − M 0 )  Applying Markov inequality for t = cm with c ∈ (0, K) and C = 2L 4/c + M 0 where c, K, L are the constants in gives
Together with Proposition 1 follows Using the fact that the spectral norm of a Toeplitz matrix is upper bounded by the sup norm of its spectral density we get sup According to the mean value theorem, for a function g between H(f ) and H(f ), it holds that some constant c 1 > 0 not depending on n, p  Chosing the same constant C as in section A 3 2 it follows
Noting that 1/f ∞ ≤ 1/δ and 2/m exp {φ(m/2)} ∈ [0 25, 1] for m ≥ 1, (A 18) implies for some constants c 2 , c 3 > 0 and n, p → ∞ Since the derived bounds hold for each Σ(f ) ∈ F β , we get all together sup This section states some technical lemmata needed for the proof of Theorem 1  The proofs can be found in the supplementary material 
The first lemma lists some properties of the kernel K h and its extension K h on the real line  The proof is based on   Lemma 2  Let h > 0 be the bandwith parameter depending on N   (i) There are constants 0 < C < ∞ and 0 < γ < 1 such that for all for x, t ∈ [0, 1] Lemma 3 states that the sum of the correlated gamma random variables in each bin can be rewritten as a sum of independent gamma random variables 
for i = 1,    , n and j = (k − 1)m + 1,    , km, and x j = (j − 1)/(2p − 2)  Finally, Lemma 4 gives explicit bounds for the stochastic and deterministic errors of the variance stabilizing transform  Thus, it quantifies the difference to an exact Gaussian regression setting  This result is a generalization of Theorem 1 of Cai et al 
(2010) adapted to our setting with n ≥ 1 observations and correlated observations  √ 2 can be written as where for the proof of the first statement  Furthermore, for x, t ∈ [0, 1] holds In particular, for some constants C 1 , C 2 > 0 depending on γ ∈ (0, 1) but not on h and x, it holds h (iii) See Lemma 15 of with p = 2q − 1 
It is sufficient to show the statement for n = 1 by independence of the Y i   Then, the number of points per bin is m = p/T   For simplicity, the index i is skipped in the following  First, we write Q k as a matrix-vector product and refactor it so that it corresponds to a sum of independent scaled χ 2 random variables 
In the second step, we calculate the scaling factors  Let E (km) be a diagonal matrix with ones on the (k − 1)m + 1,    , km-th entries and otherwise zero diagonal elements  Then, By Theorem 1 of for the gamma distribution it follows where Wi,j iid  ∼ Γ(1/2, 2 f (x * k )) and such that Cov( Wi,j , Wi,h ) = Cov(W i,j , W i,h ) for j = (k − 1)p/T + 1,    , kp/T and h ∈ {1,    , p} \ {(k − 1)p/T + 1,    , kp/T } 
Let θ be the maximum difference of the observations' means in each bin  Then, θ = max are defined via quantile coupling, it holds Z k = Φ −1 {F Q( Qk )} (see   Furthermore, define the uniform random variables Let ρ = Cov(Z k , Z l )  Then, the identity implies F Z,Z (x, y) − Φ(x)Φ(y) ≥ 0 for all x, y ∈ R ⇐⇒ ρ ≥ 0, (see  
Since Cov( Qk , Ql ) ≥ 0 and the ratio of two densities is non-negative, x = − 2/m, it follows that f Q(x) is monotone decreasing for x ≥ − 2/m  Furthermore, F Q(− m/2) ≤ 0 5 for all m ∈ N as f Q(x) is right-skewed  In particular, − m/2 ≤ F −1 Q (1/2) for all m ∈ N  Finally, since f Q(− 2/m) → φ(0) for m → ∞ there is a constant c > 0 not depending on m such that
The simulation study in Section 5 is performed in the same way, but with the uniform and the gamma distribution instead of the Gaussian distribution 
Context 3: 
Recent works have mainly focused on deep learning based language identification, where handling codemixed data is a big challenge in the domain  For our experiments, we use a version of XLM-RoBERTa finetuned on a language identification dataset 2   This model has near-perfect test accuracy of 99 6% 

Dialect Classification

Dialect classification has been previously solved using statistical methods like Gaussian Mixture Models and Frame Selection Decoding or Support Vector Machines (SVM)   It has been explored relatively sparsely, mostly in the case for local languages   Deep learning approaches have been explored in previous editions of the VarDial workshop shared tasks and otherwise  
Dialect classification was also explored previously as a part of other shared tasks   We want to stress that given the multilingual nature of the dataset, using the present methods directly was not an option  In our work, although we take inspiration from the previous works, we propose a novel system that surpasses the performance of the previous systems by a large margin 

Data

The dataset  We observed that the class PT-BR had the most number of samples (2,724) and the class EN had the least number of samples (349), and thus the imbalance ratio was almost 1:8  We have illustrated the data distribution in Figure   We tried to mitigate this imbalance using over-sampling and weighted sampling methods 
However, the improved data sampling method did not affect the performance 

System Description

This was a problem of multi-class classification having 9 classes for Track-1 and 6 classes for Track-2  The samples were belonging to 3 languages having 3 varieties each, so the classification pipeline was made in 2 stages  The Language Identification (LID) model which is the first stage classifies the sentence into 3 languages: English (EN), Spanish (ES) and Portuguese (PT) 
The LID is a pretrained XLM-RoBERTa that is fine-tuned for the task of language identification  It is able to classify the input sentence into 20 languages  We classify and separate the samples according to their language  The samples corresponding to the specific languages are then fed into the language specific models for dialect identification 
For dialect identification we have used models like BERT and RoBERTa with a linear layer connected to the pooler output of the models  Then fine-tuning is done on the models for dialect identification   using the samples corresponding to the specific languages  For the task of dialect identification we experimented with several pretrained models like XLM-RoBERTa, BERT, ELECTRA, GPT-2 and RoBERTa 
All models were fine-tuned for 20 epochs with a learning rate of 1e-6 and weight decay 1e-6 with a batch size of 8  The best performing model checkpoint was chosen according to the epoch-wise validation macro-F1 score  5 Experiments and Results

Experiments using Large Language Models

For the task of Dialect Identification we have tried various language specific models like XLM-RoBERTa, BERT, ELECTRA, RoBERTa and GPT- 2  The base variant of all these models were used and all the models were used through the Hugging-Face library  The pooler output of these models was passed through a linear layer and the models were fine-tuned 
First, we experimented with different models for Track-1  All the models were trained for 20 epochs with learning rate 1e-6, weight decay 1e-6 and a batch size of 8  We used XLM-RoBERTa as the baseline for all 3 languages  The best performing models for the English language were RoBERTa and BERT whereas GPT-2 was the worst performing 
Similarly the language specific versions of RoBERTa and BERT performed well for the Spanish and Portuguese respectively  Overall the worst performing model was GPT-2 across all 3 languages  The validation F1 scores are present in Table   The two best-performing models for every language were chosen for Track-2 
The same procedure as specified above was used and the F1 scores are present in Table   The train and validation F1 scores for 2-class classification are higher for all models as compared to the F1 score of the same models for 3-class classification  This was mainly due to the poor representation and accuracy of classification of the third class 
We observed symptoms of overfitting in all models after 12-15 epochs and the best validation F1 score was obtained in the range of 4-8 epochs 

LID experiments

The pipeline for dialect identification is divided into two parts as the sentences in the dataset belong to different languages  The stages are described in Section 4  The XLM-RoBERTa we have used for language classification has a test accuracy of 99 6% meaning it correctly classifies all input sentences and hence, can be considered as a perfect classifier 
For the final pipeline we experimented using the two best performing models for each language in Track-1 and Track-2  For both the tracks we experimented with all 8 (2 3 ) possible combinations of models and calculated the validation F1 score for the combined validation dataset which had sentences belonging to all languages 
The validation scores for Track-1 and Track-2 are shown in Table and Table respectively  For both the tracks, the three pipelines with the best validation F1 scores were chosen for submission 

Using 3-way classifier as a 2-way classifier

In Track-1, participants are expected to train a classifier which classifies amongst 9 classes, and in Track-2, participants are expected to train a classifier which classifies amongst 6 classes



ground_truth: Decorrelating the data before running the PLS algorithm improves the performance of the algorithm.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What was the best performing model for the Spanish language in Track-1?

Read the following text and answer briefly.

Context 3:  These 6 classes are a proper subset of the 9 classes from Track-1  Thus, an intuitive baseline for Track-2 is to use the model finetuned for Track-1, whilst considering only the relevant classes for the latter task 
The classes EN , ES and P T , i e  the classes without any national dialect associated with them are not included in Track-2 as compared to Track-1  Thus, we calculate the predictions for the Track-2 validation dataset using the models for Track-1 and exclude the metrics for Track-1 specific classes to get the metrics for this "adapted" 2-way classification 
We show the results of this experiment in Table and observe that, as expected, the adapted 2-way classification performs worse compared to the explicitly finetuned variant 

Results for Track-1 and Track-2

We now present our experiments and their performance for both tracks  Our experiments for Track-1 are described in Table and our experiments for Track-2 are described in Table   The participants were allowed three submissions for evaluation on the test set, so we submitted predictions using the three systems which performed the best on the validation set 
As mentioned in Section 5 2, we performed 2 3 , i e  a total of 8 experiments using the two best models for each language  We observed that RoBERTa base on English, Spanish BERT base on Spanish and Portuguese BERT base performed the best on the testing set for Track-1  The same combination, with RoBERTa base for English, worked best for Track-2 
All of our submissions were the top submissions for each track, which surpassed the next best competitors by a margin of 4 5% and 5 6% for Track-1 and Track-2 respectively 

Ablation of best submissions

We hereby make some observations of our submissions and other experiments  To assist this, we plot the confusion matrices of our best submissions for Track-1 and Track-2 in Figures respectively  Note that these confusion matrices have their rows (i e  true labels axes) normalized according to the number of samples in the class 
Here are observations from our experiments: 1  BERT-based models outperform other models across all languages: We observe that BERT-based models outperform ELECTRA-based and GPT-2-based models, as shown in Table   We speculate this is because of the inherent architecture of BERT, which combines semantic learning with knowledge retention 
This combination of traits is particularly useful for this task  2  Common labels perform the worst across all languages: We observe that the common labels EN , ES and P T perform the worst, both in the individual as well as the two-stage setup  We hypothesize this is because of the absence of dialect specific words, or words that are specific to the geographical origin of the national dialect (for example, "Yankees" for EN-US and "Oxford" for EN-GB) 
3  English models work better than models of other languages: It can be noted from Figures 4 and 3 that the English models have the best performance across all classes  This can be attributed to two reasons: absence of national dialect specific words and lesser pretraining data in the case of Portuguese 
4  British English is most correctly classified class: We can observe that the Spanish or Portuguese models make equal number of mistakes in the case of either national dialect, in the case of Track-2 (see Figure )  However, in the case of English, the label EN − GB is correctly classified for more than 95% of the cases 
We speculate this is because British English involves slightly distinctive grammar and semantics, which help the model separate it from other classes  5  The proposed 2-step method is scalable for multiple language dialect classification: We can strongly assert that the novel 2-step deep learning method for multilingual dialect classification is a scalable method for the task due to two specific reasons: firstly, the multilingual models (like XLM-RoBERTa) might not have the vocabulary as well as the learning capabilities to learn the minute differences between individual dialects 
Secondly, this system can be quickly expanded for a new language by simply adding a language specific dialect classifier, provided the language identification model supports that particular language 

Conclusion

In this paper we propose a two-stage classification pipeline for dialect identification for multilingual corpora  We conduct thorough ablations on this setup and provide valuable insights  We foresee multiple future directions for this work  The first is to expand this work to many languages and dialects 
Secondly, it is a worthwhile research direction to distill this multi-model setup into a single model with multiple prediction heads  The obvious limitation of this system is the excessive memory consumption due to the usage of language specific models  For low resource languages this system is difficult to train and scale 
We hope that these problems will be addressed by researchers in future works 



ground_truth: The best performing model for the Spanish language in Track-1 was Spanish BERT.