# C_2020424EN.01000101.xml

| 8.12.2020 | EN | Official Journal of the European Union | C 424/1 |
| --- | --- | --- | --- |

8.12.2020

EN

Official Journal of the European Union

C 424/1

---

COMMISSION NOTICE

Guidelines on ranking transparency pursuant to Regulation (EU) 2019/1150 of the European Parliament and of the Council

(2020/C 424/01)

Table of contents

| 1. | INTRODUCTION | 3 |
| --- | --- | --- |

1.

| 1.1. | Purpose of the guidelines | 3 |
| --- | --- | --- |

1.1.

| 1.2. | Ranking transparency – background | 4 |
| --- | --- | --- |

1.2.

| 1.3. | Objectives and general considerations | 4 |
| --- | --- | --- |

1.3.

| 1.3.1. | User-oriented approach | 4 |
| --- | --- | --- |

1.3.1.

| 1.3.2. | Individual assessment and technologically neutral approach | 5 |
| --- | --- | --- |

1.3.2.

| 1.3.3. | The right level of detail | 5 |
| --- | --- | --- |

1.3.3.

| 2. | NATURE AND SCOPE OF THE REQUIREMENTS OF ARTICLE ? | 6 |
| --- | --- | --- |

2.

| 2.1. | Integrated services | 6 |
| --- | --- | --- |

2.1.

| 2.1.1. | Paid ranking | 6 |
| --- | --- | --- |

2.1.1.

| 2.1.2. | Describing ranking | 7 |
| --- | --- | --- |

2.1.2.

| 2.1.2.1. | General approach | 7 |
| --- | --- | --- |

2.1.2.1.

| 3. | WHAT ARE MAIN PARAMETERS AND HOW TO SELECT THEM? | 8 |
| --- | --- | --- |

3.

| 3.1. | Examples of ranking parameters | 8 |
| --- | --- | --- |

3.1.

| 3.2. | Selecting the main parameters | 8 |
| --- | --- | --- |

3.2.

| 3.3. | Specific considerations when identifying the main parameters | 9 |
| --- | --- | --- |

3.3.

| 3.3.1. | Personalisation | 9 |
| --- | --- | --- |

3.3.1.

| 3.3.2. | Consumer search behaviour and intent | 10 |
| --- | --- | --- |

3.3.2.

| 3.3.3. | User’s history | 10 |
| --- | --- | --- |

3.3.3.

| 3.3.4. | Default settings, sorting & filtering mechanisms | 10 |
| --- | --- | --- |

3.3.4.

| 3.3.5. | Cross-platform presence | 11 |
| --- | --- | --- |

3.3.5.

| 3.3.6. | Other external factors | 11 |
| --- | --- | --- |

3.3.6.

| 3.3.7. | Third party notifications | 11 |
| --- | --- | --- |

3.3.7.

| 3.3.8. | Randomisation | 11 |
| --- | --- | --- |

3.3.8.

| 3.3.9. | Housekeeping/tidying up | 12 |
| --- | --- | --- |

3.3.9.

| 3.3.10. | Relationship with ancillary services | 12 |
| --- | --- | --- |

3.3.10.

| 3.3.11. | Use of providers’ technical tools | 12 |
| --- | --- | --- |

3.3.11.

| 3.3.12. | Effect of machine learning | 12 |
| --- | --- | --- |

3.3.12.

| 3.3.13. | Evaluation of websites | 13 |
| --- | --- | --- |

3.3.13.

| 3.3.14. | Measures taken to avoid third party bad faith manipulation of ranking results | 13 |
| --- | --- | --- |

3.3.14.

| 3.3.15. | User reviews | 13 |
| --- | --- | --- |

3.3.15.

| 3.3.16. | Providers’ measures against illegal content | 13 |
| --- | --- | --- |

3.3.16.

| 4. | HOW TO SELECT THE MAIN PARAMETERS AND STILL PREVENT BAD FAITH MANIPULATION OF RANKING | 13 |
| --- | --- | --- |

4.

| 5. | SPECIFIC DESCRIPTIONS REQUIRED – DIRECT & INDIRECT REMUNERATION | 14 |
| --- | --- | --- |

5.

| 5.1. | General approach | 14 |
| --- | --- | --- |

5.1.

| 5.2. | Direct remuneration | 15 |
| --- | --- | --- |

5.2.

| 5.3. | Indirect remuneration | 15 |
| --- | --- | --- |

5.3.

| 6. | HOW SHOULD MAIN PARAMETERS BE DESCRIBED? | 15 |
| --- | --- | --- |

6.

| 6.1. | General approach | 16 |
| --- | --- | --- |

6.1.

| 6.2. | Level of detail | 16 |
| --- | --- | --- |

6.2.

| 6.3. | Level of detail for direct and indirect remuneration | 16 |
| --- | --- | --- |

6.3.

| 6.4. | Requirement for the description to be drafted in plain and intelligible language | 17 |
| --- | --- | --- |

6.4.

| 6.5. | Presentational tools | 17 |
| --- | --- | --- |

6.5.

| 7. | WHERE THE MAIN PARAMETERS SHOULD BE DESCRIBED | 17 |
| --- | --- | --- |

7.

| 7.1. | Online intermediation services | 17 |
| --- | --- | --- |

7.1.

| 7.2. | Search engines | 18 |
| --- | --- | --- |

7.2.

| 8. | WHEN SHOULD THE DESCRIPTION OF THE MAIN PARAMETERS BE CHANGED? | 18 |
| --- | --- | --- |

8.

| 8.1. | Keeping the descriptions up to date | 18 |
| --- | --- | --- |

8.1.

| 8.2. | Temporary changes | 19 |
| --- | --- | --- |

8.2.

| 8.3. | Experiments | 19 |
| --- | --- | --- |

8.3.

| 9. | STEPS TO BE TAKEN WHEN THE DESCRIPTION OF THE MAIN PARAMETERS IS UPDATED | 20 |
| --- | --- | --- |

9.

| 9.1. | Requirement for online intermediation services to notify proposed changes | 20 |
| --- | --- | --- |

9.1.

| 9.2. | Requirement for online search engines to keep description up to date | 20 |
| --- | --- | --- |

9.2.

| 10. | SPECIFIC OBLIGATIONS ON SEARCH ENGINES TO ENABLE CORPORATE WEBSITE USERS TO INSPECT THE CONTENTS OF THIRD PARTY NOTICES | 20 |
| --- | --- | --- |

10.

1.   INTRODUCTION

1.1.   Purpose of the guidelines

| 1. | These Guidelines aim to facilitate the compliance of providers of online intermediation services (1) and providers of online search engines (2) with, and the enforcement of, the requirements provided in Article 5 of Regulation (EU) 2019/1150 of the European Parliament and of the Council of 20 June 2019 on promoting fairness and transparency for business users of online intermediation services (3) (hereafter – the ‘Regulation’). In line with Article 5(7) and recital 28 of the Regulation, these Guidelines also aim to assist providers in applying the requirements and help optimise the manner in which the main parameters determining ranking are identified and presented to business users and corporate website users (4). |
| --- | --- |

1.

| 2. | Article 5(1) and (2) require, in essence, providers to set out the main parameters that determine ranking and the reasons for the relative importance of those parameters as opposed to other parameters. |
| --- | --- |

2.

| 3. | Article 2(8) defines ‘ranking’ as ‘the relative prominence given to the goods or services offered through online intermediation services, or the relevance given to search results by online search engines, as presented, organised or communicated by the providers of online intermediation services or by providers of online search engines, respectively, irrespective of the technological means used for such presentation, organisation or communication.’ |
| --- | --- |

3.

| 4. | According to Article 5(5), each provider is to provide a description that gives users an adequate understanding of whether – and if so, how and to what extent – the ranking mechanism takes account of: (a) the characteristics of the goods or services offered through the provider’s service; (b) the relevance of those characteristics to the consumers using that service; and (c) solely as regards providers of online search engines, the design characteristics of the website used by the corporate website users. |
| --- | --- |

4.

| 5. | As explained in Recitals 24 and 26, the description to be given by providers aims to improve predictability and help users improve the presentation of their goods and services, or a characteristic of those goods and services. |
| --- | --- |

5.

| 6. | At the same time, the Regulation seeks to achieve this objective without requiring providers to disclose algorithms or any information that, with reasonable certainty, would result in the deception of consumers or consumer harm through the manipulation of search results (Article 5(6)). Thus, providers are not required to disclose the detailed functioning of their ranking mechanisms, including algorithms, and their ability to act against bad faith manipulation of ranking should not be impaired (Recital 27). |
| --- | --- |

6.

| 7. | Considering the partially diverging legal requirements for providers of online intermediation services on the one hand and providers of online search engines on the other hand, which are set out in the provisions of Article 5, and given the different nature of the services concerned, the content of the required description of the main ranking parameters will necessarily have to differ between the two types of services. Moreover, as acknowledged in Recital 25, the content, including the number and type of main parameters, can also vary strongly between providers of online intermediation services themselves. |
| --- | --- |

7.

| 8. | The guidance contained in these Guidelines should not be applied mechanically, but rather with due consideration to the relevant facts and circumstances at issue in each individual case. The examples given are for illustrative purposes to aid understanding. Where they describe specific situations, they should not be interpreted as limiting the scope of the obligation concerned to the particular situation described. In addition, examples may be given for a specific sector, but the concepts may be transferable and could be understood to apply to similar methods/situations used in other sectors. |
| --- | --- |

8.

| 9. | These Guidelines are not legally binding. They are without prejudice to the providers’ own responsibilities to ensure compliance with the requirements of Article 5 and the powers and responsibilities of the competent authorities and courts of the Member States with respect to the enforcement of those requirements in accordance with the provisions of the Regulation and other provisions of EU law. It is ultimately solely for the Court of Justice of the EU to interpret the requirements. |
| --- | --- |

9.

| 10. | The Commission will continue to monitor the manner in which the requirements of Article 5, as clarified by these Guidelines, are applied and may decide to revise these Guidelines where necessary in the light of future developments and evolving insights. |
| --- | --- |

10.

1.2.   Ranking transparency – background

| 11. | Ranking is defined in Article 2(8), cited above, and can essentially be thought of as a form of data-driven, algorithmic decision-making. When providers present, organise or communicate information on goods or services for consumers or search results, they ‘rank’ results on the basis of certain parameters. |
| --- | --- |

11.

| 12. | As explained in Recitals 24 and 26, the ranking of goods and services by providers has an important impact on consumer choice and, consequently, on the commercial success of the users offering those goods and services to consumers. As the economy continues to move online, ranking in search and on online intermediation services becomes increasingly central to entrepreneurship and EU businesses’ commercial success will increasingly depend on their visibility and discoverability online. |
| --- | --- |

12.

| 13. | Providers design their ranking methods, including proprietary algorithms, in different ways. These approaches to ranking are frequently adapted and in general not disclosed. From the consumers’ perspective, the quality of search results may differentiate services and, therefore, the detailed functioning of ranking methods may be a provider’s competitive edge or trade secret. Businesses trading online therefore do not always know the reasons for their performance in ranking or whether and how they could perform better, potentially with the help of paid ranking, despite their ability to reach customers critically depending on this knowledge. |
| --- | --- |

13.

| 14. | Therefore, whilst the Regulation does not limit the choice of the ranking parameters that providers may want to use, Article 5 aims to improve predictability for users. |
| --- | --- |

14.

1.3.   Objectives and general considerations

| 15. | As explained in Recitals 24 and 26, the description to be given by providers pursuant to Article 5 aims to improve predictability and help users improve the presentation of their goods and services, or a characteristic of those goods and services. Recital 24, explains that predictability entails that providers determine ranking in a non-arbitrary manner. |
| --- | --- |

15.

| 16. | A number of ‘general considerations’ can be derived from this predictability standard, which can assist providers in applying the requirements of Article 5 and facilitate the enforcement of those requirements: |
| --- | --- |

16.

1.3.1.   User-oriented approach

| 17. | Given the aforementioned objective of improving predictability and protecting the legitimate interests of users as well as the requirement of enabling users to obtain an ‘adequate understanding’ (recital 27), providers should give meaningful explanations of their ranking mechanisms and, in particular, the main parameters used. For the explanations to be meaningful to users, they should take account of the nature, technical ability and needs of ‘average’ users of a given service, which may vary considerably between different types of services. |
| --- | --- |

17.

| 18. | For providers of online intermediation services, pursuant to Article 5(1), the required information has to form part of their terms and conditions. Article 3(1)(a) requires that terms and conditions are drafted in plain and intelligible language. |
| --- | --- |

18.

| 19. | In respect of providers on online search engines, Article 5(2) equally requires that the information on the main parameters are drafted in plain and intelligible language. |
| --- | --- |

19.

1.3.2.   Individual assessment and technologically neutral approach

| 20. | Providers should individually determine what measures they need to take to comply with the requirements of Article 5. Where they provide multiple services, they should in principle do so separately for each service covered by the Regulation. Likewise, users, competent authorities and courts should base themselves on an individual, case-by-case assessment as to whether or not those requirements are complied with. This individual approach is particularly important given that certain providers supply several services (online intermediation and/or online search) as part of a single corporate structure, or even a single user interface, whilst the description needed for users to adequately understand the ranking mechanisms used can vary depending on the service at issue. |
| --- | --- |

20.

| 21. | In addition, Article 2(8), which defines the term ‘ranking’, indicates that the technological means used by providers to present, organise or communicate the goods or services offered by users through the services at issue here are irrelevant. Therefore, the individual assessment should be conducted in a technologically neutral manner, in the sense that the particular technology used as part of the ranking process is not decisive. |
| --- | --- |

21.

1.3.3.   The right level of detail

| 22. | The descriptions given by providers in accordance with Article 5 should provide real added-value to the users concerned. Articles 5(1) and (2) require that providers give information not only of the main parameters, but also the reasons for the relative importance of those main parameters as opposed to other parameters. In addition, pursuant to Article 5(5), the users should be enabled to obtain an ‘adequate understanding’ of whether and if so, how and to what extent three particular factors are taken into account (5). This means that the description to be provided has to go beyond a simple enumeration of the main parameters, and provide at least a ‘second layer’ of explanatory information (6). Providers could, for example, consider describing the company-internal ‘thought process’ that was used for identifying the ‘main parameters’, as a way to also derive the ‘reasons for their relative importance’. |
| --- | --- |

22.

| 23. | Recital 27 clarifies that the description required may be general, but that it should give users an adequate understanding and, moreover, should at least be based on ‘actual data on the relevance of the ranking parameters used’. That Recital also clarifies that, whilst disclosing the detailed functioning of ranking mechanisms, including algorithms, is not required, providers’ commercials interests cannot justify a refusal to disclose the main parameters determining ranking in accordance with Article 5. |
| --- | --- |

23.

| 24. | Under Article 5(1) and (2), the description to be given by providers must relate to all main parameters determining, respectively that are most significant for, ranking. Therefore, this obligation applies also in respect of factors such as temporary changes, randomness or personalisation, in as far as such factors constitute main parameters. |
| --- | --- |

24.

| 25. | However, an excess of information can mean that, in effect, no meaningful information is provided to users. Providers should accordingly on the one hand identify and adequately explain the main ranking parameters, whilst on the other hand not overwhelm users with too lengthy or complicated descriptions, or descriptions of parameters other than the main ones. Not providing excessive details should also help avoid the risk of enabling the deception of consumers or consumer harm, as referred to in Article 5(6). |
| --- | --- |

25.

2.   NATURE AND SCOPE OF THE REQUIREMENTS OF ARTICLE 5

2.1.   Integrated services

| 26. | The requirements of Article 5 apply to providers of online intermediation services and online search engines. If providers of online search engines are (vertically or horizontally) integrated, part of what they offer may be online intermediation services as well. Similarly, ‘conglomerate’ providers of online intermediation services may incorporate several online intermediation services in a single corporate structure. As explained above, in such cases, the obligations of Article 5 will then in principle apply separately for each online search engine as well as for each online intermediation service provided, as otherwise it will normally not be possible to provide meaningful explanations to the users concerned. In this respect, the following issues could arise. |
| --- | --- |

26.

| 27. | Services offered through different devices. If providers offer services through different devices, they should assess whether separate explanations may be needed for each device. If the service operates in the same way on all devices and the same main parameters apply to a single ranking mechanism, under Article 5 no separate explanation seems, generally speaking, required. However, if the service operates independently and differently on different devices, in such a manner that different main parameters are used, then a separate explanation does seem required in principle. |
| --- | --- |

27.

| 28. | Services operating cross-border and through seamless consumer interfaces. Situations may exist where a provider operates a storefront embedded in a marketplace service (a type of online intermediation services), and where online intermediation services or online search engines operate in different territories using different or even multiple domain names. If providers incorporate such a storefront embedded in their marketplace service, they should assess whether these are a part of this marketplace service or a distinct activity. This is relevant in determining whether and how the functioning of the storefront has to be included in a description pursuant to Article 5. Depending on the situation, one or several such descriptions may be required, and the presence of these storefronts may have to be described in different ways. The same applies to the use of different or multiple domain names |
| --- | --- |

28.

| 29. | Syndication or sub-syndication of search results. Another situation that could arise is one where a provider of online search engine uses syndication or sub-syndication of search results, including ‘outsourcing’ the relevant activities to third-party online search engines (7). If search results are effectively sourced from third-party providers of online search engines, it should not relieve the providers of the online search engines concerned from complying with the obligation in Article 5, insofar as the relevant activity falls within the scope of that provision. |
| --- | --- |

29.

| 30. | Integration of business users’ offers on third-party online intermediation services. Another situation that could arise in practice is one where a provider of online intermediation services integrates business users’ offers on third-party online intermediation services, for instance ‘metasearch engines’ that integrate links to business users’ offers on ‘online travel agents’.Where applicable, providers should assess whether and how the presence of these offers has to be included in a description pursuant to Article 5. Depending on the situation, one or several such descriptions may be required, and the presence of these offers may have to be described in different ways. |
| --- | --- |

30.

| 31. | The use of ‘buy buttons’ in social media. So-called ‘buy buttons’ used by professionals can be integrated into online social media services. If this is the case, providers should assess whether these ‘buy buttons’ are a part of broader online intermediation services or a distinct activity which may constitute online intermediation services in and of itself. This is relevant in determining whether and how the functioning of the buy buttons has to be included in a description pursuant to Article 5. Depending on the situation, one or several such descriptions may be required, and the presence of these buy buttons may have to be described in different ways. |
| --- | --- |

31.

2.1.1.   Paid ranking

| 32. | In respect of direct or indirect remuneration possibly being one of the main parameters determining ranking (‘paid ranking’), Article 5(3) contains an explicit obligation to set out a description of any such paid ranking possibilities and of the effects of such remuneration on ranking – where such possibilities are among the main parameters determining ranking. |
| --- | --- |

32.

| 33. | According to Article 2(8), ranking turns around ‘the relative prominence given to the goods or services offered through the online intermediation services’ (8). Banner or display advertising can in certain cases actively influence the ‘relative prominence’ of the good or services offered by users. Where that is so and the other requirements of Article 2(8) are also met, and the relevance of this parameter is moreover such that it qualifies as a ‘main’ parameter, Article 5(3) applies. In other cases, Article 5(3) does not apply to such advertising. The fact that the same online advertising tools or online advertising exchanges, which are excluded from the scope of the Regulation, (9) may be used to facilitate ‘on-platform’ and ‘off-platform’ advertising should in general not be considered a decisive factor in this regard.






  Illustrative example

  A mobile app provider pays one and the same provider of online advertising services (advertising networks, serving tools and exchanges) to improve the relative prominence of its app in a mobile app store through ‘paid ranking’ and to simultaneously promote that app by means of a banner advertisement on a set of third-party websites that are part of the same online advertising services provider’s advertising network. The mere fact that the same advertising technology used to serve that banner advertisement on the third-party websites may be used to promote the app directly within the online intermediation services or online search engine concerned does not exclude the ‘advertising’ of that app done within the latter services from the scope of Article 5. The promotion of the app within the relevant provider’s own online intermediation services or online search engines could in principle constitute ‘paid ranking’. |
| --- | --- |

33.

Illustrative example

A mobile app provider pays one and the same provider of online advertising services (advertising networks, serving tools and exchanges) to improve the relative prominence of its app in a mobile app store through ‘paid ranking’ and to simultaneously promote that app by means of a banner advertisement on a set of third-party websites that are part of the same online advertising services provider’s advertising network. The mere fact that the same advertising technology used to serve that banner advertisement on the third-party websites may be used to promote the app directly within the online intermediation services or online search engine concerned does not exclude the ‘advertising’ of that app done within the latter services from the scope of Article 5. The promotion of the app within the relevant provider’s own online intermediation services or online search engines could in principle constitute ‘paid ranking’.

| 34. | Considering that Article 5(3) applies to all providers at issue here, the same applies in respect of providers of online search engines, in the context of which ‘paid ranking’ can cover advertising opportunities that involve paid prominence of corporate website users’ websites or digital applications on the search engine concerned |
| --- | --- |

34.

2.1.2.   Describing ranking

2.1.2.1.   General approach

| 35. | It follows from Article 2(8) that the concept of ‘relative prominence’ relates to how goods and services are offered through the online intermediation services in a variety of circumstances (10). This is confirmed in Recital 24, which states that ‘[r]anking refers to the relative prominence of the offers of business users or relevance given to search results as presented, organised or communicated by providers of online intermediation services ... resulting from the use of algorithmic sequencing, rating or review mechanisms, visual highlights, or other saliency tools, or combinations thereof’. |
| --- | --- |

35.

| 36. | Thus, algorithmic sequencing of results in response to a search query is just one example of how goods or services can be presented, organised or communicated as part of a ranking mechanism covered by Article 5. Indeed, in this regard, goods and services can be presented, organised or communicated to consumers in numerous manners. These include ‘default’ ordering of goods and services that consumers may navigate without using search queries, through the seamless use of different online intermediation services through ‘buy buttons’ (see above), visual exposure (including in online intermediation services that take the form of maps or directories), highlighting, lead generation and editorial interventions, etc. |
| --- | --- |

36.

| 37. | Depending on what consumers see when they look for goods and services on a given service covered by the Regulation, the information presented, organised or communicated may be basic or complicated. For example, at the most basic level, this may be one list of goods with no other features. For other services, this may be more diverse and consumers may see an array of goods or services set out and grouped in a multitude of different ways when they look at any given page on the service, or when they apply different ‘filters’. Providers of online intermediation services may, for example, combine ordered lists of search results with paid ranking results and editorial features such as recommendations, all in a single, seamless interface. Or they may differentiate such ordered lists by, for example, using different markings, annotations or headings, at different positions on the page (i.e. regardless of the ‘position’ of the products or services concerned in the list). |
| --- | --- |

37.

| 38. | Given the varying degrees of complexity of ranking and given the potentially broad scope of the technologically neutral notion of ‘relative prominence’, providers should take all relevant features of ranking into account when determining what the ‘main parameters’ are for their specific service. |
| --- | --- |

38.

3.   WHAT ARE MAIN PARAMETERS AND HOW TO SELECT THEM?

3.1.   Examples of ranking parameters

| 39. | Once providers have carried out the case-by-case assessment of what are the parameters determining ‘ranking’ as defined in Article 2(8), they will need to identify the main parameters, as referred to in Article 5(1) and (2). Examples of types of ranking parameters are set out in Annex A. |
| --- | --- |

39.

3.2.   Selecting the main parameters

| 40. | For some services, there may be a large number of ranking parameters. For others, it may be more limited. Especially where there is a large set of parameters, the provider will need to select the most relevant ones (cf. Recital 24). More specifically, pursuant to Article 5(1), relating to online intermediation services, the providers concerned have to set out the ‘main parameters determining ranking’, as well as ‘the reasons for the relative importance of those main parameters’. Pursuant to Article 5(2), relating to online search engines, the providers concerned have to set out the ‘main parameters, which individually or collectively are most significant in determining ranking’, as well as the ‘relative importance
  of those main parameters’. |
| --- | --- |

40.

| 41. | To determine which set of parameters are ‘determining’ (Article 5(1)) or ‘most significant in determining’ (Article 5(2)) ranking, and are to be considered the ‘main’ ones, providers could consider what drove the design of the algorithm in the first place. For example, whether it was driven by a desire to ensure that consumers found goods or services that were local, cheap, of high quality, etc. This approach reflects one of the objectives of Article 5, which is explained in Recital 24: ‘help business users to improve the presentation of their goods and services, or some inherent characteristics of those goods or services’. As a best practice, this company-internal reflection could be combined with giving thought to what the provider considers is the top type of result on its service. Similarly, the provider could consider what would make the services’ consumers most satisfied. This may relate to what the provider does to serve the interests of the consumers on its service, for example by considering why it chose the filters that are offered to consumers to sort results on its services or other similar techniques. |
| --- | --- |

41.

| 42. | Where a provider identifies a large number of ranking parameters, it could consider dividing them into several broad categories and then consider what are the subcategories or nuances that fall under them. If numerous different categories of parameters are concerned, providers could consider working backwards to eliminate those that only have a peripheral role in determining ranking. This would then leave only the main parameters, which would then need to be categorised and subcategorised as necessary. This could enable the provider to determine and explain what is the ’relative importance’ of the ‘main parameters’ identified. |
| --- | --- |

42.

| 43. | A best practice that can help users understand how ranking functions is to examine the more unexpected elements that determine ranking. These could also be factors that a user may assume are irrelevant as they are unrelated to the quality of the good or service they offer through the service. |
| --- | --- |

43.

| 44. | A best practice is to help users’ understanding by including a high-level description of the company-internal process conducted by the provider concerned that led it to conclude what the ‘main’ parameters are. This could also help providers meet the requirement in Article 5(5) of enabling users to obtain ’an adequate understanding‘ of certain particular factors relating to ranking (see Section 6.2 below). |
| --- | --- |

44.

| 45. | In a similar way, in making the decision on what are the main parameters, providers could consider what would be most useful to know for the users concerned. For business users, the transparency in respect of ranking serves to improve predictability and to help them improve the presentation of their goods and services, or some inherent characteristics of those goods or services. For corporate website users this is to obtain an adequate understanding of whether, and if so how and to what extent, certain design characteristics of the website used, such as their optimisation for display on mobile telecommunications devices, is taken into account. |
| --- | --- |

45.

| 46. | It is important to note that ‘relative importance’ does not require the exact weighting of the main parameters, or indeed the providers’ algorithms, to be disclosed, (cf. Article 5(6) and Recital 27). However, providers should capture to some extent the inherently dynamic nature of (typically algorithm-driven) ranking in a more static description. Temporary changes that may be foreseeable and regular, such as sales promotions or the influence of the weather, could accordingly, as a matter of principle, be described in a general manner, without necessarily triggering a need to adapt the description of ranking each time the temporary change kicks in. Major alterations to the ranking mechanisms leading the ‘main parameters’ to change, including ‘evolutionary’ changes that occur over time as a result of a degree of deep-learning, would trigger the need to adapt the description given (11). |
| --- | --- |

46.

| 47. | At a more general level, the main parameters identified should be a true reflection of what determines ranking under the mechanism in question. What are described as the main parameters should be what is actually most important in determining ranking. Therefore, it is irrelevant whether or not the parameters are factors that can be influenced by users. Similarly, providers cannot just refer to the fact that ranking is determined by algorithms that use artificial intelligence. To understand further the relationship between machine learning and the obligation to disclose the main parameters, see 3.3.12 below. |
| --- | --- |

47.

3.3.   Specific considerations when identifying the main parameters

| 48. | When conducting the necessary assessment to identify the main ranking parameters, in line with the general principles outlined above, providers and other parties could specifically take account of the following considerations. |
| --- | --- |

48.

3.3.1.   Personalisation

| 49. | Personalised ranking of offers is a widely used feature. Although the ranking outcomes can hypothetically differ for each individual consumer, parameters used to personalise results (which can be multiple) normally apply in the same way to all the offers of goods or services by users on the services concerned. In other words, the parameters that personalise results are objectively determined beforehand even if their application results in one consumer seeing a different ranking compared to another consumer using the same online intermediation services or online search engine. |
| --- | --- |

49.

| 50. | As explained in Recital 24, predictability entails that providers determine ranking in a non-arbitrary manner. Where this is a main parameter, an explanation of the use of personalisation, its key features as well as its impact on ranking, including the volatility (i.e. the extent of the difference in ranking for different consumers), (12) could help improve predictability and avoid the risk of ranking being determined in an arbitrary manner. Providers may therefore have to reflect on how and if so, to what extent personalisation affects ranking in the case of their specific online intermediation services or online search engines. |
| --- | --- |

50.

| 51. | This could require that providers analyse the potentially very long list of factors that are used for this personalisation, such as consumers’ personal profiles, interests, search behaviour, their actual geographic location, the time of day the search takes place, their use of cookie blockers or other technical tools and more generally, the wealth of data held on the specific consumer as well as their use of default settings (i.e. their ability to undo default setting or to apply filtering mechanisms (see 3.3.4 below)). |
| --- | --- |

51.

| 52. | Given that certain providers may try to capture unique audiences for unique purposes in order to differentiate their respective business models, (13) they could also consider the composition of the consumers using the specific services when considering the impact of personalisation on ranking. |
| --- | --- |

52.

| 53. | Providers should consider whether and, if so, how consumers can and do use privacy protection settings on their services, given that the use of such settings can affect the possibility to personalise ranking.






  Illustrative example

  An online marketplace ‘personalises’ ranking by assigning up to 10 000 predefined ‘features’ to individual consumers. The provider finds that this can be highly effective in ordering its business users’ offers (i.e. in ranking), although the effectiveness of its personalisation mechanism is heavily affected by the privacy settings used by its consumers. Depending on the number, heterogeneity and privacy preferences of consumers active on the service concerned, ranking outcomes can therefore vary greatly. Also, the provider may have designed its ranking mechanism to weigh other factors, such as paid ranking more heavily in cases or at times where personalisation is less effective. In such a situation, the provider may need to include these factual insights in the description of its main parameters determining ranking and the reasons for their relative importance, so as to allow the business users to properly understand the ranking mechanism, without however overburdening or confusing the business users by listing thousands of individual features of the mechanism used. |
| --- | --- |

53.

Illustrative example

An online marketplace ‘personalises’ ranking by assigning up to 10 000 predefined ‘features’ to individual consumers. The provider finds that this can be highly effective in ordering its business users’ offers (i.e. in ranking), although the effectiveness of its personalisation mechanism is heavily affected by the privacy settings used by its consumers. Depending on the number, heterogeneity and privacy preferences of consumers active on the service concerned, ranking outcomes can therefore vary greatly. Also, the provider may have designed its ranking mechanism to weigh other factors, such as paid ranking more heavily in cases or at times where personalisation is less effective. In such a situation, the provider may need to include these factual insights in the description of its main parameters determining ranking and the reasons for their relative importance, so as to allow the business users to properly understand the ranking mechanism, without however overburdening or confusing the business users by listing thousands of individual features of the mechanism used.

3.3.2.   Consumer search behaviour and intent

| 54. | Where relevant for meeting the requirements of Article 5, in particular in respect of the required description of the main parameters, providers could consider how and if so, to what extent consumers’ search behaviour affects ranking, potentially as an element in personalised ranking (see 3.3.1 above). |
| --- | --- |

54.

3.3.3.   User’s history

| 55. | Where relevant for meeting the requirements of Article 5, in particular in respect of the required description of the main parameters, providers could consider how and if so, to what extent certain factors that are not directly related to the goods or services that a user offers through the services in question, such as a user’s history or past performance, affect ranking.






  Illustrative example

  In mobile application stores, developers or publishers may be rated separately from their applications, so that, for example, a newly launched application of an experienced developer with an existing offer in that particular application store may perform better in ranking than the application of a newcomer developer. In this case, in as far as such factors constitute main parameters determining ranking, the provider concerned will have to ensure that business users understand that this is the case. The provider may, where relevant, also inform business users of possible ‘correction’ mechanisms, such as temporary boosting of newcomers’ apps. |
| --- | --- |

55.

Illustrative example

In mobile application stores, developers or publishers may be rated separately from their applications, so that, for example, a newly launched application of an experienced developer with an existing offer in that particular application store may perform better in ranking than the application of a newcomer developer. In this case, in as far as such factors constitute main parameters determining ranking, the provider concerned will have to ensure that business users understand that this is the case. The provider may, where relevant, also inform business users of possible ‘correction’ mechanisms, such as temporary boosting of newcomers’ apps.

3.3.4.   Default settings, sorting & filtering mechanisms

| 56. | As a type of personalisation, default settings which can be rearranged, undone, or ‘overridden’ by consumers using sorting or filtering tools can have an important impact on the ranking of offers of goods and services of business users. These mechanisms could be ‘main parameters’ in and of themselves, for example where their use by consumers is very frequent and they are particularly relevant to ranking. If so, it can be important for business users to understand, for example, whether the ranking after application of the filter is based on all offers or if ranking then depends on whether the offer of goods or services meets certain criteria for inclusion, such as a minimum number of reviews in order for goods or services that are referenced under the default settings to be featured under a ‘quality’ filter. The same goes for the size of the possible impact of the different filters on the ranking. |
| --- | --- |

56.

| 57. | In addition, given the logically increased importance of the parameter used (i.e. the filter) in specific filtering mechanisms (e.g. price or review score), other parameters that apply besides the filter might equally increase in importance in terms of the weighting applied to them relative to the situation where no filter is applied. This heavier weighting may namely be necessary to allow a large number of offers that may not differ significantly on elements such as price or review score (i.e. the filter), to still be ranked effectively. Such other parameters may then, depending on the case, constitute ‘main parameters’ under Article 5. Besides those situations, it can be important for business users to understand the possible implications of the use of filters for the relative weight of parameters other than the filter that is applied. |
| --- | --- |

57.

3.3.5.   Cross-platform presence

| 58. | Business users often tend to simultaneously offer their goods or services across different online intermediation services in order to maximise sales (‘multi-homing’). In addition, independent third-party providers exist that collect user reviews of businesses. In practice, some providers may consider the activities or the presence of business users on third party services an indicator of quality or relevance of business users on the online intermediation services in question. Where that is the case, and the relevance of this factor for ranking is such that it is to be considered a ‘main parameter’, those providers should inform users that, and how, this factor is taken into account in ranking. |
| --- | --- |

58.

3.3.6.   Other external factors

| 59. | Users and the goods and services that they offer can also be assessed using other factors that are external to the given online intermediation services or online search engines. Factors such as hotel star ratings, ‘brand appeal’ measured through surveys of fashion professionals, third-party trust marks awarded to a business users’ own retail webshop (e.g. ‘trusted store’) or industry awards (e.g. ‘best cafe in x’) or features in third-party media (e.g. newspaper reports or magazine features) could be relevant for ranking as ‘main parameters’. If so, such factors have to be described, too. |
| --- | --- |

59.

| 60. | Conversely, it cannot be excluded that it may in certain cases be ‘surprising’ to some business users or corporate website users that certain external factors that would appear to be of great significance in view of the nature of the service are not taken into account in determining ranking. For example, not considering a privacy-rating assigned to websites on a privacy-focused online search engine or not considering food labels on a health-oriented food marketplace could be considered surprising for users. The description of the reasons for the relative importance of the main parameters, or of the main parameters which are collectively most significant in determining ranking and the relative importance of those main parameters, should be sufficiently clear so that users can understand that those other factors are not taken into account in ranking. |
| --- | --- |

60.

| 61. | Providers of online intermediation services and online search engines should also consider, as a best practice, whether the description should refer to the sectoral rules, which may have directly or indirectly an impact on the determination of ranking parameters. This includes rules of EU law and of national law reflecting various public interest objectives in various sectors. |
| --- | --- |

61.

3.3.7.   Third party notifications

| 62. | Where third party notifications about offers of goods and services are relevant to ranking as ‘main parameters’, the descriptions given by the providers concerned have to take account of this factor as well. Such relevance could be described in terms of its potential importance in individual cases, as well as in terms of any continuous effects that may result, for example, from a high volume of notifications or from the providers’ policies adopted in relation to the types of illegal content to which third party notifications may relate. As regards the separate obligation for online search engines to allow the inspection of the content of notifications under Article 5(4), see Section 10. |
| --- | --- |

62.

3.3.8.   Randomisation

| 63. | In practice, certain providers may use techniques to reorganise the relative prominence given to offers of goods and services (partly) in a random manner, for example, in order to refresh the look of a landing page. Randomisation can also be applied, for example, to make ranking more dynamic, to distinguish the relevant online intermediation services or online search engine from others. Where such factors constitute ‘main parameters’, the providers concerned have to take this into account in their descriptions. They could, for example, explain how such techniques are applied (by reference to when they are used, how long they are used, how significant the use is, etc.), as well as their impact. Such explanations could provide a facts-based indication of the ‘amplitude’, i.e. the size of the potential impact of randomisation on ranking, which would also help users understand the importance of adopting ranking optimisation techniques for different online intermediation services and online search engines on which they may be present. |
| --- | --- |

63.

3.3.9.   Housekeeping/tidying up

| 64. | Some providers may edit or ‘tidy up’ offers of goods and services by users as presented, organised or communicated on their services. For example, they may ‘treat’ or qualify offers of goods that have featured on their ecommerce online intermediation services without being purchased for a long time as being ‘old’ and thus featured lower for the purposes of ranking. Where such factors amount to ‘main parameters’, a description of the relevant practices will need to be provided. This could be important for predictability purposes, not least given the practices’ potentially large impact on ranking and therefore on the users’ commercial success. |
| --- | --- |

64.

3.3.10.   Relationship with ancillary services

| 65. | In certain cases, providers design their ranking mechanism so that the use by users of ancillary services of the same provider (offered together with the relevant online intermediation services or online search engines, or those offered separately) can affect these users’ ranking. This situation can be an example of indirect remuneration, which requires an additional description pursuant to Article 5(3). Yet even apart from that, the use of ancillary services can also be one of the main parameters that are used to determine ranking as referred to in Article 5(1) and (2). Where it is so, providers have to explain, with an appropriate degree of detail, that this is the case, so that the users can decide whether to use such ancillary services, knowing that these ancillary services, for example, offer not just ease (i.e. all being available from the same provider in a ‘one-stop-shop’) but that their use may be important in optimising sales. For example, this may be the case when fulfilment services offered by the relevant provider outperform regular delivery or postal services in terms of speed, with delivery speed being a ‘main parameter’ determining ranking. Understanding the importance of the use and effect of ancillary services as ‘main parameters’ can therefore be important to improve predictability for users. |
| --- | --- |

65.

3.3.11.   Use of providers’ technical tools

| 66. | As with ancillary services, some providers may offer technical tools to users, possibly in return for remuneration and therefore be covered by Article 5(3). The use or effect of such tools may equally have an impact on ranking and – if the relevance is such that this factor constitutes a ‘main parameter’ within the meaning of Article 5(1) and (2) – therefore require the same approach as described in the previous section. |
| --- | --- |

66.

| 67. | For example, in certain circumstances the use of a data analytics tool available on an online intermediation services or online search engine may enable users to significantly improve their ranking. In this case, in as far this factor is a ‘main parameter’, it will be important for providers to explain whether the effect on ranking is merely due to the insights gained and subsequently applied by users, or whether it is the mere fact they use the analytics tool that is also taken into account. In the latter case, the use of the analytics tool could be a form of indirect remuneration, in which case the specific requirements of Article 5(3) apply as well. The significance of such an analytics tool may, for example, derive from the fact that the insights offered would be unique and enable users to improve their goods or services beyond what these users could do based on own research or other market solutions, or because the tool could help optimise their online presence, on or off-platform. |
| --- | --- |

67.

3.3.12.   Effect of machine learning

| 68. | Providers will have to determine whether machine learning is a ‘main parameter’. If so, even if this is dynamic, it could be captured by a more ‘static’ description. In line with the guiding principles to adopt a user-oriented approach, which provides the right level of detail, providers could focus on explaining the expected ‘amplitude’ i.e. size of the impact of machine learning on ranking. This may include whether it affects all main parameters equally and for example, give an indication of the timing and frequency with which machine learning may lead to ‘main parameters’ changing. If such main parameters do change as a result of machine learning, providers need to adjust the description required under Article 5. |
| --- | --- |

68.

3.3.13.   Evaluation of websites

| 69. | A provider of an online search engine may take into account certain characteristics of websites in order to evaluate them in terms of the trust they enjoy among users or their safety, authenticity, popularity or technological features. If a provider does so, it should consider and explain to what extent its evaluation of a website’s characteristics affect ranking, to the extent that such factors constitute ‘main parameters’. |
| --- | --- |

69.

| 70. | This entails that providers assess on an individual basis what they do to evaluate websites and how this affects ranking. This may include, for example, how providers develop specific scores of websites calculated based on several variables, such as whether a website is trusted by users and whether it has a history of sharing links or misinformation. Thus, the quality of the content or how authoritative the website is may be relevant for its performance in ranking. Alternatively, the focus may be on the popularity of the website measured through, for example, the number of unique visitors and page views in a given period. Similarly, certain technical aspects may be taken into account for example, load speed, mobile-friendliness, domain age or security and accessibility of the website. |
| --- | --- |

70.

| 71. | If certain characteristics of websites are more important than others, and the evaluation of these aspects is reflected in ranking to such an extent that it qualifies as a ‘main parameter’, this is important for users to understand. This should also be reflected in the level of detail that is provided in the description to ensure that users have sufficient clarity. |
| --- | --- |

71.

3.3.14.   Measures taken to avoid third party bad faith manipulation of ranking results

| 72. | Providers tend to use sophisticated fraud prevention mechanisms, which may constitute ‘main parameters’ under Article 5. |
| --- | --- |

72.

| 73. | Providers should therefore carefully assess whether they need to describe the constituent elements of their fraud fighting mechanisms. While by definition those mechanisms cannot be revealed in detail without potentially undermining their purpose and effectiveness, providers should at least inform about their existence and provide high-level information about how they can influence ranking – where these amount to ‘main parameters’. |
| --- | --- |

73.

3.3.15.   User reviews

| 74. | If consumer reviews are a ‘main parameter’ for a given ranking mechanism, the providers concerned should include this factor in their descriptions required under Article 5 with a sufficient level of detail and clarity. For example, it could be explained that the reviews are generated on the relevant online intermediation service or online search engine, or, it may be reviews hosted outside those services. In such cases, it can also be important for users to know what, if any, steps are taken to verify the veracity of such reviews. |
| --- | --- |

74.

3.3.16.   Providers’ measures against illegal content

| 75. | Where providers take measures to tackle illegal content online, they need to consider the effect that these measures may have on ranking and what may need to be included in the description of ‘main parameters’ required under Article 5. The effect on ranking can differ between providers, but in general both the presence and temporary presence of illegal content as well as the removal of illegal content can have a direct impact on the visibility of business users' legitimate offers. Where such measures amount to ‘main parameters’, in deciding what to describe, providers should consider what information and transparency about, for example, the types and prevalence of illegal content on the service in question could help business users better understand the functioning of the ranking mechanisms in question. |
| --- | --- |

75.

4.   HOW TO SELECT THE MAIN PARAMETERS AND STILL PREVENT BAD FAITH MANIPULATION OF RANKING

| 76. | Article 5(6) recognises that providers are ‘not required to disclose algorithms or any information that, with reasonable certainty, would result in the enabling of deception of consumers or consumer harm through the manipulation of search results’. |
| --- | --- |

76.

| 77. | This provision should be read in conjunction with Recital 27, where it is recognised that, ‘[t]o ensure that the objective of the Regulation is achieved, consideration of the commercial interests of providers of online intermediation services or online search engines should, therefore, never lead to a refusal to disclose the main parameters determining ranking’ (14). |
| --- | --- |

77.

| 78. | Accordingly, there are certain limits to the disclosure requirements in relation to ranking set out in Article 5. However, those limits relate not to providers’ commercial interests as such, but rather to potential negative effects on consumers. Nonetheless, providers have several safeguards for their ‘commercial interests’. |
| --- | --- |

78.

| 79. | Firstly, the requirement under Article 5 is to provide a description of the main parameters only (as well as the reasons for their relative importance), without providers being required to disclose the detailed functioning of their ranking mechanisms (recital 27). |
| --- | --- |

79.

| 80. | Secondly, as set out in Articles 1(5) and 5(6), the Regulation in general and Article 5 in particular is without prejudice to Directive (EU) 2016/943 of the European Parliament and of the Council (15) (the ‘Trade Secrets Directive’). |
| --- | --- |

80.

| 81. | To explain this second point, it is important to note that Article 2(1) of the Trade Secrets Directive defines a ‘trade secret’ as information which meets all of the following requirements: (a) it is secret in the sense that it is not, as a body or in the precise configuration and assembly of its components, generally known among or readily accessible to persons within the circles that normally deal with the kind of information in question; (b) it has commercial value because it is secret; (c) it has been subject to reasonable steps under the circumstances, by the person lawfully in control of the information, to keep it secret. |
| --- | --- |

81.

| 82. | Providers therefore cannot refuse to disclose, for example, the main parameters based on the sole argument that it has never revealed any of its parameters in the past or that the information in question is commercially sensitive. |
| --- | --- |

82.

| 83. | Thirdly, providers can act to address bad faith manipulation of ranking by third parties, including the risk of deception of consumers (see recital 27). |
| --- | --- |

83.

| 84. | In this respect, it should be noted that this possibility relates to bad faith manipulation, just as Article 5(6) refers to providers not being required to disclose any information that, 'with reasonable certainty', would result in such deception of consumers or consumer harm. Therefore, a balance needs to be struck between countering manipulative and harmful behaviour on the one hand and the transparency required under Article 5 on the other hand. |
| --- | --- |

84.

5.   SPECIFIC DESCRIPTIONS REQUIRED – DIRECT & INDIRECT REMUNERATION

| 85. | Article 5(3), explained in Recital 25, specifically requires providers to describe the possibility for users to influence ranking against any direct or indirect remuneration, where such possibility constitutes a ‘main parameter’. |
| --- | --- |

85.

| 86. | As specified in Article 5(3), in those cases, providers need to include in their description, in accordance with the requirements of Article 5(1) or (2) as applicable to the provider in question:






  a. | an explanation of any possibility for business users to actively influence ranking against remuneration; as well as |
| --- | --- | --- |

86.

a.

an explanation of any possibility for business users to actively influence ranking against remuneration; as well as

| b. | an explanation of the relative effects of such remuneration on ranking. |
| --- | --- |

b.

an explanation of the relative effects of such remuneration on ranking.

5.1.   General approach

| 87. | In the impact assessment made by the Commission for the purposes of its proposal for the Regulation, a lack of meaningful accountability and predictability for business user and corporate website users with regard to the ranking mechanisms used by providers was identified. In the case of paid ranking, it was recognised that businesses, and small businesses in particular, could benefit by understanding how these opportunities work. It could enable them to either choose not to participate where the resulting ranking is unlikely to be satisfactory, thus saving them the cost of participation, or choose to participate, and gain increased exposure. (16) |
| --- | --- |

87.

| 88. | In this respect, the term ‘remuneration’ is to be understood in a broad sense. In recital 25, it is described as ‘payments made with the main or sole aim to improve ranking, as well as indirect remuneration in the form of the acceptance by a business user of additional obligations of any kind which may have this as its practical effect, such as the use of services that are ancillary or of any premium features’. In addition, recital 25 gives an example of what the explanation of the relative effects of remuneration could cover, namely, ‘the amplitude of the impact of remuneration on ranking’. |
| --- | --- |

88.

| 89. | A best practice when providing the information required under Article 5(3) could involve a combined use of a written explanation with technological tools such as, for example, a dynamic simulator of the anticipated effects of remuneration on ranking. |
| --- | --- |

89.

5.2.   Direct remuneration

| 90. | Direct remuneration is described as ‘payments made with the main or sole aim to improve ranking’ (17). Providers thus should consider what opportunities are offered to users to pay for improving their ranking and how any such opportunities operate. Examples of possible types of direct remuneration are set out in Annex B. |
| --- | --- |

90.

5.3.   Indirect remuneration

| 91. | Indirect remuneration is described as ‘remuneration in the form of the acceptance by a business user of additional obligations of any kind which may have this [improved ranking] as its practical effect’ (18). Providers thus should consider what opportunities are offered to users that could have the practical effect of improving ranking in return, and how any such opportunities operate. |
| --- | --- |

91.

| 92. | The use of ancillary services such as payment, escrow, fulfilment, etc. by users could have an impact on the ranking of their goods or services, for a variety of reasons. The description required pursuant to Article 5 should include these possible uses of ancillary services where they involve indirect remuneration within the meaning of Article 5(3) and constitute ‘main parameters’. In particular, where any or all of these effects on ranking result from the mere participation of the business user in a particular ancillary service - in isolation of any indirect effect the ancillary service may have on the user’s performance measured by other parameters - the additional obligations of Article 5(3) may apply. Other examples of possible types of indirect remuneration are set out in Annex B. |
| --- | --- |

92.

6.   HOW SHOULD MAIN PARAMETERS BE DESCRIBED?

| 93. | Article 5(5) requires the descriptions of the ‘main parameters’ referred to in Article 5(1), (2) and (3) to be, ‘sufficient to enable the business users of corporate website users to obtain an adequate understanding of whether, and if so, how and to what extent, the ranking mechanism takes account of the following:







  a. | the characteristics of the goods and services offered to consumers through the online intermediation services or the online search engine; |
| --- | --- | --- |

93.

a.

the characteristics of the goods and services offered to consumers through the online intermediation services or the online search engine;

| b. | the relevance of those characteristics for those consumers; |
| --- | --- |

b.

the relevance of those characteristics for those consumers;

| c. | as regards online search engines, the design characteristics of the website used by corporate website users’ (19). |
| --- | --- |

c.

as regards online search engines, the design characteristics of the website used by corporate website users’ (19).

| 94. | As a general remark, it is important to note that by obtaining an adequate understanding of how ranking functions, users will be able to compete on a more level playing field when deciding their sales strategies. These strategies potentially encompass both good and service optimisation as well as investment in online visibility, including search engine optimisation and paid ranking. Users should reasonably be able to decide whether or not to invest more in particular elements of their goods and services and to make a better determination of whether and if so, how to invest in ‘ranking strategies’. |
| --- | --- |

94.

6.1.   General approach

| 95. | Providers of online intermediation services must, in accordance with Article 5(1), set out the ‘reasons for the relative importance of those main parameters as opposed to other parameters’. Similarly, under Article 5(2), providers of online search engines must set out the ‘relative importance of those main parameters’ (20). |
| --- | --- |

95.

| 96. | This entails that providers describe why the specific parameters were selected as the main parameters (see Section 3.2 above). A best practice could be for providers to explain how and what in their individual assessment led them to decide how ‘relative importance’ is determined on their services and what are the ‘main parameters’. |
| --- | --- |

96.

6.2.   Level of detail

| 97. | Providers should determine the appropriate level of detail considering the objective of the obligation in Article 5 and include as much detail as necessary and appropriate for their specific, professional user base, without overburdening or confusing them. The level of detail provided should go beyond a simple enumeration of main parameters and provide at least a ‘second layer’ of explanatory information. |
| --- | --- |

97.

| 98. | Providers should not give too brief descriptions or descriptions that could be misleading. For example, if a provider determines that ‘quality’ is a main parameter and knows that what makes up ‘quality’ is a complicated analysis of multiple factors, the description they provide should reflect this fact, whilst using plain and intelligible language. Similarly, merely describing that the main parameter may be affected by ‘search history’ could also be considered insufficiently specific, especially where the provider knows that only certain particular elements of the search history are taken into consideration. |
| --- | --- |

98.

| 99. | When drafting the description of the main parameters, including the underlying reasons for their relative importance, it is important for providers to be clear and to avoid confusion. For example, if one of the main parameters is ‘quality’, providers should consider describing exactly what constitutes ‘quality’ for the purposes of their ranking mechanisms, as it is likely to be understood and measured differently from service to service. Another example could be where a parameter is based on a score received, in which case providers should consider describing what factors are taken into consideration when awarding the score. These descriptions should, whenever possible, refer to objective factors, for example, product categories or price units/bands. Similarly, when referring to reviews, the weight of different types of reviews in the overall score and ranking of the user may be different depending on, for example, the specific product or service that the consumer is looking for. An example could be restaurants offering both table service and take away offers, for which services different factors may be more or less important. If applicable, this should be explained in order to go beyond the obvious. The same applies to a scenario where more recent reviews are given more weight over older ones. |
| --- | --- |

99.

| 100. | Where relevant in order to meet the requirements of Article 5, providers could also include explanations based on how a company-internal strategy may affect what consumers see on the service, for example, where the main parameters may be affected by boosting of offers of certain goods based on the providers’ own assessment of what will be the next big consumer trend. The description provided should go into sufficient detail on what is taken into account in making such decisions, so as to provide users with meaningful explanations of the main parameters. |
| --- | --- |

100.

6.3.   Level of detail for direct and indirect remuneration

| 101. | Where the main parameters used include the possibility to influence ranking against remuneration, the providers concerned should consider how they designed these options. In such cases, a best practice could be to accompany the description of the ‘paid ranking’ options and of the amplitude of their impact with an explanation of the business rationale behind that option and its potential impact. |
| --- | --- |

101.

| 102. | Where applicable, the description of paid ranking options could, as a best practice, cross-reference the descriptions provided pursuant to Article 7(3)(c) and Article 7(3)(d), on differentiated treatment. Differentiated treatment can potentially constitute a ‘main parameter’ under Article 5. In addition to its description under that provision, the cross-reference will point users to the ‘main economic, commercial or legal reasons’ behind the differentiated treatment applied to providers’ own services or goods, which providers have to set out under Article 7(1). |
| --- | --- |

102.

6.4.   Requirement for the description to be drafted in plain and intelligible language

| 103. | As noted before, pursuant to Article 5(1) and (2), read in conjunction with Article 3(1), the descriptions of the main ranking parameters need to be drafted in plain and intelligible language. |
| --- | --- |

103.

| 104. | Each provider should assess whether its description meets that requirement. Giving some thought to how the description is presented will generally also contribute to the clarity and readability of the description. |
| --- | --- |

104.

| 105. | Providers should also consider who their audience is when drafting the description. In certain cases, more technical descriptions may be appropriate and required, bearing in mind that the descriptions are intended for professional users. Without prejudice to the requirement of using plain and intelligible language, professionals may in principle be assumed to require and be able to understand more detailed and more technical information than consumers. |
| --- | --- |

105.

6.5.   Presentational tools

| 106. | Each provider needs to decide on its own how best to tailor the description of main ranking parameters on its services. This may require it to consider techniques that go beyond a written description and include supplementary actions that can help users obtain an adequate understanding of how the ranking mechanism in question operates. These supplementary actions can be tailored to meet the different needs of different users. |
| --- | --- |

106.

| 107. | To explain the effect of ‘direct and indirect remuneration’ on ranking in situations where this constitutes a ‘main parameter’, a written explanation could, for example, be combined with technological tools such as a dynamic simulator of the anticipated effects of remuneration on ranking and the extent of its effect. The explanation given would in principle need to clearly explain what the effects of using the different technique individually or in combination with others/the normal ranking mechanism are, if that is how the options operate on the service in question. It may be useful for providers to consider if this can be done by reference to a baseline. |
| --- | --- |

107.

| 108. | Basic techniques may help users navigate through the information provided. This can be as simple as using a clear structure, with headings and subheadings, that can link through to the supplementary actions that are described above. |
| --- | --- |

108.

| 109. | Providers may want to consider using available tools or testing a possible approach with a panel of users to ensure that the description is understood in the way it is intended, or incorporating a means to obtain feedback from users on whether the description is useful and contains the rights amount of detail. Another way to get this feedback can be to test the description with a representative panel of users. While not specifically required, such measures may help ensure that providers comply with the requirements of Article 5. |
| --- | --- |

109.

7.   WHERE THE MAIN PARAMETERS SHOULD BE DESCRIBED

7.1.   Online intermediation services

| 110. | Article 5(1) requires providers of online intermediation services to set out the description of the main parameters, including the reasons for the relative importance of those main parameters, in their terms and conditions. |
| --- | --- |

110.

| 111. | It should be noted that ‘terms and conditions’ is a broad notion. It is defined in Article 2(10) as covering: ‘all terms and conditions or specifications, irrespective of their name or form, which govern the contractual relationship’ between the provider of such services and the users that are unilaterally determined by that provider. In other words, the description may take different forms and still be considered as having been set out in the provider’s terms and conditions, in compliance with Article 5(1). |
| --- | --- |

111.

| 112. | Whilst the description of the main ranking parameters must be part of the terms and conditions, providers of online intermediation services can choose how to best communicate to their business users and give prominence to the required description. They could, for example, take steps that direct business users to the exact location of the description and/or include it in Q&A sections, tutorials, guidelines, pop-up windows, video messages or in other forms. However, the information should not be inconsistent nor spread out over different tools or media, if that has the effect of making the information in question no longer easily available or plain and intelligible. |
| --- | --- |

112.

| 113. | Without prejudice to case-by-case solutions that providers may use, providers could consider establishing a single touchpoint (for example in a user ‘dashboard’) that could reference or index all the relevant informational tools available to explain ranking transparency. Alternatively, the information could be duplicated across different informational tools – provided that the legal requirements of Articles 3 and 5 are met. |
| --- | --- |

113.

| 114. | In accordance with Article 3(1)(b), the terms and conditions of providers of online intermediation services must be easily available to business users at all stages of their relationship, including before they enter into a contract with those providers. Therefore, all of this information should also be available to prospective business users. |
| --- | --- |

114.

7.2.   Search engines

| 115. | Article 5(2) requires providers of online search engines to provide ‘an easily and publicly available description (...) on the online search engines of those providers’. |
| --- | --- |

115.

| 116. | This entails that the description is placed in an easily accessible location on the online search engine’s webpage. This may be an area that does not require users to log in or register to be able to read the description. |
| --- | --- |

116.

| 117. | Within the limits set by Article 5(2), it is for the provider of online search engine to assess and decide where it provides the required description on ranking. In making that decision, the provider could consider how users of its service behave to find a solution that matches this behaviour. For example, if users use the service as it provides a result in one click, the same principle could be applied to the description. This could mean that if a description is accessed via a link, the link should lead directly to the description and not require further navigation to find it. |
| --- | --- |

117.

| 118. | Similarly, the provider of online search engine could also consider the methods successfully used on its service to attract users’ attention to particular features that already exist. This may be, for example, through the use of icons, tabs or banners. Using a similar method could therefore ensure that the description is easily available and positioned in a way that users of the service expect or are accustomed to. |
| --- | --- |

118.

| 119. | Providers of online search engines could also consider how their services are accessed by users. If this occurs, for example, using voice assistants, then providers could consider how users of their service find a solution to access other information about the service. If this is an effective way, providers may want to replicate it to enable corporate website users to find the description in the same way. |
| --- | --- |

119.

8.   WHEN SHOULD THE DESCRIPTION OF THE MAIN PARAMETERS BE CHANGED?

8.1.   Keeping the descriptions up to date

| 120. | Whilst only Article 5(2) contains an express requirement for the description of the main ranking parameters to be kept up to date, a similar requirement is implicit in Article 5(1). The latter provision requires, after all, a description of the main parameters determining ranking – not of the main parameters that determined ranking. Moreover, in this situation the description is to be set out in the terms of service, which the provider will keep up to date. Otherwise, the objective of improving predictability could not be achieved and the description would not be based on actual data, contrary to what recital 27 explains. |
| --- | --- |

120.

| 121. | Accordingly, providers are to keep the required descriptions up to date. To that aim, they may need to regularly consider whether they still meet this legal requirement, or whether the descriptions need to be updated. Providers are free to devise their own systems to determine whether, when and how they will do this, considering elements such as how frequently changes are made, the effects of any such changes and when experiments (e.g. A/B testing) and other techniques (e.g. machine learning) used on their services may trigger changes to the ‘main parameters’ determining ranking. |
| --- | --- |

121.

| 122. | If the outcome of the individual assessment shows that an update is needed to the description of the main ranking parameters, then providers of online intermediation services must notify business users of those changes in advance in accordance with Article 3 (see 9.1 below). No such requirement exists for providers of online search engines, as they are to ensure the public availability of their descriptions (Article 5(2)). |
| --- | --- |

122.

8.2.   Temporary changes

| 123. | Providers should determine on a case-by-case basis whether modifications to their ranking mechanisms, including those of a temporary nature, result in a change to the ‘main parameters’ and necessitate a change to the descriptions thereof to ensure compliance with Article 5. |
| --- | --- |

123.

| 124. | Providers should not assume that the temporary nature of changes made to parameters, weighting or other features of ranking necessarily means that the description of the main parameters does not need to be changed. For example, one-off events such as sporting events may come with significantly increased consumer demand, which may lead to temporary changes to the main parameters that determine ranking. The same applies to promotional events like ‘Black Friday’, holiday seasons, or seasonal sales periods such as pre-Christmas. When providers alter their ‘standard’ ranking methods in such cases, for example, by giving more weight to certain parameters (e.g. ‘delivery speed’, ‘price’ or other notions such as ‘brand appeal’ or ‘quality’, possibly combining several factors), adding new ones or suppressing some, and these modifications affect the main parameters, providers should adapt the description thereof. |
| --- | --- |

124.

| 125. | It may also be good practice for providers to try and capture in their description of the main ranking parameters any ‘regular’ dynamic features of their ranking mechanisms. The fact that these are a ‘regular’ feature can mean they are a ‘main parameter’ or a reason for the main parameters’ relative importance. For example, a provider of an online e-commerce marketplace may wish to structurally increase the weighting of the ‘price’ parameter during a specific recurring sales event. |
| --- | --- |

125.

8.3.   Experiments

| 126. | If providers use experiments, including A/B testing of changes or evolving fraud fighting mechanisms, they need to consider on a case-by-case basis whether and if so, how this affects ranking on their services and, in particular, whether it requires a change to the description of the main ranking parameters required under Article 5. As a general rule, providers could in this context consider elements such as the frequency and impact of the experiments, as well as the size of the test audience or of the geographic region affected by the testing. |
| --- | --- |

126.

| 127. | In cases where the experiments are ‘live’ (meaning that ‘test audiences’ cannot necessarily be distinguished from the ‘rest’ of the user base), it is possible that the experiments can affect the relative prominence of the goods and services offered by all users. Accordingly, as a first step, providers may have to reflect on whether or not their experimentation is conducted in a genuinely isolated testing environment. If individual experiments are not isolated, providers may have to consider in particular (a) whether the experiment itself has an impact on ranking and if so, whether the parameter being tested is a ‘main parameter’ that needs to be described and; (b) whether and when the full roll-out of the experiment will mean the parameter being tested will change their ranking mechanism and be a ‘main parameter’ that needs to be described. |
| --- | --- |

127.

9.   STEPS TO BE TAKEN WHEN THE DESCRIPTION OF THE MAIN PARAMETERS IS UPDATED

9.1.   Requirement for online intermediation services to notify proposed changes

| 128. | The description of the main parameters determining ranking must be included in the terms and conditions of providers of online intermediation service (Article 5(1)). Pursuant to Article 3(2), when such providers intend to change their terms and conditions, they must notify their business users of the proposed changes at least 15 days before applying them. The only exception to the application of the notice period is where the provider of online intermediation services: (a) is subject to a legal or regulatory obligation which requires it to change its terms and conditions in a manner which does not allow it to respect the notice period; or (b) has exceptionally to change its terms and conditions to address an unforeseen and imminent danger related to defending the online intermediation services, consumers or business users from fraud, malware, spam, data breaches or other cybersecurity risks. In addition, the notice period does not apply to changes that are merely of an editorial nature (i.e. changes that do not alter the content or meaning of terms and conditions). The notion of ‘editorial changes’ necessarily excludes changes to ‘main parameters’, as they will alter the content or meaning of terms and conditions. |
| --- | --- |

128.

| 129. | This means that if the description of the main parameters changes, in principle, at least 15 days’ notice must be given to business users before the changes can take effect. It is important to note that 15 days prior notification is the legal minimum requirement. Under Article 3(2), longer notice periods are required if necessary to allow business users to make technical or commercial adaptations to comply with the changes in question. |
| --- | --- |

129.

| 130. | Furthermore, under Article 3(2), providers of online intermediation services have to inform business users on a durable medium of any such proposed changes. ‘Durable medium’ is defined in the Regulation as: ‘any instrument which enables business users to store information addressed personally to them in a way accessible for future reference and for a period of time adequate for the purposes of the information and allows the unchanged reproduction of the information stored’ (21). |
| --- | --- |

130.

| 131. | In order to improve clarity, providers of online intermediation services may wish to ensure that the changes to the description of the main parameters are easily identifiable. Best practices in this regard could include keeping a dedicated page that gives access to previous versions of the descriptions or keeps a record of the changes made over time. Separately, a best practice that can help business users obtain an adequate understanding is to explain the practical implications of the changes. |
| --- | --- |

131.

9.2.   Requirement for online search engines to keep description up to date

| 132. | Pursuant to Article 5(2), providers of online search engines must make sure that the description of the main ranking parameters is kept up to date. |
| --- | --- |

132.

| 133. | As regards any such updates of the descriptions, reference is made to the best practices referred to in 9.1 above. |
| --- | --- |

133.

10.   SPECIFIC OBLIGATIONS ON SEARCH ENGINES TO ENABLE CORPORATE WEBSITE USERS TO INSPECT THE CONTENTS OF THIRD PARTY NOTICES

| 134. | Pursuant to Article 5(4), providers of online search engines must offer corporate website users the possibility to inspect third-party notifications where these led the provider to change the ranking order or to delist the relevant website. |
| --- | --- |

134.

| 135. | Recital 26 explains that this obligation will help mitigate potential abuses that take the form of anti-competitive notifications, and that it reflects the potential difficulty providers of online search engines may face in notifying all relevant corporate website users in the absence of contractual relationships. |
| --- | --- |

135.

---

(1)  ‘Providers of online intermediation services’ is defined in Article 2(3) of the Regulation as ‘any natural or legal person which provides, or which offers to provide, online intermediation services to business users’. ‘Online intermediation services’ are defined in Article 2(2) as ‘services which meet all of the following requirements: (a) they constitute information society services within the meaning of point (b) of Article 1(1) of Directive (EU) 2015/1535 of the European Parliament and of the Council;(b) they allow business users to offer goods or services to consumers, with a view to facilitating the initiating of direct transactions between those business users and consumers, irrespective of where those transactions are ultimately concluded; (c) they are provided to business users on the basis of contractual relationships between the provider of those services and business users which offer goods or services to consumers.’

(2)  ‘Providers of online search engines’ is defined in Article 2(6) of the Regulation as ‘any nature or legal person which provides, or which offers to provide, online search engines to consumers.’‘Online search engines’ are defined in Article 2(5) as ‘a digital service that allows users to input queries in order to perform searches of, in principle, all websites, or all websites in a particular language, on the basis of a query on any subject in the form of a keyword, voice request, phrase or other input, and returns results in any format in which information related to the requested content can be found.’

(3)  OJ L 186, 11.7.2019, p. 57.

(4)  For reasons of brevity, in these Guidelines the term ‘providers’ is used to refer to both providers of online intermediation services and providers of online search engines, unless stated otherwise. Likewise, the term ‘users’ refers to both business users and corporate website users, as defined in Article 2(1) and (7) of the Regulation respectively, again unless otherwise stated. Unless indicated otherwise, the articles and recitals referred to in these Guidelines are those of the Regulation.

(5)  Recital 25 reflects the effects to be achieved by the ‘adequate understanding’ benchmark: ‘The content of the description, including the number and type of main parameters, can accordingly vary strongly depending on the specific online intermediation services, but should provide business users with an adequate understanding of how the ranking mechanism takes account of the characteristics of the actual goods or services offered by the business user, and their relevance to the consumers of the specific online intermediation services’.

(6)  Recital 25 in this regard sets out several examples of design elements of a ranking mechanism that should assist business users in obtaining the required adequate understanding, including: the ‘indicators used for measuring the quality of goods or services of business users’, the ‘use of editors and their ability to influence the ranking of those goods or services’, the ‘amplitude of the impact of remuneration on ranking’ and ‘elements that do not or only remotely relate to the good or service itself, such as presentational features of the online offer’.

(7)  For more information, see, for example: https://en.wikipedia.org/wiki/Search_syndication

(8)  Underlining added.

(9)  Article 1(3) of the Regulation.

(10)  As part of the ranking transparency obligation, providers are also obliged to describe the ‘relative importance’ of the main parameters; this is dealt with in Section 6.2.

(11)  As noted above, for providers of online intermediation services, which are to include the main parameters and the related reasons in their terms and conditions, the requirements of Article 3 apply to any such changes to the terms and conditions.

(12)  This may range from, for example, several products or services seeing only a marginal change in their relative prominence on only a limited number of occasions, to entirely different products or services being featured for different consumers on a regular basis.

(13)  See also COMMISSION STAFF WORKING DOCUMENT IMPACT ASSESSMENT Accompanying the document Proposal for a Regulation of the European Parliament and of the Council on promoting fairness and transparency for business users of online intermediation services, SWD/2018/138 final, Sections 1.6, 2.2 and 7.1.

(14)  Underlining added.

(15)  Directive (EU) 2016/943 of the European Parliament and of the Council of 8 June 2016 on the protection of undisclosed know-how and business information (trade secrets) against their unlawful acquisition, use and disclosure (OJ L 157, 15.6.2016, p. 1).

(16)  Ibid 5.

(17)  Recital 25 of the Regulation.

(18)  Recital 25 of the Regulation, insertion added for clarity.

(19)  Underlining added.

(20)  Underlining added.

(21)  Article 2(13) of the Regulation.

---

ANNEX I

Illustrative examples of ranking parameters

| Quality of sitemap |
| --- |
| Page-loading speed |
| Security (e.g. HTTPS) |
| Images (e.g. type, number, quality) |
| Consumer reviews (e.g. number, rating, recent) |
| Trader-Consumer Interaction (e.g. answered queries, responsiveness) |
| Dispute settlement history (e.g. number of consumer complaints, solutions found) |
| Completed sales (e.g. number, recent) |
| Price |
| Internet traffic, performance in search |
| ‘Offline’ service quality indicators (e.g. hotel star rating, delivery performance, the degree to which places, brands etc. are familiar or well known in society) |
| Trust measures (e.g. participation in platforms' escrow services, registered identity, industry certification schemes/icons, data protection seals / certifications) |
| Data protection ‘score’, e.g. based on reviewing the privacy policies of apps by an app store |
| Web accessibility |
| (Multi-)Device adaptability |
| Content quality (e.g. based on website linking, richness, language quality, number of languages, etc.) |
| Key word tagging (positive – number, level of detail, and negative – ‘stuffing’) |
| Title accuracy and relevance (e.g. brand, technical specifications, etc.) |
| Date of market entry |
| Concise answers, for example as regards products or services offered, or in response to FAQs |
| Paid ranking (including by online intermediation services on online search engines)






  — | bid |

Quality of sitemap

Page-loading speed

Security (e.g. HTTPS)

Images (e.g. type, number, quality)

Consumer reviews (e.g. number, rating, recent)

Trader-Consumer Interaction (e.g. answered queries, responsiveness)

Dispute settlement history (e.g. number of consumer complaints, solutions found)

Completed sales (e.g. number, recent)

Price

Internet traffic, performance in search

‘Offline’ service quality indicators (e.g. hotel star rating, delivery performance, the degree to which places, brands etc. are familiar or well known in society)

Trust measures (e.g. participation in platforms' escrow services, registered identity, industry certification schemes/icons, data protection seals / certifications)

Data protection ‘score’, e.g. based on reviewing the privacy policies of apps by an app store

Web accessibility

(Multi-)Device adaptability

Content quality (e.g. based on website linking, richness, language quality, number of languages, etc.)

Key word tagging (positive – number, level of detail, and negative – ‘stuffing’)

Title accuracy and relevance (e.g. brand, technical specifications, etc.)

Date of market entry

Concise answers, for example as regards products or services offered, or in response to FAQs

Paid ranking (including by online intermediation services on online search engines)

—

bid

| — | quality of ad, context |
| --- | --- |

—

quality of ad, context

| — | reliability of advertiser |
| --- | --- |

—

reliability of advertiser

Size of offer (e.g. breadth of product or service offer, or in-app purchases featured separately in app stores ranking)

Editorial processes and specific selection criteria (e.g. platform-reviewed apps or art projects)

Adjustment algorithms (concerning, for example, spam, freshness, quality)

Mobile app uninstall rates

Bounce rates

A/B testing (impact can be influenced by certain elements such as timing – peak demand, duration and sample size)

Randomisation

Personalisation

| — | the extent of personalisation (number and types of features that come into play), whether and how it may be dependent on users’ privacy settings, etc. |
| --- | --- |

—

the extent of personalisation (number and types of features that come into play), whether and how it may be dependent on users’ privacy settings, etc.

| — | geographic location, timing of search |
| --- | --- |

—

geographic location, timing of search

| — | user search history, acquisition history |
| --- | --- |

—

user search history, acquisition history

| — | undoing default settings |
| --- | --- |

—

undoing default settings

| — | impact of filters (and remaining parameters) |
| --- | --- |

—

impact of filters (and remaining parameters)

Multi-platform presence (e.g. negative impact of lower pricing on competing platform on ranking, social media ‘scoring’, referencing/linkage rate of the site, frequency of referencing, reputation of referencing sites or blogs)

‘Brand appeal’ of new entrant business users being measured using factors external to the online intermediation services concerned (e.g. surveys among a specialised community)

Strength of the offer, e.g.:

| — | comparable competitiveness (e.g. ratings, consumer reviews) |
| --- | --- |

—

comparable competitiveness (e.g. ratings, consumer reviews)

| — | relevance |
| --- | --- |

—

relevance

| — | availability |
| --- | --- |

—

availability

| — | rejection tracker |
| --- | --- |

—

rejection tracker

| — | cancellations |
| --- | --- |

—

cancellations

| — | double bookings (sector specific) |
| --- | --- |

—

double bookings (sector specific)

Conversion rates

Geographic proximity

Level of commission paid

Non-participation in certain programme (causing degradation in a ranking) or participation in a certain program/purchase of additional services (leading to improved ranking)

Business user’s stock depth

Seasonality and temporary deviations (e.g. a one-day-long sales event)

National approach (Cross-border cultural differences)

Particular methodology of weighing user reviews (e.g. usage of trusted reviewers)

Experimentation (e.g. platforms pro-actively pushing new entrant business users)

‘Mobile-friendliness’ of a website

Editorial interventions (editor’s ‘top picks’, ‘deals of the day’, corrections made to search results that are specific to an individual business user or corporate website user, regardless of whether these are ‘manual/human’ or ‘algorithmic’, etc.)

Housekeeping practices (e.g. deletion of old apps)

Click-through rates

Access rules

Fraud prevention mechanisms

Bundled services of business or corporate website users (e.g. free shipping, easy returns policy, etc.)

Quality of the description of the offer (written description, usage of pictures, etc.)

Use of options for premium visibility/temporary visibility boosting (e.g. the use of such mechanism to generate additional income and/or to facilitate product/service launches or market entry) possibly combined with, or in the form of, mechanisms or actions that reduce the importance for relative prominence of ‘otherwise applicable’ parameters

Importance of reputation/trust based on e.g. user reviews or ratings

Quality of content including regular update of content

Use of clear and brief titles

The speed and all-device user–friendliness

Number of listings matching the buyer's query

Domain age

Uniqueness of content

Input provided by the user (words typed or spoken, gender, age, culture, language, address, previous interactions, etc.)

‘Objective’ platform-external data (date, time, weather, etc.)

Personal data related to other users (likes, most searched words, products most sold, etc.)

Number of ‘shares’/ number of views / number of ‘saves’, ‘favourites’

Technological means/medium (i.e. how the ranking is being accessed by the consumer)

Multi-links

Remuneration

Business relationship with the business user (length of historical relationship, privileged business relationship, any investment related to the platform, etc.)

Quality of the customer service

Ratio of cancellations

Market-related features (general demand, competitive prices, competitive availability, etc.)

Negative criteria such as incidence scores for consumer complaints or assumptions that certain features are disliked by consumers

Scores for resolutions of consumer issues

Parity (e.g. comparison of intermediary rates with rates in other channels of distribution)

Store conversion

Application Not-Responding (ANR)

Retention

Installs and back-links to external websites

Stock availability

Quality and characteristics of the product

Relevance of the product to the assortment

Filters applied by consumers to narrow down their search

Search-related inputs (listing content, listing attributes, keywords/tags/labels)

Relevance based on the match of end user and business user inputs

Popularity of offers

End user demonstration of ranking order preference (most recent, most relevant, highest reviewed, etc.)

Legal requirements (including fraud/anti-counterfeiting issues)

Newly listed

Preference towards business users opting for the possibility to use MFN clauses, e.g. if these are not generally imposed

Offers ending soon

Nearest first

Auction vs buy it now

Brand

Item condition (new vs used)

Example of specific parameters that may be used in the accommodation sector:

Minimum availability indicated by the accommodation

The volume realised by the accommodation

Services and amenities (e.g. dining, parking, fitness facilities, front desk and concierge services, WiFi and business centres, proximity to transportation and local attractions)

The number of bookings related to the number of visits to the relevant accommodation page on the platform (conversion rates)

Quality of the hotel compared to the quality advertised

Confirmation of the reservation of the hotel or restaurant

Distance of the restaurant to the customer or previous user patterns

Policies (regarding e.g. early check-in, late departure, and cancellation privileges)

Property-related (number and quality of consumer reviews, commission level, punctuality of invoice payments, promotions, participation in preferred programmes, etc.)

Purpose of travel (business/leisure)

---

ANNEX II

Illustrative examples of direct and indirect remuneration

| Payment of a higher amount of commission by a business user for increased relative prominence, for example in auctions run as part of premium/partner programmes |
| --- |
| Business users’ or corporate website users’ periodic commitments to paid ranking or joint promotional efforts within or on the online intermediation services or online search engines concerned |
| Direct payment for endorsement or preferred placement by the relevant provider of online intermediation services or online search engines |
| Direct payment for ‘boosting’ visibility of business pages on social media, online directory services or other online intermediation services |
| (One off) Direct payment for increased visibility during particular periods |
| (Regular) Direct payment leading to a product featuring prominently in a ranking |
| Loyalty or premium programmes
  Business users can pay for certain premium features through loyalty or premium programmes offered by the platforms. The mere fact of membership of such programmes, rather than possibly objective effects of that membership on performance measured under different main parameters (e.g. delivery speed in e-commerce), may indirectly improve a business users' ranking because of the configuration of the ranking mechanisms that may favour those businesses that are members. |
| In-app purchases
  In-app purchases are extra content or subscriptions within apps on mobile devices or desktops available after the original download. Not all apps offer in-app purchases, but some or all features of an app may require users to make a one-time in-app purchase or sign up to a (paid) subscription. Examples include extra app functionalities, premium and add-free app versions, etc.
  Business users that offer apps with in-app purchases, usually pay commission to the platform on these purchases. This can improve the visibility of their apps in search results. In some cases, in-app purchases can even be referenced in the ranking by the app stores operators, separately from, and in addition to, the underlying app, which equally enhances the overall ‘relative prominence’ of the app concerned. |
| Ancillary services
  Similar to loyalty or premium programmes, the mere fact of using ancillary services of the relevant provider of online intermediation services, such as upfront payment, escrow, fulfilment, cloud services, rebates, logistics, insurance, etc., could have an impact on ranking. |
| Reward/incentive structures
  Reward programmes granting credits for free on-platform advertising – which impacts relative prominence – in return for, for example, business users expanding their product or service offerings on the online intermediation services concerned, in terms of breadth and/or scale, or in return for committing to mark-downs during specific period where the resulting increased relative prominence does not result merely from the relative price differences - where price is a main parameter |
| Trust marks
  Licence fees for trust marks, where the relevant provider of online intermediation services or of online search engines is in some way involved with the specific trust mark |
| Exclusivity
  Exclusivity deals if they can lead to a better ranking. This could be the case where such deals are voluntary – as opposed to unilaterally imposed on business users of the online intermediation services – and only taken up by a subset of business users. |
| Profit-sharing
  Indirect remuneration may exist in a situation of profit-sharing when compensation is, for example, paid as a percentage of the links clicked on the referring online search engine. A business may, for example, be ranked higher in the list of search results if it agrees to allow the provider of the online search engine to display the business’ content in a news aggregation service or a podcasting service offered by that same provider. |
| Access and use of data
  Business users may agree to granting enhanced access to, or other contractual/legal rights regarding the data they generate by consuming the online intermediation services, to the providers of these services. Such a contractual commitment concerning data, if it is made only by a subset of business users, could impact ranking in different ways, including as a result of superior ranking optimisation tips/insights provided by the provider in return. These data are valuable information that can also give online intermediation services a strong commercial or competitive advantage, including when they are themselves actively selling on the same online intermediation services. |
| Bundled offers / commercial relations
  Remuneration being part of broader commercial arrangements with providers of online intermediation services. In such a situation, direct payment for a single service, which may even be unrelated to the online intermediation services or online search engine concerned, could impact the status – and possibly ranking – of the relevant business across more or all of the services exchanged between the two entities. |

Payment of a higher amount of commission by a business user for increased relative prominence, for example in auctions run as part of premium/partner programmes

Business users’ or corporate website users’ periodic commitments to paid ranking or joint promotional efforts within or on the online intermediation services or online search engines concerned

Direct payment for endorsement or preferred placement by the relevant provider of online intermediation services or online search engines

Direct payment for ‘boosting’ visibility of business pages on social media, online directory services or other online intermediation services

(One off) Direct payment for increased visibility during particular periods

(Regular) Direct payment leading to a product featuring prominently in a ranking

Loyalty or premium programmes

Business users can pay for certain premium features through loyalty or premium programmes offered by the platforms. The mere fact of membership of such programmes, rather than possibly objective effects of that membership on performance measured under different main parameters (e.g. delivery speed in e-commerce), may indirectly improve a business users' ranking because of the configuration of the ranking mechanisms that may favour those businesses that are members.

In-app purchases

In-app purchases are extra content or subscriptions within apps on mobile devices or desktops available after the original download. Not all apps offer in-app purchases, but some or all features of an app may require users to make a one-time in-app purchase or sign up to a (paid) subscription. Examples include extra app functionalities, premium and add-free app versions, etc.

Business users that offer apps with in-app purchases, usually pay commission to the platform on these purchases. This can improve the visibility of their apps in search results. In some cases, in-app purchases can even be referenced in the ranking by the app stores operators, separately from, and in addition to, the underlying app, which equally enhances the overall ‘relative prominence’ of the app concerned.

Ancillary services

Similar to loyalty or premium programmes, the mere fact of using ancillary services of the relevant provider of online intermediation services, such as upfront payment, escrow, fulfilment, cloud services, rebates, logistics, insurance, etc., could have an impact on ranking.

Reward/incentive structures

Reward programmes granting credits for free on-platform advertising – which impacts relative prominence – in return for, for example, business users expanding their product or service offerings on the online intermediation services concerned, in terms of breadth and/or scale, or in return for committing to mark-downs during specific period where the resulting increased relative prominence does not result merely from the relative price differences - where price is a main parameter

Trust marks

Licence fees for trust marks, where the relevant provider of online intermediation services or of online search engines is in some way involved with the specific trust mark

Exclusivity

Exclusivity deals if they can lead to a better ranking. This could be the case where such deals are voluntary – as opposed to unilaterally imposed on business users of the online intermediation services – and only taken up by a subset of business users.

Profit-sharing

Indirect remuneration may exist in a situation of profit-sharing when compensation is, for example, paid as a percentage of the links clicked on the referring online search engine. A business may, for example, be ranked higher in the list of search results if it agrees to allow the provider of the online search engine to display the business’ content in a news aggregation service or a podcasting service offered by that same provider.

Access and use of data

Business users may agree to granting enhanced access to, or other contractual/legal rights regarding the data they generate by consuming the online intermediation services, to the providers of these services. Such a contractual commitment concerning data, if it is made only by a subset of business users, could impact ranking in different ways, including as a result of superior ranking optimisation tips/insights provided by the provider in return. These data are valuable information that can also give online intermediation services a strong commercial or competitive advantage, including when they are themselves actively selling on the same online intermediation services.

Bundled offers / commercial relations

Remuneration being part of broader commercial arrangements with providers of online intermediation services. In such a situation, direct payment for a single service, which may even be unrelated to the online intermediation services or online search engine concerned, could impact the status – and possibly ranking – of the relevant business across more or all of the services exchanged between the two entities.

---