# Investigating the use of file advisory hints on lustre and GPFS

Louis-Marie Nicolas, Salim Mimouni, Philippe Couvée, Jalil Boukhobza

► To cite this version:

Louis-Marie Nicolas, Salim Mimouni, Philippe Couvée, Jalil Boukhobza. Investigating the use of file advisory hints on lustre and GPFS. SSDBM 2025 - 37th International Conference on Scalable Scientific Data Management, Jun 2025, Columbus, United States. pp.1-6, 10.1145/3733723.3733745. hal-05271597

HAL Id: hal-05271597

https://hal.science/hal-05271597v1

Submitted on 22 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Investigating the Use of File Advisory Hints on Lustre and GPFS

### Louis-Marie Nicolas

Lab-STICC, CNRS UMR 6285, ENSTA, Institut Polytechnique de Paris 29806 Brest, France
louis-marie.nicolas@ensta.fr

### Philippe Couvée

Atos BDS R&D Data Management 38100 Grenoble, France
philippe.couvée@eviden.com

## Abstract

Optimizing the I/Os is essential to avoid performance bottlenecks in High-Performance Computing. Advisory hints, such as POSIX fadvise and Lustre ladvise, are designed to improve I/O performance by allowing applications to provide the file system with additional information on their access patterns. These mechanisms are supposed to enable more efficient cache utilization and reduce I/O performance bottlenecks. However, in practice, their effectiveness depends on how individual file systems interpret these hints and on how developers use them. In this paper, we offer a comprehensive investigation of the impact of advisory hints on Lustre and GPFS parallel file systems. Our contributions include: (1) a methodology to evaluate the performance impact of such hints on these widely-used parallel file systems, (2) the introduction of a benchmark that implements this methodology, and (3) we analyze the result of our benchmark, leading to practical guidelines for application developers to improve I/O performance through proper use of those hints. Our findings demonstrate that the effect of these hints varies significantly between evaluated state-of-the-art parallel file systems and highlight the need for tailored approaches in leveraging advisory hints in order to achieve optimal I/O performance.

## CCS Concepts

* Information systems → Information storage systems; Storage management; Hierarchical storage management;

## Keywords

Advisory Hints, I/O patterns, Parallel File System, Data Prefetching, High Performance Computing

## ACM Reference Format:

Louis-Marie Nicolas, Salim Mimouni, Philippe Couvée, and Jalil Boukhobza. 2025. Investigating the Use of File Advisory Hints on Lustre and GPFS. In *The International Conference on Scalable Scientific Data Management 2025*

<figure><img src="image_2.png" /></figure>

This work is licensed under a Creative Commons Attribution International 4.0 License.

SSDBM 2025, Columbus, OH, USA
© 2025 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-1462-7/25/06
https://doi.org/10.1145/3733723.3733745

### Salim Mimouni

Atos BDS R&D Data Management 38100 Grenoble, France
salim.mimouni@eviden.com

### Jalil Boukhobza

Lab-STICC, CNRS UMR 6285, ENSTA, Institut Polytechnique de Paris 29806 Brest, France
jalil.boukhobza@ensta.fr

(SSDBM 2025), June 23–25, 2025, Columbus, OH, USA. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3733723.3733745

## 1 Introduction

Efficient data access and caching are critical for the success of large-scale High Performance Computing (HPC) applications. Parallel file systems (PFS) such as Lustre [15] and GPFS [14] are designed to handle massive amounts of data, providing the necessary throughput and scalability for these applications. However, due to the variability in application access patterns, optimizing the PFS performance remains a challenge for both users and developers [13].

To address this challenge, file systems support advisory mechanisms like POSIX fadvise [4] and Lustre ladvise [3], which allow application developers to provide hints to the file system about how they intend to access files. For instance, the WILL_NEED hint can be used to indicate some data will be used in the near future. The DONT_NEED hint is the opposite: it marks data as potentially no longer useful. Application developers usually expect the data that has been marked with the former to be prefetched by the file system [9], and the data marked with the latter to be removed from the cache [11]. Various other hints exist, and are intended to improve I/O performance by allowing the file system to optimize caching, read-ahead, and other strategies based on the expected patterns.

In the literature, these hints are mostly used for prefetching [9] and for eviction [10]. They can also be used with the intention to disable optimizations such as the read-ahead [9], or as a way to help the file system manage the cache, for instance by preventing the log files that are written from remaining into the cache [11]. In general, they are widely used in real-world I/O intensive applications such as RocksDB [8], GridGraph [16] or SQLite [5].

However, while application developers and researchers have clear expectation on what a file system should be able to do when given hints such as "WILL_NEED" or "DONT_NEED", the behavior of a file system when receiving such advices is often not documented. This might lead to some users choosing not to use any hints, potentially at the expense of some I/O performance drop, and to some other users investing time adding advices that happen to be ignored by the file system used in their applications.

In this paper, we propose to investigate the impact of advisory hints on real-world PFS and to clarify the consequences of their use on I/O performance. We provide application developers with guidelines to maximize performance. Our contributions are threefold: (1)
<table><caption>Table 1: I/Os advising functions for 4 common I/O hints</caption><thead><tr><th></th><th>WILL_NEED</th><th>DONT_NEED</th><th>SEQUENTIAL</th><th>RANDOM</th></tr></thead><tbody><tr><td>POSIX ladvise</td><td>POSIX_FADV_WILLNEED</td><td>POSIX_FADV_DONTNEED</td><td>POSIX_FADV_SEQUENTIAL</td><td>POSIX_FADV_RANDOM</td></tr><tr><td>Lustre ladvise</td><td>LU_LADVISE_WILLNEED</td><td>LU_LADVISE_DONTNEED</td><td>NOT SUPPORTED</td><td>NOT SUPPORTED</td></tr></tbody></table>

we propose a methodology for evaluating the effectiveness of these advisory hints on various PFS, and (2) we present a simple benchmark<fnref n="1" /> that implements this methodology, enabling users to assess whether such hints can lead to tangible I/O performance gains on the PFS they use, and (3) we analyze the results from our benchmark, offering insights and practical guidelines for developers.

We show that some hints, like the SEQUENTIAL I/O pattern hint, shows no effect on both PFS evaluated. Some other hints like the RANDOM I/O pattern hint, the DONT_NEED hint and the WILL_NEED hint have a visible effect on either Lustre or GPFS, but none on the other. Overall, our findings indicate that the impact of these hints varies significantly across PFSs, emphasizing the need for customized approaches to effectively leverage advisory hints.

# 2 Background

## 2.1 Parallel File Systems

The Lustre file system [15] is an open-source PFS commonly used in high performance computing. As of October 2024, it is listed 23 times in the top 100 of IO-500 [2], a ranking of the top 500 best performing PFS used for research in the world. The Lustre PFS has a distributed approach, with specialized metadata nodes hosting the file's metadata, data nodes hosting the file's content as objects, and management nodes most notably handling the PFS configuration.

GPFS [14], also called Spectrum Scale, is a closed-source PFS developed by IBM. It is listed 8 times in the top 100 of IO-500. While very similar to Lustre, the nodes hosting the GPFS file system are not specialized, with nodes dynamically reassigning themselves roles on hardware and network failures, making it highly resilient.

## 2.2 Advisory Hints for Parallel File Systems

To optimize data access patterns and resource utilization, file systems can rely on mechanisms that allow users to provide hints about expected I/O behavior. We identify two of them in Table 1.

**POSIX fadvise** is a system call that allows applications to provide advice to the kernel about the expected usage of file data. By using fadvise, applications can inform the file system about how data will be accessed, which can help optimize caching and prefetching strategies. It supports two types of hints: I/O pattern hints and cache hints. For the first one, a user can specify that certain file regions will be accessed sequentially or randomly using the SEQUENTIAL and RANDOM hints. For the second one, the user might help the file system caching policy by specifying that certain file regions will be used in the future using the WILL_NEED hint, that they will no longer be used with DONT_NEED, or that they will be used in the future but only once with NO_REUSE.

**Lustre ladvise** extends the concept of POSIX fadvise, enabling applications on a compute node to provide equivalent hints to Lustre data nodes. However, as illustrated in Table 1 Lustre ladvise only supports cache hint, and does not support I/O pattern hints [3].

# 3 Benchmarking Methodology

## 3.1 Overview

Our study focuses on understanding the impact of advisory hints on the I/O performance of PFSs. To achieve this objective, we perform four experiments on a synthetic application to characterize the effect of different I/O hints on I/O performance.

The main evaluation metric we used to describe the I/O performance is the throughput. The throughput is a common I/O performance metric [12] defined as the I/O volume divided by the I/O time. For our experiments that are done through a synthetic application, the I/O volume and I/O time instrumentation is integrated directly into the source code.

At the core of our methodology is the study of the effect of the different hints on a configurable single-process synthetic application. The main advantage of using such a synthetic application is that it makes possible the comprehensive analysis of the effect of predictive hints by varying I/O sizes, inter-arrival times, I/O patterns, and file sizes, covering most real-world I/O behaviors.

**Experiment 1: Storage system characterization:** The first synthetic experiment that we present in this methodology aims to characterize the storage system. We measure the baseline read performance when the data are either not cached (1), cached in compute nodes (2), or cached in data node only (3). We expect to observe the best performance when the data are cached in the compute nodes, the lowest performance when it is not cached, and an average performance when the data are cached in the data nodes.

**Experiment 2: Evaluating the effect of I/O patterns hints,** In the second experiment, we evaluate the effect of the I/O patterns hint, namely SEQUENTIAL and RANDOM, on the read performance. We expect a performance increase due to a more accurate readahead and caching when the hint matches the I/O pattern.

**Experiment 3: Evaluating the effect of the DONT_NEED hint on cache eviction:** In the third synthetic experiment we evaluate the effect of the DONT_NEED hint on the read performance. We expect data marked with this hint to be removed from the cache.

**Experiment 4: Evaluating the effect of the WILL_NEED hint on cache prefetching:** In the fourth synthetic experiment, we evaluate the effect of the WILL_NEED hint on the read performance, and compare its effect with the forceful data prefetching that can be done through asynchronous reads. We expect the effect of this hint to be a prefetch. As such, if the hint arrives early enough for the prefetch to be entirely performed before the I/Os start, we expect the data to be fully cached and the performance to be maximized.

As introduced in the previous section, source code for the synthetic workloads used in these experiments is available on github.

## 3.2 Hardware and Software Environments

A small-scale HPC cluster was used in this study, with 20 computes nodes with 2x AMD EPYC 7282 16-Core Processor available.

Two PFS were used in this study. The first one is under GPFS [14] version 5.1.8.0, with 8 volumes of 50 TB, for a total capacity of 400 TB. We note that GPFS comes with its own page pool, separate from the Linux page cache. This page pool has a fixed size of 4 GB. The second PFS used is under Lustre version 2.12.5.0 for the server and version 2.12.9 for the client, with 8 OST of 5 TB and 1 MDT of 500 GB, for a total capacity of 40 TB.

<footnote n="1">¹https://github.com/ShinySilver/IO-Hints-Benchmark</footnote>
## 3.3 Restoring the cache to its initial state

A special attention is given to ensure the cache is cleared between each measurement in order to obtain accurate and consistent performance metrics. The Linux page cache on compute nodes can be cleaned by writing to `/proc/sys/vm/drop_caches`.

In addition, when using the Lustre file system, the Linux page cache on the data nodes can be interacted with and cleaned using Lustre ladvise, which we confirm in the first experiment.

At last, GPFS do not utilize the Linux page cache and do not provide a straightforward method to interact with its cache. To clear its cache, we read a large volume (compared to GPFS page pool size parameter) of unrelated data to replace our data in the page pool.

## 3.4 Experiment 1: Storage system characterization

We consider the following three baseline configurations leading to different identifiable performance values:

(1) When the data are **not cached**.

(2) When the data are **cached in the compute node**.

(3) When the data are not cached in the compute node, but are **cached in the data node**'s main memory.

In order to measure the read performance when the data is not cached (1), we clean both the compute node cache and (on Lustre) the data node cache using the method described in the previous section. We can then proceed to read a target file, of configurable size. While doing so, we keep track of both I/O time and I/O volume, ultimately returning an I/O throughput for our experiment.

For the second item of the list (2), we aim at measuring the throughput when data are cached in the compute node. To do so, it is necessary to read the target file a first time for its content to enter the compute and data nodes caches, and then read the target file again, the "cached" throughput being the one measured while reading the file for the second time.

For the third item in the list (3), we aim at measuring the throughput when data are cached in the data nodes only. Its procedure is very similar to the previous one, with an additional client cache clean before reading the file for the second time.

Each of these three baseline procedures are executed with both a sequential and a random access pattern, as it is usually expected to observe a different I/O performance depending on the I/O pattern.

We expect to obtain the highest I/O performance when the data are cached on the compute nodes, slightly lower I/O performance when data are cached in data nodes (Lustre only), and the lowest performance when the data are not cached at all.

## 3.5 Experiment 2: Evaluating the effect of the I/O pattern hints

I/O pattern hints, namely SEQUENTIAL and RANDOM, can be used by developers to inform the PFS of the I/O pattern that will be used to read some data. To evaluate the impact of these two I/O pattern hints on I/O performance, we propose a procedure below. We note that while these two I/O pattern hints are supported by POSIX fadvise, they are not supported by Lustre ladvise.

The first step of this procedure is cleaning the cache. This is followed by the use of an I/O pattern hint on the whole file with

either fadvise or ladvise. The last step is reading the file while keeping track of I/O time and volume that can be combined into an I/O throughput. If the PFS is taking these I/O pattern hints into account, we expect to measure a similar or better I/O performance when the I/O pattern hint is matching the I/O pattern of the synthetic application and potentially worse performance when we send a wrong I/O pattern hint.

## 3.6 Experiment 3 methodology: Evaluating the effect of the DON'T_NEED hint

As illustrated in Table 1, the DON'T_NEED hint is supported by both POSIX fadvise and Lustre ladvise. It is used to inform the PFS that some data - that might have been previously cached - will no longer be needed, and that it can as such be removed from the cache. In order to evaluate the effect of this hint, we first clean the cache to have reproducible results, then read the file a first time for it to be cached, apply the hint to the file and at last read the file again while instrumenting I/O volume and I/O time. By comparing the I/O throughput obtained by doing so with the baseline, we can determine if the data or part of the data were effectively removed from the cache or not. This second option may signify that the PFS either ignored the hint, or marked the data for removal but did not act on it for various reasons, such as the cache not being full or the PFS not having enough time to do so.

## 3.7 Experiment 4 methodology: Evaluating the effect of the WILL_NEED hint

The WILL_NEED hint is supported by both POSIX fadvise and Lustre ladvise. It is used to inform the PFS that some data will be needed in the near future. As such, it is usually expected that the reception of this hint by the PFS will be followed by an asynchronous prefetching of these data.

In order to evaluate the effect of this hint, we apply it to the whole file some time before the reading begins, with varying file sizes (from very small to very large) and delays between the prefetch and the reading operations. It can be expected that the earlier the prefetch, the more data will be prefetched at the time the reading starts, and the better the I/O performance.

As an additional baseline in this experiment, the WILL_NEED hints that were added to our synthetic application in the above experiments are replaced with explicit prefetching primitives. In our case, we used POSIX asynchronous reads as they offer the guarantee that the data will be read by the file system. Doing so gives us a baseline on the performance that could be expected if PFS triggered a prefetch upon receiving the WILL_NEED hint.

# 4 Results

All experiments detailed in the methodology were performed with varying I/O sizes, inter-arrival times, I/O patterns and file sizes. Due to the large number of experiments, we cannot show all the figures in this paper, we will focus on the most relevant ones. We chose to set the file size to 16 GiB and the inter-arrival time to 0 (in case changing those values has an impact on the results, it will be mentioned in the analysis).
<figure><img src="image_3.png" /><figcaption>Figure 1: Lustre sequential read</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 2: GPFS sequential read</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 5: Lustre sequential read with I/O pattern hints</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 6: GPFS sequential read with I/O pattern hints</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 3: Lustre random read</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 4: GPFS random read</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 7: Lustre random read with I/O pattern hints</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Figure 8: GPFS random read with I/O pattern hints</figcaption></figure>

## 4.1 Experiment 1: Storage system characterization

The objective of the first experiment is to measure the baseline I/O performance, i.e. the throughput, for all three baseline situations: when the data are not cached, when it is cached in the compute node, or when it is cached in the data nodes for Lustre. In Fig. 1 and 2, the I/O pattern in set to sequential. We observe on Lustre an improved read performance when the data are cached in the compute node main memory (up to 7 GiB/s) versus when it is cached in the data nodes main memory (up to 2 GiB/s) or when it is not cached at all (up to 1,7 GiB/s). This observation holds for GPFS, and for the various file sizes and inter-arrival time that we tested for this experiment. In Fig. 3 and 4, when using a random I/O pattern, the resulting I/O performance is slightly different. While accessing uncached data is never really faster than accessing cached data, the I/O throughput increase obtained by caching the data is minimal for very small I/Os, and maximal for higher I/O sizes.

We note that on both Lustre and GPFS, the throughput during sequential read is close to optimal for an I/O size of 1 MiB, and lower for both higher and lower I/O sizes. Similar observation is made on Lustre during random read. We hypothesize that this lower I/O performance could be caused by network and driver overhead on lower I/O sizes, and by the stripe configuration on higher I/O sizes.

**Takeaway:** Overall, we observe that accessing data cached in the compute nodes is the fastest, followed by accessing data cached in data nodes, if any, and at last, accessing uncached data is the slowest.

## 4.2 Experiment 2: I/O pattern hints effect

The objective of the second experiment is to evaluate the effect of the SEQUENTIAL and RANDOM I/O pattern hints.

In Fig. 5 and Fig. 6 various I/O pattern hints were evaluated while reading the data sequentially. In these graphs, in addition to this experiment's measures, we refer to two baselines from the previous experiment: the measures where the data were not cached is used as our baseline with no I/O pattern hint, while the measures where the data were cached in the compute nodes cache gives the highest expected throughput for the given I/O pattern and I/O size. We observe that sending an I/O pattern hint does not seem to significantly affect the sequential read performance on either GPFS and Lustre. On Lustre, the performance with the hint is the same as the baseline performance where the data is not cached and without the hint. On GPFS, we observe a small deviation of at most 100 KiB/s in the I/O throughput depending on the I/O hint used. No clear trend was observed.

In Fig. 7 and Fig. 8 however, the I/O pattern is set to random. On Lustre, we confirm that I/O pattern hints are ignored as the I/O throughput is the same again regardless of the I/O pattern hint passed. On GPFS, we observe that while the SEQUENTIAL hint provides no performance difference compared to the baseline and is therefore probably ignored, the RANDOM hint (slightly) negatively affects the performance (green curve showing lower throughput in Fig. 8). This can be explained by GPFS not caching the data marked with the RANDOM hint at all, while our random reads still shows a small temporal locality, thus generating some cache misses and lowering the throughput.
<figure><img src="image_11.png" /><figcaption>Figure 9: Using DON'T_NEED on cached data before sequentially reading on Lustre
Figure 10: Using DON'T_NEED on cached data before sequentially reading on GPFS</figcaption></figure>

**Takeaway:** Overall, we observe that the SEQUENTIAL hint does not seem to affect I/O performance on any of the two PFS. This is to be expected on PFS as sequential read is the most common I/O pattern in HPC, even though this tends to change due to new HPC usages such as machine learning [6, 7]. The RANDOM hint, while similarly ignored on Lustre, seems to be taken into account by GPFS. However, we observed that the use of this hint can negatively affect performance, which we explain as GPFS expecting randomly-read data not to be accessed more than once; thus, they are not cached (while few data blocks were re-accessed in the benchmark).

## 4.3 Experiment 3: DON'T_NEED hint effect

When using the DON'T_NEED hint of POSIX fadvise on Lustre, we observe in Fig. 9 the same performance as the baseline where the data are in the data node cache but not in the compute node cache. This can be interpreted as the data have been evicted from the compute node cache instantly upon receiving the hint, but have been kept in the data node cache where it could still be useful for other nodes, which is an interesting behavior. In Fig. 10, we observe that the same hint used on GPFS has no clear impact on I/O performance compared to the baseline where the data were cached. This means that unlike Lustre, GPFS is not directly evicting the data upon receiving this hint. It could be that either GPFS will evict the data later, or that it ignores the hint.

In both Fig. 9 and 10, when using the DON'T_NEED hint of Lustre ladvise, we observe the same performance as when the data are cached in the compute node. This can be explained by ladvise only targeting the data node cache and not the compute node cache.

**Takeaway:** Overall, we see that the DON'T_NEED hint has a different effect on different PFS. On Lustre, we observed that data were directly evicted from the compute node cache when using POSIX fadvise, and from the data node cache when using Lustre ladvise. On GPFS, we observed that data were not directly evicted, but determining whether the hint was ignored or not would require additional experiments.

## 4.4 Experiment 4: WILL_NEED hint effect

The objective of the fourth experiment is to evaluate the effect of the WILL_NEED hint. This hint is usually used to prefetch data asynchronously a small delay before it is used.

<figure><img src="image_12.png" /><figcaption>Figure 11: Full file WILL_NEED hint 1 ms before sequential read start on Lustre</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Figure 13: Full file WILL_NEED hint 1 ms before sequential read start on GPFS</figcaption></figure>

<caption>Figure 14: Just-in-time full file WILL_NEED hint on GPFS</caption>

We evaluated the use of the WILL_NEED hint on the whole file with varying delays between the use of the advising function and the start of the reading. In Fig. 11 and 13, this delay is set to one millisecond (larger delays gave similar results), while in Fig. 12 and Fig. 14 it is set to 0. Moreover, to establish an additional baseline, we added an asynchronous prefetching primitive (forced prefetch) and included its I/O throughput in Figs. 11 and 13.

When using the POSIX fadvise WILL_NEED hint, we observe in Fig. 11-13 no difference in the I/O performance compared to the uncached baseline that was introduced in Fig. 1. This could be interpreted as either the WILL_NEED hint being ignored, or the PFS not having enough time to cache the data. However, when using asynchronous I/Os to force a prefetch with a delay of 1 ms, we can observe a large performance gain of up to x3 on Lustre and x2 on GPFS. This shows that the WILL_NEED hint is likely to be ignored (or is not reactive enough).

In the same figures, we see that the Lustre ladvise WILL_NEED hint can be used to reach the same level of performance as when the data has been prefetched on the data node. This level of performance, however, is still below the level of performance that was observed when data are cached in the compute nodes.

We note that while a forced prefetch gives a good baseline, it cannot be used as a perfect replacement to the WILL_NEED hint: we see that for Lustre it can lead to a x0.5 reduction in I/O throughput when there is no delay between the prefetch and I/Os start.
Moreover, we note that in this experiment, I/O interarrival time has an effect on I/O performance, with higher interarrival time improving the I/O throughput when doing a forced prefetch. For instance, we observed on Lustre that forcing a prefetch with an asynchronous I/O with no delay and an I/O interarrival time of 0 had a negative impact on performance. However, by adding an interarrival time of 10 µs, we obtained an increase in throughput of up to x3. This can be explained by the PFS having more time to prefetch before being overpassed by the I/Os.

**Takeaway:** Overall, we observe that contrary to expectations, application developers should not expect the PFS to prefetch any data when using the POSIX fadvise WILL_NEED hint. Instead, a forced prefetch with a small delay of 1 ms before the I/O starts could increase the I/O performance by a factor of up to x3.

# 5 Discussion and Guidelines

In this section, we provide guidelines to help application developers maximize performance, and identify limitations in the way these advisory hints are interpreted and used by PFSs.

Using I/O pattern hints. From our observation on GPFS and Lustre, the I/O pattern hints do not have a noticeable effect on performance when doing sequential I/Os. This is likely because the sequential I/O pattern is still the most common I/O pattern [6, 7]. GPFS documentation goes as far as explicitly saying that by design GPFS page cache (called the page pool) with its default size of a few GiB could only accelerate sequential I/Os [1]. However, when evaluating the effect of I/O pattern hints on random reads, we observed that while performance was not affected on Lustre, it was negatively impacted on GPFS when using the RANDOM hint. It is likely because when receiving this hint and observing a random read pattern, GPFS stops caching the data in order to avoid wasting space in its very small cache, which slightly reduces the performance because our random read benchmark can read part of the data more than once. As such, we suggest that application developers do not focus on adding hints on sequential reads as it can be assumed PFS will handle them efficiently, and only add hints on random reads after benchmarking their PFS or ensuring there is no temporal locality.

Using the DONT_NEED hint. From our observation, data marked with this hint using POSIX fadvise on Lustre are directly evicted from the compute node page cache. This results, in essence, in a more targeted cache eviction than simply calling the `/proc/sys/vm/drop_caches` Linux interface. Moreover, we observe that using the DONT_NEED hint using Lustre ladvise resulted in the data being evicted from the data node page cache, but not from the compute node page cache, which is an interesting feature. On GPFS however, we observe no effect on I/O performance after applying this hint, suggesting that this hint is either ignored or that the eviction is not instantaneous. We show that using the DONT_NEED hint, it is possible on some file systems including Lustre to free unneeded data from the cache, either instantly or with a delay. As such, we suggest application developers to consider using this hint on data that will not be reused, like logs or checkpoints in order to reduce cache usage.

Using the WILL_NEED hint. From our observation, data marked with this hint with the POSIX fadvise is not guaranteed to be prefetched on the tested PFSs. However, we observed that Lustre ladvise could be used to successfully increase the I/O throughput by prefetching data to the data node cache. As such, we suggest that application developers targeting Lustre focus on adding WILL_NEED Lustre ladvise hints. As for application developers that are not targeting Lustre, we suggest to possibly consider forcing asynchronous data prefetching as long as there is a delay between the prefetch and the data access.

# 6 Conclusion

In this paper, we proposed a methodology for evaluating the effectiveness of advisory hints on Lustre and GPFS. Our investigation has revealed critical insights for application developers and indicate that the effect of these hints varies significantly between state-of-the-art PFSs, and that their effect was often not straightforward. We provided guidelines for developers to handle these hints. Overall, our study highlights the necessity for deeper understanding and tailored approaches in leveraging advisory hints to achieve optimal I/O performance. As a perspective for future work, we consider extending our study to more file systems.

# References

[1] 2024. IBM Storage Scale. https://www.ibm.com/docs/en/storage-scale/5.1.9?topic=configuring-parameters-performance-tuning-optimization [Online; accessed 12. Nov. 2024].

[2] 2024. IO500 - ISC24 - Research List. https://io500.org [Online; accessed 10. Oct. 2024].

[3] 2024. Lustre* Software Release 2.x. https://doc.lustre.org/lustre_manual.xhtml#idm140494033959744 [Online; accessed 12. Nov. 2024].

[4] 2024. posix_fadvise. https://pubs.opengroup.org/onlinepubs/9699919799/ functions/posix_fadvise.html [Online; accessed 12. Nov. 2024].

[5] 2024. SQLite Home Page. https://sqlite.org/index.html [Online; accessed 3. Oct. 2024].

[6] Jean Luca Bez, Suren Byna, and Shadi Ibrahim. 2023. I/O Access Patterns in HPC Applications: A 360-Degree Survey. *ACM Comput. Surv.* 56, 2 (Sept. 2023), 1–41. https://doi.org/10.1145/3611007

[7] Jalil Boukhobza and Pierre Olivier. 2017. *Flash Memory Integration: Performance and Energy Issues*. Elsevier.

[8] Facebook. [n.d.]. RocksDB | A persistent key-value store. https://rocksdb.org

[9] Shaleen Garg, Jian Zhang, Rekha Pitchumani, Manish Parashar, Bing Xie, and Sudarsun Kannan. 2024. CrossPrefetch: Accelerating I/O Prefetching for Modern Storage (ASPLOS '24). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3617232.3624872

[10] Dusol Lee, Inhyuk Choi, Chanyoung Lee, Sungjin Lee, and Jihong Kim. 2023. P2Cache: An Application-Directed Page Cache for Improving Performance of Data-Intensive Applications (HotStorage '23). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3599691.3603408

[11] Eunji Lee, Hyokyung Bahn, Minseong Jeong, Sunghwan Kim, Jesung Yeon, Seunghoon Yoo, Sam H. Noh, and Kang. G. Shin. 2016. Reducing Journaling Harm on Virtualized I/O Systems (SYSTOR '16). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2928275.2928289

[12] Jakob Lüttgau, Michael Kuhn, Kira Duwe, Yevhen Alforov, Eugen Betke, Julian Kunkel, and Thomas Ludwig. 2018. Survey of storage systems for high-performance computing. *Supercomputing Frontiers and Innovations* 5, 1 (2018).

[13] Sophie Robert, Soraya Zertal, and Philippe Couvee. 2020. Shaman: A flexible framework for auto-tuning hpc systems. In *Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems*. Springer, 147–158.

[14] Frank Schmuck and Roger Haskin. 2002. {GPFS}: A {Shared-Disk} file system for large computing clusters. In *Conference on file and storage technologies (FAST 02)*.

[15] Philip Schwan et al. 2003. Lustre: Building a file system for 1000-node clusters. In *Proceedings of the 2003 Linux symposium*, Vol. 2003. 380–386.

[16] Xiaowei Zhu, Wentao Han, and Wenguang Chen. 2015. {GridGraph}: {Large-Scale} Graph Processing on a Single Machine Using 2-Level Hierarchical Partitioning. 375–386 pages.