# libMeshb: a Simple, Fast and Versatile Library to Handle HPC Meshes and Solutions with a Dedicated File Format

Loïc Maréchal

► To cite this version:

Loïc Maréchal. libMeshb: a Simple, Fast and Versatile Library to Handle HPC Meshes and Solutions with a Dedicated File Format. AIAA SCITECH 2026 Forum, Jan 2026, Orlando, FL, United States. hal-05469677

HAL Id: hal-05469677

https://inria.hal.science/hal-05469677v1

Submitted on 21 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# libMeshb: a Simple, Fast and Versatile Library to Handle HPC Meshes and Solutions with a Dedicated File Format

Loïc Maréchal *

GammaO, Equipe Inria-ONERA, Centre Inria de Saclay, Université Paris-Saclay, 91120 Palaiseau, France

## Abstract

Problem statement: a full HPC toolchain is made of numerous pre/post-processing tools and solvers, all communicating with each other either through file-based I/O or in memory exchanges. The first solution is less invasive in terms of software development but requires lots of external libraries, components and knowledge to use them (i.e. MPIO, HDF5 ([5]), GPFS ([6]), Lustre ([11]), CGNS ([2])). The latter is more invasive as internal data structures need to be adapted and exposed to the communication layer but is much faster both in terms of throughput and latency. Furthermore, most standard HPC I/O libraries and file formats have been developed in the time of spinning hard drives and fail to extract the full potential of today's SSDs.

Solution: we propose a lightweight library made of a single C file, without external dependency aside the Linux kernel's async IO and pthreads, whose API is based on tables and scalars so it can be easily accessed from any kind of languages and that is inherently parallel and SSD-native.

## Motivation

When it comes to file I/O in the fields of numerical simulations, the coming of fast SSD able to sustain around 10 GB/s is a game changer. We quickly realized that our old in-house file format and associated libraries were just fast enough to extract the full performance of spinning hard drives (200 MB/s) but topped at one or two GB/s with SSD, failing to extract their full speed. It turned out that there was no single culprit to this speed limit but the whole I/O chain was not up to the task. It was sequential, it used the high-level LIB-C library interface and it used blocking I/O, reading data then processing it afterward.

Our first goal was to make the best out of a single fast SSD and prepare the work for distributed access to and from a single file in order to increase the speed while maintaining the ease of use for the user. Hence our choice to enable concurrent access to a single file rather than distribute access to separate sub files, even if going this route was more challenging.

As a result, we decided to develop a concurrent read and write access capacity, to use asynchronous I/O to process data while reading/writing it, and to use low-level disk access to extract the full raw speed of SSD.

The first development step presented in this paper offers all the above functionalities and validated them on a single consumer grade SSD using multithreaded parallelism. On a second step, we plan on testing the library with distributed MPI parallelism on a multi-SSD file server whose theoretical speed is around 100 GB/s. We also validated accesses on local disks and network disk connected through Ethernet with the NFS protocol and evaluate the speed penalty deriving from the full network stack.

## File Format

An INRIA *meshb* file consists of a single linked list of data fields, each containing a single kind of data associated with a mesh entity. Each field is called a *keyword*, a concept that is the building blocks of our format and is made of four fields:

* a unique 32-bit integer that identifies the keyword kind (up to 4 billion possibilities)

* a 64-bit pointer to the next keyword position in the file

* a header describing the data field, it is made of a single 64-bit integer that stores the number of lines of the data field (i.e., number of vertices or tetrahedra), some fields may require a more complex header like physical solutions (number of physical fields per mesh entity, field dimension, etc.)

<footnote>*Research Engineer, loic.marechal@inria.fr</footnote>
* the data field itself: it is a 2-D table whose width and height are fully defined by the header and the keyword code (for example, a tetrahedron is made of four integers to store its nodes and a fifth integer to store a physical attribute; hence the data field contains one million tetrahedra and is made of 5 million integers.)

This simple keyword-based approach is highly customizable and we made it easy for programmers to add new keywords with the help of a single character string that defines its whole structure. This string is interpreted by the library at run time, each letter being interpreted this way:

* i: a single integer (32 or 64 bits depending on a file-wide setting)

* r: a single real value (32 or 64 bits)

* c: a single character used for comment).

Additionally, each single letter in the description string above can be preceded by a modifier that will duplicate the scalar value several times. It is useful to describe complex physical fields or high order meshing elements without specifying tens of "i" or "r" characters.

* d: the following letter is multiplied d times, d being the mesh dimension. For example, "dr" will be expanded into "rr" in 2D or "rrr" in 3D which is useful to describe coordinates vectors.

* s: the following letter is multiplied by the number of solutions real values defined by the solution header.

* h: the following letter is multiplied by the number of nodes in the high-order element specified by the keyword (10 for a 2nd order tetrahedron or 64 for a 3rd order hexahedron)

With this very basic format string, it is easy to build more complex data structures with the help of a single line descriptor. Here are a few illustrating examples:

<table><caption>**Fig. 1** Some fields definitions among the 250+ available. Any new field can be easily defined with a single character string.</caption><thead><tr><td>unique iD</td><td>name string</td><td>header</td><td>field</td><td>comment</td></tr></thead><tbody><tr><td>3</td><td>GmfDimensions</td><td>0</td><td>i</td><td>empty header = information field made of one line</td></tr><tr><td>4</td><td>GmfVertices</td><td>i</td><td>dri</td><td>header "i": the header only contains the number of lines</td></tr><tr><td>8</td><td>GmfTetrahedra</td><td>i</td><td>iiiii</td><td>a tetrahedron is made up of four node indices and a material tag</td></tr><tr><td>68</td><td>GmfSolAtHexahedra</td><td>i</td><td>sr</td><td>multiple real values as defined by the solution descriptor</td></tr></tbody></table>

The complete keywords list can be found on the project's GitHub page [9]. The initial field definition was given in the book *Mesh Generation* ([12]) and subsequent definitions are listed in the library's documentation ([8]). At the time of writing, 240 keywords have been defined and new ones can be submitted to the authors.

The advantage to this linked list of keywords is that the file format is upgradable and offers both backward and upward compatible between different versions of the library. A code compiled with the oldest version of the library will always be able to open a file generated with the latest one. It will be able to cycle through all keywords, reading and processing the ones it knows, and simply skipping the ones that had been invented after its compile time. Thanks to this capacity, we have been able to maintain strong compatibility between our software tools for more than 25 years.

## The libMeshb library

To ease the integration of this file format, an ad hoc library is freely provided under the MIT license, so that anyone can integrate it into any commercial or free software.

The whole library is made of a single ANSI C file and a header file to be included and compiled along the including code. The source code is self-contained and its few dependencies are available in the Linux or macOS kernel (AIO: asynchronous input/output and PTHREADS: posix threads).

Its API is simply based on tags, scalars and tables so that it may be very easily interfaced with any kind of language (C, F77 and F90 interfaces are provided). Accessing data on the disk is done through a single call that takes a file tag, and keyword tag, the number of entities to be accessed and a table containing these entities.
A few commands are provided by the library to handle most use cases: open, read/write a line of data, read/write a whole block of data, get the file information and close a file. As a result, a sample code that reads a tetrahedral mesh is only made of a few dozens of lines.

Since the purpose of this paper is not to duplicate the library's documentation and GitHub Wiki, the readers are encouraged to visit the project's page ([9]). Anyway, here is the shortest code needs only four instructions to access a file which makes for a very easy startup.

Opening a mesh file:

```R
FileID = GmfOpenMesh("manytets.meshb", GmfRead, &version, &dimension);
```

Getting the number of tetrahedra if present in the file:

```R
NbTets = GmfStatKwd(FileID, GmfTerahedra);
```

Asking the library to read all the tetrahedra and store them in the user's tables:

```R
GmfGetBlock(FileID, GmfTerahedra, 1, NbTets, 0, 0, 0, GmfIntVec, 5, tets[1], tets[NbTets]);
```

Close the mesh file *et voilà*:

```R
GmfCloseMesh(FileID);
```

Remark: opening a mesh file returns a unique file ID that you can pass on to any further operations on the file until it is closed. You may open as many files as you need at the same time, each ID and its associated internal structure being completely separate.

## Low-level File Access

Every file access from the C language can either use the low-level API (open, read, close) or the high-level operations (fopen, fread, fclose). The high-level operations are indeed based on the lower-level ones and add some useful features like automatic file buffering and file positioning. Consequently, this I/O pipeline is compute-bound and single-threaded and its useful speed hovers around one or two GB/s depending on the processor's speed. Low-level operation requires more work as buffering and byte precise positioning of data to be read or written must be handled by the programmer. The libmeshb uses this low-level API to transfer data directly from the file to the user's data structures without moving them through the C high-level file handling pipeline, hence making the transfer faster.

## Asynchronous I/O

Regular, high or low level I/O operations, are synchronous, which means that a call to read a given block of data must be fully completed before the data can be processed. In MPI parlance, they are called blocking I/O. The optional async I/O provided by today's unices (Linux and macOS) can perform asynchronous I/O, also known as non-blocking I/O. In this mode, a call to read a block of data returns control to the calling code immediately, without waiting for the transfer to complete. It is up to the user to wait for completion before manipulating the data. The libmeshb uses this capacity to further increase preprocessing speed by splitting the data field to read into smaller chunks that are red and processed in a pipelined way.

For example, the library can read the vertex coordinates and compute the mesh bounding box at the same time. To do so, it starts reading the first chunk of 40,000 vertices, wait for completion, then start reading the next chunk and at the same time starts computing the partial bounding box of the first chunk while reading the second one. When chunk 2 is read, its processing starts while the library starts reading the chunk 3, and so on until completion of the last chunk. This way we can overlap the pre/post-processing with the read or write operations, speeding I/O up to a factor 2.

## Parallelism

As each numerical field stored in the file is a direct bijective map from memory to disk, it was easy to divide operations into chunks that do not overlap each other. A single mesh file can be opened at the same time by multiple threads or tasks, either local to the machine that contains the storage hardware or from distant machines connected through a network. Each process initializes it own instance of the libMeshb and all instances can access the same file at the same time, offering parallelism while keeping the simplicity of single file storage.
Parallel read is fairly easy as no write contention can arise, so several reading processes may even overlap each other without compromising the I/O validity. It is more challenging when it comes to writing data concurrently and it is up to the user to make sure that ranges of items written by all threads or MPI ranks do not overlap each other. A naive but efficient way to make sure they do not overlap is to split the data into consecutive chunks and assign them to each process: task 1 could write data from vertex 1 to n, task 2 could write vertices from n+1 to 2n, and so on, thus avoiding any write contention.

Hardware matters: it is important to note that performing concurrent accesses at different file locations on a hard drive will not increase transfer speed in any way as the HD has a single read/write head! Indeed, as the HD's head will frantically move to and from distant positions on the platters, it will even slow things down. Such parallel I/O are only useful with fast SSD whose throughput is greater than what a sequential CPU can handle (around 1GB/s). As most recent gen 5 PCIe NVME SSD can reach up to 14 GB/s, our parallel development is useful even on a single drive. Of course, much higher speed could be reached with parallel SSD servers, some of them reaching over 200 GB/s either locally or through multiple 400 Gb/s network interfaces.

## Use cases

The INRIA *meshb* file format has been used in all our team's software for many years, consequently it has been fully tested and tried. We believe its performance and ease of use could make it a useful tool in the field of HPC compared to many available solutions that may be more scalable but at the cost of a much higher complexity. We put great emphasis on ease of integration so that every single project will use *libMesh*. Indeed, we witnessed that research team that selected a complex I/O software tended to have a lot of stray codes that skipped the official team's choice and used a makeshift solution, inconsistent, but much faster to integrate. We think there is a balance to strike between the I/O library's functionalities and its ease of use.

Here are some notable Gamma team's software that makes use of the *libMesh*:

* Vizir ([3]) a mesh and solution visualizer

* Wolf ([1]) a CFD solver

* Hexotic ([10]) a hexahedral mesh generator

* GHS3D ([4]) a tetrahedral mesh generator

## Performance

Over the past decades, all HPC related I/O libraries and file formats have been designed around hard-disks based storage. As these kinds of devices had a throughput ranging from a few MB/s in the 1990' to a few hundreds of MB/s as of today, the full process stack needed to be just as fast to handle such speed. Thus, the sequential data processing speed of most I/O software was in the hundreds of MB/s, which was fast enough to get the best of any hard-drive speed. Higher speed was achieved through the concurrent access to many hard drives (RAID, Redundant Array of Interleaved Disks).

When very fast SSD (PCIe attached NVME devices) appeared around 2020 that offered transfer speed of up to 15 GB/s, these old I/O subsystems turned out to be a limiting factor and did not fully benefit from these newer SSDs speed. Hence came the need to design a new SSD-native software that both focused on high sequential speed and the use of multithreading to saturate a single SSD's throughput.

The table 2 sums up a simple performance test that reads a single mesh file made of one billion tetrahedra and 160 million vertices. This test was conducted on a 2021 MacBook Pro laptop just to emphasize how fast a consumer SSD can be. It turns out that it takes four concurrent access to reach the SSD theoretical speed (5.7 GB/s according to the manufacturer) which is expected as the device is made of four parallel flash modules. The fastest server-grade SSD can have as many as 16 modules so they will probably need as many threads to fully exploit their speed.

**Remark:** the *meshb* exists also in ASCII format but only for the sake of debugging and experimenting. The ASCII format is deprecated for HPC usage as it is compute-bound (lots of strings need to be interpreted) and cumbersome to parallelize as there is no direct bijective mapping between the file and the memory (a double precision number always uses 8 bytes in binary but its size can vary in ASCII).

The table 3 compares the scalability of binary access in read and write modes, locally and through a slow 1 Gb/s Ethernet network. Surprisingly, concurrent access helped to increase the speed through the slow 1 Gb/s network by making the byte flow more constant throughout the transfer. Single threaded networked transfer oscillates between 30 and 80 MB/s probably due to the flow moving through the multiple TCP/IP layers, buffers and packet sizes.
<table><caption>Fig. 2 Performance comparison between ASCII, binary and parallel binary. As the test MacBook Pro's SSD is made of four cells, it takes that many threads to reach the full SSD's throughput.</caption><thead><tr><td>Format</td><td>File size in GB</td><td>Nb of threads</td><td>Read time in s</td><td>Speed in MB/s</td><td>Speed in Million Tetrahedra/s</td></tr></thead><tbody><tr><td>ASCII</td><td>47</td><td>1</td><td>750.43</td><td>63</td><td>1.2</td></tr><tr><td>Binary</td><td>23</td><td>1</td><td>13.32</td><td>1741</td><td>70</td></tr><tr><td>Binary</td><td>23</td><td>2</td><td>7.14</td><td>3221</td><td>129</td></tr><tr><td>Binary</td><td>23</td><td>4</td><td>4.13</td><td>5569</td><td>223</td></tr><tr><td>Binary</td><td>23</td><td>8</td><td>4.09</td><td>5623</td><td>226</td></tr></tbody></table>

<table><caption>Fig. 3 I/O throughput in GB/s depending on the storage hardware and access method, Read/Write, Serial/Parallel with four threads.</caption><thead><tr><td>Hardware</td><td>Serial Read</td><td>Serial Write</td><td>Parallel Read</td><td>Parallel Write</td></tr></thead><tbody><tr><td>Single HDD</td><td>322</td><td>279</td><td>257</td><td>312</td></tr><tr><td>RAID5 with 6 HDD</td><td>1200</td><td>1065</td><td>984</td><td>625</td></tr><tr><td>PCIe4 NVME SSD</td><td>1483</td><td>2582</td><td>5084</td><td>6014</td></tr><tr><td>1 Gbps network HDD</td><td>66</td><td>90</td><td>81</td><td>85</td></tr><tr><td>1 Gbps network SSD</td><td>69</td><td>89</td><td>81</td><td>85</td></tr></tbody></table>

## Conclusion and Perspectives

We have successfully developed, used and validated this basic, yet versatile and efficient, file format for over 25 years at the Gamma team and we are offering it to the HPC community in the hope that many programmers could benefit from its ease of use. The best way to try and use it is to start from a sample code available from the GitHub page and adapt it to suit your needs. If that is the case, you may go further by reading the complete documentation and peer through the more complex examples provided. You may go even further by requesting new keywords to the authors through the same GitHub platform.

We are currently finalizing the work on parallel write access on a single file, which is made possible with low-level access and sparse file capability of Linux and macOS. As for the v8.0 beta, parallel writes work but are limited to a single keyword per mesh file.

We also plan to validate our approach on faster multi-SSD file servers that can reach up to a hundred GB/s throughput. Being able to read or write to a single mesh or solution file at this kind of speed would fulfill all our HPC needs. Eventually, we will run the new IO500 ([7]) test suite to compare the libMesh's performance against other HPC I/O subsystems.

## References

[1] F. Alauzet and L. Frazza. Feature-based and goal-oriented anisotropic mesh adaptation for RANS applications in aeronautic and aerospace. JCP, 439(110340):77 pages, 2021.

[2] The CGNS Steering Committee. The official cgns home page.

[3] R. Feuillet, M. Maunoury, and A. Loseille. On pixel-exact rendering for high-order mesh and solution. JCP, 424:109860, 2021.

[4] P.L. George, F. Hecht, and E. Saltel. Automatic mesh generator with specified boundary. CMAME, 92:269-288, 1991.

[5] The HDF Group. High-performance data management and storage suite.

[6] IBM. The general parallel file system.

[7] IO500. The io500 benchmark has been developed together with the community and its development is still ongoing.
[8] Loïc Maréchal. An easy way to access files in gamma mesh format: The libmeshb library.

[9] Loïc Maréchal. A library to handle the *.meshb file format.

[10] Loïc Maréchal. Advances in octree-based all-hexahedral mesh generation: handling sharp features. 2009.

[11] OpenSFS. The lustre® file system is an open-source, parallel file system that supports many requirements of leadership class hpc simulation environments.

[12] Paul-Louis George Pascal J Frey. *Mesh Generation*, chapter 1, pp32-38. ISTE - WILEY, London, 2 edition, 2000.
**Dimension:** must be provided at file creation, only dimensions 2 and 3 are supported.

**Returns:** zero on failure or the open mesh index otherwise. This index should be properly stored since it must be provided to any further `libMesh` commands working on this file.

**Example:** create a new three-dimensional mesh file storing double precision numbers.

```cpp
MeshIndex = GmfOpenMesh("newfile.meshb", GmfWrite, 2, 3);
```

### GmfCloseMesh

A mesh file must be properly closed in order to release any allocated memory and to write tailing information.

```cpp
GmfCloseMesh( int64_t MeshIndex );
```

**MeshId:** the index returned by `GmfOpenMesh()` must be provided for the file to be closed.

### GmfStatKwd

This command queries the mesh file for the presence of a given keyword and the number of associated lines.

*Getting information on a mesh keyword*

```cpp
int GmfStatKwd( int64_t MeshIndex,
                 int Keyword );
```

**MeshIndex:** index of referenced mesh.

**Keyword:** the keyword tag you are requesting information on (check the GitHub page [9] for a full list of available keywords).

**Example:** check out and print the number of triangles in a mesh file.

```cpp
int NumberOfTriangles;
```

```cpp
NumberOfTriangles = GmfStatKwd( MeshIndex, GmfTriangles );
```

```cpp
if(NumberOfTriangles)
    printf("This file contains %d triangles\n.", NumberOfTriangles);
```

```cpp
else
    puts("This file does not contain any triangle.");
```

*Getting information on a solution keyword*

In this case, additional information will be provided: the number of fields per solution, the number of real numbers a solution line occupies and a table of solution types.

```cpp
int64_t GmfStatKwd( int64_t MeshIndex,
                     int Keyword,
                     int *NumberOfTypes,
                     int *SizeOfSolution,
                     int *TableOfTypes );
```
**MeshIndex:** index of referenced mesh.

**Keyword:** the keyword tag you are requesting information on (check the GitHub page [9] for a full list of available keywords).

**NumberOfTypes:** pointer to an integer, it will be set to the number of fields in the solution.

**SizeOfSolution:** pointer to an integer, it will be set to the number of real numbers (float or double depending on the file version) used by a solution line for memory allocation purpose.

**TableOfTypes:** pointer to a previously allocated table which will be filled with the type of each solution field.

**Example:** check out and print the number of solutions and their kinds associated to vertices.

```c
int NmbSol, NmbTypes, NmbReals, TypesTab[ GmfMaxTyp ];

NmbSol = GmfStatKwd( MeshIndex, NmbSol, &NmbTypes, &NmbReals, TypesTab );

if(NmbSol)
{
    printf("This file contains %d solutions at each vertex\n.", NmbSol);
    printf("Each solution contains %d fields:\n", NmbTypes);

    for(i=0; i<NmbTypes; i++)
    {
        switch(TypesTab[i])
        {
            case GmfSca : printf("scalar,\n"); break;
            case GmfVec : printf("vector of %d scalars,\n", dim); break;
            case GmfSymMat : printf("upper triangular part of a symmetric \
                                %d x %d matrix,\n", dim, dim); break;
            case GmfMat  : printf("full %d x %d matrix,\n", dim, dim); break;
        }
    }
}
else
    puts("This file does not contain triangles.");
```

## GmfSetKwd

Prior to writing each line of a keyword with the GmfSetLin() command, the keyword header should be written along with the number of lines.

### Writing a mesh keyword

```c
int GmfSetKwd( int64_t MeshIndex,
               int Keyword,
               int NumberOfLines );
```

**Mesh Idx:** the index returned by GmfOpenMesh() must be provided.

**Keyword:** code of the keyword whose data are to be written.
**NumberOfLines:** number of data lines which are to be written.

**Returns:** zero if the data could not be written, and the number written lines otherwise.

### Writing a solution keyword

When it comes to solution keywords, two extra arguments must be passed on: a table of solution types and its size.

```c
int GmfSetKwd( int64_t MeshIndex,
               int Keyword,
               int NumberOfLines,
               int NumberOfTypes,
               int *TableOfTypes );
```

**Mesh Idx:** the index returned by GmfOpenMesh() must be provided.

**Keyword:** code of the keyword whose data are to be written.

**NumberOfLines:** number of data lines which are to be written.

**NumberOfTypes:** the number of fields stored for each line of this solution. It sets the size of the following TableOfTypes containing each field type.

**TableOfTypes:** pointer to a table of integers, each entry setting the type of each solution field: 1 for a scalar, 2 for a vector, 3 for symmetric matrix and 4 for a full matrix.

**Returns:** zero if the data could not be written, and the number written lines otherwise.

### GmfGetBlock

GmfGetBlock() is a variable argument command, it reads all the lines of data from the file and stores each item in the provided pointers to users' data structures. The user's data structure has to be fully described in order for the library to fill all the lines automatically.

```c
int GmfGetBlock(int64_t MeshIndex, int Keyword, int64_t BeginLine,
               int64_t EndLine, int MapType, void *RenumberingMap,
               void *Procedure, arguments...);
```

**Mesh Idx:** the index returned by GmfOpenMesh() must be provided.

**Keyword:** code of the keyword whose lines of data are to be read.

**BeginLine:** starting line in the mesh file, it enables partial reading for parallelism.

**EndLine:** ending line in the mesh file.

**MapType:** set the integer type (GmfInt or GmfLong) of the following renumbering table.

**RenumberingMap:** pointer to a renumbering table that gives the position to store each mesh entity in the user's table (for example: give the old to new renumbering through a Hilbert curve).

**Procedure:** pointer to an optional user's procedure that will be called in parallel after each block has been read. If a procedure is given, a second pointer on users' data must be provided right after.
**arguments**: for each type of data as stated by the keyword definition, three arguments must be provided. First, the user's type of data in which the file's data will be stored (four kinds are available: GmfFloat, GmfDouble, GmfInt and GmfLong). Second, a pointer to the first line of this data type in the user's structure. Third, the same pointer but on the last line. The example below is more telling.

**Example:** reading all vertices in three-dimensional case.

```c
int ref[nv];
double x[nv], y[nv], z[nv];

GmfGetBlock(MeshIndex, GmfVertices, 1, nv, 0, NULL, NULL,
             GmfDouble, &x[1], &x[nv],
             GmfDouble, &y[1], &y[nv],
             GmfDouble, &z[1], &z[nv],
             GmfInt, &ref[1], &ref[nv]);
```

## GmfSetBlock

Works exactly as GmfGetBlock except that all lines are written, you cannot specify starting and ending lines in the file since concurrent writing is not supported for now. Note that you still need to set the keyword first with the help of GmfSetKwd() prior to writing the whole data lines with GmfSetBlock().

**Example:** applying a pre-processing function on vertices before writing them on the disk.

```c
int ref[nv];
double x[nv], y[nv], z[nv];

GmfSetBlock(MeshIndex, GmfVertices, 1, nv, 0, NULL, FlipRefs, ref,
             GmfDouble, &x[1], &x[nv],
             GmfDouble, &y[1], &y[nv],
             GmfDouble, &z[1], &z[nv],
             GmfInt, &ref[1], &ref[nv]);

FlipRefs(int64_t begin, int64_t end, void *data)
{
    int *ref = (int *)data;
    int64_t i;

    for(i=begin; i<=end; i++)
        if(ref[i] == 1)
            ref[i] = 2;
    else
        ref[i] = 1;
}
```

## GmfSetHONodesOrdering

This command sets a special high-order nodes ordering for the specified element. There is, unfortunately, as many HO nodes ordering in each kind of elements as there are programmers! It is then useless to try to find a common node ordering in any high-order element so it was decided to add a new set of keywords (like GmfTrianglesP2Ordering), that map the sequential finite element ordering with the Bezier indices. When reading a HO mesh, you may provide the libMeshb with your own ordering table, as defined below, along with the one that you may have found in the processed mesh file, so that the GmfGetBlock() function will transparently reorder the nodes with each element in your own way.
```c
int GmfSetHONodesOrdering( int64_t MeshIndex, int Keyword,
                          int *YourOrdering, int *FileOrdering );
```

**MeshIdx:** the index returned by GmfOpenMesh() must be provided.

**Keyword:** code of the keyword whose lines of data are to be read.

**YourOrder:** a pointer to your ordering table.

**FileOrdering:** a pointer to the ordering table found in the mesh file.

**Example:** reading and reordering Q2 quads.

This example reads the ordering information stored in the *GmfQuadrilateralsQ2Ordering* keyword, calls the *GmfSetHONodesOrdering()* procedure to link the required ordering with that from the file, and finally reads the Q2 quads.

```c
int quad[nq];
int FileOrdering[9][2];
int MyOrdering[9][2] = {
    {0,0}, {1,0}, {2,0},
    {0,1}, {1,1}, {2,1},
    {0,2}, {1,2}, {2,2} };

GmfGetBlock(MeshIndex, GmfQuadrilateralsQ2Ordering, 1, 9, 0, NULL,
            NULL, GmfIntTab, 9, FileOrdering[0], FileOrdering[8]);
GmfSetHONodesOrdering( MeshIndex, GmfQuadrilateralsQ2,
                      MyOrdering, FileOrdering );
GmfGetBlock(MeshIndex, GmfQ2Quadrilaterals, 1, nq, 0, NULL,
            NULL, GmfIntTable, 9, &quad[1], &quad[nq]);
```