Scheduling Working Group Alain Roy

Scheduling Working Document: 4 The University of Chicago

Category: Best Current Practice Volker Sander

Forschungszentrum Jülich GmbH

> June 2000

Advance Reservation API

Status of this Draft

> This document specifies a best current practices for the grid
> scheduling community. Discussion and suggestions are requested.
> Distribution of this memo is unlimited.

Copyright Notice

> Copyright (c) The Grid Forum (2000). All rights reserved.

1\. Introduction

The Grid Scheduling architecture should provide programmers with
convenient access to end-to-end quality of service (QoS) for programs.
To do so, mechanisms for making QoS reservations for different types of
resources, including computers, networks, and disks are required. A
reservation is a promise from the system that an application will
receive a certain level of service from a resource. For example, a
reservation may promise a certain bandwidth on a network or a certain
percentage of a CPU.

A Grid resource reservation API should provide two interesting
advantages. First, it should allow to make reservations either in
advance of when the resource is needed or right at the time that the
user needs theman *immediate reservation*. Second, the same API should
be capable of making and manipulating a reservation regardless of the
type of the underlying resource, thereby simplifying the programming
when an application needs to work with multiple kinds of resources.

The proposed Grid Advance Reservation API can be considered a remote
procedure call mechanism to communication with a reservation manager. A
reservation manager controls reservations for a resource: it performs
admission control and controls the resource to enforce the reservations.
Some resources already have the ability to work with advanced
reservations, so the reservation manager is a simple program. Most
resources cannot deal with advanced reservations, so the reservation
manager tracks the reservations and does admission control for new
reservation requests.

To create a flexible architecture, supporting co-allocation, resource
location and resource acquisition steps as described in S-RFC 3, a
layered architecture with three levels of APIs and one level of
low-level mechanisms is proposed:

The remaining document describes the intermediate API, the Grid Advance
Reservation API (GARA).

2\. Reservations

Reservations have five important attributes:

> *Start Time:* The time that the reservation begins. A reservation
> always has a start time, even if it is an immediate reservation, which
> begins as soon as you make the reservation. The start time is in
> seconds since 00:00:00 UTC, January 1, 1970. For example, if you want
> an immediate reservation, you can just call the Unix time() function.
>
> *Duration:* How long the reservation lasts, in seconds. All
> reservations must specify how long they will last, so that GARA can do
> appropriate admission control for reservations granted in advance.
>
> *Reservation Type:* The type of underlying resource, such as a
> network, a computer, or a disk.
>
> *Reservation Subtype*: A particular kind of reservation. See *Types of
> Network Reservations*, below.
>
> *Resource-Specific Parameters:* Parameters that are unique to each
> type of resource, such as bandwidth for a network reservation.

When a reservation is requested, one must specify these attributes. If
the reservation request is accepted, a *reservation handle* is returned
from the system. This is an opaque string that uniquely identifies your
reservation. All future operations require you to provide this handle.
The internal format of this string will be described in a different
S-RFC.

Once the application has received a reservation handle, it can perform
several operations described in S-RFC 3 with that handle:

-   *Modify Reservation:* One can request a modification to an existing
    reservation. For instance, one can increase the bandwidth that has
    already been requested.

-   *Cancel Reservation:* One can inform the reservation manager that
    the reservation is no longer needed, by canceling it.

-   *Claim Reservation:* When the application is ready to use a
    reservation, it must claim the reservation. This is known as
    *binding* a reservation because one has to specify run-time
    parameters that were not known when the reservation was created,
    such as ports being used for the network reservation.

-   *Query Reservation:* One can discover the status of a reservation by
    polling it. The status includes whether the start of the reservation
    has begun and whether the reservation has been claimed.

-   *Register Callback:* One can provide a function that will be called
    when the status of a reservation changes or when the reservation
    manager wishes to provide extra information to the application. This
    information may include notification that the related reservation
    appears to be too small. Callback implement the monitor function
    described in S-RFC 3.

3\. Using the API

The API should be provided as a library that can be linked by programs
written in C. Additional bindings, as Java implementation, are not
described here. Notice that this API is derived from the GARA API which
and has some references to Globus. A final version of the API may not
include any such references to Globus.

3.1 Initialization

Before one can use the API, an initialization of the module is required.

module_activate(GARA_CLIENT_MODULE);

3.2 Describing a Reservation Request

Reservation attributes or allocation properties are described using the
Resource Specification Language (RSL). An RSL string is simply a list of
attribute-value pairs that looks like

&(attribute-1=value-1) (attribute-2=value-2) ... (attribute-N=value-N)

An example RSL string for requesting a network reservation for 150Kbps
between looks like this:

&(reservation-type=network)

> (start-time=953158862)
>
> (duration=3600)
>
> (endpoint-a=140.221.48.146)
>
> (endpoint-b=140.221.48.106)
>
> (bandwidth=150)

Note that this string was spaced out on several lines for readability,
while RSL strings do not have newlines in them.

Below is a list of attributes that may be used to specify a reservation.
The universal attributes are for all types of reservations, while the
other attributes are for specific types of resources. Note that the
compute resource attributes are mutually exclusive, and currently only
the percent-cpu attribute is used.

<table>
<colgroup>
<col style="width: 21%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 1%" />
<col style="width: 7%" />
<col style="width: 1%" />
<col style="width: 1%" />
<col style="width: 5%" />
<col style="width: 14%" />
<col style="width: 1%" />
<col style="width: 29%" />
</colgroup>
<tbody>
<tr class="odd">
<td><strong>Attribute</strong></td>
<td><strong>Units</strong></td>
<td colspan="3"><strong>Default</strong></td>
<td colspan="3"><strong>Req?</strong></td>
<td colspan="3"><strong>Description</strong></td>
</tr>
<tr class="even">
<td colspan="2">Universal Attributes</td>
<td colspan="6"></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td>reservation-type</td>
<td></td>
<td colspan="3"></td>
<td colspan="3">Y</td>
<td colspan="3">Allowable values: “network”, “compute”, or “disk”.</td>
</tr>
<tr class="even">
<td>reservation-subtype</td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3">Currently valid only for network reservations. If it is
not specified, it is a foreground reservation. Otherwise it is one of
“background” or “low-latency”. For more information, see <em>Types of
Network Reservations</em> above.</td>
</tr>
<tr class="odd">
<td>start-time</td>
<td>secs</td>
<td colspan="3"></td>
<td colspan="3">Y</td>
<td colspan="3">What time the reservation starts in seconds since
00:00:00 UTC, January 1, 1970. If you specify “now”, then the
reservation will begin immediately.</td>
</tr>
<tr class="even">
<td>duration</td>
<td>secs</td>
<td colspan="3">100</td>
<td colspan="3"></td>
<td colspan="3">Length of the reservation, in seconds.</td>
</tr>
<tr class="odd">
<td colspan="3">Compute Resource Attributes</td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td>percent-cpu</td>
<td>%</td>
<td colspan="3">20</td>
<td colspan="3"></td>
<td colspan="3">Percentage of the CPU’s time given to the reserved
process.</td>
</tr>
<tr class="odd">
<td colspan="4">Network Resource Attributes</td>
<td colspan="3"></td>
<td colspan="3"></td>
<td></td>
</tr>
<tr class="even">
<td>endpoint-a</td>
<td></td>
<td colspan="3"></td>
<td colspan="3">Y</td>
<td colspan="3">The machine at one end of the network flow. This must be
specified as a dotted IP address, such as 140.221.48.162.</td>
</tr>
<tr class="odd">
<td>endpoint-b</td>
<td></td>
<td colspan="3"></td>
<td colspan="3">Y</td>
<td colspan="3">The machine at the other end of the network flow. This
must be specified as a dotted IP address, such as 140.221.48.162.</td>
</tr>
<tr class="even">
<td>bandwidth</td>
<td>Kbps</td>
<td colspan="3">8</td>
<td colspan="3"></td>
<td colspan="3">How fast a flow can transfer data.</td>
</tr>
<tr class="odd">
<td>directionality</td>
<td></td>
<td colspan="3">bidirectional</td>
<td colspan="3"></td>
<td colspan="3"><p>unidirectional-ab: reservation for traffic from a to
b.</p>
<p>unidirectional-ba: reservation for traffic from b to a.</p>
<p>bidirectional: reservation for traffic in both directions.</p></td>
</tr>
<tr class="even">
<td colspan="4">Disk Resource Attributes</td>
<td colspan="3"></td>
<td colspan="3"></td>
<td></td>
</tr>
<tr class="odd">
<td>size</td>
<td>KB</td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3">The storage space needed for a single file.</td>
</tr>
<tr class="even">
<td>bandwidth</td>
<td>Kbps</td>
<td colspan="3">8</td>
<td colspan="3"></td>
<td colspan="3">How fast data can be read/written to a file.</td>
</tr>
</tbody>
</table>

3.3 Creating a Reservation

Before a reservation can be created, its needs have to be specified as
described above. To request a reservation the application has to call
gara_reservation_create ():

int error;

char \*request_rsl = "&(reservation-type=network)

(start-time=953158862) (duration=3600)

(endpoint-a=140.221.48.146)

(endpoint-b=140.221.48.106)

(bandwidth=150)";

char \*reservation_handle;

error = gara_reservation_create( reservation_manager_contact,

request_rsl,

&reservation_handle);

Note that the reservation manager contact is a string obtained from
another location, such as the Grid Information Service.

3.4 Modifying a Reservation

Modifying a reservation is similar to creating a reservation, except
that instead of providing a reservation manager contact, the application
provides the handle to the reservation that was created earlier:

int error;

char \*request_rsl = "&(reservation-type=network)

(start-time=953158862) (duration=7200)

(endpoint-a=140.221.48.146)

(endpoint-b=140.221.48.106)

(bandwidth=200)";

error = gara_reservation_create(reservation_handle, request_rsl,

&reservation_handle);

3.5 Querying a Reservation

To find out the status of a reservation, it can be queried by:

int error;

int status;

error = gara_reservation_status(reservation_handle, &status);

If there is not an error, the status will be one of

GARA_RESERVATION_STATUS_NOT_STARTED

GARA_RESERVATION_STATUS_NOT_STARTED_BOUND

GARA_RESERVATION_STATUS_READY_NOT_BOUND

GARA_RESERVATION_STATUS_ACTIVE

GARA_RESERVATION_STATUS_FINISHED

A reservation is bound if a previous call to gara_reservation_bind
succeeded. A reservation is ready if the current time is later than the
start time, and the duration has not yet elapsed. A reservation is
active if it is both ready and bound. A reservation is finished if the
current time is later than the start time plus the duration.

3.6 Binding a Reservation

When the application ready to use a reservation, it need to *bind* it in
order to begin using the reservation:

int error;

char \*bind_paramters = "&(which-endpoint=a)(endpoint-a-port=1234)

(endpoint-b-port=5678)";

error = gara_reservation_bind(reservation_handle,

&bind_parameters);

Notice that the run-time parameters are specified as an RSL string. This
allows the integration of different reservation managers within one API.
Bind parameters are resource dependant. For compute reservations, for
instance, the only parameter to be specified might be the process-id,
which specifies the process ID of the process that will be receiving the
reservation. For network reservations, there are more parameters:

-   which-endpoint: If the reservation is being bound from a machine
    involved in the reservation, this specifies which machine it is. The
    machine is either "a" or "b", and it matches what was specified in
    the reservation request. If a different machine is binding the
    reservation on behalf of the processes involved, simply use "a".

-   endpoint-a-port: This is the port used by endpoint-a, as specified
    in the reservation request. The implementation assumes that data is
    being sent from endpoint-a to endpoint-b, this will be the port used
    by the sender.

-   endpoint-b-port: This is the port use by endpoint-b, as specified in
    the reservation request. The implementation assumes that data is
    being sent from endpoint-a to endpoint-b, this will be the port used
    by the receiver.

Note that a reservation is not considered active until it is bound. Once
a reservation has both begun and been bound, the reservation manager do
whatever setup is necessary in order to ensure that the reservation is
granted. It is okay if the reservation is bound before it has begun,
because the reservation manager will automatically enable the
reservation once it begins.

If the application will temporarily not be using a reservation but it
will resume using it before it has expired, it can unbind the
reservation:

int error;

error = gara_reservation_unbind(reservation_handle);

Once an application unbind a reservation, it may bind it again.

3.7 Using Callbacks

Callbacks are used to communicate monitoring functions to the user. If
the application would like to be informed whenever the status of a
reservation changes (see *Querying a Reservation* above), it can use a
callback function. Once the user registers a callback function, it will
immediately be called once, to provide the current status, and will be
called every time the status changes afterwards.

First the application needs to create a callback function:

static void callback_handler(

char \*reservation_handle,

gara_reservation_event_t event,

void \*user_parameter)

{

/\* Place code here to examine the event \*/

/\* If it is a status event, event.event_type will be

GARA_STATUS_EVENT, and the status will be in

event.event. \*/

if (event.event == GARA_STATUS_EVENT)

{

if (event.event_type == GARA_RESERVATION_STATUS_FINISHED)

{

/\* React to reservation being finished \*/

}

}

return;

}

Then the application needs to register this function. The user needs to
register the function for each reservation that he wishes to monitor:

int error;

error = gara_reservation_callback_register(reservation_handle,

callback_handler, NULL);

Note that the last parameter you pass to the registration function will
be forwarded as the user_parameter to your callback function.

If you would no longer like to have a function called when the status
changes, you can unregister it:

int error;

error = gara_reservation_callback_remove(reservation_handle,

callback_handler);

Note you can register multiple callback functions for a single
reservation handle.

3.8 Canceling a Reservation

When the application has finished using a reservation, it should cancel
it, using the reservation handle that was obtained when the reservation
was created.

gara_reservation_cancel(reservation_handle);

When the user cancels a reservation, all of the callbacks that have been
registered for that reservation will automatically be cancelled.

It is important to note that all reservation manager will clean
reservations automatically, once they are expired. However, the cancel
call MUST not fail in that case.

3.9 Deactivating GARA

When the application has finished using API, it should deactivate it, to
allow it to clean up:

module_deactivate(GARA_CLIENT_MODULE);

4 Grid Advance Reservation API Reference

4.1 Constants

This section describes the constants used by the API.

Errors

GARA_ERROR_NONE

No error has occurred.

GARA_ERROR_UNKNOWN

An error has occurred, but the reservation manager just doesn't know
what it is.

GARA_ERROR_MODULE_NOT_ACTIVE

The user has tried to use the API without activating the module first.

GARA_ERROR_BAD_PARAMETER

A bad parameter, such as a NULL reservation handle, has been passed to
an API function.

GARA_ERROR_ZERO_LENGTH_RSL,

> An RSL string was provided, but it is empty. It may be that this is
> never returned.

GARA_ERROR_BAD_RSL,

> There is an error, probably a syntax error, in the RSL string.

GARA_ERROR_BAD_RESERVATION_HANDLE

The reservation handle that was provided isn't really a reservation
handle.

GARA_ERROR_CONNECTION_FAILED

> The API was unable to connect to the reservation manager.

GARA_ERROR_AUTHORIZATION

> The API was unable to authenticate and authorize the user.

GARA_ERROR_VERSION_MISMATCH

> Protocol error with the reservation manager due to mismatch of
> version.

GARA_ERROR_INVALID_REQUEST

The request can not be handled by the reservation manager.

GARA_ERROR_UNKNOWN_RESERVATION_TYPE

> The reservation type in the RSL reservation request must be one of
> "network", "compute", or "disk", but it wasn't.

GARA_ERROR_PROTOCOL_FAILED

> There was a problem communicating with the reservation manager.

GARA_ERROR_MISSING_RESERVATION_TYPE

> The reservation type in the RSL reservation request wasn't provided.

GARA_ERROR_OUT_OF_MEMORY

> A request for memory failed.

GARA_ERROR_MISSING_ENDPOINT_A

> A network reservation request didn't specify endpoint-a.

GARA_ERROR_MISSING_ENDPOINT_B

> A network reservation request didn't specify endpoint-b.

GARA_ERROR_CANT_MAKE_RESERVATION

> The reservation can't be made. Probably there are other reservations
> already at the same time, and there isn't room for the new
> reservation.

GARA_ERROR_PROBLEM_WITH_LRAM

> The most likely cause of this error is that the reservation manager is
> not running or that communication with it has failed.

GARA_ERROR_HTTP_UNPACK_FAILED

> A serious protocol error happened, probably a programming error on our
> part, not yours.

GARA_ERROR_BAD_RESERVATION_OBJECT

> This error probably means that the user tried to make a network
> reservation for an endpoint that the reservation manager hasn't been
> configured to allow reservations for.

GARA_ERROR_GARA_SERVICE_EXECUTABLE_NOT_FOUND

> The reservation manager is misconfigured.

GARA_ERROR_CANT_CONTACT_RESERVATION_MANAGER

> The reservation manager is unavailable. Check to make sure that it's
> running or whether the correct resource location was specified.

GARA_ERROR_UNKNOWN_GRAM_ERROR

> Some error in the underlying protocol has failed.

GARA_ERROR_MISSING_RESERVATION_SUBTYPE

The request indicates that a subtype has to be specified, but it wasn´t.

Callback and Status Constants

The following events are reported to callbacks:

GARA_STATUS_EVENT

> The status of the reservation has changed. See the lists of status
> constants below.

GARA_CHANGE_EVENT

> The reservation has been preempted, or the reservation quantity (like
> bandwidth) has changed. See the list changes below.

GARA_MONITOR_EVENT

> Informs the user about specific monitor events such as that he is
> exceeding its reservation.

The following statuses can be reported to callbacks on a status event or
in response to a user calling gara_reservation_status.

GARA_RESERVATION_STATUS_NOT_STARTED

> The reservation has not yet begun (the current time is before the
> start time).

GARA_RESERVATION_STATUS_NOT_STARTED_BOUND

> Although the reservation has not yet begun, the reservation has been
> bound.

GARA_RESERVATION_STATUS_READY_NOT_BOUND

The reservation has begun (the current time is after the start time) but
can't yet be used because it has not been bound yet.

GARA_RESERVATION_STATUS_ACTIVE

> The reservation has begun and been bound.

GARA_RESERVATION_STATUS_FINISHED

The reservation is over. That is, the current time is greater than the
start time plus the duration of the reservation.

The following changes can be reported on a CHANGE_EVENT:

GARA_RESERVATION_CHANGE_PREEMPTED

> The reservation has been preempted because a more important
> reservation has occurred. Currently, this will not be reported,
> because preemption has not yet been implemented.

GARA_RESERVATION_CHANGE_QUANTITY

> The quantity (like bandwidth) has been changed. This occurs for bulk
> transfer reservations.

4.2 Data Structures

This is a description of the data structures used by the API.

The Event Data Structure

typedef struct

{

int event_type;

int event;

double quantity;

} gara_reservation_event_t;

This structure is provided to callback functions. The event type and
event are constants from the list above. The quantity is provided when
the event is a change event indicating that the quantity has changed.

Callback functions

typedef void (\*gara_reservation_callback_t)(

char \*reservation_handle,

gara_reservation_event_t event,

void \*user_parameter);

This is the type of function that must be used for callback functions.

4.3 Functions

Note that all of the functions of the API return an integer. This
integer is the error code, if any error occurred. See the list of errors
under *Constants*.

gara_reservation_create

int gara_reservation_create(

const char \*manager_contact,

const char \*reservation_specification,

char \*\*reservation_handle);

This function attempts to make a reservation.

**In:**

> manager_contact: The contact string for access to the reservation
> manager for the resource the user wishes to make a reservation with.
>
> reservation_specification: An RSL string describing the attributes the
> user wishes to have for the reservation. See *Describing a Reservation
> Request* above.

**Out:**

> reservation_handle: If the reservation was successfully made, a
> pointer to your reservation handle will be provided in this parameter.
> The memory for the reservation handle is allocated by malloc(), and it
> is your responsibility to free the memory with free() when the user is
> done.

gara_reservation_modify

int gara_reservation_modify(

const char \*old_reservation_handle,

const char \*reservation_specification,

char \*\*new_reservation_handle);

This function attempts to modify a new reservation. Note that if the
reservation is changed, you are might receive a new reservation handle.

**In:**

> old_reservation_handle: The handle for the reservation that you wish
> to modify.
>
> reservation_specification: An RSL string describing the new attributes
> you wish to have for your reservation. See *Describing a Reservation
> Request* above.

**Out:**

> new_reservation_handle: If the reservation was successfully modified,
> a pointer to your reservation handle will be provided in this
> parameter. The memory for the reservation handle is allocated by
> malloc(), and it is under the responsibility of the user to free the
> memory with free().

gara_reservation_bind

int gara_reservation_bind(

const char \*reservation_handle,

const char \*bind_parameters);

This claims a reservation by providing run-time parameters.

**In:**

> reservation_handle: The handle for the reservation that the user
> wishes to bind.
>
> bind_parameters: An RSL string describing the new attributes the user
> wishes to have for the reservation. See *Binding a Reservation* above.

gara_reservation_unbind

int gara_reservation_unbind(

const char \*reservation_handle);

This "un-claims" a reservation. The reservation is still valid and can
be used again by calling gara_reservation_bind() again.

**In:**

> reservation_handle: The handle for the reservation that the user
> wishes to bind.

gara_reservation_status

int gara_reservation_status(

const char \*reservation_handle,

int \*status;

This function queries for a reservation's status.

**In:**

> reservation_handle: The handle for the reservation that the user
> wishes to query.

**Out:**

> status: The status of the reservation. It is one of the constants
> described in *Callback and Status Constants.*

gara_reservation_callback_register

int gara_reservation_callback_register(

const char \*reservation_handle,

gara_reservation_callback_t callback_function,

void \*user_parameter);

After this function successfully completes, the specified callback
function will be called whenever the status of a reservation changes. It
will also be immediately called once to provide the current status of
the reservation. Note that multiple callbacks can be registered for a
single reservation.

**In:**

> reservation_handle: The handle for the reservation for which the user
> wishes to receive callbacks.
>
> callback_function: The function that will be called by GARA when the
> status of a reservation changes.
>
> user_parameter: The value provided here will be passed to the callback
> function unmodified.

gara_reservation_callback_remove

int gara_reservation_callback_remove(

const char \*reservation_handle,

gara_reservation_callback_t callback_function);

After this function successfully completes, the specified callback
function will no longer be called when the status of the reservation
changes.

**In:**

> reservation_handle: The handle for the reservation for which the user
> wishes to receive callbacks.
>
> callback_function: The function that will be called by GARA when the
> status of a reservation changes.
>
> user_parameter: The value provided here will be passed to the callback
> function unmodified.

gara_reservation_cancel

int gara_reservation_cancel(

const char \*reservation_handle);

This cancels a reservation. When a reservation is cancelled, the
reservation handle (and copies of it) may not be used anymore. For
example, if the user tries to bind the cancelled reservation, it will
fail.

**In:**

> reservation_handle: The handle for the reservation that the user
> wishes to cancel.

gara_reservation_version

int gara_version(void);

This returns the current version number for the reservation manager.

gara_reservation_client_debug

int gara_client_debug(void);

This enables debugging mode. Output will be printed to stderr.

**In:**

> reservation_handle: The handle for the reservation that the user
> wishes to cancel.

gara_client_error_string

const char \*gara_client_error_string(

int error_code);

For any error code returned by the reservation manager, this provides a
printable string that corresponds to the error code.

**In:**

> error_code: The error code for which the user wishes to obtain a
> string representation.

4\. Contact Author Address

Alain Roy

Building 221

Mathematics and CS

9700 South Cass

Argonne, IL 60439

\(630\) 252-5686

roy@mcs.anl.gov