# Presentation: 085523

## Grid Packaging Toolkit Tutorial

- Charles Bacon
- GRIDS Center

## Overview

- Goals of Grid Packaging Toolkit (GPT)
- Package types
- Creating a source package
- gpt_verify
- globus_makefile_header
- Metadata overview

## Grid Packaging Toolkit goals

- Easy installation of Globus
- Easy upgrades of packages
- Includes versioning
- Support complicated build processes
- Patches included software like openssl and openldap during package creation
- Ability for users/developers to create their own packages

## Life before GPT

- Monolithic globus-install
- No chance to customize components
- No easy upgrade of components
- Had to release new version of entire toolkit
- Users required to install other software
- SSLeay, openldap
- Not easy to integrate other software into a single distribution

## What is the GPT?

- A set of PERL libraries/modules
- Some simple client utilities
- Package install/uninstall
- Dependency checking
- Package creation/editing
- Can install independently of Globus
- GPT_LOCATION

## GPT Requirements

- Requires Perl 5.005 or higher
- Written as a set of PERL modules
- Can install independently of Globus
- Will install two other PERL modules
- Archive::Tar
- Compress:Zlib
- Tried to keep outside requirements to a minimum

## Flavors

- Flavors encapsulate compile-time choices
- Choice of compiler: gcc or vendorcc
- 32 or 64bit memory model
- Include debugging information?
- Type of threading – pthreads, other?
- Common flavors:
- gcc32dbg, gcc32dbgpthr
- List of all flavors:
- $GPT_LOCATION/bin/globus-flavor-configuration

## Why not RPM?

- Uses Berkely DB
- Not portable to all platforms, e.g., Unicos
- Source package metadata was lacking
- Doesn’t support multiple flavors of packages
- Difficult to combine threaded and non-threaded versions, or 32 and 64 bit versions

## Package Delivery Methods

- Source and Binary packages
- globus_gram_client_tools-2.1.tar.gz
- globus_gsi_wuftpd-0.6-1-i686-pc-linux-gnu-gcc32dbg-pgm_static
- Packages collect into Bundles
- Allows customized installations
- Able to create site-specific bundles
- globus-bundle currently only has a static set of bundle definitions

## Currently Published Bundles

- API Bundle
- Developers can stop here
- Grid Services I (non-threaded)
- Gatekeeper, Job Manager, wu-ftpd
- Grid Services II (threaded)
- MDS
- Tools Bundle
- Client tools: grid-proxy-init, globusrun, grid-info-search, . . .

## Bundle dependencies

- API Bundle
- Required for all other bundles
- Grid Services bundles do **not** depend on each other!
- Leads to some commonly included sub-packages
- GPT will not try to install a package twice for the same flavor
- Will include two or more flavored versions of libraries, but not binaries
- Bundle dependencies based on underlying package dependencies
- Not explicitly a property of the bundle itself
- filelist of packages in bundle is in dependency order

## Setup Packages

- Used to provide configuration files for other packages
- Include metadata, so that setup packages can depend on other setup packages
- Setup packages in a bundle will generate a globus-postinstall.sh in $GLOBUS_LOCATION/setup/
- This will be replaced with an automated setup dependency tool
- Allows re-installation of a package without overwriting customizations

## GPT files in $GLOBUS_LOCATION/etc/

- $GLOBUS_LOCATION/etc
- globus_packages/
- One directory per installed package
- Contains filelists and metadata
- gpt/
- Contains flavor definitions in XML
- globus_package_management/
- globus_build.conf – architecture type (e.g., i686-pc-linux-gnu)

## Uninstalling Packages

- $GPT_LOCATION/sbin/globus-uninstall
- Takes a package name like globus_gram_client_tools-gcc32dbg_dev
- Can see a list of possible package names with gpt_query
- Currently not possible to uninstall a bundle at a time

## Package Creation

- Write some code which depends on Globus
- Install GPT
- Perl 5.005+ required
- Run gpt_setup
- Edit Makefile.am
- Run gpt_edit
- Bootstrap and configure
- Make; make install

## Globus dependent code

- #include “globus_common.h”
- int main (void){
- return globus_libc_printf(“Hello, world!\n”);
- }
- Fancy way to print “Hello, world!”

## Using gpt_setup

- Current contents of directory:
- hello.c
- Now, run gpt_setup.  Creates:
- doxygen/
- pkgdata/
- Makefile.am
- bootstrap

## Using gpt_setup (cont.)

- You will need to edit Makefile.am with your desired targets
- bin_PROGRAMS = hello
- hello_SOURCES = $(Sources)
- hello_LDFLAGS = $(GPT_LDFLAGS)
- Possible targets:
- lib
- bin
- libexec
- etc

## gpt_edit

- Dependency created by gpt_setup
- Only globus_core – the root dependency
- We need to add a dependency to globus_common
- Choose “add dependency”
- Because my compile requires headers
- Introduces a compile dependency
- Can specify a version dependency if required

## Dependency types

- Compile dependency on package X
- X must be installed for me to compile
- Link dependency on package X
- For shared libraries, X must be present at runtime
- Runtime dependency
- May depend on components (executables, etc.) of another package to be present at runtime.

## Check that your package compiles

- 	-install-only
- 	-srcdir=.
- 	-installdir=$GLOBUS_LOCATION
- 	<flavor-type>

## Creating binary packages

- Will create a binary package for the current flavor and architecture
- Different package types
- pgm = Dynamically-linked binary
- pgm_static = Statically-linked binary
- rtl = Run-time library
- data = Data files
- dev = Headers, static libs, libtool library files
- doc = Documentation

## What files go to what packages?

- Override with pkgdata/MyFilelists.pm
- Uses functions from perl/lib/GPT/Filelist.pm
- Defaults
- Binaries to bin/, libraries to lib/, etc. according to regular expressions contained in Filelist.pm

## gpt_verify

- Checks both runtime dependencies and compile-time dependencies
- Runtime reported as “complete installation”
- Compile-time reported as “coherent development tree”

## gpt_verify (cont.)

- require Grid::GPT::Version;
- require Grid::GPT::Dependencies;
- foreach my $pkgdata (@packagemetadatalist){
- 	$pkg->read_metadata_file($pkgdata);
- 	my $flavor=$pkg->Flavor;
- 	my $linktype=undef();
- 	@missingbuild=(@missingbuild, $DepMod->get_dependencies($pkg, "build",$flavor,$linktype,"silent"));
- 	@missinginst=(@missinginst, $DepMod->get_dependencies($pkg, "check",$flavor,$linktype,"silent"));
- }
- 	my $rehashref=$DepMod->check_setup_dependencies;

## globus_makefile_header

- For programs that you don’t want to repackage
- Use globus_makefile_header to find out the information about a local Globus installation
- Run this at compile time
- If you run it on your computer ahead of time, it will only be valid for your installation
- Prepend it to your Makefile

## Source Package Metadata

- Aging version of the source package.
- Types of binary packages produced.
- Flavored?
- Setup requirements
- Compile Dependencies
- Linking Dependencies
- Build Env. (CFLAGS, LIBS ... )
- Runtime Dependencies

## pgm Metadata

- Dynamically linked binaries
- Aging version of the package.
- Package type
- Flavor the package was built with (or noflavor)
- Configuration requirement
- Runtime dependencies with other pgm, data, and doc packages
- Includes verisions
- Runtime linking dependencies to rtl packages if the executables were linked with shared libraries.
- Includes versions

## Aging Version

- Based on libtool version numbering
- Major.minor.aging
- The aging number specifies how backwards compatible the major number is
- E.g., version 5.4.3 is backwards compatible with 2.x.x and up, because
- 	5-3=2

## pgm_static Metadata

- Statically linked binaries
- Aging version of the package.
- Package type
- Flavor the package was built with (or noflavor)
- Configuration requirement
- Package regeneration dependencies to dev packages
- Same as the runtime linking dependencies list of the pgm packages except for the package type.
- Runtime dependencies on pgm, data, and doc packages.
- Also contains the runtime dependencies of the static libraries for the entire regeneration dependency tree.

## dev Metadata

- Headers and static libraries
- Aging Version of the package.
- Package type
- Flavor the package was built with.
- Configuration specification
-  if the package requires run-time configuration files.
- Compile dependencies if files from the package include headers from other packages.
- Linking dependencies
- Build Environment
- Runtime dependencies

## rtl Metadata

- Run-time libraries
- Aging version of the package.
- Package type
- Flavor the package was built with.
- Configuration specification package if the package requires run-time configuration files.
- Linking dependencies
- Runtime dependencies.

## data Metadata

- Non-editable data files
- Aging version of the package.
- Package type
- Flavor the package was built with.
- Runtime dependencies on any other data packages

## doc Metadata

- Non-editable data files
- Aging version of the package.
- Package type
- Flavor the package was built with.
- Runtime dependencies on any other doc packages