# Presentation: 928341

## The OSG Resource Selection Service (ReSS)

- Gabriele Garzoglio
- Fermilab, Computing Division
- March 13, 2007

## The Resource Selection Project

- The Resource Selector Service implements cluster-level Workload Management on OSG.
- The project started in Sep 2005
- Sponsors
- DZero contribution to the Common Project
- FNAL-CD (30% FTE Gabriele, 50% FTE Tanya)
- Collaboration of the Sponsors with
- OSG (TG-MIG, ITB, VDT / John Weigand)
- CEMon gLite Project (INFN)
- FermiGrid
- Glue Schema Group

## The Resource Selection Service Motivations / Deliverables

- A Resource Selector allows...
- ...expressing requirements on the resources in the job description
- ...the user to refer to *abstract *characteristics of the resources in the job description
- The Resource Selection Project has two major goals
- Enable OSG resource usage by DZero. Jobs are prepared and data is handled by the SAM-Grid.
- Develop and deploy a Resource Selection Service that VOs with requirements on job management similar to DZero can use.

## Resource Selection Example

- universe = globus
- globusscheduler = $$(GlueCEInfoContactString)
- requirements = TARGET.GlueCEAccessControlBaseRule == "VO:DZero"
- executable = /bin/hostname
- arguments = -f
- queue

- MyType = "Machine"
- Name = "antaeus.hpcc.ttu.edu:2119/jobmanager-lsf-dzero.-1194963282"
- Requirements = (CurMatches < 10)
- ReSSVersion = "1.0.6"
- TargetType = "Job"
- GlueSiteName = "TTU-ANTAEUS"
- GlueSiteUniqueID = "antaeus.hpcc.ttu.edu"
- GlueCEName = "dzero"
- GlueCEUniqueID = "antaeus.hpcc.ttu.edu:2119/jobmanager-lsf-dzero"
- GlueCEInfoContactString = "antaeus.hpcc.ttu.edu:2119/jobmanager-lsf"
- GlueCEAccessControlBaseRule = "VO:dzero"
- GlueCEHostingCluster = "antaeus.hpcc.ttu.edu"
- GlueCEInfoApplicationDir = "/mnt/lustre/antaeus/apps
- GlueCEInfoDataDir = "/mnt/hep/osg"
- GlueCEInfoDefaultSE = "sigmorgh.hpcc.ttu.edu"
- GlueCEInfoLRMSType = "lsf"
- GlueCEPolicyMaxCPUTime = 6000
- GlueCEStateStatus = "Production"
- GlueCEStateFreeCPUs = 0
- GlueCEStateRunningJobs = 0
- GlueCEStateTotalJobs = 0
- GlueCEStateWaitingJobs = 0
- GlueClusterName = "antaeus.hpcc.ttu.edu"
- GlueSubClusterWNTmpDir = "/tmp"
- GlueHostApplicationSoftwareRunTimeEnvironment = "MountPoints,VO-cms-CMSSW_1_2_3"
- GlueHostMainMemoryRAMSize = 512
- GlueHostNetworkAdapterInboundIP = FALSE
- GlueHostNetworkAdapterOutboundIP = TRUE
- GlueHostOperatingSystemName = "CentOS"
- GlueHostProcessorClockSpeed = 1000
- GlueSchemaVersionMajor = 1
- ...

**Resource Description**

**Job Description**

***Abstract *****Resource**
**Characteristic**

**Resource**
**Requirements**

## The Resource Selection ServiceArchitecture

- Condor
- Match Maker

- Info
- Gatherer

- Condor
- Scheduler

- Central Services

## ReSS Validation

**Validated that requirements of DZero are met by the ReSS central services**
- _[https://twiki.grid.iu.edu/twiki/bin/view/ResourceSelection/ReSSValidationTest](https://twiki.grid.iu.edu/twiki/bin/view/ResourceSelection/ReSSValidationTest)_
**Investigated the impact on resources (load, mem, ...) of CEMon at OSG CEs**
- _[https://twiki.grid.iu.edu/twiki/bin/view/ResourceSelection/CEMonPerformanceEvaluation](https://twiki.grid.iu.edu/twiki/bin/view/ResourceSelection/CEMonPerformanceEvaluation)_
**US CMS studied the scalability of ReSS central services for US CMS requirements**
- _[https://twiki.grid.iu.edu/twiki/bin/view/ResourceSelection/ReSSEvaluationByUSCMS](https://twiki.grid.iu.edu/twiki/bin/view/ResourceSelection/ReSSEvaluationByUSCMS)_

- Jobs Submitted = 1 job/sec for hour.
- Total Jobs Submitted = 3600
- First Job Matched = 9/8/2006 16:33:00
- Last Job Matched = 9/9/2006 02:05:53
- Resources Satisfying Jobs = 2 (1800 jobs per resource)
- Total Number Of Resources = 426
- Max Jobs Matched Per Negotiation Cycle Per Resource = 10
- Total Jobs Matched In One Negotiation Cycle = 20
- Longest Negotiation Cycle: 2 sec
- Shortest Negotiation Cycle: 0 sec
- Average Negotiation Cycle: 0.772222222222 sec

## Status

- Development is mostly done
- We may still add SE to the resource selection process
- Integration of ReSS with Fermigrid is done
- Assisting Deployment of ReSS on Production OSG
- Worked with ITB since May 06, targeting deployment for Summer 06
- Validation process very slow: OSG 0.6.0 released on Mar 07.
- Using ReSS on SAM-Grid / OSG for DZero data reprocessing for the available sites
- However, the delay in OSG deployment makes operations difficult (keeping “right” amount of idle jobs at sites)
- Working with OSG VOs to facilitate ReSS usage

## Current Deployment

- Click _[here](http://home.fnal.gov/~garzogli/ReSS/ReSS-prd-History.html)__ for live URL_

## Remaining Tasks for the Project

- Assist with OSG deployment (i.e. CEMon at sites)
- Assist OSG VOs (e.g. Engagement) to use ReSS
- Integrate ReSS with GlideIn Factory
- Check with collaborators if they are interested in SE support
- ...one of the last development activities on the table today
- Assist OSG with Truth-In-Advertisement (GIP)
- Move project from devel. to maintenance
- estimated effort reduction: from 0.8 FTE to 0.25 FTE
- Maintain CEMon in VDT reasonably up to date

## Conclusions

- ReSS Project is naturally moving from development to maintenance
- We are still involved in integration and supporting activities
- More info at _[http://](http://osg.ivdgl.org/twiki/bin/view/ResourceSelection/)__[osg.ivdgl.org/twiki/bin/view/ResourceSelection](http://osg.ivdgl.org/twiki/bin/view/ResourceSelection/)__[/](http://osg.ivdgl.org/twiki/bin/view/ResourceSelection/)_