VINCI SAS/Grid

Overvie

w

Mark EzzoVIN

CI S

A

S A

dminist

rator

Decem

VINCI SAS Administrator
VINCI SAS/Grid Agenda

• SAS Grid Introduction

– Overview– User Interfaces User Interfaces– Grid Enabling Existing SAS

• Examples

– Base SAS– Enterprise Miner– Batch Submit Batch Submit

• Summary
VA Informatics and Com A Informatics and Comp g utin Infras

Infras

(VINCI) Foundations

• VINCI Research Infrastructure VINCI Research Infrastructure

-- A Leading – Edge Research Venue utilizing SAS and SAS/

• New Data Privacy & Security Capabilities

– New and improved methods to protect the privacy and secu

used in the research environment.

• Remote Analy py sis Capability

– New methods for researchers to perform research remotely 

removing data from the VINCI system.

• Prepare for Future Data Capabilities Prepare for Future Data Capabilities

– Develop and implement methods to extract and make usea

unstructured data such as text, genomic data, and images
SAS/Grid v1.0: Pre Grid

APP04 has been upgraded to RDS01.PCS07 remains the same

30 November 2011
SAS/Grid Windows: Archi

30 November 2011
SAS/Grid Value

1. Approved Resource Control

– Resource Allocation, High Availability, Fail– User Prioritization, Queuing, Monitoring– Job Management, Scheduling, Suspension

2. Increases Performance 

– Workload Balance Users Across Available 

– Parse Jobs to Simultaneously Process on 

– Imp g gj rove lon

g runnin

g jobs as well as Ad-H

3. Batch Processing

– User can submit and forget (no need to rem– U i j b t t hil j b i i User can view jo

b ou

tpu

t while jo

b is runni

– Allows for SAS checkpoint/restart capabilit– Uses SAS Grid Manager metadata for cen
Grids are built, not boug
SAS/Grid Road Map

• SAS/Grid v1 0. Now: (10) Node Windows 20

: (10) Node Windows 20

Cluster

• SAS/Grid v2.0 SAS/Grid v2.0 Plus (6) Node RHEL Linux C

Plus (6) Node RHEL Linux C

modified clustered file systarchitecture

Run Windows and Linux in parallel for a perio

then decide on one platform going forward

30 November 2011
SAS User Interfaces

Batch Submit

Base SAS SAS

® Enterprise Miner 

®

SAS

® Enterprise Gu

30 November 2011
New Additions

• The SAS Add-In for Microsoft Office is a Component Ob

(COM) add (COM) add-in that extends Microsoft Office by enabling 

in that extends Microsoft Office by enabling 

the power of SAS analytics and access data directly froMicrosoft Excel, Word, PowerPoint, and Outlook.

• SAS 9.3 Stored Processes introduces stored process re

th STP d A t d t i bthe STP procedure. A stored process report is a new ob

that contains stored process output that is cached. The can be viewed without re-executing the stored process

STP enables users to execute a stored process from a STP enables users to execute a stored process from a 

program. PROC STP can be executed in an interactive, server SAS session and can even be executed by anoth

process. Essentially anyone with a Web Viewer can ex

Essentially, anyone with a Web Viewer can ex

view the results, without using SAS itself.

30 November 2011
SAS Administration

RTM (Report , Track, Monitor)

S h d li S

c

h

e

d

uling

SAS Grid Manage

r

Single Point of Control! Business Metadata

Shared Metadata Shared Data

S S

®

C

30 November 2011

SAS

Management 

Console
SAS Enterprise Miner 

(User Interface for SAS Data Mining and ModParallelized Workload Balancing
VINCI SAS Grid Example

30 November 2011
SAS/Grid Next Steps

• Arrange training sessions• Establish 

p p y gp artici

pator

y user 

grou

ps 

• Assist in optimization of project 

techniques

30 November 2011
VINCI SAS/Grid Summary

• Load Balancing & Parallel Processing

– Support for large SAS Community

• Reuse of existing SAS Programs

– Small learning curve for immediate value or subm

code immediately to the grid in SAS 9.3 via the GS

Server

• High Availability/Automatic Failover

– Less downtime = more research

• Eas

y Administration

– Quickly provide help and support to users

30 November 2011
Grid Enabling

• SAS Display Manager and the SAS 

Parallel Execution ( p) Exam les)

30 November 2011
Submit to Grid from SAS Display Mana

(The “Five Lines of Code”)

• options metaserver='dnnnn'; options 

meta

port=8561; %let 

rc=%sysfunc(grdsvc_enable(grid, server=SASApp)); g g ; si

gnon 

grid; rsubmi

wait=no persist=no; 

•

/

* insert SAS program here insert SAS program here

*

/

30 November 2011
Key Definition for Grid Submissio

• Save following to external file (c:\gpre.sas for ex.) 

options noconnectpersist; 

options noconnect it wa ; 

options metaserver='dnnnn'; 

options metaport

=8561;

%let rc=%sysfunc(grdsvc_enable(grid, server=Ssignon grid; 

rsubmit; 

• Create new DMS key definition (F12 for ex ) new DMS key definition (F12 for ex.)

gsubmit “%include „c:\gpre.sas

‟;”; rsubmit; 

30 November 2011
Syntax to Enable Grid 

Grid function calls 

grdsvc_enablegrdsvc_nnodesgrdsvc_getnameg _g rdsvc_getadd

r

grdsvc_getinfo

SAS Grid Computing for 9.2 doc for comsyntax (available on the SAS Sharepoint (available on the SAS Sharepoint sitehttp://vaww.vinci.med.va.gov/vincicentral/projectsites/SASGrid/default.

30 November 2011
Tell SAS to Use the Grid

options options metaserver metaserver

=

„metadata sas 

metadata.sas

options metaport=8561; 

%l t e

t rc = 

%sys

func

(grd bl svc_enabl

e 

server=“explicit project server”)); 

30 November 2011
SAS Grid Parallel Executi

Parallelized Workload Balancing 

%let rc=%sysfunc(grdsvc_enable(_all_, resource=SASMsi

gnon task1;

rsubmit task1 wait=no; 

/* code to be remote submitted */ 

endrsubmit;

signon task2; rsubmit task2 wait=no; 

/* code to be remote submitted */ 

endrsubmit; 

waitfor

_all task1 task2; all_ task1 task2;

/* continue local execution */ 

30 November 2011
Grid Enabling Existing SAS Prog

Ori

ginal Serial Pro

gram 

Grid Enabling Existing SAS ProgOri

g g inal Serial Pro

gram 

libname a "/u/users/sales"; data a.one; 

do x = 1 t 1000000 1

to 1000000; 

output; end; run; 

libname b "/u/users/sales"; data b.two; 

do y = 1 to 100000; output; end; run; 

30 November 2011
Grid Enabling Existing SAS Prog

Load Balance Parallel Work Units 

options metaserver='xxx.yyy.zzz.com‘; 

o

ptions meta

port=8561;

%let rc=%sysfunc(grdsvc_enable(_all_, resource=SASMain)); 

signon host1; rsubmit wait=no; 

libname a "/u/users/sales"; data a.one; 

do x = 1 to 1000000; 

output; end; run; ; end; run;

endrsubmit; si

gnon host2; 

rsubmit wait=no; 

libname b "/u/users/sales"; data b.two

do 

y = 1 to 1000000; 

output; end; run; 

endrsubmit;

signoff all ;signoff _all

_;

30 November 2011
The Problem

• Lots of existing SAS programs of existing SAS programs

– long and complex – Written a long time ago– O i i l th l il bl Original author no longer available

• Advances in computing architectures and SAS prod

– mu tl i-co e a d d st buted syste s re and distributed systems– SAS metadata, SAS Enterprise Miner/Guide

• Existing programs could be more efficient 

– manual improvements time consuming and costly 

30 November 2011
The Solution

• SAS code analyzer code analyzer

• SAS procedure new in 9.2 • executes an existing pg SASro ram • analyzes job steps, input/output data and depende• records information used to enhance efficiency an

manageabilit f th bility of the program 

30 November 2011
A Sample Invocation

30 November 2011
Sample Output

30 November 2011
Benefits

• Parallel execution results in accelerate

times

• Easy to use tool 

– eliminates costly manual analysis of prog

costly manual analysis of prog

– inserts necessary syntax to eliminate 

programming errors programming errors

– accelerates learning curve for creating pa

programs

30 November 2011
SAS Command-Line Grid Submission 

• Standalone utility that will allow user to Standalone utility that will allow user to

– submit SAS program to grid for processin– display display status of status of user

‟

s jobs on the grid jobs on the grid

– retrieve output from user

‟s jobs to local d

– kill jobs

30 November 2011
Advantages

• User can submit and forget can submit and forget (Batch Jobs) (Batch Jobs)

– no need to remain connected to process job 

• User can view job output while job is running can view job output while job is running

• Allows for SAS checkpoint/restart capability • Uses SAS Grid Manager metadata for centra

SAS Grid Manager metadata for centra

control 

• NOTE - requires shared file system between 

and grid 

30 November 2011
Submitting a Job

• sasgsub –gridsubmitpgm <sas_pgm> 

– other parameters stored in configuration file parameters stored in configuration file

• -GRIDWORK <shared_file_dir>• <metadata_connection_parameters> 

• -GRIDAPPSERVER <app_server_name> • [-GRIDLICENSEFILE <license_file_pathname>] • [-GRIDFILESIN < GRIDFILESIN

<file list file

_

list>]

• [-GRIDJOBNAME <job_name>] • [-GRIDJOBOPTS <job_options>] 

• [-G SO RIDRE

START

OK] 

• [-GRIDSASOPTS <sas_options_for_job>]• [-GRIDWORKLOAD <workload

_

values>]

• [-GRIDWORKREM <remote_shared_file_dir>] 

30 November 2011
Example Output

Job ID: 6772 Job directory: "/CNT/sasgsub/gridwork/sascnn1/SASGSUB-2009-03-17 14 09 52 847 t tP " 17

_

14.09.52.847

_

testPg

m"

Job log file: "/CNT/sasgsub/gridwork/sascnn1/SASGSUB-2009-03-17_14.09.52.847_testPgm/testPgm.log“

30 November 2011
Currently Installing Linux 

30 November 2011
Thank you for attending.

Please contact Mark Ezzo VINCI SAS Please contact Mark Ezzo VINCI SAS

Administrator: 

Mark.Ezzo@va.govwith any questions or comments.

30 November 2011
Enterprise Miner Interface

30 November 2011
Enterprise Miner Sample R

30 November 2011