# Presentation: 051033

## CTMS Application Development:Individual Process within aStandards-Based Process Framework(The Unified Process Framework)

*Charlie Mead, M.D., M.Sc.*
*Senior Associate*
*Booz Allen Hamilton** *

***“******What’s right is what’s left when you’ve done everything else wrong.” – ******Robin Williams***

***CTMS FtF***
***Chicago, IL***
***July 10-11, 2006***

## A Perspective to Bear in Mind

**The problems we are dealing with – clinical, scientific, *****and technological***** – are *****complex****** ***
**As such, their associated solutions are rarely simple (Ockham’s Razor not withstanding) *****(“Pluralitas non est ponenda sine neccesitate'‘)***
**Communities faced with complex problems gradually evolve Best Practices which facilitate communication within the community and advancement of the community’s agenda (e.g. clinical care, scientific methods in proteomics and genomics, etc.)**** **
**The software engineering topics in this talk are ‘simply’ Best Practices evolved over the last 15-25 years to address the complexities of building complex software/solving complex software engineering problems.**** **
**Unless you’re interested, you don’t need to know the details:  you only need know that they are *****empirically derived Best Practices***

## Complexity of Communicationand Coordination

**Number of involved parties**
**Diversity of roles, skills, perspectives, and responsibilities**
**Persons vs machines/systems**** **
**Breadth and depth of data/data types involved**** **
**Complexity of processes to be automated**
**Agility of process automation (‘contextual extensibility’)**** **
**Physical and temporal distribution of parties, data, and process start/end points**

## Overview

**caBIG****TM has**** standardized data elements using caDSR.  Why do we need to standardize application development methodology?**
**Why should members of CTMS workspace care?**
**Process Evolution in the world of software engineering:****A Brief History and Motivation**** **
**The Unified Process Framework:  key elements for standardization**
**Multi-disciplinary project teams**
**Structured Risk Management**
**Iterative/Incremental Development**
**Relationship between architecture and application functionality**** **
**Building Blocks of the UPF:  lifecycle, phases, workflows, iterations**** **
**Project Management 101**** **
**Risks (in the context of Project Management)**

## Overview (2)

**Anatomy of a Requirement**
**Functional (Use Case)**
**Quality (Architecture-sensitive)**
**Relationship over UPF Phases**** **
**UPF and BRIDG**** **
**Impact of UPF on RFP/SOW process and Project Teams**** **
**Relationship between UPF teams and SIGs**** **
**A final clarification:  process framework vs process instance**** **
***“******This is not a Field of Dreams”***
**Q/A**

## Why Standardize Methodology?

**Standardization on static elements is a *****necessary *****prerequisite for (computable) semantic interoperability (CSI)**
**It is *****not *****sufficient unless you restrict interoperability to query/response**
**Process integration requires interoperability at ‘deeper’ levels**
**Granularity of exposed interfaces**
**Quality of Service**
**Service-Level Agreements**
**Ability to exchange context/meta-data as well as data**
**(Service-Oriented) Architecture**** **
**CSI demands reliable functionality, performance, *****and *****quality**** **
**What standards *****beyond static data standards *****(e.g. caDSR) must NCICB supply to enable CSI at the ‘business process interaction’ level?**
**Standards for Application Development Methodology is the first next step**** **
**Moving caBIG****TM from pilot to enterprise requires this next step**

## Standardizing ApplicationDevelopment Methodology...

**Does *****not *****mean micro-managing application development**** **
**It *****does *****mean requiring project teams to apply software engineering Best Practices**** **
**It *****does *****mean providing a Best-Practice-driven framework in which each project’s software engineering process (SEP) will be executed**
**Defined, high-level SEP characteristics**
**Defined, high-level team characteristics **
**Defined, high-level, phase-specific milestones**
**Encouragement for project-specific SEP customization**
**One SEP does *****not *****fit all projects**
**NCICB has chosen to standardize application development *****within *****the Unified Process Framework**

## A Brief History of SEP

**The Dark Ages (195x – 1980)**
**Majority of projects were done by small groups (1-5 engineers)**
**‘****Pure’ engineering applications for use by engineers**
**Requirements defined ‘Type 1’ applications**
**Computer does something that humans can’t do (e.g. simultaneous image projection/rotation)**
**Computer does something that humans can do but does it much better and faster (e.g. real-time navigation calculations)**
**Few requirements for interoperability (most applications ran in relative isolation)**
**SEP:  Waterfall**
**Gather Requirements**
**Perform Design based on Requirements**
**Build application based on Design**
**Test application following completed build**
**Deploy application**
**Waterfall works when...**
**Requirements are known and stable**
**Deployment environment is known and *****relatively unchanging over time***

## A Brief History of SEP (2)

**The Even Darker Ages (1985 – 1995)**
**60-80% of all software projects ‘failed’**
**2x over budget**
**2x over schedule**
**Cancelled before completion**
**Never deployed after completion (‘irrelevant to users’)**** **
**A number of organizations (including) the DoD attempted to understand ‘why’ and to define ‘building applications’ as a type of engineering discipline**
**Better tools (languages, machines, etc.)**
**Better process (Waterfall no longer appropriate)**** **
**A set of Best Practices emerged from an empirical experiential base of countless projects, both successful and unsuccessful**

## A Brief History of SEP (3)

**The Age of Enlightenment and Recognition (2000 ********) ...**
**...****building *****contemporary *****software is a *****complex problem***
**...****most software being built is ‘Type 2,’ – “powertools for the mind” (“The Trouble With Computers” ****Thomas K. Landauer****)**
**...*****communication and management *****issues are at the heart of many of the problems experienced by the software engineering community ****(****“The Mythical Man Month” ****Frederick P. Brooks****)**
**...****the cost of building and maintaining software was disproportionately distributed in *****maintenance and evolution *****and *****not *****in initial *****construction *****(20/80)**
**...****application-to-application *****interoperability *****required SEP focused on *****Quality of Service *****scoping *****functionality***
**...*****standards *****could help address many of the problems facing the software engineering community (e.g. other successful engineering disciplines had developed discipline-appropriate standards of practice, etc.)**
**...****standards must take into account huge differences in project type, personnel, complexity, etc.**

## The Unified Process Framework (UPF)

**Key Concepts (i.e. the Unified Process *****Framework)***
**Iterative/Incremental development**
**Waterfall**** replaced by ‘mini-waterfalls’ which more closely resemble human cognitive processes for solving *****complex problems***
**Aggressive Risk Identification and Management**
**“****Do the hard things first”**
**Importance of architecture in Requirements gathering**
**Functional Requirements vs Quality Requirements**
***“******A requirement is a ***_***single, testable, verifiable ***_***system behavior, function, or capability.”***
**Requirement for multi-disciplinary teams meeting and working together *****throughout *****the course of a project (“horizontal rather than vertical Project Management.”)**** **
**The Unified Process is one of several variants of the UPF**
**Rational Unified Process (RUP)**
**Agile/XP Agile**
**Scrum/Spiral**

## Unified Process Building Blocks

**Lifecycle**
**Type 1:  architecture construction/modification**
**Type 2:  functional additions only (no architectural changes needed)**
**Phases**
**Inception (go/no go)**
**Elaboration (prototype bound to architecture baseline)**
**Construction (beta-release)**
**Transition (roll-out)**
**Production/Support (ongoing)**
**Workflows**
**Collections of discipline-specific tasks**
**Additional relevant workflows created as needed (e.g. Stakeholder Management, Governance)**
**Executed by persons in Roles with specific skills to produce well-defined Deliverables**

## Unified Process Building Blocks (2)

**Iterations**
**‘****mini’ Waterfall run *****continuously *****over Life Cyle of project**
**Short duration (2-6 weeks)**
**All Workflows should be represented during Iteration *****planning***
**Workflow contribution (“intensity”) varies over time/Phase**
**Most Workflows are represented in all Iterations *****execution***
**Multiple iterations per phase drive integrated ‘horizontal’ project management**
**Regression Testing**
**Requirements Traceability**
**Integrated tool support is essential**

## Project Flow and

**The Unified Process Framework**

- Project
- Closeout

- Project
- Inception

- Project Execution

**Iterative/Incremental****Development**

- Inception

- Project
- Transition

- Elaboration

- Construction

- Transition

_**Iteration Highlights: **_
**      ****Selected Use cases**
**      ****Quality Requirements (architecture-sensitive)**
**      ****CIM constructed**
**      ****Design Patterns**
**      ****Architecture baselines deployed**

_**Iteration Highlights: **_
**      ****Remaining Use Cases**
**      ****PIM->PSM (using  caCORE SDK)**
**      ****Semantically annotated XMI**
**      ****Test plans executed**

_**Iteration Highlights: **_
**      ****System verification plan (IV&V if needed)**
**      ****Deployed on Stage Tier**
**      ****Beta or Release Candidate testing and feedback**

_**Iteration Highlights:**_** **
**      ****Vision and Scope defined**
**      ****Core architecture identified**
**      ****Code dependencies (risks) identified**

## The NCICB Unified Process Framework

**Governance**
**Project Management**
**Stakeholder Management**** **
**Business Requirements**
**Requirements Analysis**
**Analysis**
**Design**
**Implementation**
**Business Testing**
**Performance Testing**
**Technical Architecture**
**Data Conversion**
**Transition**
**Operations and Support**

**Inception**

**Elaboration**

**Construction**

**Transition**

**Production / Support**

**Initial**

**Iterations**

**Milestones**

**Elab.**
**#1**

**Elab.**
**#2**

**Const.**
**#1**

**Const.**
**#2**

**Const.**
**#3**

**Tran.**
**#1**

**Tran.**
**#2**

**Prod.**
**#1**

**Prod.**
**#2**

**Phases**

**The NCICB Unified Process Framework**
***(Iterative-Incremental, Risk-Focused, Architecture-Centric)***

## Project Management 101

**Resources**** **
**Schedules**** **
**Dependencies**** **
**Risk Management**** **
**Coordination**** **
**Communication and Problem Solving**** **
**Risk Management (Mitigation and Contingency Planning)**

## An Example RUP Project Plan

**An Example RUP Project Plan**

## Risks – The Bane of aProject Manager’s Existence

**Events you don’t *****expect***** to happen**** **
**Events you don’t *****want***** to happen**** **
**Events that *****do***** happen (and often in multiples)**** **
**Events that *****can***** and *****must *****be managed if a project is to produce *****predictable results *****on *****predictable schedules****** ***
**Risk Mitigation vs Risk Contingency plans**
**Resolving a Risk vs Working around a Risk that has occurred**** **
**Keys to Risk Management**
**Continuous monitoring (think of an anesthesiologist in the OR)**
**Project Plan ‘flow-through’**

## Risks and Project Management

_**Structured Risk Management Matrix (SRMM)**_
***Project Team: ***** <<name of team assessing and completing the SRMM>>**
***Team Lead:  *****<<normally the Project Manager>>**
***Project Start Date:***** <<date that the SOW or other contract to begin the current project>>**
***Today’s Date: *****<<date of the collection of the data on the current instance of the SRMM>>**

| Risk | Type | Date
Identified/
# of wks in SRMM | Date of
Mitigation
OR
Occurrence
(indicate which) | Likelihood
(this week vs
last week) | Impact
(this week
 vs
last week) | Consequence | Structured
Mitigation
Plan | Contingency
Plan | Comments |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |

**Successful Risk Management requires integration of *****static *****Risk data with a *****dynamic *****process**

## Risks and Project Management

**Characteristics of Risk Management process**
**Regular, cross-team Risk *****identification***
**Agreement on Risk static elements (see SRMM)**
**Consistent use of *****semi-quantitative *****assessments**
**Likelihood (that Risk will occur)**
**Impact (on the Project if/when it occurs)**
**Defined, stable thresholds for Mitigation/Contingency planning**
**Well-defined criteria for Mitigation/Contingency plan ‘flow-through’**** **
**‘****Flow-through’ criteria (Mitigation or Contingency plan tasks)**
**Plan contains *****only***** tasks that can be...**
**Assigned to a *****single *****resource accountable to the Project Manager**
**Completed (“done/not done”) within *****one week****** ***
**The Risk Management process models the ‘Assess ******** Define ******** Plan ******** Intervene ******** Evaluate ******** Asses....’ cycle of human (clinical) problem solving**

## Anatomy of a Requirement

**A single, verifiable (“testable”) system capability**
**Functional Requirements**
***“******The system shall...” *****(MuSCoW ranking)**
**Often defined via Use Case Specifications**
**Verifiable by virtue of fulfilling the ‘function’ **
**Quality Requirements**
**Impact application architecture**
**Performance**
**Scalability**
**Extensibility**
**Usability**
**Etc. (~20 commonly defined although not all applicable all the time)**
**Harder (but not impossible) to quantitatively define**
***Fit Metric *****provides testable criteria**
**Fit Metric may be globally defined or vary by Use Case**
**Surfacing Quality Requirements *****early *****in the Lifecycle is critical to the success of a project**

## Example Fit Metrics

**Performance**
**“****A screen refresh time of < 2 seconds for all user requests for same-day patient data.”**** **
**Scalability**
**“****Ability to handle1,000,000 single-variable queries with no more than a 10% decrease in performance.”**** **
**Extensibility (same functionality in new context)**
**“****Once the system has been built, it will be able to be deployed in both the CCU and the ER.”**** **
**Usability**
**“****A user will be able to complete the data entry form at least twice as fast as using the current paper form and make no more data entry mistakes than are currently made using the paper form.”**

## Requirements in the UPF

**In the Inception and Elaboration Phases, Functional Requirements shape the Architecture**** **
**In the Construction Phase (i.e. once the Elaboration Phase has been completed and an Architectural Baseline has been established), the Architecture *****scopes *****the Functional Requirements**
**The UPF can help prevent ‘scope creep’ **** **
**Requirements definition (or lack thereof) is the single most important aspect of a successful SEP**
**Waterfall-style requirements definition (“gather all the requirements first”) do *****not *****work in environments with...**
**Diverse user community**
**Heavy emphasis on user interface**
**The Communications Pyramid *****must be traversed *****if CSI projects are to succeed**

## The Communication Pyramid

**The Communication Pyramid**

*** ******Information Sharing/Communication Techniques***

***Problem****** ******Space***

***Solution****** ******Space***

***Implementation-Independent***

***Implementation-Specific***

***Level of Abstraction***

## UPF Projects and BRIDG

**BRIDG has four stakeholder groups interested in/responsible for its content, support, and direction**
**CDISC**
**FDA**
**HL7**
**NCICB**** **
**BRIDG scope (July, 2006):  *****“The domain of protocol-driven research involving human, animal, or devices as subjects plus appropriate associated regulatory documentation.”****** ***
**The Regulated Clinical Research Information Management (RCRIM) TC of HL7 has adopted BRIDG as its Domain Analysis Model**
**All V3 specifications are derived from the DAM**** **
**NCICB has designed BRIDG as the DAM within the UPF for CTMS**

## UPF Projects and BRIDG (2)

**BRIDG is a DAM:  an implementation-independent representation of the dynamic and static semantics of ‘What?’ – it is *****not *****about ‘How?’**** **
**Project Teams will be expected to deliver BRIDG harmonizable artifacts’**** **
**How to use BRIDG?**
**Whenever possible, start from BRIDG elements (static and dynamic)**
**Elaborate/modify/add as project dictates**
**Submit to BRIDG Technical Harmonization Committee for harmonization in composite BRIDG model**** **
**Interaction with BRIDG will flow naturally from building and working within UPF-compliant project plans**** **
**Relationship between Web Services and HL7 V3 messages ******** in process (SOA SIG)...**
**BRIDG consistent with RCRIM V3 semantics**

## Impact of UPF Standardizationon RFP/SOW Process

**The UPF is the NCICB’s standardization on software engineering Best Practices**** **
**RFPs/SOWs will require that organizations that desire funding submit proposals that are UPF-compliant**
**All CTMS application/service development teams will be expected to produce UPF-compliant artifacts (as well as other project-specific artifacts to be outlined in SOW**** **
**Requirement to work under UPF will encourage subject matter experts (SMEs) to partner with software development organizations to form a project team composed of domain experts, NCI research community stakeholders, and software development professionals.**
***“******Cancer centers ***_***involved in***_*** building software for use by cancer centers.”***** **

## Impact of UPF Standardizationon RFP/SOW Process (2)

**UPF *****does *****work in fixed price contract space**** **
**Two possible approaches**
**Two RFPs/SOWs**
**RFP/SOW for Inception/Elaboration ******** prototype + architecture baseline**
**RFP/SOW for Construction/Transition ******** beta release ******** general release**
**Single RFP/SOW with phased/renegotiated funding**
**Detailed funding/schedule for Inception/Elaboration**
**Order-of-magnitude funding/schedule for Construction/Transition with required renegotiation after completion of Elaboration**** **
**The UPF helps eliminate ‘schedule-based project management.’**** **
**The UPF does *****not *****eliminate reliable project schedules.  In fact, in enables them!**

## Relationship Between UPFProject Teams and SIGs

**SIGs primary responsibility is *****vetting *****of deliverables produced by project teams**
**Relevancy**
**Usability**
**Errors of omission**
**Errors of commission**
**Next Steps**** **
**Standardization on the UPF will enable communication between project teams and SIGs to be...**
**More targeted to interest areas**
**More efficient in terms of utilization of individual’s time**
**Importance of SIGs is reflected in the ‘Stakeholder Management’ and ‘Governance’ Workflows is a reflection of the importance of the SIGs in the overall work of project teams**

## Process Framework vsProcess Instances

**Clinical Guideline vs patient-specific Care Plan**** **
**UPF provides *****high-level *****definition of a SEP**
**Multi-disciplinary project teams**
**Iterative/Incremental development cycle**
**Structured Risk Management and project plan ‘flow-through’**
**Focus on architecture and the relationship between Functional Requirements and ‘architecture-sensitive’ requirements (Quality Requirements)**** **
**Each Project Team will develop its own, project-specific project plans that will be compliant with UPF but will contain project specific *****data***** (e.g. Risks), have project-specific iteration lengths, develop a project-specific architecture, etc.**
**Each project team will be responsible for defining the details of its project-specific SEP (*****“One SEP does ***_***not ***_***fit all projects.”)***

## This is not a Field of Dreams

*“**Healthcare is the only business where ****information**** ****sharing ****is the norm rather than the exception.  Those of us who build systems are not used to this framework.  However, if we are to provide the healthcare system with the tools they need, we must embrace this paradigm completely, committing ourselves to defining, designing, and building fundamentally different types of system than those with which we have our predominate historical experience.” **Bob Herbold / Charles Mead*

## A