# Presentation: 476022

## Reconstruction PackageDesign Guide

***Designing data structures and algorithms***Jim KowalkowskiMarc Paterno

## Goals

- Summarize the rules for the production of “legal” reconstruction packages
- Present some suggestions for production of well-crafted reconstruction packages

**Notes:**
Go quickly over the requirements I\(available from other documents)
Stress good practice

Many people are coding by copying, and may be copying bad practice
— point out more egregious cases?

Alert people to the fact that they need more information on various topics; we aren’t in the position to present that information during the talk.

People can refer to the handout after the talk; perhaps present examples in the talk.

Aim for the intermediate designer, not the complete novice.

## The Big Picture

- An overview

## The Model

- Deleterious effects of physical coupling
- The standard solution: separation of data and algorithms
- data in one library (“chunks”)
- algorithms in another library (“packages”)
- chunk library must not depend on package library

## Algorithm Inputs and Outputs

- First design task:
- decide what inputs are needed
- decide what is produced
- Second design task: Is this algorithm...
- Part of a package?
- One package?
- Several packages?

## Algorithms and the Framework

- Inheritance from *Package*, implementation of one or more interface(s)
- What interfaces are available, and which should I use?
- Building an executable
- parts needed (objects and libraries)
- GNUmakefile (how to tell SRT what is needed)? What is the party line? How does this work in CTBUILD?

## Classes and Instances

- Instance
- a unit that combines a specific state (data) and the functions used to manipulate it (methods)
- Class
- a type that defines related instances
- a description of what the instances have in common (types of data, method definitions)
- the body of code that manipulates the data in the instances
- A program can have multiple instances of the same class, each with different values

## Parameterized Classes

- Class template
- A description for how to write a class
- Describes a family of classes that share common characteristics
- Instantiating a class template causes the compiler to write a class; one can then make instances of the class
- std::vector			— class template
- std::vector<float>		— instantiated class
- std::vector<float> vf	— class instance

## The Framework Interfaces

*More commonly used*
- Process
- Filter
- Analyze
- Tag
- JobSummary
- RunInit
- RunEnd

*Less commonly used*
- Generate
- Decide
- Dump
- Output
- Build

## Parts Needed

- Main program object
*framework.o*
- Framework related libraries
*libframework.a*
*libio_packages.a*
*libevpack.a*
*libd0om.a*
- ... Analyze package has a full list (it might have more than required)

- Package libraries
*libemreco.a*
*libcpsreco.a*
- one required for each package you use
- Package registration objects
*RegEMReco.o*
*RegCPSReco.o*
- one required for each package you use

## Package Configuration

- Parameters are introduced by RCP objects
- flexibility, automatic bookkeeping
- Framework described by RCP object
- can have multiple instances of the same class with different parameters, in the same program

## Contents of Framework RCP

- string InterfaceName = “process” // use this for reco development
- // Put here all the interfaces which you might want to implement
- string Interfaces = “generator decide filter process analyze dump                                tag output jobSummary runEnd runInit”
- // Put here the interfaces in the order  you want the Event to travel
- string Flow = “generator decide filter process analyze dump tag output”
- // Package instance names, in the order in which you want them called
- string Packages = “read_event prereq conejet_5 conejet_7”
- // Define what these package instances mean
- RCP read_event = <ReadEvent.rcp>
- RCP prereq = <SomeCalorimeterStuff.rcp> // put the real prereqs here
- RCP conejet_5 = <ConeJet5.rcp>RCP conejet_7 = <Conejet7.rcp>

## Chunk Design

- Basic chunk design
- Good chunk design

## Basic Chunk Design

- What must I do to make my chunk work?

## Persistence

- Requirements due to D0OM
- allowed types
- basic types
- classes which inherit from d0_Object
- STL containers
- no pointers
- no template classes unless a D0OM adapter exists (*e.g.*, STL containers)
- preprocessing with d0cint required, to write *x_lnk.cpp* and *x_ref.hpp* files

## Accounting Information

- Record information specific to the creation of your chunk
- values of algorithm parameters through RCPID
- commonly used parameters can be stored redundantly, *e.g.* KT algorithm ycut threshold
- Record parents
- Can always keep more than the minimum
- This information will be used by others during chunk lookup

## EDM Rules

- No direct pointers to another chunk, or things within another chunk
- instead, use *Link<C,O,I>* (under review)
- Chunk has exclusive ownership of all its data
- Provide *const* access; users get access only to *const* chunks
- Avoid *mutable* data members
- EDM tutorial online
- http://d0server1.fnal.gov/www/edm/tutorial/welcome.htm

## Good Chunk Design

- What should I do to make my chunk work *well*?

## Chunks as Collections

- Most chunks are collections. They should be:
- Coherent
- Contain and present an STL container
- consider *std::vector* first
- Present standard *typedefs*, and use them
*value_type*, *index_type*, *container_type*
- Present standard functions:
*contents( )*, *at(const index_type& i )*
- Provide predicates in the chunk library, appropriate for your users.

## Predicates

- A class (or struct) which encapsulates a unary test or binary comparison

**struct less_pt : ****public binary_function<Track, Track, bool>**** {**
**  ****bool operator()(const Track& a, const Track& b) const**
**    ****{return (a.pt( ) < b.pt( )); }**
**};**
**class high_pt : ****public unary_function<Track, bool>**** {**
**public:**
**  ****high_pt(float t) : _threshold(t) { }**
**  ****bool operator( )(const Track& a) const**
**    ****{ return a.pt( ) > _threshold; }**
**private:**
**  ****float _threshold**
**};**

## Designing for Selection

- Merely meeting requirements is insufficient; useful chunks provide more
- Chunk designers should write the selectors which users need to locate the chunk instances they want
*e.g.*  *JetCollection*: algo = “KT”, ycut = *x*
- Generally will consider “accounting” info, not “event” data
- Record name and version of algorithm library

## Do I Need a New Chunk Class?

- Use instances of the same chunk, when sensible
- different instances of *JetCollection*, made by algorithms with different parameters
- Maybe make a different chunk class when the contained data is really different
- Consider a new chunk class if your design contains multiple collections of different types

## Using Activate and Deactivate

- Transient data format can be more efficient and easier to use
- provide more convenient form of data for users
- save storage by making only non-redundant data persistent
- Only persistent data needs to meet the D0OM restrictions
- How *activate( )* works
- called after D0OM fills persistent data members
- Note there is no Event pointer available

## Avoiding if/else and case/switch

- This is generally bad design
- object has an internal state which has to be queried from outside
- this state has to be checked in many places in the code — very difficult to keep all correct
- Instead, use polymorphism
- maybe contain a base class
- maybe design different chunk class to hold each different flavor of contents

## Package Design

- Basic package design
- Good package design

## Basic Package Design

- What must I do to make my package work?

## Package Initialization

- Constructor takes a *Context**
- merely pass it to base class c’tors
- Get at parameters via method *Package::packageRCP( )*
- All parameters for initialization should come from RCP
- keep record of what was done
- Write *setParameters(RCP r)*
- call from c’tor, passing *packageRCP( )*
- can be called by Framework, to reinitialize

## Keys, Handles, and Selectors

- Use a selector (provided by the chunk designer) to choose what to extract
- Chunk class determined by *TKey<X>*
- Chunk instance determined by selector
*THandle<X>* returned; use as a pointer
**THandle<XChunk> hXch = ****	****	****            key.find(e);**
**if (hXch.isValid()) **
**	****hXch->doXChunkFunction();**

## Putting Chunks into the Event

*Event* takes ownership of the chunk
- Uses *auto_ptr<T>* to indicate this
- do not delete the chunk
- Chunk owns its contents
- do not delete the contents

## Using the Framework

- Implement the *processEvent() *interface
- From *Package*, implement
*statusReport( )*
- Report summary of state (statistics) to *cout*
*reinitialize(RCP r)*
- this should call *setParameters(r),* probably should clear histograms, etc.
*flush( )*
- Flush any buffers your package contains (histograms, log streams, etc.)

## Rules

- No caching event data between events
- No global variables
- No removing pointer from handle
- use the handle as a pointer
- Do not include headers from another package’s private directory
- Do reconstruction in the *processEvent( )* interface
- Do not cast away *const*-ness

## Good Package Design

- What should I do to make my package work *well*?

## Error Handling

- Catch exceptions thrown by your own algorithms
- Framework catches all exceptions that exit your package; logs the message and program dies
- Use return codes
- Framework group needs to decide what the return codes mean
- Currently any package failure aborts further processing of that event (from that group)

## Views: proposal

- Collection of pointers to items in a chunk; template class
- used by algorithms in preference to chunks
- decouples algorithms from chunk details
- Filled using standard chunk method *contents( )*; View remains modifiable
- can use predicates for selection
- can be sorted upon or after filling
- Usable with STL algorithms
- EDM will provide a default View template class

## template <class SRC, class DEST,

**          ****class ****PRED****>****void fillView(const ****SRC****& s, ****DEST****& d,**
**              ****const ****PRED****& p);**
**template <class ****ELEMENT****>**
**class View : public****   std::vector<const ****ELEMENT*****> > {...}**
**template <class ****ELEMENT****>****class AlwaysTrue**
**{**
**  ****bool operator()(const ****ELEMENT****&) const****    { return true; }**
**}**

## Example Use of View

**  ****TKey<XChunk> key;**
**  ****THandle<XChunk> hX = key.find(e);**
**  ****if (hX->isValid() ==false)**
**    ****return Result::failure;**
**  ****// Create an empty View**
**  ****View<X> xview;**
**  ****// Fill the View, using a predicate**
**  ****fillView(hX->contents(), xview, ****BigX(5)****);**
**  ****// Sort in place, using another predicate**
**  ****sort(xview.begin(),xview.end(),****SortAscX()****);**
**  ****... use xview like an STL container ...**
**}**

*Example Use of View*

## Example Chunk Predicates

**public:**
**  ****bool operator()(const X& a, const X& b) const**
**    ****{ return a.value()>b.value(); }**
**};**
**class BigX {**
**public:**
**  ****BigX(int t) : _thresh(t) { }**
**  ****bool operator()(const X& a) const**
**    ****{ return a.value() >= _thresh; }**
**private:**
**  ****int _thresh;**
**};**

*Example Chunk Predicates*

## Reconstructor Package Model

- Reconstructor = *Package* that implements *processEvent()* to do reconstruction
- Reconstructor gets chunks from *Event*, gets access to data (STL-like interface for chunk makes this easy); maybe makes a View
- Algorithm(s) use data, create new data — use a *View*
- Reconstructor wraps new data in a new chunk, inserts it into the *Event*

## Using the Strategy Pattern (1)

- Strategy pattern used to provide flexibility at run-time: vary algorithms, not just numerical parameters
- Example: KT jet reconstruction
- Selection algorithm: selection which objects to combine (or decides that the algorithm terminates)
- Combination algorithm: combines two objects

## Slide 40

## Using the Strategy Pattern (2)

- Many possible selection algorithms; all have a common interface  base class
- Many possible combination algorithms; all have a common interface  another base class
- Reconstructor contains pointers to base class, filled during construction as directed by an RCP object, and calls their methods in *processEvent( )*.

## Do I Need a New Package Class?

- If you are using the same inputs and making the same outputs, probably not; consider *Strategy.* Or just use another instance, with different parameters (fed in by RCP).
- If you are using different inputs, but making the same output, probably yes. If *Strategy* makes the task clearer, use it.
- If you are making different outputs, almost certainly yes.

## Effective use of STL

- Use iterators to indicate parts of collections, rather than partial copying
- Use the right collection
- consider *vector* first; *list* for mid-insertion; *set* for automatic ordering*; map* for arbitrary association; *deque* as a *vector/list* compromise
- Use available chunk predicates to perform filtering and sorting
- Learn the available algorithms
- http://www.dinkumware.com/htm_cpl/index.html

## STL Example

- #include <deque> // for deque
- #include <algorithm> // for sort, for_each
- #include <functional> // for not2, mem_fun
- #include <iterator> // for back_inserter
- using namespace std;
- void printInterestingStuff(View<Track>& tracks) {
-   // Select tracks with transverse momentum > 5.
-   deque<Track*> good;
-   copy_if(tracks.begin(), tracks.end(), back_inserter(good), high_pt(5.));
-   // Sort on pt, in decreasing order
-   sort(good.begin(), good.end(), not2(less_pt()));
-   // Print summary
-   for_each(good.begin(), good.end(), mem_fun(Track::print));
- }

## MAINTAINING A SYSTEM

- Techniques to help your code be maintainable

## Best Practice

- Do not put *using namespace* in any header
- Use forward declarations wherever possible
- Prefer *++it* to *it++*
- Do not expose details unnecessarily
*e.g*.: Do hide bit encodings from users
- Use meaningful member names, rather than slot numbers in an array

## Writing Class Tests

- Write small tests that (as fully as possible) exercise your class
- use minimal extraneous stuff
- Great aid to regression testing
- Great way to ensure that your code is not the source of a failure
- Use Purify on your tests before requesting a release