# ROSE Compiler Framework/Arithmetic intensity measuring tool

ROSE tools>

## Overview

A tool to help measure arithmetic intensity (FLOPS/Memory) of loops. It does so by
* statically estimating floating point operations and load/store bytes per iteration for user-specified loops
* instrumenting the loops with statements to capture loop iteration counts and calculate FLOPS and memory footprints (load/store bytes)
* users then run the instrumented code to generate the final reports.
Quick information
* tool location: https://github.com/rose-compiler/rose-develop/tree/master/projects/ArithmeticMeasureTool
* testing: type "make check" within the corresponding build tree

## Download and Installation

It is recommended to obtain the tool from rose-develop repo to have the latest update.
* https://github.com/rose-compiler/rose-develop
The first step is to download and install rose as usual
* Latest instructions: http://rosecompiler.org/ROSE_HTML_Reference/installation.html
Then
* cd rose-build-tree/projects/ArithmeticMeasureTool
* make && make install
An executable file named measureTool will then be installed within ROSE_INSTALLATION_PATH/bin
Now prepare your environment so the tool can be invoked

## Command line options

List
* -help: print out help information
* -debug: enable debugging mode, generating screen output showing progress and internal results
* -annot your_annotation_file: accept user specified function side effect annotations, complement compiler analysis
* -static-counting-only : a special execution mode in which the tool scans all loop bodies and write counting results into a report file
* -report-file your_report_file.txt : specify your own report file name, otherwise the default file ai_tool_report.txt is used.
* -use-algorithm-v2: using 2nd version algorithm in the static counting mode, bottomup synthesized traversal to count FLOPS, still under development

### Function side effect annotation

Compiler analysis cannot figure out side effect of all functions. This can be caused by no access to the library source code or complexity of pointer uses in the source code. To solve this problem, the tool accepts function side effect annotation file via an option --annot
Annotation file format
example command line
* measureTool -c -annot /path/to/functionSideEffect.annot your_input.c

## Execution mode 1: static analysis only

This is a special mode of the tool to only find all loops and count FLOPs for loop bodies. The reported numbers are for single iteration only.
The load/store bytes are represented in two ways
* expression format: such as 3*sizeof(float) + 5*sizeof(double)
* evaluated final integer values: 52
The result is written to a text report file.

### Example use

./measureTool -c -static-counting-only -annot ../../../sourcetree/projects/ArithmeticMeasureTool/src/functionSideEffect.annot -I. ../../../sourcetree/projects/ArithmeticMeasureTool/test/jacobi.c
Excerpt of the generated report. Note that a loop at line 129 has two Plus FP operations and 2 multiplication operations. It loads 0 bytes and store one double element (8 bytes usually). So the final arithmetic intensity (AI) is 4/8= 0.5 ops/byte
Content of generated report file: ai_tool_report.txt
Right now
* AI is set to -1.0 if it is unintialized
* AI is set to be 9999.9 if divided by zero bytes

### User pragma to verify results

In this mode, the translator can verify the tool-generated results by comparing the results to what is indicated by pragmas in the input code.
The user provided pragma has the form of
fp_total is required while the clauses of other kinds of FP operations are optional.

## Execution mode 2: analyze and instrument the code

This is the default mode .

### Manual instrument the input code

The tool currently works with collaboration with user-added code instrumentation, using the following steps:
* declare four global counters with specific variable names, which will later be recognized by the tool
* add chiterations = .. before the loops you want to count FPs and Load/store bytes
* print out the results: printf ("chflops =%lu chloads =%lu chstores=%lu\n", chflops, chloads, chstores);

### Use the tool to transform the code

./measureTool -c -annot ../../../sourcetree/projects/ArithmeticMeasureTool/src/functionSideEffect.annot nestedloops.c
The tool will
* count the FLOPs and load store bytes for the specified loop
* add counter accumulation statements, using different counters for different loops

### Compile& run the transformed code

gcc -O3 rose_nestedloops.c -o nestedloops.out -l
./nestedloops.out
The result looks like
chflops =401000 chloads =800000 chstores=8000

## Limitations

The tool does not support Fortran loops with function calls for now
* ROSE's Fortran procedure/routine representation is not accurate enough (missing parameter type info.) to hook up with function side effect annotations designed to match C/C++ functions.

## Internals

Execution model variable running_mode
* e_analysis_and_instrument
* e_static_counting

### FP operations

class FPCounters: public AstAttribute; to store analysis results
void CountFPOperations from src/ai_measurement.cpp

### Load/Store bytes

The main functions are defined in ai_measurement.cpp:
* std::pair <SgExpression*, SgExpression*> CountLoadStoreBytes (SgLocatedNode* input, bool includeScalars /* = true */, bool includeIntType /* = true */)
* SgExpression* calculateBytes (std::set<SgInitializedName*>& name_set, SgStatement* lbody, bool isRead)
return expressions to calculate the value, not the actual values, since sizeof(type) is machine dependent.
Configuration
* By default: only array references are counted. Scalars are ignored.
Algorithm
* call side effect analysis to find read/write variables, some reference may trigger both read and write accesses. If analysis is successful, proceed. Otherwise warning is sent.
* Accesses to the same array/scalar variable are grouped into one read (or write) access: e.g. array[i][j], array[i][j+1], array[i][j-1], etc are counted a single access
* Group accesses based on the types: same type access-> increment the same counter to shorten expression length
* Iterate on the results to generate expression like 2*sizeof(float) + 5* sizeof(double)
* As an approximate, we use simple analysis here assuming no function calls.

### Nested loops

Scientific applications usually have nested loops. Naive instrumentation will cause two problems
* double counting for nested loop body:
* the chiterations= .. statement is used for all levels of loop. The inner loop's chiterations will overwrite the chiterations used to indicate outer loop.
Solutions
* The translator uses a bottom-up traversal order: processing inner loops first, then outer loops.
* To avoid double counting FP operations within nested loops: all visited FP operations expressions are stored into a lookup table. Later counting will check if an operation is already counted. If so, skip it.
* To avoid double counting of variables used in nested loops when counting a outer loop body: This is slightly different from the handling of FP op expressions. Here we find all variables counted in inner loops, exclude them when do the counting for an outer loop. Note: excluding a entirely, not just flagging a reference to a and exclude such reference later.
** Note: static counting mode does not do this excluding since the assumption of redundant execution is no longer a concern. We still count loop body's FLOPS for inner and outer loops if they are nested.
* rewrite chiterations= to chiterations_loopId= .. , so each loop has its own iteration number variable.

## Testing

run all builtin tests
* make check
run tests for static analysis only
* make check-static
Manual testing
* [liao6@tux322:~/workspace/ExReDi/ai_tool.git/translator]m && ./measureTool -c -annot ./src/functionSideEffect.annot -I. ./test/jacobi-v3.c