# ROSE Compiler Framework/ROSE tools
## Overview

ROSE is a compiler framework to build customized compiler-based tools. A set of example tools are provided as part of the ROSE release to demonstrate the use of ROSE. Some of them are also useful for daily work of ROSE developers.
We list and briefly explain some tools built using ROSE. They are installed under ROSE_INSTALLATION_TREE/bin .
* some tools are not installed by default. You may have to cd into the tool's subdirectory and type "make install"
Any ROSE tool(translator) works like a compiler. You have to provide all necessary compilation flags to make it work properly. You have to specify some include paths and/or some macro to be defined.
* One way to use rose tools is to replace your default compiler (gcc for example) with the rose tool command (identityTranslator, outline, etc) in your Makefile. So the tool will be called with all correct flags when you type "make".
* Alternatively, if you are only interested in processing a single file. You can manually watch the full compilation command line (e.g. gcc .... -c ) used to compile that single file during a normal "make". Then replace your compiler (gcc) with the rose tool you are interested in (e.g. outline) in the command line.

## Prerequisites

You have to install ROSE first, by typing configure, make, make install, etc.
You also have to set the environment variables properly before you can call ROSE tools from command line.
For example: if the installation path (or --prefix path in configure) is /home/opt/rose/install, you can have the following script to set the environment variables using bash:
All ROSE tools are installed under the path specified by the --prefix option.

## identityTranslator

Source: http://www.rosecompiler.org/ROSE_Tutorial/ROSE-Tutorial.pdf (chapter 2)
This is the simplest tool built using ROSE. It takes input source files , builds AST, and then unparses the AST back to compilable source code. It tries its best to preserve everything from the input file.

### Uses

Typical use cases
* without any options, test if ROSE can compile your code: replace the compiler used by your Makefile with identityTranslator
* turn on some built-in analysis, translation or optimization phases, such as -rose:openmp:lowering to support OpenMP
** type "identityTranslator --help" to see all options
* debug a ROSE-based translator: the first step is often to use identityTranslator to rule out if it is a compilation problem using ROSE
* use the source of the identityTranslator as a start point to add custom analysis and transformation. The code in the identityTranslator is indeed the minimum code required for almost all kinds of ROSE-based tools.

### Source code

identityTranslator.c

### Plugin

Starting from Version 0.9.9.83, ROSE has a new feature to support external plugins. It borrows the design and implementation of Clang Plugins. The interface is very similar to what Clang has, with some simplification and improvements.
With this feature, you can develop your ROSE-based tools as dynamically loadable plugins. Then you can use command line options of ROSE's default translator, identityTranslator (or another ROSE translator), to
* load shared libraries containing the plugins,
* specify actions to be executed,
* as well as pass command line options to each action.
See more at:
* ROSE Compiler Framework/Plugin

### Limitations

Due to limitations of the frontends used by ROSE and some internal processing, identityTranslator cannot generate 100% identical output compared to the input file.
Some notable changes it may introduce include:
* "int a, b, c;" are transformed to three SgVariableDeclaration statements,
* macros are expanded.
* extra brackets are added around constants of typedef types (e.g. c=Typedef_Example(12); is translated in the output to c = Typedef_Example((12));)
* Converting NULL to 0.

## AST dot graph generators

Tools to generate AST graph in dot format. There are two versions
* dotGenerator: simple AST graph generator showing essential nodes and edges
* dotGeneratorWholeASTGraph: whole AST graph showing more details. It provides filter options to show/hide certain AST information.
command line:
More information about how to use the tools can be found at How_to_visualize_AST

## AST Outliner

Basic concept: outlining is the process of replacing a block of consecutive statements with a function call to a new function containing those statements. Conceptually, outlining the inverse of inlining.
ROSE provide a builtin translator called AST outliner, which can outline a specified portion of code and generate a function from it.
* Official documentation for the AST outliner is located in Chapter 37 Using the AST Outliner with the ROSE Tutorial. pdf.
* Supplemental information can be found here at ROSE Compiler Framework/outliner

## KeepGoingTranslator

Often a ROSE-based tool may encounter some issues when processing a large scale applications. Users may want to let the tool keep running until the processing is finished and they can check the final results about how many source files are successfully processed or not. This is similar to the GNU make's -k option, by using which make will try to compile every file that can be tried and show as many compilation errors as possible.
ROSE has a builtin -rose:keep_going support. If this feature is turned on and an error occurs, ROSE will simply run your backend compiler on your original source code file, as is, without modification.
To further simplify the use of -rose:keep_going, we provide the Rose::KeepGoing namespace, which internally
* uses -rose:keep_going and keeps track of successfully processed or failed files, and
* save such information into log files.
An example translator named KeepGoingTranslator is created to demonstrate the use of this namespace.
In summary, there are three levels of keep going support
* GNU make's -k option as part of the build system
* ROSE's -rose:keep_going option, independent from the build system.
* Rose::KeepGoing namespace with additional logging support
To use the builtin support for logging messages for each file

## Inliner

see more at ROSE_Compiler_Framework/Inliner
The ROSE Inliner inlines functions at function callsites.
Official documentation about the Inliner is
* Chapter 36 "Calling the Inliner of ROSE" tutorial: http://rosecompiler.org/ROSE_Tutorial/ROSE-Tutorial.pdf
Source code
* https://github.com/rose-compiler/rose-develop/tree/master/src/midend/programTransformation/astInlining
Test directory with an example translator and test input files
* https://github.com/rose-compiler/rose-develop/tree/master/tests/nonsmoke/functional/roseTests/astInliningTests
By looking into Makefile.am, the example translator's source code will generate an executable named "inlineEverything" in your buildtree.
This is the tool you can try to inline your sample code.
The same Makefile.am's make check rules contain sample command lines to use the tool.

## call graph generator

source code of this tool
* https://github.com/rose-compiler/rose/blob/release/tutorial/buildCallGraph.C
Command line:

## Control flow graph generator

Command line:

## autoPar

This is a tool which can automatically insert OpenMP pragmas into input serial C/C++ codes
This tool is an implementation of automatic parallelization using OpenMP. It is used to explore semantics-aware automatic parallelization, as described in one of our paper* .
* The source files are currently located in rose/projects/autoParallelization.
* A standalone executable program (named autoPar ) is generated and installed to the installation tree of ROSE (under ROSE INS/bin).
* you can test the tool in rose_build_tree/projects/autoParallelization by typing "make check"
* There is a section in ROSE manual pdf: 12.7 Automatic Parallelization
Publications
* Chunhua Liao, Daniel J. Quinlan, Jeremiah J. Willcock and Thomas Panas, Semantic-Aware Automatic Parallelization of Modern Applications Using High-Level Abstractions, Journal of Parallel Programming, Volume 38, Numbers 5-6, 361-378, August 23. 2010 LLNL-JRNL-421803
* Chunhua Liao, Daniel J. Quinlan, Jeremiah J. Willcock and Thomas Panas, Extending Automatic Parallelization to Optimize High-Level Abstractions for Multicore, In Proceedings of the 5th international Workshop on OpenMP: Evolving OpenMP in An Age of Extreme Parallelism (Dresden, Germany, June 03 - 05, 2009).
More info is at autoPar

## abstract handles

This work is used in the autotuning and also other tools that pass references to source code as part of an interface.
* essentially it defines ways to create a string to uniquely identify a language construct in your source code
* any tool then can locate the corresponding node in AST and do the targetted analysis or transformation for you.
Key info
* Documentation: Chapter 46 of ROSE Tutorial: http://rosecompiler.org/ROSE_Tutorial/ROSE-Tutorial.pdf
* Header file: https://github.com/rose-compiler/rose-develop/blob/master/src/midend/abstractHandle/abstract_handle.h
* Example use: https://github.com/rose-compiler/rose-develop/tree/master/projects/autoTuning
** http://rosecompiler.org/autoTuning.pdf Section 6.2 explains how parameterized loop transformations are used on specified loops, using abstract handles.
** https://github.com/rose-compiler/rose-develop/blob/master/tests/roseTests/astInterfaceTests/loopCollapsing.C a loop collapse tool accepting abstract handles
See more at
* ROSE Compiler Framework/Abstract handles

## Loop translators

List of individual loop translators which perform transformation only. So callers use make sure the transformations are semantically correct for your input code
* Loop collapsing
* loop interchanging
* loop normalization
* loop tiling
* loop unrolling
Example command lines to run these loop translators can be found in Makefile.am
* e.g.: loopUnrolling -rose:loopunroll:abstract_handle 'Statement<position,5>' -rose:loopunroll:factor 3
There is another integrated loopProcessor, which relies on sophisticated analyses to drive a range of loop optimizations.
* Loop Processor

## Declaration move tool

This tool will move variable declarations to their innermost possible used scopes.
For a declaration, find the innermost scope we can move it into, without breaking the code's original semantics.
* For a single use place, move to the innermost scope.
* For the case of multiple uses, we may need to duplicate the declarations and move to two scopes if there is no variable reuse in between, otherwise, we move the declaration into the innermost common scope of the multiple uses.
User instructions: The translator accepts the following options:
* -rose:merge_decl_assign will merge the moved declaration with an immediately followed assignment.
* -rose:aggressive : turn on the aggressive mode, which will move declarations with initializers, and across loop boundaries. A warning message will be sent out if the move crosses a loop boundary. Without this option, the tool only moves a declaration without an initializer to be safe.
* -rose:debug, which is turned on by default in the testing. Some dot graph files will be generated for scope trees of variables for debugging purpose.
* -rose:keep_going will ignore assertions as much as possible (currently on skip the assertion on complex for loop initialization statement list). Without this option, the tool will stop on assertion failures.
* -rose:identity will turn off any transformations and act like an identity translator. Useful for debugging purposes.
* -rose:trans-tracking will turn on the transformation tracking mode, showing the source statements of a move/merged declaration
source code: https://github.com/rose-compiler/rose-develop/blob/master/tests/roseTests/astInterfaceTests/moveDeclarationToInnermostScope.C
* Note: moved to tests/nonsmoke/functional/roseTests/astInterfaceTests recently
tests: make move_diff_check defined in https://github.com/rose-compiler/rose-develop/blob/master/tests/roseTests/astInterfaceTests/Makefile.am
See more at Declaration move tool

## Arithmetic intensity measuring tool

Arithmetic intensity measuring tool : measure arithmetic intensity (FLOPS/Memory) of loops

## MPI code generator

A set of code generation functions used to support translating high level domain specific languages into MPI code, with some runtime support.
* source code: https://github.com/rose-compiler/rose-develop/blob/master/projects/ShiftCalculus2/mpiCodeGenerator.C
* test: make mpiTests within https://github.com/rose-compiler/rose-develop/blob/master/projects/ShiftCalculus2/Makefile.am

## Compilation Database

A utility program to convert a compilation database json file into a single makefile.
The motivation is to avoid hacking into users' build systems in order to invoke a ROSE-based tool.
With the single makefile, we can freely replace the compiler name and options without understanding the complex build systems.
Details at: https://github.com/rose-compiler/rose/tree/develop/utilities/compilationDatabase2Makefile
Steps to use it
* generate compilation_database.json file
* convert it to a makefile
* rm all object files first from your build tree!!
** make -f your-makefile clean
* make -k -f your-makefile all # -k means keep going if somethings goes wrong

## TODO
### refactor the tool translators

Refactor the tools into a dedicated rose/tools directory. So they will always be built and available by default, with minimum dependency on other things, like which languages are turned on or off (when applicable of course)
Our current idea is we should separate translators used as examples or tutorials AND translators used for creating end-user tools.
* For tutorial translators, they should NOT be installed as tools by default. Their purpose is to be included in Manual or Tutorial pdf files to illustrate something to developers by examples. Examples should be concise and to the point.
* On the other hand, translators used to build end-user tools should have much higher standard to accept command options for different, even advanced features. These translators can be very sophisticated since they don't have the page limitation as tutorial examples do.