# ROSE Compiler Framework/Abstract handles
## Overview

This work is used in the autotuning and also other tools that pass references to source code as part of an interface.
* essentially it defines ways to create a string to uniquely identify a language construct in your source code
* any tool then can locate the corresponding node in AST and do the targetted analysis or transformation for you.
Abstract handles support the following forms for specifying language constructs:
* Source file position information including path, filename, line and column number etc. GNU standard source position from http://www.gnu.org/prep/standards/html_node/Errors.html presents some examples.
* Global or local numbering of specified language construct in source file (e.g. 2nd ”do” loop in a global scope). The file is itself specified using an abstract handle (typically generated from the file name).
* Global or local names of constructs. Some language constructs, such as files, function definitions and namespace, have names which can be used as their handle within a context.
* Language-specific label mechanisms. These include named constructs in Fortran, numbered labels in Fortran, and statement labels in C and C++, etc.
In addition to human-readable forms, compilers and tools can generate internal IDs for language constructs. It is up to compiler/tool developers to provide a way to convert their internal representations into human-readable formats.
Abstract Handles can have any of the human-readable or machine-generated forms. A handle can be used alone or combined with other handles to specify a language construct. A handle can also be converted from one form to another (e.g. from a compiler specific form to an human readable form relative to the source position; filename, line number, etc.). Abstract handles can have different lifetimes depending on their use and implementation. An abstract handle might be required to be persistent if it is used to reference a language construct that would be optimized over multiple executions of one or more different tools. Where as an abstract-handle might be internally generated only for purposes of optimizations used in a single execution (e.g. optimization within a compiler).

## Use case

A typical use can for Abstract Handles might be for a performance tool to identify a collection of loops in functions that are computationally intensive and to construct Abstract Handles that refer to this specific loops. Then pass the Abstract Handles to a second tool that might analyze the source code and/or the binary executable to evaluate if the computational costs are reasonable or if optimizations might be possible. The specific goal of the Abstract Handles is to support these sorts of uses within autotuning using diverse tools used and/or developed as part of autotuning research within the DOE SciDAC PERI project.

## Syntax

A possible grammar for abstract handles could be:
* A handle is a single handle item or a link of them separated by ::, or other delimiters
* Each handle item consists of construct_type and a specifier. Or it can be a compiler generated id of any forms.
* Construct types are implementation dependent. An implementation can support a subset of legal constructs or all of them. We define a minimum set of common construct type names here and will grow this list as needed.
* A specifier is used to locate a particular construct e.g: <name, "foo">
* Tokens for specifier types could be name, position, numbering, label, etc. specifier type is necessary to avoid ambiguity for specifier values, because a same value could be interpreted in different specifier types otherwise
* Possible values for a specifier
* A label could be either integer or string
* Start and end source line and column information
e.g.: 13.5-55.4, 13, 13.5 , 13.5-55
* Integer value: one or more digits
* String value: start with a letter, followed by zero or more letters or digits

## Reference Implementation

Abstract Handles are fundamentally compiler and tool independent, however to clarify the con- cepts, provide meaningful examples, a working reference implementation we have provided a reference implementation in ROSE. The source files are located in src/midend/abstractHandle in the ROSE distribution. A generic interface (abstract handle.h and abstract handle.cpp) pro- vides data structures and operations for manipulating abstract handles using source file positions, numbering, or names. Any compilers and tools can have their own implementations using the same interface.

### Connecting to ROSE

A ROSE adapter (roseAdapter.h and roseAdapter.cpp) using the interface is provided as a concrete implementation for the maximum capability of the implementation (within a source- to-source compiler).
The follow image shows an input source file.
The follow image shows the code (using ROSE) to generate abstract handles for loops in the input source file.
Abstract handle constructors generate handles from abstract nodes, which are implemented using ROSE AST nodes. Source position is used by default to generate a handle item. Names or numbering are used instead when source position information is not available. The Constructor can also be used to generate a handle item using a specified handle type (numbering handles in the example). The follow image is the output showing the generated handles for the loops.

## Connecting to External Tools

A third example is provided to demonstrate how to use the abstract interface with any other tools, which may have less features in terms of supported language constructs and their correlations compared to a compiler. Assume a tool operating on some simple for-loops within an arbitrary source file (the input file is not shown in this example). Such a tool might have an internal data structure representing loops; such as that in given in the next figure.
We will show how the tool specific data structure for loops can be used to generate abstract handles and output as strings that can be used by other tools which use abstract handles (which would generate the abstract handles by reading the strings).
An adapter (loopAdapter.h and loopAdapter.cpp) using the proposed abstract handle in- terface is given in src/midend/abstractHandle. It provides a concrete implementation for the interface for the simple loops and adds a node to support file nodes (Compared to a full-featured IR for a compiler, the file node is an additional detail for tools without data structures to support files). The test program is given in next Figure.
Again, it creates a top level file handle first. Then a loop handle (loop handle1) is created within the file handle using its relative numbering information. The loop handle2 is created from its string format using file position infor- mation (using GNU standard file position syntax). The loop handle3 uses its relative numbering information within loop handle1.
The output of the program is shown in the follow image. It demonstrates the generated strings to represent the abstract handles in the arbitrary code operated upon by the tool. Interoperability is achieved by another tool reading in the generated string representation to generate an abstract handle to the same source code language construct.

## Examples
### Language handles using the syntax

We give some examples of language handles using the syntax mentioned above. Canonical AST’s node type names are used as the construct type names. Other implementations can use their own construct type names.
* A file handle consisting of only one handle item:
* A function handle using a named handle item, combined with a parent handle using a name also:
* A function handle using source position(A function starting at line 12, column 1 till line 30, column 1 within a file):
* A function handle using numbering(The first function definition in a file):
* A return statement using source position (A return statement at line 100):
* A loop using numbering information (The second loop in function main):
* A nested loop using numbering information (The first loop inside the second loop in func- tion main):

### Input Code Examples

Outline the for loop at line 12 of the input code
* ./outline -rose:outline:abstract_handle "ForStatement<position,12>" -c $(srcdir)/inputCode_OutlineLoop2.c
Outline the 2nd for statement within a function named initialize from the input code
* ./outline -rose:outline:abstract_handle "FunctionDeclaration<name,initialize>::ForStatement<numbering,2>" -c $(srcdir)/inputCode_OutlineLoop2.c -rose:output rose_inputCode_OutlineLoop2b.c
Outline the statement at line 5 of the input code
* ./outline -rose:outline:abstract_handle "Statement<position,5>" -c $(srcdir)/declarations.c

## Summary

Abstract handles are low level mechanisms to support multiple tools to exchange references to source code. Several examples are used to present the different features of abstract handles. Importantly, the specification of abstract handles is tool independent. A reference implementa- tion is provided and is publically available within the ROSE compiler framework. We encourage debate on the pros and cons of this concept to support interoperability of tools which must pass references to source code between them. This work is expected to a small piece of the infrastructure to suport autotuning research.

## Key info

* Documentation: Chapter 46 of ROSE Tutorial: http://rosecompiler.org/ROSE_Tutorial/ROSE-Tutorial.pdf
* Header file: https://github.com/rose-compiler/rose-develop/blob/master/src/midend/abstractHandle/abstract_handle.h
* Example use: https://github.com/rose-compiler/rose-develop/blob/master/projects/autoTuning/tests/Makefile.am
** http://rosecompiler.org/autoTuning.pdf Section 6.2 explains how parameterized loop transformations are used on specified loops, using abstract handles.
** https://github.com/rose-compiler/rose-develop/blob/master/tests/nonsmoke/functional/roseTests/astInterfaceTests/loopCollapsing.C a loop collapse tool accepting abstract handles

## code sample

How to support abstract handles in your tool
https://github.com/rose-compiler/rose-develop/blob/master/tests/nonsmoke/functional/roseTests/astInterfaceTests/loopCollapsing.C

## reference

To cite the abstract handle work, please use the following paper in which the handles were first published:
* Chunhua Liao, Daniel J. Quinlan, Richard Vuduc and Thomas Panas, Effective Source-to-Source Outlining to Support Whole Program Empirical Optimization, The 22nd International Workshop on Languages and Compilers for Parallel Computing (LCPC), Newark, Delaware, USA. October 8-10, 2009