# ROSE Compiler Framework/PreprocessingInfo
## Overivew

In addition to nodes and edges, ROSE AST may have attributes in addition to nodes and edges that are attached for preprocessing information like #include or #if .. #else. They are attached before, after, or within a nearby AST node (only the one with source location information.)
An example translator will traverse the input code's AST and dump information which may include preprocessing information.
* https://github.com/rose-compiler/rose-develop/blob/master/exampleTranslators/defaultTranslator/preprocessingInfoDumper.C
For example
Source: http://www.rosecompiler.org/ROSE_Tutorial/ROSE-Tutorial.pdf (Chapter 29 - Handling Comments, Preprocessor Directives, And Adding Arbitrary Text to Generated Code)

## What is PreprocessingInfo

It is a type of ROSE AST meta information to store source code comments.
They do not exist independently, but are attached to located sgnode.
* before: before a code segment
* after: after a code segment
* inside: inside an empty scope
PreprocessingInfo::RelativePositionType position=PreprocessingInfo::before
typedef Rose_STL_Container<PreprocessingInfo*> AttachedPreprocessingInfoType;
* source comments: c style, c++ style
* preprocessor info: include, define, ifdef, line declaration
* and more

## Dump out PreprocessingInfo
### SageInterface::dumpPreprocInfo

void dumpPreprocInfo (SgLocatedNode *locatedNode)

### the builtin preprocessingInfoDumper

This tool is not built by default. You have to build it manaully
* cd buildDebug/exampleTranslators/defaultTranslator
* make install
ROSE_SRC/exampleTranslators/defaultTranslator/preprocessingInfoDumper.C
ROSE_INSTALL/bin/preprocessingInfoDumper
see also ROSEAttributesList::display from rose_attributes_list.C for dumping it

## Creation and Insertion into AST
### call stack

Two phases: collect them first, then attach them into matching nodes
main
* frontend
** frontend
*** SgProject
**** SgProject::parse
***** SgProject::parse
****** determineFileType
******* SgSourceFile::callFrontEnd
******** SgFile::callFrontEnd
********* attachPreprocessingInfo(sourceFile); attachPreprocessingInfo.C
class AttachPreprocessingInfoTreeTrav // attach_all_info.C/h
* AttachAllPreprocessingInfoTreeTrav::evaluateSynthesizedAttribute
* AttachAllPreprocessingInfoTreeTrav::evaluateInheritedAttribute
Fortran code
Calling attachPreprocessingInfo to attach them
* C/C++ SgFile::callFrontEnd
* Fortran: SgSourceFile::build_Fortran_AST
SgLocatedNode::addToAttachedPreprocessingInfo(PreprocessingInfo *prepInfoPtr, PreprocessingInfo::RelativePositionType locationInList )
The locationInList abuses the PreprocessingInfo::RelativePositionType, only before and after positions are meaningful here, which is used to specify the head or the rear of the info list.
PreprocessingInfo::before
gdb breakpoint:
> break PreprocessingInfo::PreprocessingInfo(PreprocessingInfo::DirectiveType, std::string const&, std::string const&, int, int, int, PreprocessingInfo::RelativePositionType)

### algorithm

src/frontend/SageIII/attachPreprocessingInfoTraversal.C
The SgFile (always) constructor calls the function
* void attachPreprocessingInfo(SgFile *sageFilePtr); which in turn calls
** getPreprocessorDirectives (see above) and then
invokes a tree traversal in order to attach the preprocessor directives (i.e., the preprocessingInfo objects) to located nodes in the AST.
For this purpose, a data member attachedPreprocessingInfoType* attachedPreprocessingInfoPtr; is available to the SgLocatedNode class. This is done in ROSETTA/src/node.C.
Furthermore, There are corresponding access functions:
* void addToAttachedPreprocessingInfo(preprocessingInfo *prepInfoPtr);
* attachedPreprocessingInfoType* getAttachedPreprocessingInfo(void);
to the SgLocatedNode class. This is done in ROSETTA/Grammar/LocatedNode.code.
The tree traversal works as follows:
* whenever it hits a located node
* it checks if there is preprocessing info the line number of which is less or equal than the line number of the current located node (currently: of the current statement).
** If this is the case, the corresponding preprocessing info is attached to the current located node (currently: before the current statement), unparse flag: "before". All this is done in the evaluateInheritedAttribute member function of the derived tree traversal class.
* The evaluateSynthesizedAttribute member function deletes the list of preprocessingInfo objects as soon as the traversal returns to a SgFile object and attaches trailing preprocessing information to the last located node (currently to the last statement) that has been visited in the file (unparse flag: "after").
Node that the preprocessingInfo objects are always attached to AST nodes. By switching the USE_OLD_MECHANISM_OF_HANDLING_PREPROCESSING_INFO flag, you only change the mechanism which the unparser is based on! If USE_OLD_MECHANISM_OF_HANDLING_PREPROCESSING_INFO is set to 1, then the unparser simply ignores the preprocessingInfo objects that have been attached to the AST nodes.
Problems with the handling of preprocessing information can be found in the directory ROSE/TESTS/KnownBugs/AttachPreprocessingInfo.

### attaching function

// frontend/SageIII/attachPreprocessingInfoTraversal.C
void AttachPreprocessingInfoTreeTrav::iterateOverListAndInsertPreviouslyUninsertedElementsAppearingBeforeLineNumber ( SgLocatedNode* locatedNode, int lineNumber, PreprocessingInfo::RelativePositionType location, bool reset_start_index, ROSEAttributesList *currentListOfAttributes)
different call sites of iterateOverListAndInsertPreviouslyUninsertedElementsAppearingBeforeLineNumber
how to decide on different locations
* before: AttachPreprocessingInfoTreeTrav::evaluateInheritedAttribute ( SgNode *n,AttachPreprocessingInfoTreeTraversalInheritedAttrribute inheritedAttribute)
** attach to the current statement's before position if traversal is in evaluateInheritedAttribute
*** calling iterateOverListAndInsertPreviouslyUninsertedElementsAppearingBeforeLineNumber(currentLocNodePtr,line,PreprocessingInfo::before, reset_start_index, currentListOfAttributes );
* inside: AttachPreprocessingInfoTreeTrav::evaluateSynthesizedAttribute( SgNode *n, AttachPreprocessingInfoTreeTraversalInheritedAttrribute inheritedAttribute, SubTreeSynthesizedAttributes synthiziedAttributeList)
** attach to the inside position of a multiline located node when leaving the node during evaluateSynthesizedAttribute
* after: AttachPreprocessingInfoTreeTrav::evaluateSynthesizedAttribute( SgNode *n, AttachPreprocessingInfoTreeTraversalInheritedAttrribute inheritedAttribute, SubTreeSynthesizedAttributes synthiziedAttributeList)
** e.g. attach to the previous located node 's after position if the traversal is leaving V_SgSourceFile during evaluateSynthesizedAttribute

### comments

Not all Located nodes can have comments!!

### arbitrary text

More versatile than comments:

### deletion

Cannot remove elements while iterating the container. Save a subset of elements into another container and remove them later on

## Move comments to new statements

Source comments and #if directives should be preserved when we remove or replace statements in the AST
A possible call stack is
* SageInterface::removeStatement from src/frontend/SageIII/sageInterface/sageInterface.C:10086
** SageInterface::moveCommentsToNewStatement from src/frontend/SageIII/sageInterface/sageInterface.C:10193

## How to debug

Debug the insertion action: SgLocatedNode::addToAttachedPreprocessingInfo in buildTree 's Cxx_Grammar.C