text: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
  <html>
  <head>
    <title>Resource change listeners in a concurrent world</title>
    <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
   
  <h1>Listeners and builders in a concurrent world</h1>
  <font size="-1">[Draft copy] </font><br><br>
  <font size="-1">Last modified: August 7, 2003</font> 
  
  <p>In the context of the work on improving platform responsiveness,
  there are some proposed changes to how resource change listeners
  and incremental project builders operate.  This document describes and 
  motivates these changes. The existing (Eclipse 2.1) behavior of listeners and
  auto-build will be described, along with a description of how this will change in Eclipse 3.0.
  Finally, this document includes a description of how existing implementations might 
  be impacted and how they can adapt to the new behavior.
  <h3>Overview and motivation</h3>
  <p>
  The principal proposed change for release 3.0 is that auto-build will be called in a 
  separate thread from which the workspace change 
  occurred.  This means builders will run concurrently with the thread that 
  caused the change, and there will be <b>no guarantee at all</b> about precisely 
  when builders will be called.  The only guarantee is that all builders
  will eventually be run when auto-build is turned on.</p>
  <p>
  There are a number of motivations for this change.  First, there is a need to minimize 
  the duration for which the workspace lock is held by an operation.  In Eclipse 2.1, 
  every workspace operation had an automatic resource change notification and 
  auto-build cycle attached to the end of it.  The operation did not complete until
  all notifications and auto-builds had finished.  For the entire duration of this cycle, 
  the workspace was locked to prevent concurrent changes in other threads.  
  This long term exclusive locking must be reduced if there is any hope to obtain 
  greater concurrency in Eclipse 3.0.  One way to shorten the duration of this locking
  is to split up the operation itself and the auto-build into separate units of work.
  </p>
  <p>
  In addition, this rigid notification and build strategy resulted in poor performance if clients
  neglected to batch groups of changes into a single operation (via 
  <code>IWorkspaceRunnable</code>).  Decoupling the workspace operation
  from the build phase allows us to employ strategies to limit the number
  of builds, and to lower the priority of builds to give precedence to other workspace 
  changing operations.
  </p>
  <p>
  <p>
  Another change that will be made to improve responsiveness is to add notifications
  <b>during</b> long running operations.  An upper bound will be introduced on the
  delay between a workspace change and the notification.  This means that even if an 
  operation takes several minutes, it will be reporting incremental resource changes 
  periodically throughout the operation.  The increased notifications will act as an added
  feedback mechanism to show the user that progress is being made on an operation.
  This also allows the user to begin working with some of the output
  of the operation before it completes.  For example, a user will be able to begin 
  editing imported files before the import is complete, if the import is taking a long time.
  </p>
  
  <h3>Old behavior of listeners and auto-build</h3>
  <p>Every workspace changing operation or group of changes batched with
  <tt>IWorkspaceRunnable</tt> had the following structure prior to Eclipse 3.0:
  <ol>
  <li><tt>PRE_DELETE</tt> or <tt>PRE_CLOSE</tt> notification if applicable</li>
  <li>Perform the operation</li>
  <li><tt>PRE_AUTO_BUILD</tt> notification</li>
  <li>If auto-build is on, perform incremental workspace build</li>
  <li><tt>POST_AUTO_BUILD</tt> notification</li>
  <li><tt>POST_CHANGE</tt> notification</li>
  </ol>
  The following characteristics are worth noting:
  <ul>
  <li>pre- and post- build notifications occurred regardless of whether auto-build was on.</li>
  <li>pre- and post- build events contained a partial delta for the changes up to that
  point.  Their deltas were always a subset of the delta supplied to <tt>POST_CHANGE</tt>
  listeners.</li>
  <li>pre- and post- build listeners are allowed to modify the workspace, but
  other listeners cannot.</li>
  </ul>
  <h3>Overview of changes for listeners in Eclipse 3.0</h3>
  <p>
  With the proposed changes for Eclipse 3.0, steps 3-5 in the above structure are
  removed from the operation.  The resulting picture looks like this:
  <ol>
  <li><tt>PRE_DELETE</tt> or <tt>PRE_CLOSE</tt> notification if applicable</li>
  <li>Perform the operation</li>
  <li><tt>POST_CHANGE</tt> notification</li>
  </ol>
  </p>
  <p>
  Periodically, the platform will perform a background workspace build operation.  The exact 
  timing of when this build occurs will not be specified.  The structure of the build 
  operation will look like this:
  <ol>
  <li><tt>PRE_AUTO_BUILD</tt> notification</li>
  <li>If auto-build is on, perform incremental workspace build</li>
  <li><tt>POST_AUTO_BUILD</tt> notification</li>
  <li><tt>POST_CHANGE</tt> notification</li>
  </ol>
  The reference point for the deltas received by auto-build listeners will be different
  from post-change listeners.  Build listeners receive notification of all changes
  since the end of the last build operation.  Post-change listeners receive a delta
  describing all changes since the last post-change notification.  This structure retains
  three characteristics of resource change listeners that have been true since Eclipse 1.0:
  <ul>
  <li><tt>POST_CHANGE</tt> listeners receive notification of absolutely all resource
  changes that occur during the time they are registered.  This includes changes made
  by builders, and changes made by other listeners.</li>
  <li><tt>PRE_AUTO_BUILD</tt> listeners receive notification of all resource
  changes <b>except</b> changes made by builders and resource change listeners.
  <li><tt>POST_AUTO_BUILD</tt> listeners receive notification of all resource
  changes <b>except</b> changes made by other <tt>POST_AUTO_BUILD</tt>
  listeners.
  </ul>
  However, there are some important differences with this approach. Prior
  to Eclipse 3.0, auto-build listeners were always called before <tt>POST_CHANGE</tt>
  listeners.  For this reason, the delta received by auto-build listeners was always 
  a subset of the delta received by the <tt>POST_CHANGE</tt> listeners.
  This relationship is now essentially reversed.  Auto-build listeners will receive
  a delta that is a super-set of all deltas supplied to <tt>POST_CHANGE</tt>
  listeners since the end of the last background build.
  
  
  
  As before, auto-build listeners will be allowed to modify the workspace, and 
  post-change listeners will not.
  </p>
  <h3>Overview of changes for builders in Eclipse 3.0</h3>
  <p>
  The proposed changes for incremental project builders are conservative.
  The task of an incremental builder is to reconcile an arbitrary set of changes
  to a densely interconnected web of source files to produce an output that 
  is a consistent representation of the source.  This is a difficult enough task to achieve 
  in a single-threaded environment that to attempt it in a concurrent environment 
  would be courting disaster.  On the other hand, building can take a long time, and
  there is a very high value in moving this work to a non-blocking background thread
  to allow the user to continue working.
  </p>
  <p>
  With these factors in mind, the proposal is to move auto-build into a background
  thread, but to preempt the build when another operation attempts to modify the
  workspace.  A typical work flow would go like this:
  <ol>
  <li>A user modifies a file and saves the change.</li>
  <li>Auto-build starts.</li>
  <li>The user modifies the file again and attempts to save.</li>
  <li>The auto-build started in step 2 is halted at the nearest convenient stopping point.</li>
  <li>The file modification is allowed to proceed.</li>
  <li>Auto-build starts running again.</li>
  </ol>
  The &quot;nearest convenient stopping point&quot; in this case is at the end
  of the current builder invocation.  No builder will be halted in the middle of building
  a single project.  This strategy ensures that the user doesn't have to wait long before they
  are allowed to make their change, and that workspace changes don't happen in the
  middle of a single builder invocation.  When a preempted auto-build resumes, it must
  start again at the beginning of the build order to ensure that interdependent projects
  are built in the correct order.  However, builders that successfully ran
  before will not be given the same delta again.
  </p>
  <p>
  This new strategy is designed to have no impact on existing builder implementations.
  In essence, this approach is equivalent to a mode where auto-build is off, but a
  background thread is performing an incremental build periodically.  In other words, since this
  strategy (apart from the preemption) can be implemented using existing API, it 
  should have no impact on existing builders.  No new API is proposed for builders,
  and no API changes are planned.
  </p>
  
  <h3>Summary of proposed API changes</h3>
  <p>
  From a binary compatibility view, this proposal includes no new API, and no
  API changes.  The changes are 100% binary compatible with all prior Eclipse releases.
  There are two proposed changes to the API contract of resource change listeners:
  <ol>
  <li>It will no longer be guaranteed that all resource changes that occur during the
  dynamic scope of an <code>IWorkspaceRunnable</code> will be batched in a single
  notification.  This mechanism can still be used for batching changes to avoid unneccessary
  builds and notifications, but the platform may decide to perform a notification during the
  operation if sufficient time has elapsed.  Note that it was previously possible for resource
  change notifications to occur during the invocation of an <code>IWorkspaceRunnable</code>
  via the <code>IWorkspace.checkpoint</code> API method.  Therefore, this API change
  is not likely to be a breaking change for existing clients.  It is equivalent to the platform
  deciding to call <code>IWorkspace.checkpoint</code> periodically during a long running
  operations.</li>
  <li>
  The javadoc of <tt>IResourceChangeEvent</tt> states that <tt>POST_CHANGE</tt>
  events happen immediately after <tt>POST_AUTO_BUILD</tt> events.  This will no
  longer always be true.  The converse will still be true: <tt>POST_AUTO_BUILD</tt>
  events will always be followed immediately by a <tt>POST_CHANGE</tt> event.
  To boil it down: auto-build events still occur at the same time in relation to builds, they 
  still receive exactly the same information, but they just occur less frequently.
  </li>
  </ol>
  </p>
  <h3>How clients might be impacted</h3>
  <p>
  From the viewpoint of a listener of <tt>POST_CHANGE</tt> notifications, this
  new structure has <b>no effect at all</b>.  This new structure is
  equivalent to a mode where auto-build is turned off, and the user is performing
  a manual workspace build once in awhile.  Most importantly, <tt>POST_CHANGE</tt> 
  listeners are still guaranteed to receive a delta that contains all changes made 
  since the last notification.
  </p>
  <p>
  For listeners to auto-build events, the new structure has some signficant differences.
  Primarily, it will no longer be true that upon completion of a workspace changing
  operation, that auto-build listeners will have been notified.  Client code that makes 
  assumptions about when auto-build listeners run are likely to be broken by this change.  
  For example, if an auto-build listener is updating a domain model to reflect changes
  to the workspace, then this update might not have happened when the workspace
  changing operation returns.
  </p>
  <p>
  It is worth noting that only UI-level code can be affected in this way.  Core-level
  code that is called via API may be called within the scope of an <tt>IWorkspaceRunnable</tt>,
  so it can never be sure about when resource change listeners will be called.  The suggested
  fix for this breakage is to use <tt>POST_CHANGE</tt> instead of build listeners if
  it is necessary to have notification occur before the operation completes.
  </p>
  <p>
  Another potential breakage is in clients that make assumptions about the ordering
  of resource change listeners.  Previously, <tt>PRE_AUTO_BUILD</tt>, <tt>POST_AUTO_BUILD</tt>,
  and <tt>POST_CHANGE</tt> events occurred in a strict linear order.  If listeners in 
  <tt>POST_CHANGE</tt> assume that the auto-build events have already fired,
  they may be broken.
  </p>
  
  <h3>Status and schedule</h3>
  <p>
  All described changes have been implemented. All work is currently in branch "Bug_36957" 
  of project org.eclipse.core.resources. Basic sanity checking of the SDK reveals no 
  significant problems, and some core team members are self-hosting with it.  
  All platform core test suites now pass with background auto-building.  There are
  some failures in JDT core test suites where they make assumptions about the timing
  of auto-build deltas.  The JDT core team is investigating this.  The hope is to release
  these changes to integration builds before the 3.0 M3 milestone.
  <p>
  Go <a href="../planning/3.0/plan_concurrency.html">back to concurrency home</a>.
  </p>
  (Plan item bug reference: <a
   href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=36957">36957</a>)<br>
  </body>
  </html>