Q:

Wait Handles and the System. Threading. Tasks namespace

When I use Parallel. ForEach or Parallel. For in the System. Threading. Tasks namespace, can I assume that all threads are synchronised before execution continues? 
Is there something equivalent to
WaitHandle. WaitAll(...);

going on? 
So If I call something like
Parallel. ForEach(collection, DoSomethingWithObject);

Can I be certain that each call to DoSomethingWithObject has completed before ForEach returns, or do I need to use WaitHandles myself?  

A:

To extend on Phong's correct answer, Parallel. For and Parallel. ForEach internally create a so called ParallelXXXReplicationTask which is run synchronously and spawns X number of child tasks where X is the effective amount of concurrency level (default Environment. ProcessorCount).  Each child task synchronizes with a Semphore which releases the Parent task once set (count to 0) which ensures that the execution of the parent task will only return once all child tasks have finished (or crashed/stopped which is determined through synchronized settings).  Since the parentTask is ran synchronously and is being waited for.  You can ensure that whenever your call to a Parallel method returns, all iterations have been finished effectively. 

A:

All the tasks will have completed by the time Parallel. ForEach() returns, unless a call to Stop() was specifically made to abort the parallel execution.  In such cases, the .IsCompleted property of the ParallelLoopState object returned by Parallel. ForEach() will be false. 
You can read about it here:
http://msdn.microsoft.com/en-us/library/dd992001.aspx
http://msdn.microsoft.com/en-us/library/system.threading.tasks.parallelloopresult.aspx
You can also write a small test application where DoSomethingWithObject() is a very long running operation, or a call to Thread. Sleep(), to verify this.  Execution will block until all threads have finished sleeping. 
Sample:
Parallel. ForEach(Enumerable. Range(1, 8),
    i =>
    {
        Thread. Sleep(5000);
        Console. WriteLine(i + " done!"); 
    });
Console. WriteLine("All done!");