The new reflection

A must read: "SIP: Scala Reflection":
https://docs.google.com/document/d/1Z1VhhNPplbUpaZPIYdc0_EUv5RiGQ2X4oqp0i-vz1qw/edit

Highlights:
* Architecture has undergone a dramatic rehash.
* Universes and mirrors are now separate entities:
universes host reflection artifacts (trees, symbols, types, etc),
mirrors abstract loading of those artifacts (e.g. JavaMirror loads stuff
using a classloader and annotation unpickler, while GlobalMirror uses
internal compiler classreader to achieve the same goal).
* No static reflection mirror is imposed on the user.
One is free to choose between lightweight mirrors and full-blown
classloader-based mirror (read below).
* Public reflection API is split into scala.reflect.base and scala.reflect.api.
The former represents a minimalistic snapshot that is exactly enough to
build reified trees and types. To build, but not to analyze - everything smart
(for example, getting a type signature) is implemented in scala.reflect.api.
* Both reflection domains have their own universe: scala.reflect.basis and
scala.reflect.runtime.universe. The former is super lightweight and doesn't
involve any classloaders, while the latter represents a stripped down compiler.
* Classloader problems from 2.10.0-M3 are solved.
* Exprs and type tags are now bound to a mirror upon creation.
* However there is an easy way to migrate exprs and type tags between mirrors
and even between universes.
* This means that no classloader is imposed on the user of type tags and exprs.
If one doesn't like a classloader that's there (associated with tag's mirror),
one can create a custom mirror and migrate the tag or the expr to it.
* There is a shortcut that works in most cases. Requesting a type tag from
a full-blown universe will create that tag in a mirror that corresponds to
the callsite classloader aka `getClass.getClassLoader`. This imposes no
obligations on the programmer, since Type construction is lazy, so one
can always migrate a tag into a different mirror.

Migration notes for 2.10.0-M3 users:
* Incantations in Predef are gone, some of them have moved to scala.reflect.
* Everything path-dependent requires implicit prefix (for example, to refer
to a type tag, you need to explicitly specify the universe it belongs to,
e.g. reflect.basis.TypeTag or reflect.runtime.universe.TypeTag).
* ArrayTags have been removed, ConcreteTypeTag have been renamed to TypeTags,
TypeTags have been renamed to AbsTypeTags. Look for the reasoning in the
nearby children of this commit. Why not in this commit? Scroll this message
to the very bottom to find out the reason.
* Some of the functions have been renamed or moved around.
The rule of thumb is to look for anything non-trivial in scala.reflect.api.
Some of tree build utils have been moved to Universe.build.
* staticModule and staticClass have been moved from universes to mirrors
* ClassTag.erasure => ClassTag.runtimeClass
* For the sake of purity, type tags no longer have erasures.
Use multiple context bounds (e.g. def foo[T: ru.TypeTag : ClassTag](...) = ...)
if you're interested in having both erasures and types for type parameters.
* reify now rolls back macro applications.
* Runtime evaluation is now explicit, requires import scala.tools.reflect.Eval
and scala-compiler.jar on the classpath.
* Macro context now has separate universe and mirror fields.
* Most of the useful stuff is declared in c.universe,
so be sure to change your "import c.universe._" to "import c.mirror._".
* Due to the changes in expressions and type tags, their regular factories
are now really difficult to use. We acknowledge that macro users need to
frequently create exprs and tags, so we added old-style factories to context.
Bottom line: almost always prepend Expr(...)/TypeTag(...) with "c.".
* Expr.eval has been renamed to Expr.splice.
* Expr.value no longer splices (it can still be used to express cross-stage
path-dependent types as specified in SIP-16).
* c.reifyTree now has a mirror parameter that lets one customize the initial
mirror the resulting Expr will be bound to. If you provide EmptyTree, then
the reifier will automatically pick a reasonable mirror (callsite classloader
mirror for a full-blown universe and rootMirror for a basis universe).
Bottom line: this parameter should be EmptyTree in 99% of cases.
* c.reifyErasure => c.reifyRuntimeClass.

Known issues:
* API is really raw, need your feedback.
* All reflection artifacts are now represented by abstract types.
This means that pattern matching against them will emit unchecked warnings.
Adriaan is working on a patch that will fix that.

WARNING, FELLOW CODE EXPLORER! You have entered a turbulence zone.
For this commit and its nearby parents and children
tests are not guaranteed to work. Things get back to normal only after
the "repairs the tests after the refactoring spree" commit.

Why so weird? These twentish changesets were once parts of a humongous blob,
which spanned 1200 files and 15 kLOC. I did my best to split up the blob,
so that the individual parts of the code compile and make sense in isolation.
However doing the same for tests would be too much work.
ClassTag.erasure => ClassTag.runtimeClass

This change is made to be consistent with JavaMirrors.
And, in my opinion, a technology-neutral term is better here.
removes tags and their incantations from Predef

All tags and reflection-related stuff requires a prefix,
be it scala.reflect for simple tags (ArrayTags and ClassTags),
or scala.reflect.basis/scala.reflect.runtime.universe for type tags.
brings macros up to speed

Before reflection refactoring, macro contexts only exposed a mirror.
Now it's time to expose both a universe (the compiler instance) and
a mirror (a macro-specific symbol resolver).

By the way, speaking of mirrors. Macro contexts have their own mirror,
which is different from compiler's rootMirror. This is done because
macros need to be able to resolve stuff from empty package.

Reflection refactoring brought major changes to runtime evaluation,
which got dropped from universes and now requires scala-compiler.jar.
However there are macro users, who would like to do eval inside macros.
To help them we expose `libraryClassLoader` to manually build toolboxes,
and also a simple-to-use `c.eval` method.

I've also sneakily introduced `c.parse`, because it's something that
has also been frequently requested. Moreover, it might help Scaladoc.
So I decided that it might be worth it to add this new functionality.
brings reification up to speed

Along with recovering from reflection refactoring, I implemented
some new features (e.g. rollback of macro expansions),
and did some stabilizing refactorings (e.g. moved mutable state into a ghetto).

Also used the refactoring as a chance to fix free and aux symbols.
Encapsulated this notion in a symbol table class, which allowed me
to address outstanding issues with symbol table inheritance and inlining.
miscellaneous cleanup, mostly fighting with feature warnings
moves positions to scala.reflect.internal.util

This is the first step of factoring out scala-reflect.jar.
repairs the tests after the refactoring spree
Fixes SI-4909 and SI-5763

Finally, -sourcepath is split into:
-Ysourcepath     - for the library bootstrapping
-doc-source-path - for scaladoc links to source code

(squished the resident compiler test fix into this commit)

Review by @jsuereth.
Introduces scala-reflect.jar
backward compatibility bridges

More information can be found here:
https://groups.google.com/group/scala-internals/msg/138efc5476751269
removes array tags

Before 2.10 we had a notion of ClassManifest that could be used to retain
erasures of abstract types (type parameters, abstract type members) for
being used at runtime.

With the advent of ClassManifest (and its subtype Manifest)
it became possible to write:

  def mkGenericArray[T: Manifest] = Array[T]()

When compiling array instantiation, scalac would use a ClassManifest
implicit parameter from scope (in this case, provided by a context bound)
to remember Ts that have been passed to invoke mkGenericArray and
use that information to instantiate arrays at runtime (via Java reflection).

When redesigning manifests into what is now known as type tags, we decided
to explore a notion of ArrayTags that would stand for abstract and pure array
creators. Sure, ClassManifests were perfectly fine for this job, but they did
too much - technically speaking, one doesn't necessarily need a java.lang.Class
to create an array. Depending on a platform, e.g. within JavaScript runtime,
one would want to use a different mechanism.

As tempting as this idea was, it has also proven to be problematic.

First, it created an extra abstraction inside the compiler. Along with class tags
and type tags, we had a third flavor of tags - array tags. This has threaded the
additional complexity though implicits and typers.

Second, consequently, when redesigning tags multiple times over the course of
Scala 2.10.0 development, we had to carry this extra abstraction with us, which
exacerbated the overall feeling towards array tags.

Finally, array tags didn't fit into the naming scheme we had for tags.
Both class tags and type tags sound logical, because, they are descriptors for
the things they are supposed to tag, according to their names.
However array tags are the odd ones, because they don't actually tag any arrays.

As funny as it might sound, the naming problem was the last straw
that made us do away with the array tags. Hence this commit.
TypeTag => AbsTypeTag, ConcreteTypeTag => TypeTag

This protects everyone from the confusion caused by stuff like this:
https://issues.scala-lang.org/browse/SI-5884
quickfix for a nasty assertion
fixes some issues with reflectField brought up today
Reverting 22c8dec5 and preventing bootstapping in scaladoc

Review by @dragos, @jsuereth. Required bootstrapping because the starr was
ant tasks were invoking locker with -Ysourcepath instead of -sourcepath.