Concatenate adjacent string tokens more efficiently.

String contents were copied multiple times if three or more string
tokens were concatenated. This patch is to do copy only once.
Cleanup.
Fix a small bug in macro token concatenation.

  #define conc(x, y) x ## y
  conc(., 1)

should be expanded to a token of type pp-number.
8cc created a new token of type identifier, which resulted in
"variable '.1' is not defined" error message.
Rewrite character input stream.

The previous implementation of File depended on FILE*.
When we want to read from a string, we needed to write the string
to a temporary file to read back.

I realized that there are many ocasssions we want to read
from a string. One example is ##. A string created by token
concatenation needs to be tokenized as if it were read from a file.
We didn't handle that well.

With the new implementation of File, reading from string is
as easy as reading from file.
Add encoding property to preprocessor string token.

A string literal may be prefixed one of u, U, u8 or L.
The lexer needs to read and add the encoding information
to a token.

Currently encoding information is ignored by the compiler
and the code generator.

When a string literal with a prefix is stringnized using #,
the prefix needs to be stringized too.

You usually can't paste anything to a string using ##, but
u, U, u8 or L are exceptions. You can paste one of these
identifiers to create a multibyte string or character literal.
Move predefined macros to an external header.
Set hash table load factor to 0.7.

0.6 was a typo.
Implement include guard detection.

A header file is usually guarded by a pair of #ifndef and #endif.
Modern compilers are able to detect the pattern to not open and
read from the same file just to skip the entire file.

This patch adds the feature to 8cc.
Add tests for fullpath().
Simplify.
Fix use-after-return bug.
Fix include guard.

If there's a macro before #ifndef, the file should not be
considered to be guarded by the #ifndef.
Define __restrict__ and __restrict as synonyms for restrict.

C99/C11 defines only restrict, but popular compilers including GCC
define these synonyms (so that you can use __restrict in C++, in
which restrict keyword is not a part of the language standard).
Move alternate keywords to a header file and add a few more.
Implement better #ifndef detection.
Fix tricky include guard bug.

If the #endif for an #ifndef is in another file, the previous
ifdef guard detector would misinterpret that as the end of
the guard.
Cleanup: remove redundant initializers.

If an initializer is given, unspecified fields will be initialized
to zero, so these fields don't have to be initailized explicitly.
Simplify token buffer API.
Add tests for wide character literal.
Removed unused code.