BlaineSch/li3_quality — pull request #4 by eLod (2012-12-28 14:05:20+00:00)
Title: Refactoring for better performance.

I managed to get better results with

* using `for` instead of `foreach` and completely dropping `range()` uses. on large token arrays it is very slow. maybe we should change `foreach`s into `for`s in rules also to further optimize.
* refactoring `Parser::tokenize()` and have information extracted in one place: i create a lineCache and a typeCache to index tokens by line number and type (id) respectively. meta holds the info parser collects (level, brackets), relationships hold relations.
* the line and type cache speeds up things greatly, i've noticed you also created something like this (see later)
* fixing rules: a couple rules was missing findAll (by type and or content), so they iterated over the full tokens array. also there were some other things

I tested the syntax for the whole lithium library, e.g.
`time ../libraries/lithium/console/li3 quality syntax --library=lithium --silent`

Before the modifications i got:
```
real	26m2.108s
user	26m0.723s
sys	0m1.208s
```

After:
```
real	0m35.321s
user	0m34.972s
sys	0m0.176s
```

The longest rule is now `OperatorSpacing`, which takes like 10secs, the other 21 or so rules take other 10 secs, 15 secs is parsing and other stuff.

Also there are a couple parse errors, namely: `lithium\tests\cases\core\LibrariesTest` and `lithium\data\source\database\adapter\PostgreSql`.

I've noticed you pushed some commits i haven't pulled yet and there's some conflict. I haven't had the time to revise the changes, but wanted to show you my modifications so we can talk about them.