sebastianbergmann/phpcpd — pull request #95 by maglnet (2014-04-13 10:51:44+00:00)
Title: Feature: hash storage sqlite to reduce memory usage

Hi,

i had some trouble running phpcpd in an environment where memory_limit was not big enough to run phpcpd over a big codebase. So i stored all hashes within a sqlite file to reduce the memory usage.

With the provided changes it is possible to switch the default storage (internal array) to a temporary sqlite database. Of cause this makes the detection slower by using a sqlite database on the filesystem, but you can run phpcpd with a very small memory usage.

Here are some examples / comparisons running phpcpd on zendframework2 code:

Running with memory (default) storage (fast but a lot of memory used):
```bash
$ php phpcpd --progress --min-tokens=70 --min-lines=5 vendor/zendframework/
phpcpd 2.0.1-3-ge862208 by Sebastian Bergmann.

[...]

2.49% duplicated lines out of 303394 total lines of code.

Time: 12.92 seconds, Memory: 177.25Mb
```

Running with sqlite storage (slower but less momory used):
```bash
$ php phpcpd --progress --min-tokens=70 --min-lines=5 --hash-storage=SQLite vendor/zendframework/
phpcpd 2.0.1-3-ge862208 by Sebastian Bergmann.

[...]

2.49% duplicated lines out of 303394 total lines of code.

Time: 36.47 seconds, Memory: 16.25Mb
```

Additionally it is now very easy to add another HashStorageAdapter by providing a class that implements `HashStorageInterface`.

Best regards,
Matthias