nocko/Cornell-Spider-for-Unix — pull request #1 by NJnoc (2012-07-24 13:35:15+00:00)
Title: Added multi-processing and line scanning.

Added the following features:
- The engine is capable of running block scans and line scans.
- Block scans are for faster scans and are done on 64K blocks of the file.
- Line scans are for more detailed scans and return the exact line number
of the suspicious content. This is CPU intensive and slow.
- The engine checks for the number of cores on the machine one which it will
run and fork into (cores-1) number of processes making the scans faster.
- Added a simpler Luhn's algorithm to validate credit cards.
- Added a new bash script that will can scan NFS mounts from a list of servers.

Details:
spider.conf has the following new directives:
- dolinescan: 0 - disable line scan, 1 - enable line scan
- hostip: IP of the host to be scan. Used when remote systems are scanned using NFS mounts.

scan.sh:
- This is a script to scan remote systems based on the NFS mounts they export.
- It takes a text file with a list of IPs to scan.
- Creates log files for every IP being scanned.
- Cleans up the mounts after the scan.

Please review and push.

Thanks,
Steve William