- functionalize curl calls so we can add pauses in between
- add isset checks in the $_POST in crawler.php