added unit tests for data processing