Set the source sets in just the core project.

Hopefully to avoid some issues.