Move library source under src dir

I've been meaning to do this, and since I'm already shuffling things
around...