Settings api cleanup

Progress so far on the settings api changes of the GSoC project.

- [x] Settings new api
  - [x] Settings priorities
  - [x] Backward compatibility support
  - [x] Testing
  - [x] Documentation
  - [x] Interface unification across code

Settings freeze support and per-spider settings will be implemented on separate pull requests.