force the DMG file extension in git-annex.rb

For non-Mavericks installs.  Also add needed nested_container
stanza.  Fixes #3650.