I see what you mean now and I think this is a limitation/intention from github. https://stackoverflow.com/questions/13808020/include-an-svg-hosted-on-github-in-markdown

Basically because it refers to a text file that is hosted on github, it treats it as a link to the source code.