Q:

Is there any performance difference between  and  when used to create link? 

  <body>
        <a href="index.html">Home</a>
    
        <form action="index.html" method="get">
            <button>Home</button>
        </form>        
  </body>

Is there any logical difference between the two anchors created above?  Any performance difference or how GET method is invoked? 
I looked the request/ response headers for both anchors, and they are almost same, only extra '?'  in request URL when <button> is used. 
Request URL:http://localhost:30272/index.html
Request Method:GET
Status Code:304 Not Modified
Remote Address:[::1]:30272
Referrer Policy:no-referrer-when-downgrade
Accept-Ranges:bytes

Request URL:http://localhost:30272/index.html? 
Request Method:GET
Status Code:304 Not Modified
Remote Address:[::1]:30272
Referrer Policy:no-referrer-when-downgrade
Accept-Ranges:bytes

Edit 1: While trying to understand the GET in-depth, I encountered the following statement in RFC documentation. 

A payload within a GET request message has no defined semantics;
  sending a payload body on a GET request might cause some existing
  implementations to reject the request. 
  https://tools.ietf.org/html/rfc7231#section-4.3.1

Does invoking GET methods using <button> make any difference in caching and/or proxying?  
Edit 2: As <button> is not intended to create anchors, is it more vulnerable to HTTP header injection? 
Edit 3: The Wikipedia article on CGI says,

In brief, the CGI program receives HTTP forms data via Unix/Linux
  standard input, and most other data (such as URL paths, URL arguments,
  and HTTP header data) via well-known Unix/Linux process environment
  variables. 

Does it mean GET request from <button> and <a> will be received differently by CGI program? 

A:

Is there any logical difference between the two anchors created above? 

Yes: only one of them is an anchor.  The other is a form, not an anchor.  An anchor is a hyperlink, and a form is a form.  Since they are logically different, performance cannot be compared. 
But, ostensibly, that's not what you're asking here.  You're asking if a hyperlink navigation and a form submission over HTTP GET are indistinguishable when no form data is involved, and the answer to that is yes. 
Before I go on answering the rest of your questions, I might as well state right now that if you really want to do like Neopets did in the late 90s/early 00s and use buttons in forms for links, then just be wary that search engines aren't going to have an easy time following your links.  Certainly not any easier than if you just used <a> elements. 

Does invoking GET methods using <button> make any difference in caching and/or proxying? 

No, it does not cause different caching behaviors compared to hyperlink navigation.  Only form submissions over HTTP POST are cached differently (usually, not at all). 

As <button> is not intended to create anchors, is it more vulnerable to HTTP header injection? 

No, unless your form accepts user input or is otherwise so poorly crafted that the URL it links to is itself vulnerable, which is not what is being discussed here. 

Does it mean GET request from <button> and <a> will be received differently by CGI program? 

Yes, if the <button> has a name attribute designating it as form data.  But if the CGI script doesn't do anything with that form data, then the difference is irrelevant.