Q:

Purpose of AtEndDocument{\def}

I am currently researching how to generate a recompilation warning from Latex if somthing in the auxfile changes and found this solution. 
However I find it rather confusing, it uses:
\AtEndDocument{\def\newreference#1#2{...}}

Which looks to me as if it simply redefines the macro \newreference#1#2 (where #1 and #2 expand to the arguments given to the surrounding macro).  However I do not know from where \newreference#1#2 is "called" (expanded?  used?)  to actually do the check. 
I tried taking a look at clsguide and at the points when the auxfile is read.  But neither was helpful (apparently the auxfile is only read at \begin{document} so this does not explain it). 

A:

Package rerunfilecheck calculates MD5 hashes (pdfTeX/LuaTeX in DVI or PDF mode)
of the auxiliary files and compares them at the very end of the document,
later than \AtEndDocument, where the .aux file is still open.  See package
atveryend for later hooks. 
Example:
\documentclass{article}
\usepackage[aux]{rerunfilecheck}

\begin{document}
\tableofcontents
\section{Hello}
\end{document}

First run:
Package rerunfilecheck Warning: File `test.aux' has changed.  Rerun. 
Package rerunfilecheck Warning: File `test.toc' has changed.  Rerun. 

A:

This is what \enddocument looks like in latex.ltx:
\def\enddocument{%
   \let\AtEndDocument\@firstofone
   \@enddocumenthook
   \@checkend{document}%
   \clearpage
   \begingroup
     \if@filesw
       \immediate\closeout\@mainaux
       \let\@setckpt\@gobbletwo
       \let\@newl@bel\@testdef
       \@tempswafalse
       \makeatletter \@@input\jobname.aux
     \fi
     \@dofilelist
     \ifdim \font@submax >\fontsubfuzz\relax
       \@font@warning{Size substitutions with differences\MessageBreak
                  up to \font@submax\space have occurred.\@gobbletwo}%
     \fi
     \@defaultsubs
     \@refundefined
     \if@filesw
       \ifx \@multiplelabels \relax
         \if@tempswa
           \@latex@warning@no@line{Label(s) may have changed. 
               Rerun to get cross-references right}%
         \fi
       \else
         \@multiplelabels
       \fi
     \fi
   \endgroup
   \deadcycles\z@\@@end}
\def\@testdef #1#2#3{%
  \def\reserved@a{#3}\expandafter \ifx \csname #1@#2\endcsname
 \reserved@a  \else \@tempswatrue \fi}

If @filesw is true (you didn't make a call to \nofiles), then LaTeX closes the main .aux file (\immediate\closeout\@mainaux), but after some redefinitions, reads the main .aux again: \makeatletter \@@input\jobname.aux. 
The redefinitions above is important in that it checks whether \labels match their definitions.  Boris' code attempts to do the same by redefining \newreference in a similar way to how LaTeX redefines \@newl@bel (via \let\@newl@bel\@testdef).