Colleges do indeed consider your freshman grades, although maybe not in the way you think. They see your freshman grades holistically, as part of your overall GPA.