Q:

Accurate Session Tracking in ASP.NET MVC

Whenever a user hits a page on my website, I run the following code to track user hits, page views, where they are going, etc...
    public static void AddPath(string pathType, string renderType, int pageid = 0, int testid = 0)
    {

        UserTracking ut = (UserTracking)HttpContext. Current. Session["Paths"];

        if (ut == null)
        {
            ut = new UserTracking();
            ut. IPAddress = HttpContext. Current. Request. UserHostAddress;
            ut. VisitDate = DateTime. Now;
            ut. Device = (string)HttpContext. Current. Session["Browser"];

            if (HttpContext. Current. Request. UrlReferrer ! = null)
            {
                ut. Referrer = HttpContext. Current. Request. UrlReferrer. PathAndQuery. ToString();
                ut. ReferrerHost = HttpContext. Current. Request. UrlReferrer. Host. ToString();
                ut. AbsoluteUri = HttpContext. Current. Request. UrlReferrer. AbsoluteUri. ToString();
            }
        }

        //Do some stuff including adding paths

        HttpContext. Current. Session["Paths"] = ut;
    }

In my Global.asax.cs file when the session ends, I store that session information.  The current session timeout is set to 20 minutes.  
    protected void Session_End(object sender, EventArgs e)
    {            
        UserTracking ut = (UserTracking)Session["Paths"];

        if (ut ! = null)
            TrackingHelper. StorePathData(ut);
    }

The problem is that I'm not getting accurate storage of the information.  For instance, I'm getting thousands of session stores that look like this within a couple minutes.  
Session #1
Time: 2014-10-21 01:30:31.990
Paths: /blog
IP Address: 54.201.99.134
Session #2
Time: 2014-10-21 01:30:31.357
Paths: /blog-page-2
IP Address: 54.201.99.134
What it should be doing, is storing only one session for these instances:
What the session should look like
Time: 2014-10-21 01:30:31.357
Paths: /blog,/blog-page-2
IP Address: 54.201.99.134
Clearly, this seems like a search engine crawl, but the problem is, I'm not sure if this is the case.  
1) Why is this happening? 
2) How can I get an accurate # of sessions to match Google analytics as closely as possible? 
3) How can I exclude bots?  Or how to detect that it was a bot that fired it? 
Edit: Many people are asking "Why"
For those of you that are asking "Why" we are doing this as opposed to just using analytics, to make a very long story short, we are building user profiles to mine data out of their profile.  We're looking at what they are viewing, how long they are viewing it, their click paths, we also have A/B tests running for certain pages and we're detecting which pages are firing throughout the user viewing cycle and we're tracking some other information that is custom and we're not able to put this into a google analytics API and pull this information out.  Once they've navigated the site, we're thing using this information to build user profiles for every session on the site.  We essentially need to then detect which of these sessions is actually real and give the site owners the ability to view the data along with our data mining application to analyze the data and provide feedback to the site owners on certain criteria to help them better their website from these profiles.  If you have a better way of doing this, we're all ears.  

A:

1) the asp.net session is tracked with the help of the asp.net session Cookie. 
But it is disabled for anonymous users (not logged on users)
You can activate sessionId creation for anonymous user's in the web.config
<configuration>
  <system.web>
    <anonymousIdentification enabled="true"/>
  </system.web>
</configuration>

A much better place to hook up your tackin is to add an global mvc ActionFilterAttribute. 
The generated SessionId is stored in the httprequest, accessed by
filterContext. RequestContext. HttpContext. Request. AnonymousID

2) You should create a feed of tracking paths to analys it asyncronly or not even in the same process.  Maybe you want to store the tracking on disk "like a Server log" to reanalyse it later. 
Geo Location and db lookup's needs some processing time and most likly you cant get the accurate geo location from the ip address. 
A much better source is to get it from the user profiles / user address later on. ( after the order submit)
Sometimes the asp.net session cookie don't work, because the user has some notracking plugin activated.  Google Analytics would fail here too.  You can increase the tracking accuracy with a custom
ajax Client callback. 
To make the Ajax callback happen globally for all pages, you can use the help of the ActionFilterAttribute to inject some Script-Content to the end of the html content stream Response. 
To map an IPv4 address to a session can help, but it should only be a hint. 
Noadays a lot of ISP supporting IPv6.  They are mapping there clients 
most of the time to a small IPv4 pool.  So one user can switch its ipv4 very fast
and there is a high possibility that visitors of the same page are using the same ISP and so share a IPv4. 
3) Most robots identify themselves by a custom user agent in the request headers. 
There are good and bad ones.  See http://www.affiliatebeginnersguide.com/articles/block_bots.html
But with the Ajax callback u can verify the browser presents, at least the present of a costly 
html-dom with JavaScript Environment. 
X) To simplfy the start and concentrate on the Analysis.  Implement a simple ActionFilterAttribute
and Register it globaly in RegisterGlobalFilters
filters. Add(new OurTrackingActionFilterAttribute(ourTrackingService));

In the filter override OnActionExecuting
public override void OnActionExecuting(ActionExecutingContext filterContext)
    {
        base. OnActionExecuting(filterContext);

        OnTrackingAction(filterContext);
    }

public virtual void OnTrackingAction(ActionExecutingContext filterContext)
    {
        var context = filterContext. RequestContext. HttpContext;

        var track = new OurWebTrack(context);

        trackingService. Track(track);
    }

To don't delay the Server Response with some tracking processing,
take a look into the Reactive package http://msdn.microsoft.com/en-us/data/gg577609.aspx
It's a good way to split the capture from the processing. 
Create a "Subject" in the TrackingService and simple push our tracking objects into it. 
You can write observers to transmit, save or process the tracking objects. 
As default the observers will only get one object at a time and so you dont need to syncronise/lock your status variables/Directory/memeory-cache and maybe u want to load the data and reprocess it with a new version of your application later on (maybe in debuging).