Lots of people here are much deeper in key-value stores, Redis, etc. than I am. So, here I outline what I wrote for a key-value store for a Web server session state store long ago and am still using and ask for expert comments on any pros/cons.

So, for my Web site, (A) when I first send a Web page to a user, I send in the HTML of that page, in an encrypted, hidden field, a key that identifies the user's session. The data I want to keep on the user's session I make a value and then write the key-value pair to my Web session state store; (B) when a user does an HTTP POST back to my Web server(s), I get the user's key and from my session state store get the value that is the user's session state. Each such value is just a byte array that is the serialized instance of my session state class, the instance particular to that user.

My session state store is just some simple .NET Framework software running as a Windows console application. The core of the store is just two instances of the standard .NET collection class, hopefully as fast as AVL trees (as in Knuth, Sorting and Searching) or red-black trees, IIRC, in Sedgwick. One collection class instance holds the key-value pairs. The other collection class instance holds for each key the time of the last access to the key-value pairs and is used to implement session time outs. The communications with the Web servers is just via simple TCP/IP sockets sending/receiving byte arrays.

The session state store is single threaded; so, there are no issues about, or efforts for, concurrency.

Then it would be good for the session state store to be sufficiently fast for the Web site and also to have a FIFO (first in, first out) queue of incoming requests. TCP/IP provides the desired FIFO queue, and, for the FIFO queue length, I'm setting that with just the standard option for TCP/IP.

So far, the session state store is all in main memory, but, of course, that memory might page as in virtual memory.

I'm guessing that there is more computing just for the TCP/IP than for the core of the session state store itself, i.e., the two collection class instances. If so, then as long as I'm using just TCP/IP communications, e.g., over my server farm LAN, the speed of the session state store code becomes a secondary issue -- the whole session state store operation as seen by the Web server(s) might not be much faster even if the session state store ran in 0 time. Maybe.

So far with my server farm software architecture, it should be easy to have as many executing instances of the session state store as needed for Web site performance and with no user affinity between a particular user and a particular Web server. There would be affinity between a particular user and a particular session state store via whatever Web server instance the user most recently got assigned to via load leveling.

Q 1. Is there a fundamental and serious flaw in my design?

Q 2. Would Redis actually be much better for me?

Q 3. If my Web site becomes very busy and has me using, say, a full rack of servers just for session state store, should I look for a still faster way to do the work?

Q 4. Maybe solid state disks (SSDs) are now so fast that I should just program my session state store to use a solid state disk instead of main memory -- e.g., can get SSDs of several terabytes, and that much main memory would be much more expensive. Of course, this assumes that the LAN and software for the TCP/IP stack are fast enough that the performance bottleneck just the work on the collection class instances and that, even with the rest of the assumptions, and SSD would be fast enough -- sounds like a strange situation.

Q 5. Should I plan on using a 10 GBps LAN for communicating with a session state store? That is, might such a fast LAN significantly reduce the latency of the communications between the Web server(s) and the session state server(s)? Have the Web server wait less time for the read/write with the session state store might speed up the whole server farm. Anyone have any actual experience about such speeds?

Thanks!!