My own system is a 486DX2-66 with 12MB of physical memory and 12MB of swap space. All the Linux kernels which come with distributions are quite bloated and contain more features then any of us will ever need or use. If you have not re-compiled your own kernel it is highly recommended that you do so. How to re-compile a kernel is beyond the scope of this document but many excellent Linux books and guides cover this subject in intimate detail. If you do re-compile your kernel remember to put in no more features then you need.
Smaller kernels require less time to load, use less memory and use less CPU cycles.
Another thing is modules. I personally do not use them because I found them to be a cumbersome. Virtual Consoles VCs are a great way to free up memory. Most Linux distributions run about 6 of them out of the box. On average running 6 VCs requires about 4MB of memory.