There are a few ways to drop items in Oblivion. The first way is to simply press the 'drop' key (default is 'G'), which will cause your character to automatically drop whatever item they are holding. You can also right-click on an item in your inventory and select 'drop', which will do the same thing.
If you want to be more precise with your drops, you can left-click on an item while holding down the 'shift' key. This will bring up a cursor that you can use to place the item exactly where you want it.