Arguably, there are acids that are more efficient but oxalic acid is specifically sold for the purpose of rust/stain removal. In fact I have a large 2kg container of crystalline (100%) oxalic acid that I bought at the local hardware store and it's labelled 'Rust and Stain Remover'. It's one of the chemicals that I deem essential to keep around my house.

One of the advantages of oxalic acid is that it's a sufficiently strong acid but still mild in comparison to many others, notably the common inorganic ones. I've not thought much about alternatives as its cheapness, easy availability and its suitable chemistry makes it a suitable choice (someone else's already done the hard thinking ;-)). Being a dicarboxylic acid—here the simplest, just two carboxyl groups bolted together—it's reasonably strong, stronger than say acetic acid but mild enough that it doesn't suffer the drawbacks of the easily-available inorganic acids. For instance, if you forget and leave the work in solution too long then not much damage is done. For instance, compare it with phosphoric acid that one often finds in vehicle rust-removal kits. I once left a large collection of rusty twist drills in a phosphoric acid solution and although it removed (converted) the rust admirably, it actually changed the crystalline structure of the HS steel in the drills to such an extent that they became brittle and broke easily (that's sort of obvious, had I bothered to think about it I wouldn't have done it). So one leaves the phosphoric for heavy-duty rust removal like when you can't tell any difference between the metal and the rust in the floor of your car (and besides technically it's more a converter than a remover).

Being comparatively mild, oxalic is also reasonably kind to other organic materials such as wood and you won't get the 'charring' damage of say sulfuric acid (no self-respecting woodworker would be without some oxalic in the workshop). For instance, the ugly bluish-black stains wood gets from being near iron, nails etc. can often be completely removed with oxalic acid. It's often combined with chlorine bleach/Na hypochlorite when there are stains from multiple sources, mold for instance (Cl works better here).

You're right about oxalic acid being toxic but one would have to be damn stupid and careless to get an LD50 dose of it, which if I recall, that figure is in excess of ingesting somewhat more than 10g. That would be rather difficult unless one did it intentionally. It's important to realize that we have a tolerance to small quantities of oxalic acid as it occurs naturally in most of our vegetables, spinach and rhubarb for instance (that's why one's told not to eat rhubarb's leaves as the acid's concentration there is much higher than in its stems, also those prone to kidney stones are told not to eat foods high in oxalic acid due to the formation of solid oxalates).

I've been using oxalic acid for years without any trouble and don't take anything other than sensible precautions when using it. In fact, I have a 500ml bottle of saturated solution (at room temp.) with a trigger pump on it that I squirt onto rusty things as well as wood stains. I'm just careful to ensure that I wash everything afterwards. Incidentally, if you're preparing an oxalic bath (say several liters of warm water with oxalic in it to remove rust off an object), then unless you do it outdoors you'll likely notice a tiny amount of bitterness in your mouth and your teeth may feel like when you bite into a lemon from the slight 'volatility' of the solution (you've probably notice the same effect when working in the lab with sulfuric and hydrochloric acids). At most, you are unlikely to be getting more than a milligram at most (which is about the amount of oxalic acid in a reasonable helping of spinach and it's not likely to be harmful unless it's a regular occurrence/part of your job etc.). If this is a worry for you or you dislike that acidic taste then wear a chemical-removing mask (the charcoal type should suit).

(BTW, it's important to note that ethylene glycol (antifreeze) is very toxic because it's both sweet and it's metabolized by the body first to glycolic acid finally to oxalic acid. I've used this oxalic acid example (along with the ethanol/methanol one) to explain to people why one shouldn't make simple assumptions about the toxicity of a chemical based on a closely related one. Here for example, its close relative propylene glycol is essentially harmless; it even has a food E number. If one came at it from only knowledge of propylene glycol and made similar assumptions about its simpler ethylene cousin then one could get into very serious trouble (unfortunately, it's happened more than once.)