As a collector of different systems, I can assure you that d20 is one of the worst systems ever created. Most people who've played a handful of other systems will prefer one of those systems to d20 in basically every scenario. The system is only popular because it is popular.

More specifically, d20 lacks a useful skill system. You can do 95+% of game mechanics from a skill system. I don't mean a handful of stuff you roll your d20 against, but something much more comprehensive. Some skills are harder to learn. Some require other skills. Some have a weakest link (eg, a surgeon skill depends on both having skilled hands and good doctoring skills). A good framework for this means your system has fewer balance issues.

Along those same lines is the effect system. In d20, a fireball spell and a grenade are very different things. They have to be independently balanced and adjusted until they "feel right".

In an effects system, you would understand that both are explosions with a specific radius, damage (with falloff), lingering fire damage, etc while some other parts are different like what limits the number of shots (magical energy vs how many grenades you can carry). The effects system would allow you to calculate the exact value of both of these things with some slightly different modifiers and different descriptions of what they look like in the game. The result in-game feels different, but is now mechanically balanced and (ideally) cost balanced too.

Next, Vancian magic is by far the least interesting magic and worst for basically any story. Players must make a blind choice and then get either a cake walk or extreme punishment based on that choice. It causes a kind of "when you are a hammer" issue as well. It also doesn't jive with what most people expect of magic.

Better systems are more flexible. They allow the user to decide which spells to use on the fly with some kind of resource limiter instead (eg, the classic mana or how long it takes to cast or how hard a spell is to cast vs how many cast dice, threshold magic where spells get riskier to the caster as you use more of them, etc).

A final aspect is tactics vs resource management. RAW (rules as written) modern d20 is a resource management game. Your team slugs it out with the other team until someone's HP runs low. You expend a resource to raise the HP and keep slogging. Rinse and repeat until there's no more way to raise the HP at which point things conclude (never mind the whole suspension of disbelief that you continue to hack the same thing over and over and over). Yes, there are better and worse choices here, but they are not of particular consequence (and tend to become obvious rather quickly). That is why DMs do battle statistics calculations instead of caring so much about stuff like positioning.

Tactical games are much more interesting. In those games (like in real life), positioning and strategy is all-important. Once someone is in your gun sights, they are almost certainly very wounded or dead. When a GM for these games is planning, they aren't worried about statistics so much as where the enemy is, why they are there, what their strategy will be, etc. This immediately shifts the focus of the game mechanics away from statistics and back to role playing. Combat will be a lot of working together on a plan and then the actual combat won't be super long because life and death results happen pretty quickly.

If you were to go with an actually different system, OpenD6 would be the place to start in my opinion (Star Wars D6 wrote a large part of the current canon and used an early version of this system). It provides a very useful skill framework that is easily extended. Because the dice mechanic is extensible in MANY different and interesting ways, it is far easier to match your particular mechanical choices to your specific universe. The default magic system is quite flexible and uses an effects system to build spells. Instead of sticking with the list of pre-approved spells, you can have your players cook up their own and be pretty sure that the new spell isn't going to break the game. And of course, the system isn't quite as lethal as some others, but it is still MUCH more lethal than d20 by default.