As a game player probably look at mod packs rather than individual mods to change up the game. The best packs put together a bunch of different mods, tweak the resulting balance of the game and might offer a "quest" system if you're the kind of person who needs to be told what your objective is rather than making their own way. Feed The Beast linked by others is the best way to find such packs these days, they make their own but also offer others.

Somewhat like Factorio, most modded Minecraft assumes you already know what you're doing so it can be pretty daunting for a beginner. Obstacles that come to mind (but I'm far from a beginner and may well have forgotten some):

JEI ("Just Enough Items" and predecessors including "Not Enough Items", "Too Many Items") knows all the "things" that exist in your game, and usually how to get them or what to do with them, but it mostly assumes you actually knew and were just looking for a reminder. I know I can probably make an electrum ingot by melting gold and silver ore in this TiC smeltery and waiting for the alloy to form, and technically the JEI says that, but chances of a beginner going from "I need an electrum ingot" to "I need to build a TiC smeltery" are almost zero without help.

Some "vanilla" Minecraft trivia takes on a great significance in most mod packs. Knowing how to make a "Cobblegen" (a structure with lava and water which produces one or more cobblestone blocks that when mined are replaced each time) and a "Mob farm" (a place where enemies are endlessly created and, perhaps later, killed automatically in order to get the "loot" they drop) are things you'll do in modded Minecraft that you may never have needed in casual play of the original game.

If it doesn't hurt your enjoyment, try watching a "Let's Play" of a pack you think you might be interested in. The nature of random seeds and different preferences means you will end up doing something different than what you see unlike in a linear narrative game, but you can pay attention to how the gamer you watch deals with problems you had, and you will likely see techniques you want to imitate or clues to how something you didn't understand works.

As to specific mods though (packs will usually advertise which mods are used, though the pack might change how they work or when you get access to them):

AE: Applied Energistics (these days most commonly Applied Energistics 2): What if instead of storing things in chests you had a storage network to keep everything in, so you can type "Gold" hit tab and see all the gold things you own? What if this network could also be used to automate crafting, now you can type "Gold" and it'll offer things you don't have yet but the network knows how to make from what you do have? Doesn't building and debugging this network sound like fun too?

RFTools: Everything should clearly be powered by electricity. Smelting, obviously, but also potion manufacture, planting and harvesting food and wood, creating and destroying enemies, making entire dimensions to explore...

Integrated Dynamics: Obviously under the hood Minecraft is just a bunch of data you could program. Some mods expose parameters (an RFTools machine with 4059RF that needs 5000RF to do something is exactly 941RF short) but many do not, however those details must exist. ID lets you dive down there. e.g. I have built a Predicate (a boolean function that given an input returns either TRUE or FALSE) that decides whether my Astral Sorcery crystals are "finished" or not and then that Predicate is installed in a machine that pulls TRUE crystals out of the liquid starlight they're bathed in for further processing. Astral Sorcery isn't a technical mod, but ID doesn't care, there's a data structure and so you can write a predicate to examine the structure and make decisions.

Extreme Reactors (previously Big Reactors): Just the fun "optimisation" part of managing a fission power plant without the explosions and deadly radioactivity (if you want those other mods can do that, I find dying in an explosion to be unsatisfying but each to their own).