The key is to have an investment strategy. That sounds very simple, but in my experience most people stumble into a portfolio. Or, to mix metaphors, they collect mutual funds and stocks as if they were butterflies. It requires some exercise in thinking about asset allocation and risk. Once you've decided on a particular variety of fund, you should look at a number of things.
Costs are a good starting point for evaluating funds.
Get the odds in your favor by picking from a pool of low-cost funds. If your account is taxable, be very careful to get a fund that doesn't generate a lot of taxable income and capital gains. I'm also a big believer in buying fund families. It's a little bit unfashionable these days, given the development of fund supermarkets. But my experience has been that good organizations breed consistently above-average funds.