Oh man, if ever there was a topic where "advice = overgeneralization + limited life experience", this is it. I can share bits and pieces of my own life story, but take any advice I give with a grain of salt. People are different; what worked for me may not work for your brother.

Like your brother, I did well in the early grades. But when I reached age 11, I completely rebelled, went on "strike", and refused to do any work. I didn't want to be successful, because I realized how empty conventional definitions of success are. By 13, I refused to go to school at all. I just walked out of the building when they made me - had the cops called on me several times.

The best decision I ever made in my lifetime was to go to a brand-new charter school for high school. I didn't want to at first - I was incredibly cynical at that point, and believed that every place would be the same and the world just sucked. (My dad is like your mom in some respects, though he has redeeming qualities as well.)

But when I got there, I found that was I was surrounded by people who, when faced with a problem, said "Let's fix it" instead of "The system sucks, so why should I bother doing anything?" Over time, that rubbed off. It didn't happen immediately - it was about 3-4 years before I said to myself "I want to be like them", and about 10 years before I actually started acting like them. I'm still working on it.

I guess one piece of "advice" I'd give you is to do your best to surround your brother with people who are actively choosing to make the world a better place. This is hard - most of us forget how small our worlds were when we were in school. It's hard to choose the sort of life you want to live when you're not given a lot of choice in anything.

Something else to think about:

By the time I was 16, my parents and nearly all the adults in my life had given up on me, and gave me a free pass to not do any work. Remember how the Randy Pausch lecture said "When you screw up, and people close to you don't tell you you've screwed up, that's a very bad sign, because that means they've given up on you"? Well, for me, it was actually the most liberating thing in my life. Because it meant that whatever I did, I was doing for me, and not to please somebody else's expectations of me. (Okay, I did go to college to please my parents, and that was nearly a disaster...) Once I'd thoroughly demolished everyone's dreams of me becoming a famous mathematician or physicist, I could do what I wanted to do, and do it with far more passion than if I was just fulfilling my parent's broken dreams.

As much as you want your brother to succeed, it has to come from him. It's his life to lead, and his life to screw up if he chooses. You can point the way, but you can't drag him there.

So - I dunno what advice to give you. I don't think I'm even qualified to give advice on this. I guess if it were my kid, I'd try to show him just how big the world is outside the gates of the schoolyard. There're so many niches, professions, and hobbies that you don't even know about when you're in high school, and many of them can be quite exciting. Get him into an internship or two, or a non-McDonalds job. Introduce him to folks in different fields. Let him surf the web and explore what's out there.