If any Canadians disagree with this post, just keep in mind these are my own impressions based on incomplete data. If you have an opposite opinion I would love to hear it to expand my knowledge.

Toronto, Waterloo, Vancouver, Ottawa, Montreal. All have their pros and cons.

Toronto:
My town. People who say there isn't money here are full of shit. I know someone who is raising a million without revenue or a finished product (in a solid space, and the team is awesome, but still).

Benefits of Toronto:
Lots of money, especially when you consider MaRS funding and all the angel networks around.
Solid Rails and JS scene.
Solid mobile app scene.

Problems with Toronto:
Too much social media focus. Half the startups here are twitter clones (sprouter) or twitter add-ons (crowdreel), groupon/other coupon tools (can't remember the groupon clone, but there is: clip mobile for location based coupons and pinpoint social (by orange rhino media) for coupons to help track your facebook/social media campaigns), multiple "we make it easy to find a place to rent" apps (zoocasa), there is a eventbrite clone (guestlistapp (FD: my roommate is one of the founders)). I can only think of a few startups that really invented anything in Toronto (like FreshBooks, my former employer, or Bumptop, I guess).
There is also a lack of really hardcore hackers. Maybe I'm just going to the wrong events, but when I was studying at Waterloo, there were so many people soldering together circuit boards AND writing their own operating systems for them. In Toronto I've only really scene someone program together a computer driven laser cutter.
Oh and the banks eat up a good portion of really smart hackers with their 20 year old data warehouses.

tl;dr: too much social media & finance, too little hard core hackers outside of the mobile and webapp space.

Waterloo:
I studied engineering at U of Waterloo. Some insanely smart people. Smarter than you can imagine. If you hang out in the right places you'll find guys that were patching the kernel at 14. Amazingly smart and connected profs too, only too willing to help you hire alumni and coop students.

Waterloo pros:
Extremely talented hackers.
With two premier universities, as well as a world class research center (perimeter institute) and a focus on education the whole city feels smart.
CHEAP rent. Extremely cheap. I knew a guy that rented a room for $200 a month, utils included.

Waterloo cons:
The public transit isn't the greatest. Doesn't really matter because the town is small, but a pain to get groceries. Definitely a car town.
I've never really seen anyone get funding for anything, besides university expansions. $50k here or there until they are good enough to make the trek to Toronto (about 1.5 hour drive away).
No where near the community of Toronto. In Toronto I could find somewhere to go every single week. Rails night every third monday, JQuery night once a month, Democamp, Podcamp, hacklabs, Open Web standards night, Toronto Hacker News Night (by yours truly). There is nothing like that in Waterloo.

Ottawa:
My mom's side of the family is from there. I've spent many summers hanging out near or in Ottawa.

Ottawa pros:
It is fun. Clean. Well laid out.
Enough hackers, no where near as good as Toronto or Waterloo (or Vancouver for that matter), but damn are they cheap. Most of the coders I know there make between $30k - $60k. Contrasted to Toronto where a really, really good rails hacker can make about $130k and where most fall in the $60k to $100k range.

Ottawa cons:
Tons of political contracts and work. No real innovation. A couple really good startups though, like Shopify.

Montreal:
Some family lives in Montreal, but I don't know that much about it. Been there many times, almost got engaged to a girl that is fairly influential there, but she wasn't in the tech space.

Montreal pros:
Money.
Game dev community, so you get some really talented coders.

Montreal cons:
Everything feels run down. People can call it European, but that is ONLY in the historical and art district. Everywhere else feels halfway between Toronto and Detroit. Their highways are so bad (gigantic potholes) the speedlimit has been lowered to 60km an hour, for what clearly should be a 100km/h road.
Make sure you speak French (or at least TRY, they want to see you try) if you want to go to certain areas, otherwise you just get bad service from the hardcore separatists.
Also, compared to Ontario drivers, Montreal drivers are really mean.

Vancouver:
Lived here for about a year and a half. Was too young to work at the time so I can't comment on much besides the non-startup factors.

Vancouver pros:
Extremely lax. Feels like Seattle, but prettier and with more weed. (I list weed as a pro, despite the fact that I do not partake strickly because to most people it would be).
Very beautiful. Just amazing mountains and valleys and rivers and fish and bears. Everything is nice about nature in Canada is there.

Vancouver cons:
Expensive. Try buying a livable place there. Almost NY prices, without the NY salaries to compensate. Highly regulated government, Ontario is much more business friendly.

Overall I recommend Toronto if you are a web startup or Waterloo if you are in a different startup space. You say you want low costs? Go to Waterloo and if you need to get capital you can always drive to Toronto.