Question: Should I write it like this? Or perhaps like this? Should I write it like this? or perhaps like this? What about after an exclamation mark or semicolon? 
Answer: Did the sentence end after the question mark? Yes. Then you need a capital letter to start the new sentence, just as usual. No. Then the question mark shouldn't have been there, since it ends a sentence. Right, I should expand on that rather than just be a grumpy old man. The capitalization rule that we care about here is that the first word of a sentence starts with a capital letter, so the question is really about what ends a sentence. The answer to that is easy: terminal punctuation, i.e. a full stop (or period if you're American), question mark or exclamation mark. There's a visual clue in that '?' and '!' are decorated full stops; you just have to remember that a colon (':') isn't really a decorated full stop, not that you'd ever know by looking at it. Colons, semicolons and commas aren't terminal punctuation, so they don't end a sentence and so don't force the next letter to be a capital. It may be a capital letter for some other reason such as being the start of a proper name, but not because it is starting a sentence. There are exceptions to this rule, occasions when '?' and '!' become non-terminal punctuation. The most obvious is in quoted speech: if the speaker asks a question or makes an exclamation, the '?' or '!' doesn't have to be terminal if the sentence carries on after the quote. "Should I write it like this?" he asked. "Or perhaps like this?" The other class of exception is for what are probably really parenthetical comments. If you have a short phrase that you could have put aside in parentheses or dashes, then a question mark or exclamation mark can be used at the end of that phrase without ending the sentence. Be sparing with this. It looks wrong at a first read, and engenders the sort of argument I had with Jason in the comments below! Should I write it like this, or abracadabra! like this? Going back to the examples, and adding a few more for fun: Should I write it like this? Or perhaps like this? Should I write it like this? or perhaps like this? Should I write it like this, or perhaps like that? "Should I write it like this?" he asked, "or perhaps like that?" Example 2 with the lowercase "or" is just plain wrong. Crusty old grammarians who disapprove of starting sentences with conjunctions may frown at example 1 all they like, but it's a perfectly acceptable fragmentary sentence. Whether it's the right answer or not is another question entirely. Example 1 makes the point that the questions are distinct, though they are strongly linked otherwise the whole structure wouldn't work. Example 3 on the other hand emphasizes that the two questions are options in a common situation, as well as reflecting a different way of saying them. That is obvious in this case because the two questions really are tightly coupled alternatives. However, consider the following: Are the lights green? Or is the switch up? Are the lights green, or is the switch up? Both of these examples imply that the state of the lights and the state of the switch are related somehow. Version 2 couples them more tightly; I would usually assume (without more context) that either this is the same question being asked in two different ways (i.e. that the switch being up should cause the lights to be green), or that they are an exhaustive list of possibilities (either the switch is up or the lights are green, but not both or neither). This isn't an absolute rule, but it is quite strongly implied. Example 4 is also wrong, though it has a better disguise. If you unwrap the quotes, what you get is: Should I write it like this? or perhaps like this? which is our duff example 2 back again. What you actually want is one of: "Should I write it like this?" he asked. "Or perhaps like this?" (i.e. example 1) "Should I write it like this," he asked, "or perhaps like that? (i.e. example 3) depending this time on how much of a break there is supposed to be in the middle of what was said. But speech-marking is a topic for another question :-) Exclamation marks work just like question marks for this purpose! Semicolons don't; they end a clause, not a sentence.
Output: yes