Maybe something funny is going on? My simple console experiment works as expected (for Javascript-levels of expected):
```
_.template("<%= a || b %>")({a: 'a', b: 'b'})
"a"
_.template("<%= a || b %>")({a: undefined, b: 'b'})
"b"
_.template("<%= a || b %>")({a: undefined, b: undefined})
""
```