  Machine learning has achieved tremendous success in a variety of domains in
recent years. However, a lot of these success stories have been in places where
the training and the testing distributions are extremely similar to each other.
In everyday situations when models are tested in slightly different data than
they were trained on, ML algorithms can fail spectacularly. This research
attempts to formally define this problem, what sets of assumptions are
reasonable to make in our data and what kind of guarantees we hope to obtain
from them. Then, we focus on a certain class of out of distribution problems,
their assumptions, and introduce simple algorithms that follow from these
assumptions that are able to provide more reliable generalization. A central
topic in the thesis is the strong link between discovering the causal structure
of the data, finding features that are reliable (when using them to predict)
regardless of their context, and out of distribution generalization.
