While brioche and challah breads are similar in taste and texture, brioche bread is made with butter while challah bread is made with oil.