No, there is no compulsion of pursing math in intermediate i.e. 11th or 12th since there are most of the universities that accept you from any stream.