Just follow the usual path for education in relatively applied math.

Here is a nutshell description:

The standard high school level subjects are algebra, plane geometry, second year algebra, trigonometry, and solid geometry.

The standard college level subjects are calculus, abstract algebra, linear algebra, advanced calculus, ordinary differential equations. Might also take elementary courses in probability and statistics.

Standard graduate school topics are point set topology, measure theory, functional analysis.

For more in applications, can do (1) optimization -- linear, network linear, linear integer, quadratic, non-linear, and dynamic programming -- and (2) probability, statistics, and stochastic processes based on measure theory.

Here is an overview:

The course in abstract algebra will make you familiar with a long list of topics that come up in computing including sets, Boolean algebra, logic, relations, mappings, integers, prime numbers, modular arithmetic, rational numbers, the fundamental theorem of arithmetic, the fundamental theorem of algebra, elementary number theory, and the Euclidean greatest common divisor algorithm. You will also see finite field theory which has long been important in algebraic coding theory. You may touch on the question of P versus NP. You may see some of the work of Goedel, etc. and model theory. You will get good with math based on definitions, theorems, and proofs and develop 'mathematical maturity', i.e., ability to read, understand, and do abstract mathematics. Your ability to describe logical material in writing will improve.

Famous authors in abstract algebra include Birkhoff, MacLane, Herstein, Lang. More recent authors likely touch on algebraic geometry.

Linear algebra, mostly about 'vectors' and linear transformations, is the core of an ocean of applications in multivariate statistics (regression, factor analysis, analysis of variance, discriminate analysis, and more and, thus, applications in ad targeting, machine learning, data mining, anomaly detection, recommendation engines, etc.), optimization, group representation theory as in molecular spectroscopy, Shannon's information theory, and more. E.g., linear algebra is the place to learn about solving systems of linear equations, e.g., by Gauss elimination, and, thus, the place to get started with matrix inversion and the simplex algorithm of linear programming (optimization) and network linear programming.

A good first text in linear algebra is by Ben Noble. The crown jewel is by Paul Halmos, 'Finite Dimensional Vector Spaces', written when Halmos was an assistant to von Neumann; the book approaches linear algebra much like it was functional analysis and, thus, is a baby version of Hilbert space theory and, thus, sometimes used to teach Hilbert space theory to physicists for quantum mechanics. Richard Bellman wrote tons. See also Roger Horn's books. Evar Nering is a good first book (although his treatment of linear programming is not good); so is Hoffman and Kunze (apparently now available on the Internet in PDF for free). For numerical linear algebra, see Forsythe and Moler.

The standard advanced calculus text to teach the theorems and proofs of calculus is Walter Rudin's 'Principles of Mathematical Analysis'. After that book, notation such as O( n ln(n) ) as in Knuth's TACP will be child's play.

Note: At one time the Halmos 'Finite Dimensional Vector Spaces' and Rudin's 'Principles' were used in Harvard's Math 55 with a colorful description in

http://www.american.com/archive/2008/march-april-magazine-contents/why-can2019t-a-woman-be-more-like-a-man/?searchterm=Sommers

A course in trigonometry, the use of trigonometry in calculus, and Rudin's treatment of Fourier series will help in data compression (e.g., JPG), time-invariant linear systems, digital filtering, and Shannon's information theory.

Also good coverage of the binomial theorem in high school or abstract algebra and an elementary course in probability will be a help in following Knuth's TACP.

Measure theory will take a huge load off your back: Calculus, that is, Riemann integration theory as taught through Rudin's 'Principles', has some rough edges, understood back to at least 1900, and Lebesgue and Borel and measure theory (heavily due to Lebesgue) provide a clean solution. As from Kolmogorov, the solution also makes a clean foundation for probability, stochastic processes, and statistics.

Likely the nicest first text in measure theory is Royden's 'Real Analysis'. Also good is the first, real, half of Rudin's 'Real and Complex Analysis' where he also gives nice introductions to both Hilbert and Banach spaces and a nice treatment of the Fourier transform.

My favorite authors in probability based on measure theory are J. Neveu (long in Paris) and L. Breiman (long at Berkeley). Or just read from their teacher, M. Loeve (long at Berkeley but with a writing style that looks like some cross between English and French).

This background will also let you do much more in both pure and applied math in many directions and for many applications.

For learning math, it is "not a spectator sport". Most of the work is between your ears as you think about the material. A good teacher in abstract algebra is likely necessary to get you started. Also without at least occasional contact with a solid university department it is too easy to get off track. Still, nearly all the work is to be done alone in a quiet room, and there a book is fine. In principle videos could help, but so far I've never seen even one that I would recommend for any utility at all in learning math.