Black Hole Machine Learning. Neural Networks vie Theoretical Physics | by Amelie Schreiber | Towards Data Science

Hosted on Sketchviz
# http://www.graphviz.org/content/clustergraph G {    node [style=filled,color=pink];    q0 -- H0 -- U -- V0 -- W0 -- A;    q1 -- H1 -- U -- V1 -- W0 -- B;    q2 -- H2 -- U -- V2 -- W1 -- C;    q3 -- H3 -- U -- V3 -- W1 -- D;    q4 -- H4 -- U -- V4 -- W2 -- E;}
In this article, we provide some links to significant and interesting code and software for using tensor networks, a mathematical tool used in gravitational physics and the study of black holes. The code we will be interested in will involve machine learning and applications to AI. We will go over a brief history of tensor networks and explain a little bit about what they are, and we’ll explain how they connect theoretical physics to machine learning and give some insight into why giants like Google are hiring physicists like Leonard Susskind to consult on their AI and machine learning algorithms. We’ll also show how quantum computing plays a role and how using quantum computers to create quantum enhanced AI and quantum machine learning algorithms forces us to rethink the way we understand information processing.
Earlier this year (June 2019), the AI blog for Google released a post about a library called “TensorNetwork” built for its infamous TensorFlow machine learning platform. It was built for the manipulation of tensor networks (we will go into a brief history of tensor networks and explain what they are below). Google’s work on developing this library was done in conjunction with X, Google’s moonshot factory for innovative tech, and the Perimeter Institute for Theoretical Physics. The Perimeter Institute was involved in the imaging of a black hole earlier this year. Along with their release of the code on Github, they wrote three research paper on applications and performance benchmarks for the TensorNetwork library:
TensorNetwork: A Library for Physics and Machine Learning
TensorNetwork on TensorFlow: A Spin Chain Application Using Tree Tensor Networks
TensorNetwork for Machine Learning
If you are interested in reading the documentation for TensorNetwork and digging into the code you can check out the Read the Docs page. So, why all the fuss? What’s so special about tensor networks, and how do black holes have anything to do with machine learning? What are... tensor networks?
Tensor Networks are a graphical notation for tensors used by physicists at least as far back as the 1970s by Roger Penrose in his graphical notation proposed in the paper in the paper “Applications of negative dimensional tensors”. He also mentions how tensor networks are applied to physics multiple times in his book The Road to Reality. Another somewhat obscure but incredibly thorough text written specifically on the topic of utilizing graphical tensor notation in representation theory is the book Group Theory: Birdtracks, Lie’s, and Exceptional Groups, by Predrag Cvitanovic. This book actually caught my attention years ago as an undergrad, not long after it was first released, specifically because I wanted to understand quantum gravity and how one could use such fascinating visual tools to do some very deep mathematics.
These graphical representations of tensors aren’t just pretty though, they actually form the foundation of some of the deepest and most fundamental ideas in theoretical physics today. They are used by physicists and mathematicians like Leonard Susskind, Roger Penrose, Juan Maldcena, and John Preskill to describe things like quantum information theory, the AdS/CFT correspondence, string theory, and the holographic principle.
Supervised Learning with Quantum-Inspired Tensor Networks
In this paper, E. Miles Stoudenmire, and David J. Schwab use tensor network models to perform machine learning classification of the handwriting data MNIST, using “matrix product states”. They are able to obtain <1% error. They also “ discuss how the tensor network form imparts additional structure to the learned model and suggest a possible generative interpretation”. Their code can be found in the Github repository.
Quimb is a python library for quantum information and many-body calculations, including with tensor networks. It is fast and optimized for efficiency. It is primarily intended for applications to physics problems, but can be adapted to other applications as well. It can run on a GPU and can compute tensor operations and representations in a geometry free way. It can optimize any tensor network with tensorflow or pytorch
It is useful for simulation of quantum computers since it is already setup for computations in IBM Qiskit. For many problems the simulation is efficient, but for high dimensional computations where the solution space of interest is not restricted by some kind of symmetry, quantum computers will of course be required in place of the classical simulations provided by Quimb.
In his article Quantum Computing in the NISQ era and beyond, John Preskill describes the current state of quantum computing on “Noisy Intermediate Scale Quantum (NISQ)” hardware. In his lectures, he discusses a “quantum information theory”, similar to the paper Physics, Topology, Logic and Computation: A Rosetta Stone, written by Google in collaboration with the mathematical physicist John C. Baez, and the computer scientist Mike Stay. One of the running themes in the paper is that we should rethink Information Theory.
In particular, the Google paper explains how to think of information theory in terms of quantum physics via tensor networks. It is not surprising that the standard notation in quantum computing, the quantum circuit diagram, is itself a tensor network.
In a short tutorial on the Google TensorNetwork library, I show how using the simultaneous contraction of edges can provide substantial speedups in computation. The code computes the runtime of a sequential contraction of edges, then a simultaneous contraction of the edges.
def one_edge_at_a_time(a, b):  node1 = tn.Node(a)  node2 = tn.Node(b)  edge1 = node1[0] ^ node2[0]  edge2 = node1[1] ^ node2[1]  edge3 = node1[2] ^ node2[2]  tn.contract(edge1)  tn.contract(edge2)  result = tn.contract(edge3)  return result.tensordef use_contract_between(a, b):  node1 = tn.Node(a)  node2 = tn.Node(b)  node1[0] ^ node2[0]  node1[1] ^ node2[1]  node1[2] ^ node2[2]  # This is the same as  # tn.contract_between(node1, node2)  result = node1 @ node2  return result.tensora = np.ones((1000, 1000, 10))b = np.ones((1000, 1000, 10))%%timeone_edge_at_a_time(a, b)CPU times: user 15.7 s, sys: 1.88 s, total: 17.6 sWall time: 14.3 s
[29]:
array(10000000.)
This contracts the three edges sequentially, first red, then green, then blue, as visualized in the image generated using Graphviz:
# http://www.graphviz.org/content/clustergraph G0 {    node [style=filled,color=pink];    edge [color = red];    a -- b;    edge [color = green];    a -- b;    edge [color = blue];    a -- b}
Hosted on Sketchviz
%%timeuse_contract_between(a, b)CPU times: user 495 ms, sys: 145 ms, total: 639 msWall time: 1.75 s
[33]:
array(10000000.)
Mathematically this translates into an inner product of rank three tensors:
Having algorithms such as this which perform more efficient computations on high dimensional tensors provides significant speedup (up to 100 times) as mentioned in this article. Add to the mix the new “Tensor Processing Units (TPUs)” that Google has designed to perform more efficient computations with tensors for machine learning purposes, and you have an extremely powerful machine learning framework, from hardware to cloud computing software to end-user application.
So far tensor network methods have found applications in image recognition, multi-class classification problems, computer vision, physics, materials science, bio-tech, quantum chemistry, and biophysics. Since TensorNetwork provides an easy way to integrate tensor networks into layers of standard neural networks, they can be used in any machine learning task involving neural networks.
While these methods may seem new to most of the machine learning community, they have a long history of applications in physics and mathematics, so the theory and conceptual understanding is well established in the mathematics and theoretical physics communities. This offers a new approach to understanding machine learning, and provides new intuition in an area that is plagued with “black box” methodology. Struggling to understand why AI makes certain decisions and comes to particular conclusions is a problem that is leading to concerns on how biased AI can be in certain applications to business solutions such as hiring assisted by AI. With law enforcement now applying AI in many cities, ensuring that AI is fair and is not making inhumane decisions is also becoming a concern on many minds. Add to that the improvements that tensor network methods can provide in machine learning tasks and computational efficiency, and it seems prudent to invest in researching and implementing these methods on a larger scale.
If you have ideas to share, questions, or if you are in need of consulting services for quantum machine learning contact the author via LinkedIn, visit The Singularity website, or checkout the Github tutorials.