The Best Way to Pick a Unit Vector | by Don Cross | Towards Data Science

Sometimes a simulation or numeric solver requires generating random unit vectors that fairly represent all directions in an n-dimensional space. Perhaps you are simulating molecules in a gas, and you want to model their random collisions. Maybe you want to explore the neighborhood of a point in a highly-dimensional space to find the local minimum of a function. Or you could be rendering a raytraced scene that includes scattering of photons from a matte surface.
In cases like these, it’s handy to have a random unit vector generator in your scientific programming toolbox. The ideal algorithm will pick unit vectors that point in random directions in the n-dimensional space, without any bias. That means that, over time, the unit vectors will be scattered evenly over the surface of an n-dimensional unit hypersphere centered at the origin.
The most obvious solution is to choose n random numbers uniformly from the range [−1, +1] to form a vector:
Then divide by the length of the vector to normalize to a unit vector:
This naive algorithm is biased toward certain directions and away from others. More specifically, it over-represents directions diagonal to your chosen Cartesian axes. To prove this assertion, consider the two-dimensional case. Let’s start with a grid having a unit circle inscribed on it:
Picking a pair of random vector components is like picking a pair of intersecting lines from the grid. The point where the lines intersect defines a direction vector from the origin.
Now consider two different pie slices emanating from the origin, one surrounding the positive x-axis, the other roughly 45 degrees away from the x-axis:
Suppose both slices contain an equal angular span. Assume we choose random x and y values from this grid, and both from the range [−1, +1]. Then each random point (x, y) will be somewhere inside or on the purple square that fits just outside the unit circle.
There are more points that fall between the green diagonal lines than between the red lines that straddle the x-axis. This is because there are more points outside the circle in the diagonal direction than in the horizontal direction.
As you make the grid smaller and smaller, with the gap between the lines approaching zero, the number of points in each region becomes arbitrarily proportional to the respective enclosed areas. The diagonal region contains more area than the horizontal region. If you imagine randomly throwing a dart at a dartboard, it is more likely to strike a region having larger area.
A point chosen using uniform Cartesian coordinates is thus more likely to occur along a diagonal than an orthogonal. The simple approach fails the bias test: it does not pick from all directions with equal probability.
In this two-dimensional case, an effective alternative is to pick a random angle θ in the half-open range [0, 2π). Then the unbiased unit vector becomes (cos θ, sin θ). However, this only works in the two-dimensional case. We want a solution that works in any positive number of dimensions n.
Looking again at the diagram above, we see that the bias occurs because the number of points inside the square but outside the circle varies with angle. If we could always pick a point inside the circle, the bias would disappear.
This suggests a modified algorithm: pick a series of random coordinates for each vector component, but reject the pick if it lies outside the circle, sphere, or hypersphere for the n-dimensional space. In other words, keep picking another batch of n random numbers until they satisfy
Once you find a point inside the bounding radius, every direction is represented equally, and the unit vector has an unbiased direction.
This approach works well for two or three dimensions, but there is a problem in the general case.
The problem with looping until we find a point inside the n-dimensional hypersphere is that it becomes increasingly inefficient as n increases. Each squared term in the above sum can take on a value from 0 to 1. With n terms, the sum of squares can range from 0 to n.
As n increases, it becomes increasingly likely that the sum will exceed 1. With a high enough dimension count, your program will spend more and more time looping until it finds a point inside a unit radius.
(As an interesting aside, this reveals a surprising fact about higher dimensional geometry: the ratio of the volume of a hypersphere to the volume of its enclosing hypercube approaches zero as the dimensionality n increases without bound.)
It would be much better to have an algorithm that picks a random direction vector without burdensome looping.
Because a uniform random distribution isn’t suited to picking an unbiased direction in space, the solution to the problem is to find some non-uniform distribution that compensates for any directional bias. The desired distribution will cancel out any artificial bias introduced by the choice of coordinate axes.
It turns out that the answer is familiar to any student of statistics: the Gaussian (or “normal”) distribution. This is a random distribution whose probability density function assumes the famous bell curve shape.
Probability density functions may be unfamiliar to you. Let’s take a step back and revisit the original uniform distribution we started with. That uniform distribution is a machine that outputs a series of random numbers that are always in the range [−1, +1]. Being uniform means that the probability of the number being in any small range inside [−1, +1], say [+0.37, +0.38], is exactly the same as it being in any other such range of the same size, for example [−0.67, −0.66].
Because each uniformly-selected random number must be somewhere within [−1, +1], we say that the probability of being in that range is 1. Likewise, the probability of the number being outside that range is 0, meaning that never happens.
Divide the total probability 1 by the size of the range (+1)−(−1)=2, to get a probability density of 1⁄2. This density number tells us that the probability of a random number being inside the range [+0.7, +0.8] is half the size of the range, or 1⁄2(0.8−0.7) = 0.05.
The graph of the uniform probability density looks like this:
It stands to reason that if using a uniform distribution to pick vector components causes the resulting vector to be biased away from the Cartesian coordinate axes, the desired non-uniform distribution must have a compensating bias of each component toward zero. The Gaussian normal distribution function is:
where σ is the standard deviation of the distribution away from the mean value 0. As we will see later, we can choose any positive number for σ to obtain an unbiased direction, once we normalize to a unit vector. The graph of the normal distribution produces the following bell-shaped curve:
This distribution certainly picks numbers closer to 0 more often than numbers far from zero, with a tapering probability as x moves away from 0 in either direction.
The above reasoning is fuzzy and doesn’t prove anything. How do we know a vector whose components are picked from a normal distribution necessarily points in a perfectly unbiased direction? There is a simple way to prove it.
The first step is to relate the probability density function of each vector component to the probability density of the whole vector. The normal function shown above, p(x), describes the relative likelihood of choosing a value near any particular value of x. In the two-dimensional case, we apply the same function to the other coordinate y. Since we choose both x and y using the same normal distribution, p(y) describes the probability density of choosing y near any given value.
When you have two independent events, the probability of both happening is the product of their individual probabilities. Therefore, the probability of choosing a vector near any given value (x, y) is described by the product p(x)p(y).
Before we go any further, let’s choose a value σ that makes things as simple as possible:
Then p(x) becomes:
Take a look at what happens with the product p(x)p(y):
Here’s where the magic happens. The quantity x2+y2 can be replaced with r2, where r is the length of the vector (x, y). In other words, when using a normal distribution to choose x and y, the likelihood of choosing a vector near a given pair (x, y) is
The key insight: the likelihood of landing in the infinitesimal neighborhood of (x, y) depends only on the length r of the vector (x, y). The direction the vector is pointing does not affect how likely you are to choose it.
Put another way, we have found a way to convert the product p(x)p(y) from Cartesian coordinates (x, y) to polar coordinates (r, θ), where r is the distance from the origin and θ is the counterclockwise angle from the positive x-axis. Because θ never appears in the probability density function, it cannot affect how likely we are to choose a vector at any given angle θ. All angles are equally likely! That’s exactly the property we wanted all along.
Even better, this angular irrelevance works for any number of dimensions. In the general case, the n-dimensional probability density product becomes
Because the Pythagorean distance formula works in any number of dimensions, we can replace the sum of n squared components with r2, resulting in a probability distribution that depends only on the length of the vector.
The Gaussian normal distribution sounds great, but how does one generate random numbers having such a distribution? Uniform random number generators are available in most programming languages. It would be helpful to transform a uniform distribution into a normal distribution.
In 1958, George E. P. Box and Mervin E. Muller published a two-page Note on the Generation of Random Normal Deviates that showed how to do exactly that. Their method generates normally-distributed random numbers in pairs. Given two uniformly-distributed random numbers A and B, both from the half-open range (0, 1], the Box–Muller transform calculates a pair of independent, normally-distributed numbers as follows:
It’s worth emphasizing that A must never be zero because the logarithm of zero is not defined. Keep generating pairs of numbers using these formulas until you have n numbers to complete a set of vector components. If n is odd, you can keep the leftover generated value and use it as the first component of the next vector you generate, or you may simply discard it. Either way, as a final step, divide through by the length of the resulting vector to create your unit vector v:
Unit vectors generated this way will be completely unbiased, having an equal probability of pointing in any direction in the n-dimensional space.
I have provided a sample implementation in the C programming language:
github.com
For Python programmers, the numpy project provides a ready-made random generator that produces normally-distributed values:
>>> import numpy>>> numpy.random.normal()0.3850365612479212>>> numpy.random.normal()0.8785540204204144>>> numpy.random.normal()-0.32533367867722596
The random unit vector generator is very simple. Just call randomvector passing in the number of dimensions as n and you will get back a list containing the components of an unbiased unit vector.
Hypersphere Point Picking [Wolfram MathWorld].Box–Muller transform [Wikipedia].Sample C source code by the author [GitHub].
Hypersphere Point Picking [Wolfram MathWorld].
Box–Muller transform [Wikipedia].
Sample C source code by the author [GitHub].