Programming Linear Algebra in Java: Vector Operations
By: Dan Hales
Tags: Mathematics, Linear Algebra, Programming, Data Science, Java

Two helper methods we need to discuss before we can do anything too linear algebra-y are isZero() , which returns true if all entries are 0 and false if there are any non-zero entries in the vector. An isZero method is important because some vector operations––like normalizing––will result in division by zero if we don’t check this first. Some operations are perfectly valid with the zero vector, so this check is only done under specific circumstances.

Second, we have checkLengths(Vector u, Vector v) . All of the important vector operations––for instance, addition or dot products––are defined only for vectors u and v that have the same number of elements. checklengths compares the two vectors. If they have the same length, nothing happens. If they have different lengths, an IllegalArgumentException is thrown:

public static void checkLengths(Vector u1, Vector u2) {

if (u1.length() != u2.length()) {

throw new IllegalArgumentException(

"Vectors are different lengths");

}

}

Basic Operations

Vectors are famous having two primary operations: scalar multiplication (multiplying each entry by the same scalar) and vector addition (adding the corresponding entries of two vectors), so our Vector class would be remiss not to support these. For each, I’ve created a static method that can be called directly from the class, and then an instance method that can be called directly on the Vector itself. Note that all of these operations create a new Vector object, rather than modifying the existing one.

public Vector add(Vector u) {

return Vector.sum(this, u);

} public static Vector sum(Vector u1, Vector u2) {

Vector.checkLengths(u1, u2); // ** see comment



double[] sums = new double[u1.length()];



for (int i = 0; i < sums.length; i++) {

sums[i] = u1.get(i) + u2.get(i);

}



return new Vector(sums);

}

The static method sum does all of the work, and the instance method add simply passes this , which refers to the calling object, and u , a passed vector, to sum . Pythonistas will recognize this as the Java cousin of self . There are some important differences between the two, like how instance methods take self as an argument on Python but simply require us to drop static in Java, but if you understand one, it’s easy to wrap your brain around the other.

I draw attention to this in order to show the contrast between calling a method on an object itself ( this.method() ) and calling the Vector class’ static method with Vector.sum() .

Also notice that in Vector.sum , the very first thing we must do is check to see if the vectors are the same length by calling Vector.checkLengths(u1, u2) . Because we’re adding the first element of u1 onto the first element of u2 , then their second elements, then their third elements, and so on, it’s important that they actually have the same number of elements.

There’s actually a bit of redundancy built into this code––because checkLengths is inherently a method of the Vector class, we could have simply written the line marked with // ** see comment as

checkLengths(u1, u2);

To make it clear that we’re not calling any method that specifically requires an instance’s data, I’ve written it explicitly as Vector.checkLengths(u1, u2); Because this method simply performs a check and throws an IllegalArgumentException if the condition is not met, we don’t have any return value, and we can think of it almost as an assert statement.

We can take a similar approach with scalar multiplication:

public Vector multiply(double scalar) {

return Vector.product(this, scalar);

} public static Vector product(Vector u, double scalar) {

double[] products = new double[u.length()];



for (int i = 0; i < products.length; i++) {

products[i] = scalar * u.get(i);

}



return new Vector(products);

}

As well as dot products:

public double dot(Vector u) {

return Vector.dotProduct(this, u);

}



public static double dotProduct(Vector u1, Vector u2) {

Vector.checkLengths(u1, u2);



double sum = 0;



for (int i = 0; i < u1.length(); i++) {

sum += (u1.get(i) * u2.get(i));

}



return sum;

}

We can apply this same logic to cross products, but we need to make sure the cross product is actually defined first––that means verifying that both vectors are actually of length 3:

public Vector cross(Vector u) {

return Vector.crossProduct(this, u);

}





public static Vector crossProduct(Vector a, Vector b) {

// check to make sure both vectors are the right length

if (a.length() != 3) {

throw new IllegalArgumentException("Invalid vector length (first vector)");

}

if (a.length() != 3) {

throw new IllegalArgumentException("Invalid vector length (second vector)");

}

Vector.checkLengths(a, b); // just in case



double[] entries = new double[] {

a.v[1] * b.v[2] - a.v[2] * b.v[1],

a.v[2] * b.v[0] - a.v[0] * b.v[2],

a.v[0] * b.v[1] - a.v[1] * b.v[0]};



return new Vector(entries);

}

I’ve accessed the elements of the arrays with slightly different syntaxes in these operations, both to make the cross product more readable and to highlight a feature of Java that can be confusing the first time you come across it. Notice how in dotProduct , I called the instance method u1.get(i) in order to access elements of the array, while in crossProduct , we accessed the elements directly with a.v[0] . In Java, any code in the Vector class has access to the private members of any Vector object, but we can also call those members’ instance methods.

Direction and Magnitude

Often, we want to know the magnitude (length) of a vector, which is really a special case of the p-norm where p=2. Since L1 and L2 norms come up in machine learning contexts (for instance, Lasso and Ridge regression), let’s go ahead and make a generalized function for computing the p-norm, given a vector and a value of p:

// static method

public static double pnorm(Vector u, double p) {

if (p < 1) {

throw new IllegalArgumentException("p must be >= 1");

}



double sum = 0;



for (int i = 0; i < u.length(); i++) {

sum += Math.pow(Math.abs(u.get(i)), p);

}



return Math.pow(sum, 1/p);

} // instance method

public double pnorm(double p) {

return Vector.pnorm(this, p);

} // magnitude

public double magnitude() {

return Vector.pnorm(this, 2);

}

Having both a static method and an instance method for pnorm gives us a little bit of flexibility in how we compute the norm of a vector. We can call the method either from the Vector class itself (e.g. Vector.pnorm(u, 2) ), or we can call it directly on an existing Vector object (e.g. u.pnorm(2) ).

Once we’ve defined the pnorm method, we can wrap it up in the magnitude method, to give us the magnitude of a vector. And now that we can compute that, we can normalize a vector. Normalizing is accomplished by dividing the entries of the vector by the vector’s magnitude, which is why we need the isZero check before we proceed:

public static Vector normalize(Vector v) {

if (v.isZero()) {

throw new IllegalArgumentException();

} else {

return v.multiply(1.0/v.magnitude());

}

}



public Vector normalize() {

return Vector.normalize(this);

}

Notice the difference in how we’ve handled a zero Vector and Vector objects of different lengths––any time we attempt an operation on Vector s with different lengths, we’re entering undefined territory, and we need to throw an IllegalArgumentException to abort, but there are some operations (such as pnorm ) that are perfectly valid for all-zero entries, so we wouldn’t want to build the exception into isZero() .

More Complex Operations: Enclosed Angles and Scalar Triple Products

Several operations rely on dot products, cross products, and magnitude calculations, and we can now perform using the methods we’ve built.

First, we want to be able to compute the angle enclosed by two Vector s, which requires the dotProduct method:

public static double angleRadians(Vector u1, Vector u2) {

Vector.checkLengths(u1, u2);

return Math.acos(Vector.dotProduct(u1, u2) /

(u1.magnitude() * u2.magnitude()));

}

And next, we want to be able to compute the , scalar triple product (which requires both the dotProduct and the crossProduct methods:

public static double scalarTripleProduct(Vector a,

Vector b,

Vector c) {

return Vector.dotProduct(a, Vector.crossProduct(b, c));

}

If you’ve ever had to do this by hand (for instance, on a linear algebra exam), you can probably understand how satisfying it was to watch these pieces fall into place with so little effort.