Deciding Finiteness of Matrix Groups in Deterministic Polynomial Time

Let G be a group of matrices with entries over an algebraic number field F (given symbolically). The group G is given by a list of generators. In this paper we give several algorithms, both deterministic and randomized, which can decide in polynomial time whether or not G is jinzte. It is easy to reduce the problem to the case F = Q. As a next step, we present a polynomial time algorithm which transforms G into a group of integral matrices whenever possible. Having done so, the main results of the paper are several polynomial time algorithms to handle the case of integral matrices. We give both randomized and deterministic algorithms to decide finiteness for finitely generated integral matrix groups. Although we are able to prove much better upper bounds for the complexity of the deterministic algorithms, in practice, the randomized algorithms support a much more efficient implementation. Thus, both kinds of algorithms are presented here but only the implementation of the randomized algorithm is explored. All algorithms in some way depend on a characterization (effectively due to Burnside) c)f finite integral matrix groups as those subgroups G s GL(n, Z) which preserve some positive definite quadratic form. The randomized (Monte Carlo) algorithms use recent random walk techniques to obtain nearly uniformly 1Department of Computer Science, Universilt y of Chicago, 1100 E 5t3th St, Chicago, IL 60637 2Eot vbs University, Budapest, Hungary 3Department of Mathematics and Computer Science, Dartmouth College, Hanover, NH 03755 4Partially supported by NSF Grant CCR 9014562 and OTKA Grant 2581. 5Partially supported by NSF Mathematical Sciences Postdoctoral Fellowship. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of tha publication and ita date appear, and notice is given that copying ia by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires s fee and/or specific permission. ACM-ISSAC ‘93-7/93/Kiev, Ukraine 01993 ACM 0-89791-604-219310007101 17...$1.50 distributed random elements in G (in a precise sense, whether G is finite or not). The subsequent analysis rests on a new estimate of the norm of the matrices in finite groups of integral matrices in terms of the norms of the generators. If G is infinite, one of our Monte Carlo algorithms is likely to produce an element with larger norm, thus certifying infiniteness. If G is finite, our other Monte Carlo algorithm is likely to produce a positive definite G-invariant quadratic form, thus certifying finiteness. Consequently, the combination of these two give an eficient and very simple Las Vegas algorithm (randomized algorithm that never errs) which we have successfully implemented. The deterministic algorithms are of two kinds. One algorithm uses some basic facts from the representation theory of finite groups to construct a G-invariant quadratic form (if it exists), by efficiently averaging the matrices A~A. The other deterministic algorithm uses the ellipsoid method to search for a positive definite quadratic form in a linear space U of quadratic forms. The new estimate on the norm of elements of G is used to ensure that if G is finite, then 1( contains a sizable ball of positive definite forms, as required for the ellipsoid method to be applicable.