Comment on " Basis Restricted Transformations and Performance Measure for Spectral Representations "

In this correspondence, we comment on some of the shortcomings, for source encoding problems, of the measure of distance between orthonormal representations proposed by Pearl [1]. We further examine, using rate distortion theory, whether rates achievable for a given distortion level by the use of basis-restricted transformations have some inherent dissimilarity independently of input Si.dtisi. Ic5" aiiu 6f oL/Jcrr iiii Research sponsored in part by the National Aeronautic and Space Administration, Grant NASA-NGR-05-003-538. Comment on "Basis Restricted Transformations and Performance Measure for Spectral Representations". Pearl [1] has proposed a distance measure to characterize "inherent dissimilarities" between orthonormal transformations. With this measure the distance between orthonormal transformations is "independent of input statistics". One of the applications mentioned for basis restricted transformations is to the source encoding of random processes. In this correspondence we comment on some of the shortcomings of Pearl's distance measure when applied to source encoding problems. We further examine, using rate distortion theory, whether rates achievable for a given distortion level by the use of basis-restricted transformations have some inherent dissimilarity independently of input statistics and of other implied parameters. We note first that dissimilarity betv/een two transformations is defined by Pearl for the class of random processes which have a diagonal correlation matrix -* for one of the transformations. The transformation is thus the Karhunen-Loeve expansion of a random process adapted to the specific transformation. Under these circumstances independence on input statistics achieved by averaging over a class of processes cannot be dissociated of the class of processes considered. Moreover this construction does not provide any insight into a common question of interest in the use of basis-restricted transformations for source encoding: Given a random process with known correlation matrix how do different transformations rank in the encoding of this random process? This is the specific question that Pearl refers to when he comments on the asymptotic distance between the Walsh-Hadamard and the Fourier transformation for large block size. However Pearl's distance measure also indicates a diverging distance between the Karhunen Loeve (which is then the Kronecker or natural transformation) and Fourier transformation for large block size. On this basis one cannot feel confident that Pearl's distance between Walsh and Fourier transforms provides an answer to a common conjecture. The evaluation of the relative performance of orthonormal transformations in source encoding problems can be done by assuming that the expansion coefficients are encoded independently and the rate distortion bound evaluated for each coefficient. This approach was used by Goblick and Holsinger [2] to evaluate the performance of the Fourier expansion for source encoding and has been extended to other "basis-restricted transformations" by Pearl. For example we show in Figure 1 such rate distortion functions for a Gaussian Markov random process with correlation function R(n) = e"'' (1) n = 0, +_1, + 2 --. These curves are for a = + 0.05 and clock size N = 32. Similar curves are given by Pearl, Andrews and Pratt for N = 256 [3]. Note that the Haar and Walsh-Hadamard transformations behave similarly, in contradiction to Pearl's distance. The similarity of performance can be explained by exploiting the formal relations between the Haar and Walsh Hadamard transformations [4]. Note also that for the semi-logarithmic scale used for the mean square distortion the rate-distortion per sample curves are parallel straight lines. This occurs for mean-square distortions lower than the smallest variance of expansion coefficients since we have then N 2 n of R(e) . L-logg^j— (2) 1 N£ Therefore dR1 R2 = R^e) R2(e) = ̂ log,, n (°î /°2î in '" subscripts 1 and 2 denote two distinct orthonormal transformations. .Mote that dR, R2 1S no longer dependent on the specific e . To illustrate the dependence of source statistics we show in figure (2) dR as a function of a , defined in (1) for the fixed block size N =. 32. It is noteworthy that the curves for Walsh Hadamard and Fourier transformations intersect. Thus no general conclusion can be obtained on the relative merit of these two orthoriormal transformations in source encoding problems. Averaging on some set of statistics is possible but the use of the results is dependent on the specific application. As shown in Figure (3), when the block size increases, distance dR, Ro between the Fourier and Karhunen Loeve transforms decreases to zero as expected. It can be shown that the distance between the Haar and Karhunen-Loeve transforms increases to a limit and it is conjectured that the Walsh Hadamard transform has a similar limiting behavior. V. Ralph Algazi Department of Electrical Engineering University of California, Davis Bernard J. Fino Department of Electrical Engineering and Computer Science University of California, Berkeley References: : [1] J. Pearl, "Basis-Restricted Transformations and Performance Measures for Spectral Representations", IEEE Trans, on Inf. Theory, Vol. IT-17, November 1971, pp. 751-752. [2] T. Goblick and J. Holsinger, "Analog Source Digitization: A Comparison of Theory and Practice", IEEE Trans, on Inf. Theory, Vol. IT-13, April 1967, pp. 323-326. [3] J. Pearl, H. C. Andrews, W. K. Pratt, "Performance Measures for Transform Data Coding", IEEE Trans, on Communications, Vol. COM-20, No. 3, June 1972, pp. 411-414. ^ ' ' ' [4] B. Fino, "Relations between Haar and Walsh Hadamard Transforms", Proceeding of the IEEE, Vol. 60, May 1972, pp. 647-648.