Authors: Inge S. Helland
Source: Scandinavian Journal of Statistics (article, 2025-12-12)
Field: Chemistry — Analytical Chemistry

Rejoinder I am grateful to the discussants for their insights and for the valuable points they have made in connection with a theme that I myself find difficult to dig deep enough into. R. Dennis Cook and Liliana Forzani: First, I want to thank the discussants for calling my aim to pursue a synthesis of different scientific cultures a laudable goal. At this moment, I see “synthesis” as too strong a word. What I really have wanted to work for is to build some bridges between the different cultures in order to promote a better understanding across the scientific disciplines. Ideally, this could serve as an example for human endeavors in other fields, where building bridges between cultures may serve as a way to work for better understanding among people with different backgrounds. But perhaps, in the long run, one could really find a synthesis between the three disciplines. As I see it, a synthesis between statistical theory and PLS regression already exists, through our joint work in Helland (1990) and Cook et al. (2013), and through other contributions. What is lacking, is a really good link between statistical theory and quantum theory. The recent work by Andrei Khrennikov, see in particular Khrennikov (2023), may be a stepping stone for such a link, together with recent works on the foundation of quantum decision theory. See Helland (2025c) and, for instance, Busemeyer and Bruza (2012). This is an area where more foundational research has to be done, and this may take some time. My article has been concentrating on prediction with univariate response y $$ y $$ , called PLS1 by chemometricians, while much of the discussants' work on envelope models, in particular the work by Dennis Cook (see Cook, 2018), extends to the case of multivariate response, called PLS2 by the chemometricians. While this is of great value, my own motivations have been partly to try to understand the simplest case first and partly the intuitive idea that, for the purpose of prediction, sometimes PLS2 can better be replaced by simply doing PLS1 separately on each component of the response vector. Having said this, it is indeed very valuable to see PLS as a sufficient dimension reduction method that can be compared to other dimension reduction methods like SIR and SAVE. I also highly appreciate the remark that the PLS foundation may be extended to the nonlinear case and that the envelope language can be used to study this extension. Coming back to my article, it is not quite true that Theorems 3, 4, and 5 only serve as stepping stones leading to Theorem 6. For instance, Theorem 5 and its corollary do have some independent value. By first stating conditions under which the PLS model/envelope model gives an optimal model reduction, one can then begin to look for the best possible method that is compatible with this model. I agree that the inequality (19) of Theorem 6 may be difficult to use as guidance in practice. However, I see it as being of some value that the righthand side only depends on the properties of the PLS-like regression vector, while on the lefthand side, the vector β ^ $$ \hat{\boldsymbol{\beta}} $$ may be connected in principle to any regression method, for instance, OLS, ridge regression, or principle component regression, which has several variants. If the corresponding regression vector is far enough from the PLS-like regression vector, the last method gives the best prediction. Theorem 6 is an exact theorem in that connection, but on the other hand, it gives only a sufficient condition for a PLS-like method to be optimal. It may be very important, as Cook and Forzani do, to use asymptotic results to gain intuition. In doing their asymptotics, Cook and Forzani consider the special case where β ^ $$ \hat{\boldsymbol{\beta}} $$ is the OLS regression vector, and this is compared to PLS regression with a = m = 1 $$ a=m=1 $$ . For this case, they have a very simple necessary and sufficient for PLS to dominate over OLS asymptotically. In my language, the condition says that the squared ratio between the relevant eigenvalue and the maximal irrelevant eigenvalue of Σ x x $$ {\boldsymbol{\Sigma}}_{xx} $$ should be greater than the scaled variance of y $$ y $$ . The simple condition found in this case may indicate that my sufficient condition in Theorem 6 can be improved. In particular, the factor 4 is disturbing. The second asymptotic result mentioned by Cook and Forzani is more difficult to link to my results. As I understand it, the vector G $$ \boldsymbol{G} $$ may here be any p $$ p $$ -dimensional vector. I have tried to take G = Σ x x Γ $$ \boldsymbol{G}=\sqrt{{\boldsymbol{\Sigma}}_{xx}}\boldsymbol{\Gamma} $$ and take the square of the variable on the left-hand side, but this still does not lead to a statement that can be compared directly to my theory. The discussion ends with a question for me. As in the article, I will concentrate on prediction with univariable response. Then the PLS model prediction/envelope model prediction is equivariant under the following transformation: There is an orthonormal matrix O $$ \boldsymbol{O} $$ transforming the relevant eigenvectors of Σ x x $$ {\boldsymbol{\Sigma}}_{xx} $$ : ( d 1 , … , d m ) → O ( d 1 , … , d m ) $$ \left({\boldsymbol{d}}_1,\dots, {\boldsymbol{d}}_m\right)\to \boldsymbol{O}\left({\boldsymbol{d}}_1,\dots, {\boldsymbol{d}}_m\right) $$ , and positive scalars α j $$ {\alpha}_j $$ transforming the corresponding regression coefficient: γ j → α j γ j $$ {\gamma}_j\to {\alpha}_j{\gamma}_j $$ . But this gives transformations defined by the group G $$ G $$ of Definition 4 of Section 3 in my article and discussed in Subsection 4.1. This group is acting on the PLS parameter θ $$ \theta $$ as defined in my article, is transitive on its range, and has a trivial isotropy group, and as discussed in Subsection 4.1, is associated with a left-invariant measure ν $$ \nu $$ . Anticipating an extension of Theorem 1 of Subsection 2.1 from the invariant case to the equivariant case, this indicates that the group G $$ G $$ satisfies the conditions of such a Theorem 1. The theorem in its present form requires that the two variables θ $$ \theta $$ and η $$ \eta $$ are related, but if the two have similar ranges, this mathematical condition can be replaced by a condition saying that there is a group M $$ M $$ defined on the total parameter ϕ = ( β , Σ x x , σ 2 ) $$ \phi =\left(\boldsymbol{\beta}, {\boldsymbol{\Sigma}}_{xx},{\sigma}^2\right) $$ , see Helland (2025a). This latter group can be defined by β → Cβ $$ \boldsymbol{\beta} \to \boldsymbol{C}\boldsymbol{\beta } $$ , Σ x x → ( C ′ ) − 1 Σ x x C − 1 $$ {\boldsymbol{\Sigma}}_{xx}\to {\left({C}^{\prime}\right)}^{-1}{\boldsymbol{\Sigma}}_{xx}{C}^{-1} $$ , σ 2 → σ 2 $$ {\sigma}^2\to {\sigma}^2 $$ for non-singular matrices C $$ \boldsymbol{C} $$ . What is the essence of Theorem 1? It says that under very weak conditions, including the symmetry conditions above and a basic condition that θ $$ \theta $$ and η $$ \eta $$ really are different and are maximal as accessible parameters/variables, the basic elements of the quantum formalism appear. There is a Hilbert space describing the situation. Every accessible variable, including θ $$ \theta $$ and η $$ \eta $$ in this situation, has a unique Hermitian operator in this Hilbert space associated with it. The possible values of the variable equal the set of eigenvalues of the operator, and an accessible variable is maximal as if and only if all eigenvalues are single. In this article, Theorem 1 is only used as a background for Corollary 3 of Section 6, but combined with conditions that give the Born Theorem, a theorem that describes quantum probabilities, it has important consequences also in statistical settings; see Helland (2025b). In fact, the theory can be said to have consequences for many different macroscopic disciplines. In Helland (2025c), and in agreement with the philosophy of the theoretical physicist Andrei Khrennikov and several other recent quantum theorists (see Bagarello, 2019; Busemeyer & Bruza, 2012; Haven & Khrennikov, 2013; Khrennikov, 2010, 2023; Plotnitsky & Haven, 2023), these disciplines are shown to include cognitive psychology, in particular aspects of decision theory, and also areas like social sciences and political sciences. I also sketched tentative conclusions in the field of theology. I hope that all this can be discussed somehow in a wider setting. Xuran Meng and Yi Li: It is very interesting if my approach can be generalized and applied to problems in machine learning. In general, I have defined variables like θ $$ \theta $$ and η $$ \eta $$ as accessible theoretical variables, a very wide notion; see Helland (2025a) and references therein. When considering statistical applications (Helland, 2025b and the present paper), I have mostly been thinking of these variables as statistical parameters. Meng and Li discuss the case where there are measures of data, that is, measures of pixels in a data vector describing some picture. This is indeed interesting, but unfortunately, one can question whether this generalization can be covered by my theory. The mathematical foundation of my theory is given in Helland (2024), which, taking into account an improvement in Helland (2025a), contains the proof of Theorem 1 of the present paper. Inspecting this proof, how is the Hilbert space constructed? It is the space associated with an irreducible representation of a certain group acting on the inaccessible variable ( θ , η ) $$ \left(\theta, \eta \right) $$ , a group constructed from 1) the transitive group G $$ G $$ acting upon θ $$ \theta $$ and 2) the notion of “relatedness.” In this theory, a main conclusion is that each accessible variable is associated with a Hermitian operator in the Hilbert space, giving a foundation of essential parts of quantum theory. These deep results are not essential for the present paper, however, and this opens for a simpler and more direct construction of the Hilbert space. As I see it, this is just what Meng and Li do in their discussion contribution but this construction, while related to the purpose of the present paper, is essentially different from the construction behind Theorem 1. Translated to my notation, θ = f S $$ \theta ={f}_S $$ and η = h # f S $$ \eta ={h}_{\#}{f}_S $$ , in this case, two different measures, the first one on 𝒳 x 𝒴 , the second one on 𝒵 x 𝒴 , where 𝒵 = h ( 𝒳 ) . These two measures are related, and one can, in some sense, see an analog to Theorem 1, but note that in Theorem 1 as such, it is assumed that θ $$ \theta $$ and η $$ \eta $$ are real-valued or real vectors. Going further, Ming and Li consider prediction and define optimality of the transformation h $$ h $$ . This may be said to mirror my approach, but no attempts to find sufficient criteria for optimality are given. Nevertheless, I really appreciate this contribution by Meng and LI, and I hope that their approach in the future will be further developed. David J. Olive: Various results on PLS and on the PLS model are mentioned in this contribution. It is not quite clear how these results can be related to my article, but some of them enlighten the role of the PLS model with m $$ m $$ components. Rolf Sundberg: I highly appreciate Rolf Sundberg's contribution. He is the only one of the discussants who really goes into the results of my article and gives constructive criticism. It is true that I in Appendix 1 say that the ordinary PLS algorithm can be reformulated in terms of the Krylov sequence, and I agree with Sundberg that care should be used in practice when using this reformulation. In the article, however, I do not discuss practical algorithms but mostly concentrate on the population model, where the Krylov formulation may be really useful, or look at the whole class of PLS-like methods. I do not agree with the statement that this makes Theorem 3 trivial, however. The inequality (1) says that the PLS model is best in terms of variance. The theorem states that it is then the best in terms of prediction error. Corollary 1 and Corollary 2 are immediate consequences. Throughout the article, I fix the number m $$ m $$ of components in the population PLS and in the competing model reduction. It is, of course, true that other methods can be better than PLS if one of these methods allows another choice of the theoretical number of components. All the results of Section 6 are formulated in terms of conditions under which the parameter θ $$ \theta $$ has a probability distribution and expectations under this distribution are considered. Given this distribution, the irrelevant regression coefficients γ j ; j > m $$ {\gamma}_j;\kern0.3em j>m $$ must be considered as unknown constants, as are all the eigenvalues of the x $$ x $$ -covariance matrix. However, the result of Theorem 4, and the consequences of this, are equally true if one replaces E θ $$ {E}_{\theta } $$ with an expectation that includes a distribution of all the regression coefficients γ j $$ {\gamma}_j $$ . As I see it, this does not matter, but it gives another class of results. Now turn to Theorem 6 and the condition (19) that is said to imply optimality of the given PLS-like method. Sundberg argues that this condition is quite narrow. I do not agree. The right-hand side of (19) is only concerned with β a ^ $$ \hat{{\boldsymbol{\beta}}_a} $$ and its closeness to the true regression vector β $$ \beta $$ . Whether this right-hand side is large or small, the criterion says that if a competing β ^ $$ \hat{\boldsymbol{\beta}} $$ is far from β a ^ $$ \hat{{\boldsymbol{\beta}}_a} $$ , then the PLS-like method dominates in terms of prediction error. Finally, I completely agree with Sundberg that the choice of scale may often be more important than the choice of method. But this was just not the theme of the article. Errata: Rolf Sundberg has pointed out two misprints in my article: 1) 2 lines after the hypothesis H m $$ {H}_m $$ of Subsection 4.2: “Section 4” should be “Section 3”; 2) The line before Corollary 1 near the end of Section 5: “Corollary 3” should be “Theorem 3.” None of the authors have a conflict of interest to disclose. Data sharing is not applicable to this article, as no datasets were generated or analyzed during the current study. Theme (computing) ; Foundation (evidence) ; Order (exchange) ; Work (physics) ; Connection (principal bundle) Spectroscopy and Chemometric Analyses ; Machine Learning in Materials Science ; Statistical Methods and Inference