Vol. 18, No. 1, pp. i vi, 2022

PREFACE: SPECIAL ISSUE ON NUMERICAL OPTIMIZATIONIN DATA SCIENCE

Zheng-Hai Huang∗ , Lingchen Kong† , Yiju Wang‡
With the development of science and technology and the internet, various industries areproducing a large amount of data every day, and their eﬀective processing constitutes thebasis of realizing intelligence. The main purpose of big data analysis and processing is todig out useful information and discover basic rules from massive and complex data, so as tomaximize the beneﬁts of data resources. In this sense, optimization provides a foundationin big data analysis and processing. In particular, the development of optimal theory andeﬀective method plays an important role in big data analysis and processing.
In recent years, many optimization methods have been used to solve data-driven optimization problems, and great progress has been made. However, due to the explosive
growth of data and other reasons, some new structured data processing problems continue
to emerge. This ﬁeld is still facing great challenges.
This special issue aims to reﬂect the latest advances in numerical optimization methods
in data science. A total of 25 papers accounted below, which were reviewed according to theusual high standards of the journal, cover a wide range of theoretical, practical, and applied
topics in numerical optimization methods in data science. This special issue is published intwo parts.
S. J. Bi, T. Tao and S. H. Pan [1] concern with the factorization form of the rank regularized loss minimization problem, which composes with an ℓ2,0 -norm regularized term, abalanced term and the factored loss function. For the least squares loss, under a restrictedcondition number assumption on the sampling operator, they establish the KL property of
exponent 1/2 of the nonsmooth factored composite function and its equivalent DC regularized surrogates in the set of their global minimizers. They also conﬁrm the theoreticalﬁndings by applying a proximal linearized alternating minimization method to the regularized factorizations.
S. Y. Chang and Y. M. Wei [2] study the Sherman-Morrison-Woodbury identity fortensors with Moore-Penrose generalized inverse by utilizing orthogonal projection of thecorrection tensor part into the original tensor and its Hermitian tensor. Based on the new
established identity, the sensitivity analysis for multilinear systems can be performed byderiving the normalized upper bound for the solution of a multilinear system. Furthermore,∗ Corresponding author. School of Mathematics, Tianjin University, Tianjin, 300500, China

E-mail: huangzhenghai@tju.edu.cn† School of Science, Beijing Jiaotong University, Beijing, 100044, China

E-mail: lchkong@bjtu.edu.cn‡ School of management Science, Qufu Normal University, Rizhao Shandong 276800, China

E-mail: wyijumail@163.com

iiZ.-H. HUANG, L. KONG AND Y. WANG

the authors present several numerical examples to demonstrate how the normalized error
upper bounds are aﬀected by perturbation degree of tensor coeﬃcients.
Y. N. Chen, L. Q. Qi and X. Z. Zhang [3] propose a lower rank quaternion matrix
approximation algorithm and apply it to color image completion. They also introduce
a concise form for the gradient of a real function in quaternion matrix variables. The
optimality conditions of our quaternion least squares problem have a simple expression withthis form. The convergence and convergence rate of the algorithm are established with thistool. Numerical tests on a set of color images demonstrate the eﬃciency of the algorithm.
L. B. Cui and Q. Hu [4] ﬁrst present a Zhang neural network model for solving theabsolute value equation and then establish a chord-Zhang neural network model. The convergence of the proposed models is studied. Some numerical experiments are presented toillustrate the eﬃciency of the proposed schemes.
M. Y. Deng, W. Y. Ding and X. P. Guo [5] prove the existence and uniqueness of thepositive solution to a non-homogenous M-tensor equation with a positive right-hand side,and extend the classical splitting methods, including the Jacobi-like, Gauss-Seidel-like, simpliﬁed Gauss-Seidel-like, and SOR-like methods, to solve this class of tensor equations andshow their convergence. Finally, a large number of numerical results show the performanceof these methods.
W. X. Du and S. L. Hu [6] present an algorithm for solving the problem of the symmetric
low rank orthogonal tensor approximation for a given symmetric tensor. Proximality technique and shifted power technique are tailored into this algorithm. In particular, the authors
show that this algorithm converges globally without any assumption once the parameters
are chosen appropriately, and moreover the convergence rate is sublinear with an explicitly
given rate and it is better than the usual O(1/p) of ﬁrst order methods in optimization.
C. J. Fang, R. R. Zhang and S. L. Chen [7] introduce an inertial Tseng’s extragradientmethod for solving multi-valued variational inequalities, in which only one projection isneeded at each iteration. They also obtain the strong convergence results of the proposed
algorithm, provided that the multi-valued mapping is continuous and pseudomonotone with
nonempty compact convex values. Moreover, numerical simulation results illustrate theeﬃciency of the method when compared to existing methods.
R. X. Feng, P. W. Zhong and Y. T. Xu [8] introduce a support matrix machine basedon squared hinge loss (L2 -SMM), which employs a smoothed substitution formula. They
construct a subspace elimination strategy for L2 -SMM (SES-L2 -SMM) to expedite the speedof its training process. At each step, an active subspace is chosen so that they can solve alower-dimensional optimization problem. Later, they use the alternating direction methodof multipliers algorithm to resolve the problem. Extensive comparative experiments on
multiple real-world datasets demonstrate the eﬃcacy of SES-L2 -SMM.
B. D. Huang and P. P. Shen [9] consider the problem of sum of linear ratios on a polyhedron. the authors show that the problem can be transformed into a two-layer optimization
problem. By using the special structure of the transformed problem, a novel convex relaxation technique is proposed to design a new branch and bound algorithm. Furthermore, theconvergence analysis and computational complexity of the presented algorithm are given.
Finally, numerical experimental results illustrate the eﬀectiveness of the algorithm.
H. W. Jiao, J. Q. Ma and Y. L. Shang [10] present an image space branch-and-boundalgorithm for a minimax linear fractional programming problem. Based on an equivalenttransformation and a new linearizing technique, the original problem can be approximatedby a linear relaxation programming problem. By subsequently reﬁning the initial image
space region and successively solving a series of linear relaxation problems, the proposed
algorithm converges globally to an optimal solution of the original problem. NumericalPREFACEiii

experimental results demonstrate the feasibility and eﬀectiveness of the algorithm.
L. J. Kong, A. L. Yan, Y. Li and J. Fan [11] introduce a novel method which consists of Least absolute deviation loss function and an L1/2 regularizer. It is a nonconvex,nonsmooth, and non-Lipschitz optimization problem. They design an eﬃcient alternating
direction method of multipliers to solve the problem and establish its convergence. Extensive numerical experiments demonstrate that the proposed method can recover sparse signal
with less measurements and is robust to dense bounded noise as well as Laplace noise.
M. Z. Li, H. B. Chen and G. L. Zhou [12] give a survey on recent advances of fourth-orderpartially symmetric tensors such as M-eigenvalue inclusion intervals, M-positive deﬁniteness,strong ellipticity condition and algorithms for computing the largest M-eigenvalue. Somefurther research questions are also released.
H. W. Li, H. B. Zhang and Y. H. Xiao [13] use an accelerated gradient method (AGM)
to solve the sparse subspace clustering problem, and establish its convergence in the senseof converging to a critical point with a certain stepsize policy. They show that closed-form
solutions are enjoyed for each subproblem by taking full use of the constraints’ structure.
Finally, they do numerical experiments by the using of two real datasets, which illustratethat AGM performs better than BDR and ABCGD evidently.
K. P. Liu, and H. T. Che [14] establish some new eigenvalue localization sets for fourthorder partially symmetric tensor. It is revealed that the new eigenvalue localization sets aretighter than some existing results. Numerical examples demonstrate the eﬀectiveness of theresults obtained. As applications, some bound estimations for the M-spectral radius andsome checkable suﬃcient conditions for the positive deﬁniteness of the fourth-order partiallysymmetric tensor are obtained based on the new eigenvalue localization sets.
Y. J. Liu and Q. X. Zhu [15] propose a semismooth Newton based augmented Lagrangian(Ssnal) algorithm for solving Weber problem, which is an important problem arising in thefacility location problems. The global convergence and locally asymptotically superlinear
convergence of the Ssnal algorithm are characterized under mild conditions. Numericalexperiments conducted on synthetic data sets demonstrate that the Ssnal algorithm outperforms several state-of-the-art algorithms in terms of eﬃciency and robustness.
Z. Y. Peng, X. J. Long, J. Zeng and Z. Lin [16] concern with the stability for a generalized
Ky Fan inequality when it is perturbed by vector-valued bifunction sequence and constrain
set sequence. By removing the assumptions of the strictly proper quasi C-convexity and thecontinuous convergence, they establish the Painlevé-Kuratowski convergence of the approximate solution mapping of a family for perturbed problems to the corresponding solution
mapping of the original problem. The obtained results are new and improve the corresponding ones in the literatures. Some examples are also given to illustrate the results.
Y. F. Shao, Q. S. Wang and D. R. Han [17] derive several new BB step sizes formulae
based on a more general distance measure, and apply them to the mirror descent method andthe Frank-Wolfe method. Compared with the two algorithms with traditional BB step sizes,
the preliminary numerical experiments demonstrate that the algorithms with the proposed
step sizes can achieve better performance.
J. Sun, P. Shang, Q. Y. Xu and B. Z. Chen [18] propose a new matrix regression model
which can induce a low-rank and sparse estimator in the sense of row-group with the helpof nuclear norm and ∥ · ∥2,1 norm. In order to obtain such an estimator, the authors develop
a linearized alternating direction method of multipliers and prove its global convergence.
Numerical experiments are carried out to demonstrate the properties of the new model andthe accuracy of the proposed algorithm.
C. M. Tang, H. Y. Chen, J. B. Jian and S. Liu [19] propose a bundle-type quasi-Newton
method for minimizing a nonconvex nonsmooth function. Global convergence of the al-iv

Z.-H. HUANG, L. KONG AND Y. WANG

gorithm is established in the sense that there exists an accumulation point of the serious
iterations such that it is a stationary point of the function. Superlinear convergence is proved
under suitable assumptions. Preliminary numerical results are reported to illustrate that
their method is eﬃcient and has advantages over the redistributed bundle method.
J. Wang [20] presents some generalizations of Von Neumann’s trace inequality for matrices to the content of tensors. The focus is on completely orthogonal decomposable tensors,
and the angle between two completely orthogonal decomposable tensors is taken into Von
Neumann’s trace inequality. Moreover, the properties of spectral functions in the case ofasymmetrically and symmetrically completely orthogonally decomposable tensors are discussed , respectively.
T. X. Wang, X. J. Cai, Y. Z. Song and X. Gao [21] study a class of diﬀerence-of-convex
programming. Based on the primal-dual reformulation of the problem, by adopting theframework of the double-proximal gradient algorithm (DPGA) and the inertial techniquefor accelerating the ﬁrst-order algorithms, the authors propose a double-inertial proximal
gradient algorithm. Under the suitable assumptions, each bounded sequence generated by
the algorithm is shown to converge globally to a critical point of the objective function.
Finally, the authors apply the algorithm to image processing model and compare it withDPGA to show its eﬃciency.
Y. X. Wei, Z. Y. Luo and Y. Chen [22] present a momentum accelerated version of theblock-randomized stochastic gradient descent algorithm for low-rank tensor CP decomposition. Under mild conditions, the authors show the global convergence to the stationary point.
Compared with the algorithms without momentum, the preliminary numerical experiments
demonstrate that the proposed accelerated algorithm is eﬃcient.
Y.R. Xu and S. J. Li [23] consider a class of parametric set optimization problems,
where both objective functions and constraint functions are perturbed by diﬀerent parameters. Some upper and lower set orderings with respect to improvement sets are introduced
and used to deﬁne solution mappings. Then, some assumptions including strong domination
properties are proposed to study the Hölder continuity of solution mappings and corresponding optimal value mappings. The obtained results generalize the upper Hölder continuity ofeﬃcient solution mappings for parametric vector optimization problems.
J. W. Zhang and Y. N. Yang [24] consider modiﬁed versions of the alternating leastsquares (ALS) and the regularized alternating least-squares (RALS) algorithms for tensor
decomposition. They propose two hybrid alternating methods by combining the extragradient method with Newton’s method. Theoretically, the step-size of the correction step
can be possibly chosen in a wide range. The global convergence of the algorithms are discussed under certain assumptions. Preliminary numerical experiments show the eﬀectivenessof the proposed methods, compared to the standard ALS and RALS algorithms.
D. Q. Zhou, X. K. Chang and J. F. Yang [25] propose and analyze a golden ratio primaldual algorithm for solving structured optimization problems involving the sum of three
convex terms. The proposed algorithm is of primal-dual and full-splitting type as it solves theprimal and the dual problems simultaneously and does not rely on solving any subproblems
or linear system of equations iteratively. The convergence rates O(1/N ) and O(1/N 2 ) areestablished for convex and strongly convex cases, respectively, which diﬀerentiate themselvesfrom existing results in terms of the adopted optimality measures. Finally, preliminary
numerical results are presented to demonstrate the eﬃciency of the algorithm.
We would ﬁrst like to thank the Editors-in-Chief, Prof. Masao Fukushima, Prof. Liqun
Qi, Prof. Jie Sun, Prof. Michel Théra and Prof. Xinmin Yang, who kindly approved thisspecial issue and also provided us valuable guidance in the editorial process. We would liketo thank all authors and referees for their contributions and help in publishing this special

PREFACEvissue.

References[1] Shujun Bi, Ting Tao and Shaohua Pan, “KL property of exponent 1/2 of ℓ2,0 -norm andDC regularized factorizations for low-rank matrix recovery”.
[2] Shih Yu Chang and Yimin Wei, “Sherman-Morrison-Woodbury identity for tensors”.
[3] Yannan Chen, Liqun Qi, Xinzhen Zhang, “Color image completion using a low-rankquaternion matrix approximation”.
[4] Lubin Cui, Qing Hu, “A chord-Zhang neural network model for solving absolute valueequations”.
[5] Mingyu Deng, Weiyang Ding and Xue Ping Guo, “Modiﬁed splitting methods for solving non-homogenous multi-linear equations with M-tensors”.
[6] Wenxin Du and Shenglong Hu, “A convergence analysis for an algorithm computing asymmetric low rank orthogonal approximation of a symmetric tensor”.
[7] Changjie Fang, Ruirui Zhang and Shenglan Chen, “An inertial Tseng extragradientmethod for solving multi-valued variational inequalities with one projection”.
[8] Renxiu Feng, Peiwei Zhong, Yitian Xu, “A subspace elimination strategy for accelerating support matrix machine”.
[9] Bingdi Huang and Peiping Shen, “Global optimization algorithm for solving sum oflinear ratios problems”.
[10] Hongwei Jiao, Junqiao Ma and Youlin Shang, “Image space branch-and-bound algorithm for globally solving minimax linear fractional programming problem”.
[11] Lijiao Kong, Ailing Yan, Yan Li and Jun Fan, “L1/2-regularized least absolute deviation
method for sparse phase retrieval”.
[12] Mengzhen Li, Haibin Chen, Guanglu Zhou, “Fourth-order partially symmetric tensors:
theory and algorithm”.
[13] Hongwu Li, Haibin Zhang, Yunhai Xiao, “An acceleratedgGradient method for nonconvex sparse subspace clustering problem”.
[14] Kaiping Liu, Haitao Che, “Tighter eigenvalue localization sets for fourth-order partiallysymmetric tensor and its applications”.
[15] Yongjin Liu and Qinxin Zhu, “A semismooth Newton based augmented Lagrangianalgorithm for Weber problem”.
[16] Zaiyun Peng, Xianjun Long, Jing Zeng and Zhi Lin, “On the stability of approximate
solution mappings to generalized Ky Fan inequality”.
[17] Yifan Shao, Qingsong Wang and Deren Han, “Eﬃcient methods for convex problems
with Bregman Barzilai-Borwein step sizes”.

vi

Z.-H. HUANG, L. KONG AND Y. WANG

[18] Jun Sun, Pan Shang, Qiuyun Xu and Bingzhen Chen, “Multivariate linear regression
with low-rank and row-sparsity”.
[19] Chunming Tang, Huangyue Chen, Jinbao Jian and Shuai Liu, “A bundle-type quasiNewton method for nonconvex nonsmooth optimization”.
[20] Jie Wang, “Von Neumann-type inequality for completely orthogonally decomposabletensors”.
[21] Tanxing Wang, Xingju Cai, Yongzhong Song and Xue Gao, “Double-inertial proximal
gradient algorithm for diﬀerence-of-convex programming”.
[22] Yangxin Wei, Ziyan Luo and Yang Chen, “Low-rank tensor Huber regression”.
[23] Yingrang Xu and Shengjie Li, “Holder continuity results for parametric set optimization
problems via improvement sets”.
[24] Junwei Zhang and Yuning Yang, “Hybrid alternating extra-gradient and Newton
method for tensor decomposition”.
[25] Danqing Zhou, Xiaokai Chang and Junfeng Yang, “A new primal-dual algorithm forstructured convex optimization involving a Lipschitzian term”.