Semantic Similarity Covariance Matrix Shrinkage
Authors: Guillaume Becquin
Saher Esmeir
Venue: EMNLP 2023 Findings (2023)
Keywords: semantics
embeddings
knowledge graphs
covariance
finance
portfolio optimization
TL;DR: Robust estimators for covariance matrices based on semantic similarity, with applications to portfolio optimization

Abstract: An accurate estimation of the covariance matrix is a critical component of many applications in finance, including portfolio optimization. The sample covariance suffers from the curse of dimensionality when the number of observations is in the same order or lower than the number of variables. This tends to be the case in portfolio optimization, where a portfolio manager can choose between thousands of stocks using historical daily returns to guide their investment decisions. To address this issue, past works proposed linear covariance shrinkage to regularize the estimated matrix. While effective, the proposed methods relied solely on historical price data and thus ignored company fundamental data. In this work, we propose to utilise semantic similarity derived from textual descriptions or knowledge graphs to improve the covariance estimation. Rather than using the semantic similarity directly as a biased estimator to the covariance, we employ it as a shrinkage target. The resulting covariance estimators leverage both semantic similarity and recent price history, and can be readily adapted to a broad range of financial securities. The effectiveness of the approach is demonstrated for a period including diverse market conditions and compared with the covariance shrinkage prior art.

Semantic Similarity Covariance Matrix Shrinkage
Guillaume Becquin Bloomberg gbecquin1@bloombergr net
Saher Esmeir Bloomberg
sesmeir Setmeirr22bloombergg
Abstract
Previous work
Our method
Text
An accurate estimation of the covariance matrix is critical component many applications in finance, including portfolio optimization. The sample covariance suffers from curse di- mensionality when the number of observations is in the same order or lower than the number of variables. This tends to the case in portfolio optimization, where portfolio manager can choose between thousands stocks using his- torical daily returns guide their investment decisions. To address this issue, past works proposed linear covariance shrinkage to regu- larize the estimated matrix. While effective, the proposed methods relied solely on histori- cal price data and thus ignored company fun- damental data. In this work, propose to utilise semantic similarity derived from textual descriptions or knowledge graphs to improve the covariance estimation. Rather than using the semantic similarity directly as a biased es- timator the covariance, we employ it shrinkage target. The resulting covariance es- timators leverage both semantic similarity and recent price history, and can be readily adapted to a broad range of financial securities. The ef- fectiveness the approach demonstrated a period including diverse market conditions and compared with the covariance shrinkage prior
Emil
Similanty matrix
Strinkage ponometers
Shrinkage
Shrihiage factor
Stock
Sample covarianc
Strunk
dats flow
prices
Literr shrivlage
Figure I: Instead of using the semantic similarity direct approximation to the covariance matrix, we use it build a target for linear shrinkage. This estimator can then be used for convex portfolio optimization.
general stock market conditions. Building upon advances in language modeling and graph embed- dings, recent work has investigated how to leverage semantic representations to guide investment deci- sions by estimating expected returns.
The volatility and the expected return of an as- set are both critical ingredients when forming an investment strategy. Risk-return trade-off is fun- damental trading principle describing the typical inverse relationship between given investment risk and investment return. In the common case of a multi-asset portfolio, an investor with a given risk tolerance seeks to maximise their return within the allowed risk level. In this case, the degree covariance between asset returns can be utilised to achieve an aggregate portfolio risk that is lower than its components. Modern portfolio theory for- malises this trade-off (Markowitz, 1952). The Mean Variance Portfolio aims at maximizing pected returns while minimizing expected variance:
1 Introduction
A wide range of domains rely on the analysis of high-dimensional time-series data. For example. this includes medical systems with MRT denoising (Honnorat and Habes, 2022), radar sensors (Kang et al., 2019) post-processing, physics and chem- istry, engineering, neuroscience, speech recogni- tion, and quantitative finance (Ledoit and Wolf, 2020b). These applications often require a reliable estimate of the covariance matrix. In the financial domain, the price of company's stock influ- enced, among other factors, by the fundamental characteristics of the company, news articles, and
w argmax (wμr Aw Iw)
(1)
where denotes the weights of assets in port- folio, μ the price returns of these assets, Σ the covariance matrix of asset prices and a param- eter to denote the level of risk aversion of the in- vestor. This optimization problem can solved effectively using convex solvers the covariance

---

matrix positive definite (and therefore invertible). Nevertheless, the covariance matrix of the assets prices is unknown and must therefore be estimated. Kan and Zhou (2007) show that the estimation er- ror for the covariance matrix has a higher impact than the mean estimation error when the number of assets considered is large compared to the number of observations.
In this work, we highlight the limitations of us- ing semantic similarity as direct estimation of the covariance matrix. We address these limitations by extending the established framework of linear shrinkage to semantic similarity matrices as seen in Figure 1. The semantic representations can be de- rived from both structured data stored in curated knowledge base or unstructured data in the form of natural language text. We share Python imple- mentation' of our method, allowing its integration as a simple post-processing step when building co- variance matrix estimators from semantic similarity matrices. We aim at validating the following:
Existing approaches leveraging semantic stock representations have focused on improving the esti- mate of the expected returns fa, using rough ap- proximation of the covariance matrix Σ by directly using the semantic similarity matrix as an estima- tor for portfolio optimization (Du and Tanaka-Ishin 2020; Li et al., 2022). However, these similarity matrix estimators may not be morphologically sim- ilar to a covariance matrix and are prone to bias. Indeed, while semantic models are effective at rank- ing similarities, their absolute similarity value is typically uncalibrated and depend on the fraction of the hypersphere occupied by the embeddings. Fur- thermore, their expected cosine similarity value de- pends on the dimension of the embeddings (Elekes et al., 2017). Since the true covariance matrix is obviously independent of the chosen embedding dimension, this would imply that single embed- ding dimension is valid for building semantic estimator, which is unacceptable. Finally, they give no guarantee of being positive semi-definite and ignore recent stock price observations.
Hypothesis 1: Semantic similarity depends on the embedding dimension and therefore cannot be used directly covariance matrix estimator.
Hypothesis 2: Semantic similarity can be used as an effective regularization target for covariance matrix estimation.
The proposed shrinkage target includes fundamental information about the random variables and is therefore less sensi- tive to sudden changes in volatility regime.
Hypothesis 3:
Related Work
Portfolio optimization using semantic represen- tations: Du and Tanaka-Ishii (2020) investigate the potential of using news data to train a classifier make buy and sell decisions over time. They take the cosine similarity matrix of intermediate stock representations as a direct estimator for the co- variance matrix to perform portfolio optimization. Similarly, Li et al. (2022) use normalized stock em- beddings trained on fundamental and price data to build similarity matrix and use it as estimate of the covariance matrix for aean-variance portfolio optimization. Both approaches suffer from the as- sumption that the similarity matrix is an unbiased estimator of the covariance. Our work instead uses semantic similarity as a shrinkage target to the sam- ple covariance matrix resulting in estimators that are asymptotically unbiased. Sawhney al. (2021) leverage knowledge graph (KG) to model the correlation between stocks through temporal hy- perbolic graph learning on Riemannian manifolds. They train a model to directly output a ranked list
An alternative estimator for the covariance ma- trix is the sample covariance matrix. While un- biased, the sample covariance matrix is singular when the number of observations N is lower than the number of random variables (preventing the use of convex solvers for equation 1) or ill-defindd when N e p. making the portfolio optimization task inaccurate (Ledoit and Wolf, 2004). Covari- ance shrinkage refers family of regularization methods aiming at combining the maximum likeli- hood sample covariance matrix with a structured, diwer-dimensionalttyy regularizing model (shrink- age target). The regularizing targets are built on a set of assumptions (e.g., constant variance and O-covariance between variables) whose degree of validity varies between asset classes and market regimes. Furthermore, the shrinkage estimators can be interpreted as empirical Bayes estimators where the prior is data-dependent, and computed from the same data as the sample covariance matrix (Ledoit and Wolf, 2020b).
'https: thub. ctm/cc htmm///oonbbrg///// semantic-sing larity covar weeovvaaanneessriinnee

---

of trade candidates, and therefore do not rely on the Modern Portfolio Theory mean-vart vreanvvrrannce optimiza- tion problem. Our approach instead leverages es- tablished portfolio optimization routines allowing better interpretability and modularity (decoupling the semantic representations generation from the investment strategy).
age (regularization) factor coefficient. Let XN be de-meaned price returns matrix of N observations for p assets (we omit the p for notation simplicity). The sample covariance matrix is a maximum like- lihood estimator (MLE) of the covariance and is defined as:
1 N
Covariance shrinkage: Linear shrinkage meth- ods, taking a convex linear combination between the sample covariance matrix and regularization target were originally proposed by Stein (1956) for the estimation of high dimensional multivariate mean, choosing the O-vector shrinkage target. Efron and Morris (1973) built upon this approach by choosing 1-parameter structured target built as the identity vector multiplied by the sample mean. This work was extended to a robust estimation of the covariance matrix in a high dimensional space by Ledoit and Wolf (2004), proposing a single- parameter shrinkage target built as the scalar prod- uct of the identity matrix and the average sample variance. Their choice of shrinkage target made the strong assumption that the stock price returns had a constant, common variance and no covari- ance. Ledoit and Wolf (2003a) instead assumed a constant and positive correlation between stock price returns, resulting a 2- P-aaamettrrs shrink- age target. The single-factor model (Ledoit and Wolf, 2003b) assumes the asset prices can be ap- proximated using the capital asset pricing model (CAPM) and linearly depend on the average market returns with sensitivity 8; and an offset o; captur- ing the stock idiosyncratic returns. The resulting shrinkage target contains p parameters to estimate, higher than the previous two models, but typically significantly lower than the p(p 1)/2 parameters of the full (symmetric) covariance matrix. These regularization targets are estimated using the same data used to compute the sample covariance matrix and are therefore prone estimation error if the number of samples is low relative to their num- ber of parameters. Our approach instead utilises similarity- based targets that are not derived from recent observations but rather trained on large seale semantic datasets. These therefore do not suffer from this parametrization trade-off and are less sen- sitive to market regime changes.
SN
X
(2)
The sample covariance matrix is symmetric and has p(p-1) parameters to estimate. When the number of observations N is lower than the number of fea- tures p, this matrix is non-invertible. Furthermore, unless < N, it is numerically ill-conditioned and the instabilities in the estimate lead to the underes- timation of the smallest eigenvalues and overesti- mation of the largest ones (Ledoit and Wolf, 2004). An alternative is to replace this MLE by lower capacity model making a series of assumptions on the random variables observed (for example con- stant variance, no correlation or factor model as introduced previously). This option is also unsatis- factory as it would suffer from a high bias because of the strong modeling assumptions implied. The approach of linear shrinkage is build a new esti- mator from a convex linear combination of (1) the high variance, low bias the sample covariance matrix; and (2) the low variance, high bias the shrinkage target (lower capacity model) T:
2
ΣN XT +11 A)SN A [0.1]
(3)
The objective is to minimize the mcan square error between this estimator and the population covari- ance Σ" The loss function is therefore given by
Lx =JÊN Σ I + (1 X)SN 13
(4)
The expected risk is
R(A) E(Cx) = E(JAT + +(1 X)SN E1
(5)
3 Preliminaries
The population covariance unknown and not observable but Ledoit and Wolf (2003b) show that the optimum X coefficient is asymptotically con-
this section, we formally describe the framework linear shrinkage and derive the optimum shrink-

---

stant over N and has the following form:
4.1
Semantic shrinkage target
Our approach for building a shrinkage target from (6) embeddings matrices is as follows. We first normal- ize the embeddings matrix E along the semantic dimension, bounding the dot product between two asset representations in [-1; 1]. We then build (p, p) similarity matrix Sim by taking the product of the embeddings matrix with itself.
π
P
A/
N
7
p
Σ
AsyVar(VNS))
AV-
i=1 P Σ>
1=1
AsyCCou(VNI (
PN
j=1
Sim EE"
(10)
P YN- Σ (Ty - Sij)
Σ j=
(9)
This results in a matrix where the diagonal en- tries are 1 while each off-diagonal entry (i,5) repre- sents the degree of semantic match between assets i and j. Rather than directly considering Sim approximation of the covariance matrix as in (Li et al., 2022), we instead consider Sim to be an approximation of a correlation matrix (based on its constant valued diagonal and entries bounded in 1; 1]. We map this semantic correlation to a covariance matrix using the covariance definition and the asset variance estimated from the sample covariance matrix. The correlation between two random variables x and y is defined as:
where AsyVar ar and AsyCov respectively refer to the asymptotic variance and covariance when the number of observations N grows larger.
Intuitively, N measures the degree of variance in the sample covariance matrix, with high values indicating high variance for the sample covariance estimator. A high TN would therefore result a higher amount of required shrinkage. PN measures the level of covariance between the sample covari- ance matrix and the shrinkage target: a high covari- ance indicates the chosen target provides limited additional information and therefore the optimum shrinkage amount should be reduced (Ledoit and Wolf, 2020b). Finally, "N measures the distance between the unbiased sample covariance matrix and the target. A higher value indicates a biased shrinkage target, and therefore leads to a lower optimum shrinkage factor.
coolx, v) corr(x, v) V
(11)
By analogy, we define the similarity- co- variance as:
SimCov
Sim =
(12)
v diiggSS))diag(Sy
4 Semantic Similarity Shrinkage
SimCov = Sim O
V /dag(SS)) diag(Si))
diagg (SN)
Our proposed approach is based on the availabil- ity of embeddings (vector representation) for the random variables considered. In the context of financial markets, this translates into the availabil- ity of a (p. k) embeddings matrix E built from k-dimensional semantic vectors for the p financial assets considered for the portfolio optimization. These embeddings may be generated using text description of the assets (e.g.. description of the company that issued stock) and sentence em- beddings model such as (Reimers and Gurevych, 2019), KG embeddings, intermediate represen- tation generated from a downstream task such buy/sell classifier (Du and Tanaka-Ishii, 2020). We first propose semantic shrinkage target built us- ing these embeddings and then derive the optimum shrinkage factor.
(13)
where @ is the Hadamard product and diag (Sx) is (1, vector made of the diagonal entries of the sample covariance matrix Sx- This approach ensures that the scale of the semantic-based shrink- age target is on the same scale of the sample co- variance matrix, regardless of the original stock embeddings space. This estimator therefore has p price-dependent parameters to estimate (the vari- ances), similar to the CAPM-based single factor model (Ledoit and Wolf, 2003b).
Finally, ensure that the shrinkage target is positive definite, a desirable property that allows the use of the shrunk covariance matrix with convex optimizers. While the covariance matrix SimCov is by construction symmetric, it is not necessarily positive definite. We introduce small perturbations

---

on the eigenvalues following spectral decomposi- tion of SimCov based on (Gill et al., 2019). Let U and be the eigenvectors and eigenvalues from the the spectral decomposition of SimCov:
with respect to the observations and symmetric, a full derivation is available in Appendix A.
p
PN
N
SimCov = UUUT
(14)
(20)
P Σ iデj
Bjj
(133)
Dani +
8is
f3.65
If (A) < 0, we shift the eigenvalues so that they are all strictly positive with margin e:
2
(V
8t
8jj
Where the dis are consistent estimators of SsyCoo(VNSMa VNSN,5) given by:
A+ A + min(abs(a))) A)) +e
(15)
We then recover the semi-definite similarity- based covariance shrinkage target:
1 N
Jas
Σ (o - (N Sw.j)) (21)
SimCov+ = U
(16)
Finally, a consistent estimator of YN is given by:
We do not use this value directly to approximate the covariance matrix; instead we build a linear shrinkage estimator from the convex linear combi- nation of the target and sample covariance matrix per equation 3:
YN i= = =|SimCov+ +i
(SimCov+i; SN.is²
(22)
St.
(23)
This fully specifies the optimal shrinkage factor based on
Σ Esim ASimCov N
(17)
4.2 Semantic shrinkage factor
Experiments
As mentioned in Section 3, the optimum shrinkage factor for the similarity- based covariance estimator can be derived analytically using equations 7 to 9. The following provides consistent estimators when assuming a semantic similarity target. Index refers time, while and j refer to the position of the random variables in the covariance matrix. The derivation is similar to constant correlation shrink- age (using similarity values instead of constant value) and is provided in Appendix A. A consistent estimator of N is given by (Ledoit and Wolf, 2020b):
5.1 Datasets
We evaluate the performance of the proposed similarity-based covariance matrix shrinkage the task of portfolio selection. In particular, we consider the universe of historical members for 3 major stock indices: the Standard & Poor's 500 (S&P 500), the NASDAQ-ION and the Nikkei 225 (Nikkei). We choose this data based on the avail- ability of price data and high quality company fun- damental data that can be used to generate stock embeddings. We report results for the S&P 500 for conciseness; the full set of results for all 3 indices available Appendices C and D.
P
ĀN
N..ij
(18)
5.1.1 Stock price return data We gather price data over a period of 16 years from 2007 to 2023, This extended period includes a variety of market regimes including high volatility events, such as the global financial crisis in 2007- 2008, the sovereign debt crisis in 2011, and the coronavirus pandemic in 2020. It also includes periods of low volatility such 2017, when the volatility index (VIX) was historic lows. We build the dataset follows:
i=1
)=1
#
(INWii - SN.ig²
N
1=1
X2 - S2.0
(19)
N
with x2, = (XN xw) 02 the matrix of element- wise squared de-meaned returns.
We obtain a consistent estimator fv based the constant variance target proposed in (Ledoit and Wolf, 2003a), where the constant correlation value is replaced by entries the similarity matrix. We use the fact that the similarity matrix is constant
We split the data in rolling windows of 3 months for estimator fitting and month for evaluation.

---

2. The stock indices composition evolved over the period considered. Within each window, we keep only index members with price data throughout the entire period.
3.
We convert the end-of-day price into daily price returns, removing the effect of the abso- lute stock price.
Litilees
Consumeet honeyctical
lechrology
This results in 190 slices, each covering 4 month span (the first 3 months are used for fitting and the last for evaluation).
40
consumer Cydlical
Frincel
Conmunccatins
Eneryy
Industiall
Basic Matorios
5.1.2 Semantic representations
Figure 2: 2-D projection of the KG embeddings using t-SNE. Each point corresponds to historical S&P 500 component colored by BICS industry sectors.
We compare the effectiveness of the proposed ap- proach using three methods for computing stock semantic representations. The hyperparame- ters used for the training of the semantic models is available in Appendix E.
5.2 Performance of semantic-based shrinkage targets
The first method leverages the text description for the companies that issued stock. We build se- mantic representation for the companies using 768 dimensions sentence embeddings transformer model (Reimers and Gurevych, 2019). The experi- ments were conducted using a DistilBERT model (Sanh et al., 2019) finetuned on the Natural Ques- tions dataset (Kwiatkowski et al., 2019). The sec- ond method leverages relational facts from knowl- edge graphs. We use publicly available 512 dimen- sions RotatE pre-trained embeddings on the Wiki- data5m dataset generated using Graph Vite (Zhu al., 2019), referred to W5m KG. For both methods match the stock price data Wiki- data entry using the Wikidata5m dataset (Wang et al., 2021) containing entity names and aliases. We successfully match 98.3%, 97.9% and 99.3% of index members for the S&P 500, NASDAQ-100 and Nikkei 225, respectively. The unmatched en- tries are excluded from both training and evaluation rolling windows.
We build the covariance matrix for each evaluation period in 5.1.1 and annualize the results by multi- plying the entries by the number of trading days per year (252). For each of the 3-month training peri- ods (first 3 months), we fit the following covariance estimators:
Sample covariance matrix Ssample-
Constant-variance shrunk covariance matrix Ecvar using 1-parameter shrinkage target assuming constant variance and covari- ance (Ledoit and Wolf, 2004).
Constant-correlation shrunk covariance ma- trix Êccor using 2-parameters shrinkage target assuming constant correlation (Ledoit and Wolf, 2003a).
Single- factor shrunk covariance matrix Esr assuming single factor CAPM model for the stock price returns (Ledoit and Wolf, 2003b).
We then consider third model using a subgraph from large scale, financial knowledge graph in Bloomberg herein referred to as Fin. KG. This includes triples involving the domicile country, in- dustry, sector, board members, supply chain, sub- sidiaries, fund and index inclusion. We train KG embeddings using RotatE (Sun et al., 2019) with 128 dimensions embeddings size.
Scaled Text similarities TText, using the Wikipedia text embeddings similarity matrix transformed using 13.
Scaled KG similarities Ew5m KGCsclled and ΣFm. KGGacoled using the KG similarity ma- trices transformed using 13.
We observe that the representations produced by these methods capture the overall semanties of the company as illustrated in Figure We build the semantic similarity matrices for all semantic models following the method described 4.1.
Shrunk covariance matrix ΣTExt, using the Wikipedia text embeddings similarity matrix.
Shrunk covariance matrices Σwsm KG and Epm.. KG using the KG similarity matrices.

---

0.0030
9.75
Fin, KG embedddings similarity
embeddings simiarity Text embeddings similanity
0.0025
8
Constant variance target
Shrinkage target covariance
Single factor target
6.73
0.0020
PRIAL
4.47
0.0015
3.68
3.64
2.59
0.0010
2.22
2
1.43
0.0805
0.0000
Condtant Variance
Constant Conelation
Single factor
Sealed Rin. KG Sim.
Scaled W5m Sim.
Scaled Text Sim
Shrunk Fin. KG Sim.
Shrunk W55 Sim.
Shrunk Text Sim.
-0.0005
000005
2000
0.00s
0010
0.8015
0,2020
0.0025
0.0030
-0
Sample covariance
Figure 3: Off-diagonal entries of shrinkage targets rela- tive the sample covariance.
Figure 4: PRIAL for the S&P 500 for 2007-2023
Estimator
Mean ||-1|x
following 1-month sample test covariance matrix an approximation the population covariance. We evaluate a given estimator Σ using the Percentage Relative Improvement in Average Loss (PRIAL) defined as (Ledoit and Wolf, 2020a):
Sample covariance
0. 0.128
Text Sim.
0.066
W5m KG Sim.
0.101
Fin. KG Sim.
0.114
Table 1: Average off-diagonal Frobenius norm
PRIAL(E) = 110 (([(Stampt)) -B()))
B(C(E))
E(C(SSammle)
(24)
We begin by observing the impact of the shrink- age model assumptions on the resulting target bias. We sample 500 (time, stock pair) observations for all estimators for the S&P 500 and compare their value to the corresponding entry in the sample covariance matrix for the same (time, stock pair) (see Figure 3). Values along the diagonal indicate shrinkage target with bias. By construction the constant variance model has 0 value on the off-diagonal and therefore the most biased esti- mator considered. The semantic estimators show a variable level of bias. The average Frobenius norm of the similarity estimators (excluding the di- agonal variances) is given in table 1. The norm of the unbiased sample covariance matrix is given for reference. It shows that different semantic models may result in significant differences in similarity values and therefore are prone to bias when esti- mating the covariance, validating hypothesis 1.
C refers to the MSE loss introduced in equation 4. Figure 4 shows the PRIAL for all estimators the entire period (190 data points) for S&P 500. The constant variance model is the best per- forming price-based shrinkage target. Using the scaled similarity matrix shows a high level of per- formance variability and shows the impact the bias for the text-based embeddings (resulting in a worse estimator than the sample covariance matrix). The higher performance of the scaled KG similarity is highly deppseet and driven by high volatility events during the period considered (see Appendix B). Our shrunk similarity- based models are outperforming price-based shrinkage methods and their scaled variant. Figure shows the out- performance of shrunk similarity-based methods is consistent across time windows and not driven by isolated high volatility outliers. We perform a sta- tistical test of the null hypothesis (the performance of similarity-
In order evaluate the performance of the shrinkage targets, we compute the optimum shrink- age factor based on 4.2 and shrink the sample co- variance matrix for each 3-month fitting period to build an estimator, We evaluate these against the
models and constant variance shrinkage method is equal) that is rejected with p-values of 1.3 10 12 and 2.5 KG-based and text-based similarity targets respec- tively, validating hypothesis 2.
for the

---

S&I 500
Nikkei 225
Nasdaq-100
Sample-
28
22
400
30
1.0
71
25
Constant Variance-
54
70
57
73
65
73
Single factor
87
46
100
70
27
66
50
57
47
91
43
100
80
74
51
55
46
94
45
100
68
76
70
59
56
51
Method
Scaled Fin. KG Sim.-
34
34
30
100
52
43
22
26
30
40
34
36
100
88
45
29
31
28
33
31
32
100
41
34
23
28
28
w5m Sim.-
23
16
18
10
29
1.00
Scaled Text Sim.
34
27
1000
Shrunk fin. KG Sim.
72
51
50
78
100
51
44
73
49
49
90
76
100
53
65
46
77
79
60
100
45
43
Shrunk W5m KG Sim.
78
51
43
74
84
66
49
100
43
80
45
45
69
88
77
47
100
20
76
40
44
12
81
63
55
100
40
Shrunk Sim,
68
58
70
56
57
100
63
57
54
90
79
57
100
75
59
72
77
75
60
100
Constant Vertance
Sumple
Single factor
Stated Fin KG Stmn
Scated W5m KG Sin-
Scalnd Tert Sim-
Shrunt Fin KG Sim.
Shrunt W5m Sim.
Shrunk Test Sim.
Constant Var leriance
Sample
Single factor
Scaled Fin. KG Simt
Scated W5m KG Sim.
Scaled le Sim-
Strunt Fin Simt
Shrunk 45m KG Sint
Shrunt Text Sim:
Constant Variarce
Sample
Single foctor
Stated Frn. KG Sim.
Scated WSn KG Sim
Scaled Text Sim.
Shrunk Fin KG Sini
Shrunk W5m KG Sini
Strunk Test Sim
Reference method
Figure 5: Percentage of rolling windows when method (left) MSE is lower than reference method (bottom). The
semantic shrinkage methods error is consistently lower than their scaled variant and estimators using price data only.
5.3 Impact of volatility regimes on shrinkage target performance
PRILIEN so aman? (RIL[
40
30
We further investigate the impact of market regime change. In order identify the contribution of specific samples the, we define the Percentage Relative Improvement in Loss (PRIL) based on 24:
PRUL
-10
Market volatility
PRIL()) = 100 L(S (SSsmplee) - L(E)
-20
2018
2019
2020
2021
2022
2023
(25)
(aSmmll)
Figure 6: S&P 500 PRIL for the KG-based similarity and constant variance shrinkage targets. The shaded area shows the difference between the PRIL values. An indicator of market volatility is shown on the right axis.
Figure 6 focuses on years 2017 to 2023 for the S&P 500 during which several market regimes were observed, including: low volatility (2017 the end of 2018); two high volatility periods (end of 2018, 2022); and a very high volatility event (first half of 2020). We show PRII. val- ues for both the KG-based similarity and con- stant variance shrunk covariance models together with the realized annualized log market volatility (log([)SSamptcll) * # trading days)) on the right axis. The outperformance of the mumilarity-bssed estima- tor is higher during periods (or transitions between periods) of high market volatility.
tail risk" (short-term move of more than 3 stan- dard deviations). This validates the Hypothesis 3 and shows that the proposed covariance matrix esti- mators are more robust during transitions between market volatility regimes.
Discussion
In this work, we present family of covariance shrinkage estimators leveraging semantic similar- ity between the random variables. We show that semantic similarity cannot be used as a direct estimator of the covariance, but instead as regu- larization for the sample covariance using linear shrinkage. The shrunk estimators include price- independent semantics and are superior in periods of high volatility, where an accurate estimation of the covariance is typically more critical. This demonstrates the potential of semantic models for covariance matrix estimation and extends beyond
We compute the PRIL difference between the KG-based similarity and constant variance shrunk covariance models (shaded area in Figure 6) for all test periods between 2007 and 2023 to validate this effect, and present the results in Figure 7. We note that the degree of outperformance of the similarity- based approach correlates with the level of market volatility. The effect is particularly pronounced for the very high volatility levels, indicating the potential of the approach to mitigate market "fat

---

period backtest. Future work may use historical Wikipedia snapshots to measure the impact of this assumption. However, we estimate this risk be low given the focus on company fundamentals. The price data frequency is typically much higher than frequency of change of the fundamental data (de- scription or KG triples): the company characteris- tics (e.g., industry sector) of the large capitalization investigated did not change significantly over the period considered. Furthermore, the analysis does not show drop in performance before and after the datasets snapshot date. Future work including dynamic semantic components (e.g., news) should however ensure that only news released prior to an experiment time window are used, since news do have significant impact on short term price data (and correlation) as illustrated by recent work referenced in our work. Similarly, the rolling win- dow evaluation excludes companies that stopped trading during the period considered: this inclu- sion/exclusion process is a-priori unknown.
09x 0.26
mittieen eol (ooe))
-0.1
-0.2
-0.3
-0.4
-0.5
Realized annualized lag vol.
Figure 7: PRIL difference between KG and constant variance shrunk estimators: performance benefits in- crease with the overall market volatility
traditional applications to predicting returns. The proposed method is agnostic to the embed- dings generation process and can be applied to most existing semantic representation algorithms. This includes semantic representations that capture static (company description) and dynamic (com- pany news, recent price data) components from text, knowledge graphs or multi-modal datasets. By decoupling the semantic model from the port- folio optimization, the proposed method provides robust and modular mechanism for integrating semantic representations into established portfolio management and investment decision systems.
The approach requires both rich semantic repre- sentations and an effective way to match the stock price returns to their embeddings. The matching process may be more challenging for international companies smaller capitalization stocks than for the S&P 500, NASDAQ-1O0 components. Finally, while the approach could be extended to any financial asset class including commodities or derivatives, obtaining a valid text description may prove more challenging. Note that the knowledge-gng fer from this limitation and can generalize to any asset class where structured data is available, and future work may investigate this method potential for multi-asse asset portfolio (i.e., equities, fixed income and commodities) covariance estimation.
Nikkei 225
7 Limitations
embeddings do not suf-
While showing encouraging results, the proposed similarity-basee targets are built from static se- mantic information. potential future work di- rection could the extension these static em- beddings with dynamic components, including for example news data or dynamic knowledge graphs. Text embeddings from news data (capturing com- pany recent events) can be combined with the static embeddings (capturing company fundamentals) to compute either single similarity matrix target for shrinkage, or use techniques such as multiple gets shrinkage. Building semantic representations from news data will involve challenges related to sampling for companies with high news coverage, the lack of news for some companies and the choice of importance given to the static versus dynamic components.
The input the semantic models (text and KG triples) was gathered as of 2021 and 2022, poten- tially leading to data leakage for the 2007-2023

---

References
Olivier Ledoit and Michael Wolf. 2020a. Analytical nonlinear shrinkage of trrge-dimensional covariance matrices. 3065.
Xin Du and Kumiko Tanaka-Ishii. 2020. Stock embed- dings acquired from news articles and price history, and an application to portfolio optimization. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational L Linguistics, pages 3353- 3363, Online. Association for Computational Lin- guistics.
The Annals of Statistics, 48(5):3043
Olivier Ledoit and Michael Wolf. 2020b. The Power of
Shrinking: A Review and Guide to Covariance Matrix Estimation. Journal of Financial Econometrics, 20(1):187-215 218.
John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(61):21212159.
Shicheng Li, Wei Li, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, and Keiko Harimoto. 2022. No stock is island: Learning internal and relational attributes of stocks with contrastive learning. In Pro- ceedings of the Fourth Workshop on Financial Tech- nology and Natural Language Processing (FinNLP), pages 147-153, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Bradley Efron and Carl Morris. 1973. Stein's estima- tion rule and its competitors-ai an empirical bayes approach. Journal of the American Statistical Asso- ciation, 68(341); 1): 1 17-130.
Harry Markowitz. 1952. Portfolio selection. The Jour- nal of Finance. 7(1):77-91
Ábel Elekes, Martin Schäler, and Klemens Böhm. 2017. On various semantics similarity word embed- ding models. In Proceedings of the 17th ACM/IEEE Joint Conference Digital Libraries, JCDL 17, page 139- 139. IEEE Press.
Nils Reimers and Iryna Gurevych. 2019. Sentence- BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCN) pages 3982 3992, Hong Kong, China. Association for Com- putational Linguistics.
Philip E. Gill, Walter Murray, and Margaret H. Wright. 2019. Practicul Optimization. Society for Industrial and Applied Mathematics, Philadelphia, PA.
Nicolas Honnorat and Mohamad Habes. 2022. Covari- ance shrinkage can assess and improve functional connectomes. Neurolmage, 256: 15921.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019, Distilbert, distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, 1ts/1990.01I 108.
Raymond Kan and Guofu Zhou. 2007. Optimal port- folio choice with parameter uncertainty. Journal of Financial and Quantitative Analysis, +2(3):621-656
Ramit Sawhney, Shivam Agarwal, Arnav Wadhwa, and Rajiv Shah. 2021. Exploring the scale-free nature of stock markets: Hyperbolic graph learning for algo- rithmic trading. In Proceedings of the Web Confer- ence 2021, WWW '21, page 11 11-22, New York, NY, USA. Association for Computing Machinery.
Naixin Kang. Zheran Shang, and Qinglei Du. 2019, Knowledge-aided structured covariance matrix esti- mator applied for radar sensor signal detection. Sen- sors, 13)..
Charles M. Stein. 1956. Inadmissibilityy of the usual estimator for the mean of multivariate normal dis- tribution. In Berkeley Symp. on Math. Statist. and Prob., 1956: 197-206.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming- Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natu- ral questions: a benchmark for question answering research. Transactions of the Association of Compu- lational Linguistics.
Zhiqing Sun, Zhi-Hong Deng, Jian- Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. CoRR, abs/1992.
Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. 2021. Kepler: A unified model for knowledge embedding and pre-trained language representation. Transac- tions of the Association for Computational Linguis- tics, 9:176-194.
Olivier Ledoit and Michael Wolf. 2003a. Honey, i shrunk the sample covariance matrix. In UPF Eco- nomics and Business Working Paper No. 697.
Olivier Ledoit and Michael Wolf. 2003b. Improved estimation of the covariance matrix of stock returns with am application to portfolio selection. Journal of Empirical Finance, 1065):603-621.
Zhaocheng Zhu, Shizhen Xu, Meng Qu, and Jian Tang. 2019. Graphvite: high-performance cpu-gpu hy- brid system for node embedding. In The World Wide Web Conference, pages 2494-2504. ACM.
Olivier Ledoit and Michael Wolf. 2004. A well- conditioned estimator for large-dimensional covari- ance matrices. J. Multivar, Anal,, 88(2):365-111.

---

A Semantic Target Shrinkage Factor Estimation
The first term is equal the diagonal of the N.j estimator calculated previously. As per (Ledoit and Wolf, 2003a), using the delta-method an es- timator for AsyCou( SSSSis Vij) is given by:
For notation simplicity we omit in the following the . index from the sample covariance matrix.
A.1 Derivation of the n estimator
A consistent estimator of T is given by (Ledoit and Wolf, 2003b):
Si AsyCoo(VVNeggii VN S
1
S AsyCou(V VNSs)
HN
WN.
(26)
V
Snj
1=1
(31)
N
1 ANJj N
Since the Ejj are constant with respect to the observations and symmetric (eig = eji), this can be re-written:
(aear S,,)
(27)
NN can be rewritten allow for factorized computation, using the fact that SN,i is constant with respect to the sum over the samples:
Sii AsyCou(VNSi, VNSig) S
(32)
N
Si AssyCou( VNS
1
#N.6
Siz)²
N
1
N
2
N 25
FriTtj S
A AsyCov( SNS V is given by (Ledoit and Wolf, 2003a):
consistent
estimator
for
N
N
t=1
1
2
Sij $
N
Tit
Ja,ú N
1
Σ
(1 Sa)( Si,)) (33)
N
1 N
2
St
And by symmetry:
1
X2N
X
(28)
N ح3ر = N
N
(2l - (ay)a Siz) (34)
with X2N = XN) the matrix of element- wise squared de-meaned returns.
The resulting consistent estimator for pN is given by:
A.2 Derivation the p.v estimator
We follow the derivation an estimator for 8 as- suming shrinkage target built on a constant corre- lation matrix provided in (Ledoit and Wolf, 2003a), replacing the constant correlation value Fx by indi- vidual semantic similarities matrix entries eij. By definition (see 8):
AN diag (fN)
Eij
Sit 1 jj.25
(
853
(35)
p P
PN AsyCou(V VNS)) (29)
AsyVar(VNS,,
+
Σ
AsyCou(V (NS)))
(30)

---

B Comparison of Scaled and Shrunk Estimators for 2018-2023 period
It can be seen that the estimator under performs both the constant variance shrunk estimator (Figure 9) and the shrunk covariance using KG similar- ity matrix target (Figure 10). A similar effect for the text encoder semantic similarity can be seen in Figure 11 to Figure 13, showing the systematic un- derperformance of uncalibrated semantic similarity matrices for covariance estimation.
This uppendix provides additional experimental results for the similarity-based estimators for the 2018 to 2023 period. Figure shows that while the Percentage Relative Improvement in Average Loss is higher for a naive estimator built using the scaled similarity as a covariance estimator, the outperformance is concentrated in single period of very high volatility. Since the loss is mean squared error, the large covariance values (and their error) seen during this period dominate the general evaluation period.
50-
PRILI
PRIL (RIL(
2ed zed
PRIL
50
-10
PRILIZes xs mmse!
40
PRILL
Real leed annua aze log vol
-20
Market volatiity
2018
2019
2020
2021
2002
2023
30
20-
Figure 11: Relative performance of the Text similarity shrunk estimator to the constant variance estimator
10
-10
Market vola atility
100.
PRILL PRIL(Éew
2018
2019
2020
2021
2022
2023
Reallaed naruaalaa log
Figure 8: Relative performance of the Fin. KG similar- ity shrunk estimator to the constant variance estimator
PRIL
-100
PRILI asavel PRILI
150
Roal ledd annualized log vol
Market volatility
2020
2021
2022
2023
Figure 12: Relative performance of the Text similarity scaled estimator to the constant variance estimator
-100
-150
-200
Market
100
PRIL Ereet PRILI aoadd
2019
2019
2020
2023
Figure 9: Relative performance of the Fin. KG similar- ity scaled estimator to the constant variance estimator
Realized urnualized
PRUL
-50-
100
50
PRILIEES ance)
Rcalized annual aed log vol
150
volatility
PRIMI E ap Read
2:02
-50
Figure Relative performance of Text similarity
-100
shrunk estimator to the Text similarity scaled estimator
150
-200
Market volatility
2018
2019
2023
Figure 10: Relative performance of the Fin. KG similar- ity shrunk estimator to the KG similarity scaled estima- tor

---

C Experimental results for NASDAQ-100
Sample
67
71
55
24
25
This appendix provides provides the experimental results for the NASDAQ- 100 components. Figure 14 shows the PRIAL (defined in equation 24) over the entire experimental period. The results align with the S&P 500, showing a strong variance in the average performance of scaled semantic estimators (with the text semantic similarity under-performing the sample covariance estimator). This variance is significantly lower for the shrunk estimators (our method).
Constant Veriance
69
13
52
Single factor
Scaled Sim.
28
Method
Scaled W5m KG Sim.
28
27
24
50
21
14
23
Scaled Texx Sim.
37
Shrunk Fin. KG Sim.
Shrunk Sim.
44
Shrunk Text Sim.
75
66
60
Constant Variance
Sample
Single factor
Scated fin. KG Sint
Scated W5m KG Sim:
Scaled Text Simi
Strunk fin. KG Sim.
Strunk wsm KG Simi
Shrumk Text Sim.
Reference method
6.84
Figure 15: Percentage rolling windows when a method (left) MSE is lower than reference method (bottom) for the NASDAQ-I00 index.
5.53
4.63
PRIAL
3.10
00.0
2.32
PRILLien ve.trnom
1.98
-0.5
1.56
PRILLion!
-1.0
20
-1.
10
-2.5
0-
Constant Variance
Constant Correlation
Singls factor
Stated Fift. KG Sim
Scaled WSm KG Sim.
Scated Toxt Sim.
Shrunk Fin. KG Sim
Shrunk W5m KG Sim.
Strunk Tixt Sim.
-00
-20
Markot volatilty
2018
2019
2020
2023
2022
2023
Figure 16: Relative performance of the Fin. KG similar- ity shrunk estimator to the constant variance estimator
Figure 14: PRIAL for the NASDAQ-100 for 2007-2023
The improved consistency of the proposed esti- mators is highlighted Figure 15. This shows that even scaled estimators showing a high PRIAL score (such as the scaled KG similarities) do not consis- tently perform better than other estimators. The PRIAL metric is an average metric and therefore sensitive to outliers. The scaled similarity methods are biased towards lower covariance values (see Ta- ble leading very significant error reduction during volatility regime transition periods when the sample covariance is poor estimator of future covariance (such as the low to high volatility tran- sition during the coronavirus pandemic), but worse performance than other estimators overall.
PRILI mnstrokk
-0.5
PRILI sosscase
-1.5
PAIL
-50
2.0
-100
3.0
volatility
200
2019
2021
Figure 17: Relative performance the Fin. KG similar- ity shrunk estimator to the KG similarity scaled estima- tor
A similar effect can be seen for the text-based estimators in Figures 18 to 19.
The PRIL (defined in equation 25) allows visual- ization the contribution of each time window to the PRIAL. Figure 17 confirms the previous observa- tion: the semantic shrinkage estimator outperforms its scaled similarity variant over the entire period, except for the transition into the high volatility events 2020.
Similarly to results presented for the S&P 500 in 7, the degree of improvement from semantic shrink- age estimators for the covariance prediction shows a positive correlation with a measure of overall market volatility, as illustrated in Figure 20.

---

0..0
40-
PRILI Erawe monn
30
PRILI Ecyerl
-1.0
10g
1.5
-2.
Realizeo
-10-
-20
Market volatiltty
2018
2019
2020
2021
2022
2023
Figure 18: Relative performance of the Text similarity shrunk estimator to the constant variance estimator
PRILI Ei ancel
-0.5
PRU PRILlEren.ce dand
1.5
-2.0
-100
V
-150
-.00
-200
-250
volatility
2020
2022
Figure 19: Relative performance of the Text similarity shrunk estimator to the Text similarity scaled estimator
0.4
0.15x 0.39
0.2
-0.2
-0.4
-1.5
Realized annualized log vol.
Figure 20: PRIL difference between KG and constant
variance shrunk estimators for the NASDAQ-100

---

D Experimental results for Nikkei 225
This appendix provides experimental results for the Nikkei 225 components. Figure 21 shows the PRIAL (defined in equation 24) over the entire experimental period. The PRIAL value for the scaled estimators is, unlike for the other two in- dices, higher than for their shrunk variant, with all semantic estimators sigmificantly outperforming the estimators based on price only (constant variance, constant correlation and single factor shrinkage).
Sample
22
20
Constaint Variance
Single
64
55
Method
Scaled Fin. KG Sim.
40
24
36
60
24
31
20
Scaled W5m KG Sm.
20
12
10
12
Scaled Text Sim.
Shrunk
53
Shrunk W5m KG Sim.
80
45
85
60
18
47
20
14
13.34
Shrunk Text Sim.
Constant Variance
Sumple
Single factor
Scaled Rr. KG Sim.
Scaled Wam KG Sim.
Scaled Text Sim.
Shrunk fin KG Sim.
Shrunk W5m KG Sim.
Shrunk Roxt Sim.
11.66
10
9.30
Reference method
PRIAL
7.33
7.22
6.10
Figure 22: Percentage of rolling windows when a method (left) MSE is lower than reference method (bottom) for the Nikkei 225 index.
3.97
1.81
1.20
Consrant Variance
Consrant Correlation
Single factor
Scaled Rin KG Sim
Scaled W5m KG Smm
Shrunk Fin. KG Sim.
Strunk W5m KG Sim,
Shrunk Text Sim.
Scaled Text
40
PRILISOn atron
PRILIion
-1.
2.0
PRIL
Figure PRIAL for the NASDAQ-100 for 2007-2023
-10
This result can be explained by the sensitivity of the PRIAL metric outliers. Figure 22 shows that the improved PRIAL of the scaled estimators over their shrunk variant is not consistent: it can be seen that the scaled similarity estimators only outper- form the price-based and the semantic shrinkage estimators 20 40% of the time (middle 3 rows). The semantic shrinkage estimators, on the other hand, tend be consistently better than the other estimators (bottom rows). This is also illustrated by the PRIL in Figures 24 and 24: the scaled sim- ilarity outperforms its shrunk variant only in the 2020 volatility transition period. The errors made by all estimators during this period are significantly higher than average, biasing the average PRIAL metrics.
200
Markot volatilty
-4.0
-30
2018
20:19
2020
2021
2022
2023
Figure 23: Relative performance of the Fin. KG similar- ity shrunk estimator to the constant variance estimator
PRILIÉeS stoal
PRIL/
-1.
0
2.0
PRIL
50
2.5
-75
3.0
335
-25
Market volatiity
Providing reliable estimate of future covari- ance is desirable feature and the proposed shrunk semantic similarity estimators show a consistent improvement over their scaled variants.
Figure 24: Relative performance of the Fin. KG similar- ity shrunk estimator the KG similarity scaled estima- tor

---

E Semantic Models Hyperparameters
E.3 Financial Knowledge Graph Embeddings
E.1
Wikidatašm Knowledge Graph
Parameter
Value
Embeddings
Architecture
RotatE
(Sun et al., 2019)
Parameter
Value
Dataset Dimension
Proprietary
Architecture
RotatE
128
(Sun et al., 2019)
Optimizer
Adagrad
Dataset
Wikidata5m
(Duchi et al., 2011)
(Wang et al., 2021)
Learning rate
1e-1
Dimension
512
Weight decay
lc-7
Optimizer
SGD
Margin
19.9
Learning rate
le-2
Batch size Epochs
1024
Weight decay
0
20
Margin
6
Sample batch size
512
Batch size
100k
# Negative/positive
2
Epochs
1000
Adversarial temp.
1.0
Sample batch size
2000
# Negative/positive
64
Table 4: Financial KG Embeddings hyperparameters
Adversarial temp.
0.2
Table 2:
Wikidata5m KG Embeddings hyperparameters
E.2
Text Embeddings
Parameter
Value
Architecture
DistilBERT (Sanh et al., 2019)
Dataset
Natural Questions
(Kwiatkowski et al., 2019)
Dimension Hidden dim.
768
3072
# layers
6
# heads
12
Activation
GeLU
Loss
Multiple Negatives Ranking
Dropout
0.1
Cls. dropout
0.2
QA dropout
0.1
Table 3: Text Embeddings hyperparameters


Reviews:
[{"review_id": "YHT5JJggkS", "reviewer": "Reviewer_EMEZ", "summary": "", "questions": "The authors should address the weaknesses mentioned in the above section.", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the suggestions. \n\n1: **Additional stock indices**: Experiments on Nikkei 250 and Nasdaq 100 have been performed (same full experiment set provided for S&P 500). Results align with S&P 500 findings and will be included in the camera ready paper.\n\n2: **Static/Dynamic semantics** The proposed approach does not include dynamic semantic information, and future work could investigate news data as an additional semantic input. Text embeddings from news data (capturing company recent events) can be easily combined with the static embeddings (capturing company fundamentals) to compute either a single similarity matrix to be shrunk, or use techniques such as multiple targets shrinkage. Challenges with this approach will be the sampling of news data to be representative of the company context, lack of news for some companies and importance given to the static versus dynamic components. These suggestions will be added in the limitations section\n\n3: **Data leakage**: The price data frequency is typically much higher than frequency of change of the fundamental data (description or KG triples): the company fundamental characteristics (e.g. industry sector) of the large capitalization investigated did not change significantly over the period considered. Furthermore, the analysis does not show a drop in performance before and after the snapshot were selected. For future work including dynamic data (e.g. news), it will be critical to ensure that only news released prior to an experiment time window are used, since news do have a significant impact on short term price data (and correlation) as illustrated by recent work referenced in our work. These considerations will be included in the camera ready paper.\n\n5: **Hyperparameters**: The proposed method is a post-processing step to semantic models, rather than a new semantic modeling approach. Our shrinkage method’s only parameter (the shrinkage factor) is analytically calculated. Nevertheless the full set of hyperparameters for the training of the 2 KG embeddings and the sentence embedding model will be added to the appendix.\n\n6: **Further applications**: The applicability of the approach to other domains using large scale covariance matrices will be illustrated in the introduction via a few additional examples (both finance applications: company replacement in a fund, value at risk or time series methods  and non-finance related: medical imaging and sensors).\n\n7: **Redacted information** The information was redacted for anonymity purposes. The source of the knowledge graph and github repository will be un-redacted for the camera-ready version\n\nGrammar style and presentation improvements have been addressed", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes an approach to improve covariance matrix estimation. It introduces a method that extends the linear shrinkage framework to semantic similarity matrices.\n\n2. The proposed approach utilizes semantic similarity, derived from textual descriptions or knowledge graphs, as a shrinkage target for covariance matrix estimation. It leverages both semantic similarity and recent price history, unlike existing linear covariance shrinkage methods, which rely solely on historical price data.\n\n3. The paper examines and highlights the limitations of using semantic similarity matrices directly as covariance matrix estimators.\n\n4. The paper conducts an experimental evaluation of the proposed method using historical members of the S&P 500 index over a period of 16 years. The authors compare their approach with traditional ones under diverse market conditions and demonstrate its superiority, especially during periods of high market volatility.", "reasons_to_reject": "1. The paper relies heavily on historical S&P 500 data for the experimental evaluation. More experimentation on other stock indexes and regions could further validate the approach.\n\n2. The proposed similarity-based targets are built from static semantic information. They do not incorporate dynamic components, such as recent news data. The authors acknowledge this and suggest it as a direction for future work.\n\n3. The input to the semantic models (text and knowledge graph triples) was gathered as of 2021 and 2022. For the 2007-2023 period backtest, there might be a risk of data leakage. However, the authors estimate this risk to be low.\n\n4. Although the authors have mentioned the limitations in the paper, they should provide a more detailed plan on how they plan to address these drawbacks in their future work.\n\n5. The authors should provide a detailed hyperparameter analysis.\n\n6. The paper focuses on the portfolio optimization setting. Extensions to other financial applications utilizing covariance matrices could benefit the paper.\n\n7. The Python Github Repository is REDACTED on Page 2. Also, some information is REDACTED on Page 6. The authors should mention why they are redacted and when they will be available.", "questions_to_authors": "The authors should address the weaknesses mentioned in the above section.", "paper_topic_and_main_contributions": "The paper presents an approach to improve covariance matrix estimation for portfolio optimization. Covariance matrix estimation often suffers from the curse of dimensionality. This is a common scenario in portfolio optimization, where portfolio managers choose between thousands of stocks using historical daily returns.\n\nWhile existing linear covariance shrinkage methods are effective, they rely solely on historical price data and ignore company fundamental data. To overcome these limitations, the authors propose to extend the linear shrinkage framework to semantic similarity matrices. They utilize semantic similarity, derived from textual descriptions or knowledge graphs, as a shrinkage target for covariance matrix estimation. This approach leverages both semantic similarity and recent price history.\n\nThe paper examines and highlights the limitations of using semantic similarity to directly estimate the covariance matrix. It argues that while semantic models are effective at ranking similarities, their absolute similarity value is typically uncalibrated, dependent on the fraction of the hypersphere occupied by embeddings, and their expected cosine similarity value depends on the dimension of embeddings.\n\nThe authors propose three hypotheses:\n1. Semantic similarity depends on the embedding dimension and cannot be used directly as a covariance matrix estimator.\n2. Semantic similarity can be used as an effective regularization target for covariance matrix estimation.\n3. The proposed shrinkage target includes fundamental information about random variables and is less sensitive to sudden changes in the volatility regime for those variables.\n\nThey validate these hypotheses using a dataset comprising historical members of the S&P 500 index over a period of 16 years and demonstrate that the semantic similarity-based models outperform the existing price-based shrinkage methods, especially during periods of high market volatility.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Figure 2 caption: \"BICS industry sector\" should be \"BICS industry sectors\" (plural)"}, "extra_text": {}}, {"review_id": "HzhIjXNZK7", "reviewer": "Reviewer_NdPC", "summary": "", "questions": "**A.** An important quantity in the evaluation of covariance estimation methods is the MSE loss funciton (4), which is based on the population covariance $\\Sigma^*$. I couldn't find the definition or a description of population covariance in the paper. Though this is a standard concept in statistics, it is unclear what it refers to in the specific datasets in the experiments, or how its value is computed in the evaluation.\n\n**B.** Line 422: While 98.3% of the stocks are mapped to a Wikipedia entry, how did you handle the rest 1.7%?\n\n**C.** Line 495: The claim that higher dimension models have lower off-diagonal similarity values sounds premature. The models in Table 1 having different dimensions are different types of models, so the dimension is not the single variable here. You can train each type of model with different embedding dimensions. If all types of models exhibit the same pattern, the above claim will be valid then.\n\n**D.** In Table 1, should the sample covariance matrix be scaled as well (making the diagonal entries be 1), for a fair comparison of the off-diagonal Frobenius norms?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the improvement suggestions.\n\nWe believe this paper would be a valuable contribution to the NLP community for the following reasons:\n- While illustrated for finance applications, the approach is valid for all domains where highly dimensional covariance matrices must be estimated (e.g. medical imaging, sensors) and not restricted to a finance audience\n- The NLP community has a strong focus on creating state of the art semantic models, yet even recent work (2022 and later) has assumed that the resulting similarity matrices were valid estimators for the covariance matrix, which is incorrect as demonstrated by our work. We aim to grow awareness of these issues with this work.\n- Linear shrinkage is an effective and widely used method in quantitative finance for highly dimensional covariance matrix estimation, but is not widely known in the NLP field. We believe our contribution would benefit the NLP community by illustrating how an established method can be adapted for use with NLP semantic models.\n\nA: **Population covariance definition** The true population covariance risk is unobservable since the distribution of the random variables is unknown. The derivation of the optimum shrinkage factor uses asymptotic analysis. For evaluation purposes, we use the empirical risk via the realized sample covariance matrix over the test set as an estimate of the population risk. Further details have been added to the article for the definition of the loss and evaluation metric.\n\nB: **Stocks not found in Wikidata** The 1.7% remaining stocks were excluded from the experiment for both training and evaluation - note added\n\nC: **Dimensionality and Frobenius norm** The claim linked to the dimensionality of the embeddings will be removed to maintain focus, and replaced by the observation that the expected frobenius norm of the similarity matrix depends on the underlying semantic model (e.g. text or KG). \n\nD: **Scaling of the sample covariance** The sample covariance matrix is an unbiased estimator of the population covariance and should not be scaled. The aim of this table is to show other estimators are on the other hand biased (different than the sample covariance matrix) : It is a necessary condition for the estimators proposed to have a Frobenius norm close to the sample covariance matrix for them to be unbiased estimators. This point will be clarified in the camera ready version\n\n**Typos and grammar**:\nLines 105-107: Linear shrinkage estimators built using price data can be seen as empirical Bayes estimators using a data dependent prior (using the same data used to evaluate the likelihood). Thank you - a clarification note and reference were added.\nRest of suggestions has been implemented.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper could belong to the \"NLP Applications\" track, and is of specific interest to readers in the FinTech community who apply NLP techniques. The paper is an exhibition of wider adaptability of embedding models than normally presented in the conference. The paper shows that the semantic similarity matrix cannot be used directly as the covariance matrix, but as a shrinkage target.", "reasons_to_reject": "Overall, the paper would be more interesting to the EMNLP audience if it compares the embedding models in more detail and analyzes why one model has better performance than another one. The current writing uses language models and knowledge graphs as a black box and might be more suited to a FinTech conference.\n\nThe technical part is mostly sound, but I have a few questions listed below that need to be resolved. Also, the experiment stops at the estimation of covariance matrix, and the paper doesn't present real-world benefits of the methods such as in portfolio optimization. I would expect a more thorough set of experiments that shows the impact of better covariance estimation with the proposed method.", "questions_to_authors": "**A.** An important quantity in the evaluation of covariance estimation methods is the MSE loss funciton (4), which is based on the population covariance $\\Sigma^*$. I couldn't find the definition or a description of population covariance in the paper. Though this is a standard concept in statistics, it is unclear what it refers to in the specific datasets in the experiments, or how its value is computed in the evaluation.\n\n**B.** Line 422: While 98.3% of the stocks are mapped to a Wikipedia entry, how did you handle the rest 1.7%?\n\n**C.** Line 495: The claim that higher dimension models have lower off-diagonal similarity values sounds premature. The models in Table 1 having different dimensions are different types of models, so the dimension is not the single variable here. You can train each type of model with different embedding dimensions. If all types of models exhibit the same pattern, the above claim will be valid then.\n\n**D.** In Table 1, should the sample covariance matrix be scaled as well (making the diagonal entries be 1), for a fair comparison of the off-diagonal Frobenius norms?", "paper_topic_and_main_contributions": "This paper provides a method for better estimating the covariance matrix of different stocks in a stock market. The method is based on shrinkage methods in statistics and the semantic embeddings of companies generated by existing language models and knowledge graphs. Experiments show that the proposed method achieves lower estimation error, and validate a few hypothese regarding how semantic similarities should be used in the context of portfolio optimization.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The authors could use more references in some of the writing, as sources for their claims. For example, lines 105-107: where does this claim come from, and why is it a shortcoming? (readers may guess but it's better to state clearly)\n\nThe section titles \"Linear shrinkage\" and \"Semantic similarity shrinkage\" look as if the proposed method was not linear shrinkage. It may be better to change the former to \"Preliminaries\".\n\nThe definitions of some notations are missing.\n* Line 95: N and p were not defined until Line 207.\n* Line 356: Where are the definitions of ti, tj?"}}, {"review_id": "pkUlwNl2fw", "reviewer": "Reviewer_yKen", "summary": "", "questions": "(1).In line 516, it is claimed that all values are positive. However, \"scaled text simu\" is -1.57. Do I missing anything?\n(2) again, there are 190 data points. I believe only the average values of PRIAL are presented in Figure 4. Why don't you present some other statistics, such as what is the percentage of data points that the semantic methods outperform?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the suggestions.\n\nA: **All PRIAL values positive** line 516: all semantic similarity *shrinkage* estimators improve upon the baseline (the negative value is for the raw similarity matrix that was used in the prior art). This section has been updated for clarity, thank you.\n\nB: **Evaluation**: a matrix showing the percentage of data points where a method outperforms other methods has been added. This effectively captures how often a method outperforms the others and eliminates the impact of outliers on the performance difference. This shows that our method not only improves on the average values (via PRIAL), but is also more consistent and reliable (especially when compared to the raw or scaled similarity matrix estimator used in the prior art). We have run extended statistical this analysis on the S&P 500 and additional indices (Nikkei 250, NASDAQ 100). Thank you for the suggestion.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) The paper introduces a new methodology for covariance matrix estimation in portfolio optimization, which incorporates semantic similarity derived from textual descriptions or knowledge graphs. The linear shrinkage method was proposed by Stein in 1956. The innovation of this paper lies in using semantic information to estimate the shrink target matrix.\n2) The experiments shows that various semantic methods(either DistilBERT or W5m KD) improves the estimation of the covariance matrix.\n", "reasons_to_reject": "1) The exposition and clarity of the paper could be improved. Certain sections, such as the derivation of the shrinkage factor estimators, may be difficult for readers to follow without more detailed explanations or examples.", "questions_to_authors": "(1).In line 516, it is claimed that all values are positive. However, \"scaled text simu\" is -1.57. Do I missing anything?\n(2) again, there are 190 data points. I believe only the average values of PRIAL are presented in Figure 4. Why don't you present some other statistics, such as what is the percentage of data points that the semantic methods outperform?", "paper_topic_and_main_contributions": "The estimation of covariance matrix of financial assets is a notoriously difficult problem given its high dimension. This paper proposes a novel approach for covariance matrix estimation by leveraging semantic similarity derived from textual descriptions or knowledge graphs. The authors use the semantic similarity as a shrinkage target rather than a direct estimator of the covariance, resulting in covariance estimators that combine both semantic similarity and recent price history. The effectiveness of the approach is demonstrated using historical S&P 500 data and compared with other covariance shrinkage methods.\n", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. At the end of Introduction section, you may summarized the main findings of this paper instead of just enumerating the three hypothesis."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a new methodology for covariance matrix estimation in portfolio optimization, which incorporates semantic similarity derived from textual descriptions or knowledge graphs. The innovation of this paper lies in using semantic information to estimate the shrink target matrix. Overall, this could be an interesting application paper.
Decision: Accept-Findings