Exploring AI for Auto-tuning
through Sparse Matrix Image
Information
Takahiro Katagiri (Nagoya University, Japan)
Collaboratos:
Mr. Shota Aoki (ex-M2, Graduate School of Informatics),
Prof. Masatoshi Kawai (ITC, Nagoya University)
45th ASE Seminar (Advanced Supercomputing Environment): International Workshop on “Integration ofSimulation/Data/Learning and Beyond” 13:45 - 14:00, November 29 (Wednesday), 2023
東京大学柏Ⅱキャンパス 情報基盤センター 4階 T412, 柏Ⅱキャンパス 情報基盤センター

Aim of the Group1.

Development of AT framework for mixedprecision computations and power
consumption optimization
 Directive-based AT language:

Proposal of a new extension functions for
ppOpen-AT2.

Development of AT facility with AI
 Adaptation of deep learning technology for auto-tuning facility
 Adaptation of Explainable AI (XAI)ASE45

Outline Background PICCG Parameter Tuning Conclusion

ASE45

Outline Background PICCG Parameter Tuning Conclusion

ASE45

Background Utilizing unverified AI prediction results contributes tovarious social problems.
 Human sense verification is essential for ensuring theaccuracy of AI predictions.
 To minimize production costs, there is extensive researchin the field of Software Auto-tuning (AT) technology.
 The integration of AI into AT facilities is currently
advancing.
→ It is crucial to validate the “explainability of AI” bytuning performance parameters on numerical libraries.
→ Scientific XAI (SXAI)
ASE45

Explainable AI (XAI) Can we explain result from machine learning?

 Explainable AI (XAI) This is sorted as the following two categories [1] Explainability
 AI technology for reason of prediction to be understood byhumans easily. Ex) LIME, SHAP Interpretability
 AI technology to show process for prediction by analysing

inner structures. Ex) Make a decision tree.

→In this study, we focus on the “Explainability.”[1] Otsubo et. al, “XAI ( Explainable AI)︓What does AI think in that time? ”,
RIC telecom, 2021, In Japanese
ASE45

Global Explanations and Local Explanations
 Global Explanations
 To understand “whole attribution of AI model. ”[1] Ex) SHAP

 Local Explanations
 To understand “Judgement result of prediction foreach result.” [1] Ex) LIME
→ In this research, the both of all are treated for AI of ATto numerical libraries.
[1] Otsubo et. al, “XAI ( Explainable AI)︓What does AI think in that time? ”,
RIC telecom, 2021, In Japanese
ASE45

LIME (Local Interpretable Model-agnosticexplanations) [2] Local Surrogate Model

An explainable model for explanation of each prediction on Blackbox model.

 Show reason to be understood by humans.
 Explain prediction from classifier to analyze contributions for eachfactor.
Obtaining AI model by varied nearest data from target explainable data.

 Adaptable for arbitrary clarifiers.
 Drawbacks
1.
2.
3.

Explanation is not stable. (By using random factors.)
Needs tuning for hyper parameters.
There is a case that cannot explain.
→ The other approaches are required.

[2] M. T. Ribeiro, S. Singh, and C. Guestrin: Why should I trust you?: Explaining the predictionsof any classifier, Proc. of 22nd ACM SIGKDD, pp.1135-1144, 2016. ASE45

SHAP（SHapley Additive exPlanations） [3] Shapley Value, by theory of cooperative game, is adapted tomachine learning.
There is a reasonable background in viewpoint of theory.

 Approximate Shapley value is calculated.

Tree-based ensemble models : High speed and accurate Shapley value.
 Deep learning models : High speed and approximate Shapley value.
 General Algorithms : Estimated Shapley value.
 Drawback
1.

Computational complexity is high.
→ Utilize approximate value in usual.

[3] S. Lundberg, S-I. Lee, A Unified Approach to Interpreting Model Predictions, 2017https://arxiv.org/abs/1705.07874
ASE45

Outline Background PICCG Parameter Tuning Conclusion

ASE45

PICCG Parameter Tuning

Preconditioner of Incomplete Cholesky
Factorization• Incomplete Cholesky (IC) factorization preprocessing isan iterative method for solving a system of linear
×(， ，),equations, such aswith a sparse symmetric matrix as coefficients.
• IC is used as a preconditioner for conjugate gradient(CG) method. The algorithm is called PICCG method.
• A zero-valued element of may become non-zero in thedecomposition matrix . This is called fill-in.
• The basic IC decomposition rejects (treats as 0) all fill-ins.
It can keep the number of nonzero elements small.
• The basic IC is also reduce the computational complexity,
are very different, then it may not
but if and 𝑡work as a preconditioner matrix.
ASE45

IC Preconditioner with a Threshold Set the maximum fill-in level ( ) and the threshold value ( ).
 Treat fill-in smaller than the threshold value as 0 for fill-in
below the maximum fill-in level, and generate fill-in above
the threshold value.
𝑖−1𝑑𝑖,𝑗 = 𝑎𝑖,𝑗 −𝑢𝑖,𝑘 𝑑𝑖,𝑖 𝑢𝑘,𝑗
𝑘=1

𝑓𝑖,𝑗 =0,𝑓𝑖,𝑘 + 𝑓𝑘,𝑖 + 1,𝑡𝑎𝑖,𝑗 ≠ 0
𝑒𝑙𝑠𝑒，

×

𝑖−1𝑢𝑖,𝑗 =−1𝑑𝑖,𝑗(𝑎𝑖,𝑗 −𝑢𝑖,𝑘 𝑑𝑖,𝑖 𝑢𝑘,𝑗 ),𝑘=10,𝑓𝑖,𝑗 ≤ 𝑚 ∧ |𝑢𝑖,𝑗 | ≥ 𝑡𝑒𝑙𝑠𝑒

𝑎𝑖,j:𝑖,𝑗 element of𝐴，𝑑𝑖,𝑖:𝑖,𝑖 element of 𝐷，𝑢𝑖,j: 𝑖,𝑗 element of𝑈，𝑓𝑖,j: fill-in level of𝑢𝑖,j，: a threshold treating 0， : maximum fill-in level

Parameters affecting execution timeASE45

Regression Model to Predict Calculation Timefor ICCG Method with Threshold• Utilize Tensorflow• Input
1. Feature image of coefficient matrix(created by the method of Yamada et al. [4])2. Maximum fill-in level and Threshold• Output：Execution time ofICCG method with threshold

1.
2.

CNN

Learning Settings
Number of Epochs
200
Batch Size
256 convolutional layer+Activation Function
ReLU
Pooling layer: 2 layers
Optimization
Adam Method
Fully connected
mean squared
Loss Function
layers: 3 layers
error[4] K. Yamada, T. Katagiri, H. Takizawa, K. Minami, M. Yokokawa, T. Nagai, M. Ogino, ”Preconditioner AutoTuning Using Deep Learning for Sparse Iterative Algorithms”, 2018 Sixth International Symposium on
ASE45
Comput. and Networking Workshops(CANDARW), pp. 257-262 (2018).

Maximum fillin Level
Threshold
ValueMLPMaking Dataset (Training and Test Data)
Measure execution time under the following conditions withthe Supercomputer “Flow" Type I subsystem:• Coefficient Matrix• Sizes： 4096x4096, 32768x32768, 262144x262144 : 3 Kinds• Different of Condition Numbers for each size: 90 Kinds• Maximum fill-in Level• 0, 1, 2 : 3 kinds• Threshold Values• 0.001 ~ 0.02 by stridden 0.001: 199 Kinds

Training data for each size: 41,073 Kinds
Testing data for each size: 10,269 Kinds
a regression model to predict execution time of PICCG methodwith threshold for each size is generated.
ASE45

Result of Machine Learning• Minimum, average, and maximum values of measured values for all data.
• Mean absolute error between the observed execution time and the model'sprediction time on the test data
Problem sizes

Estimated
Min. Time (s)Average Time (s)Estimated
Max. Time (s)Mean Absolute Error(ratio to Ave. Time)

4096 x 4096

0.00528

0.00746

0.04230.000888 (11.9%)32768 x 32768

0.01400.0196

0.03790.000659 (3.3%)

262144x2621440.05480.09800.118

0.00409 (4.1%)• There is a mean absolute error of 10% or less with respect to the averagevalue, except for 4096x4095. Hence, the model is reasonable.
• We show SHAP's explanation of how the maximum fill-in level and threshold
affect the computation time for this model in next slide.
ASE45

Description by SHAP (In case of max fill-in level 2)
Has the effect ofincreasing execution time

4096×4096

32768×32768
Max fill-in Level 2
Max ill-in Level 1
Max fill-in Level 0

Max fill-in Level 2
Max fill-in Level 0
Max ill-in Level 1

Has the effect ofdecreasing
execution time

262144×262144
Max fill-in Level 2
Max ill-in Level 1
Max fill-in Level 0# of fill-in is smaller.
= smaller memory amount = weaker preconditioner

When the maximum fill-in level is 2 (red point)
For 4096 x 4096 and 32768 x 32768 matrices, according to smaller thethreshold, shorter execution time is observed. → ReasonableIn the 26,2144 x 26,2144 matrix, the execution time tends to be shorten when
the threshold is between 0.035 and 0.050.
→ Reasonable (in the tanning data)
ASE45

Description by SHAP (In case of max fill-inlevel 0 and 1)
Has the effect ofincreasing execution time

4096×4096

32768×32768
Max fill-in Level 2
Max fill-in Level 0
Max ill-in Level 1

262144×262144
Max fill-in Level 2
Max ill-in Level 1
Max fill-in Level 0

Max fill-in Level 2
Max ill-in Level 1
Max fill-in Level 0

Has the effect of
decreasing execution time

In case of maximum fill-in level is 0 (blue point) and 1 (purple point)
Actual: Since there is no fill-in at less than fill-in level 1 in this training data, the execution
time does not depend on the threshold. →The behavior should not change whether themaximum fill-in level is 0 or 1.
Explanation of SHAP: according to smaller the threshold, the shorter the execution time isobserved. → Not reasonableExplanation of SHAP: according to higher the maximum fill-in level, more susceptible
to the threshold is observed. → Not reasonable
ASE45

Adapting One-Hot encoding• One-Hot encoring: A method of converting qualitative data intovariables (dummy variables) that express 0 and 1.
Feature Feature Feature
Value 0 Value 1 Value 2

FeatureValue
Max fill-in
Level 00

Max fill-in
Level 0100

Max fill-in
Level 1

1Max fill-in
Level 1010

Max fill-in
Level 22Max fill-in
Level 2001

• The maximum fill-in level is represented by three features.
• The corresponding feature quantity is set to 1, and the others areset to 0.
• One-hot encoding is adapted for the same training data, then learn
data without changing the conditions for “the number of featuresin the numerical data.”
ASE45

Comparison with Machine Learning Result4096 x 4096

0.000888

Mean Absolute Error
without One-Hot
Encoding(Improvement %)0.000204 (435%)

32768 x 32768

0.000659

0.000526 (125%)

262144 x 262144

0.00409

0.00370 (110%)

Problem
Sizes

Mean Absolute Error
without One-Hot
Encoding

For all problem sizes, the mean absolute error decreased
by One-Hot encoding.
→ The model performance is improved.
ASE45

Comparison of descriptions by SHAP
32768 x 32768 without One-Hot Cording

32768 x 32768 with One-Hot cording

Max fill-in Level 2
Max ill-in Level 1
Max fill-in Level 0The erroneous explanation that “the larger the maximum
fill-in level is, the more likely it is to be affected by the 0threshold” has been removed.
→ Closer to human interpretation.
ASE45

Outline•Background•PICCG Parameter Tuning•Conclusion

ASE45

Conclusion Remarks
Explainable AI (XAI) tools can be adapted to AI outputs in the context ofanomaly detection. Even within a sparse iterative algorithm, they can provide
coherent explanations.
Utilizing XAI tools such as LIME and SHAP, we derive sensible explanations
based on the distinctive features of the targets.

We have identified a pivotal traditional technique for refining AI models:the implementation of One-Hot encoding.

Future work

1.

Development of AT method with XAI.
 Reinforcement learning for explainable variables with strong
factors.
 Reduce AT time by reducing explainable variables with weakfactors.

2.

Adaptation of AT for mixed-precision computations.
 Automatic selection of double and single
computations in PICCG method.
ASE45

ASE45