サイエンティフィック・システム研究会 科学技術計算分科会 2017年度会合

マテリアルズインフォマティクスの最前線

吉田 亮

1,2,3,4

yoshidar@ism.ac.jp

1 情報・システム研究機構 統計数理研究所 モデリング研究系 准教授
2 情報・システム研究機構 統計数理研究所 ものづくりデータ科学研究センター センター長
3 総合研究大学院大学 複合科学研究科 統計科学専攻 准教授
4 物質・材料研究機構 特別研究員

1

“外挿的予測と発見” のデータ科学
Data Science towards Extrapolative Prediction and Discovery

 データサイエンスの予測は “内挿的” (材料開発者の経験や勘も同様に)
 “外挿的予測と発見” を実現するシナリオ？

ORGANIC SOLAR CELLS×SPACIERDFT properties of 106 compounds with amodel calibration to experimental data
HOMO-LUMO gap in [1.3, 1.5] (eV), PCE(power conversion efficiency) >> 11%
Aspuru-Guzik, A(Harvard Univ.)2Bayesian Inverse Materials Design

ベイズ推論で新物質を発見する

3

MACHINE LEARNINGSAILING THE BIG MATERIAL SPACE
e.g. 1060 small-sized organic molecules (GO game: 250150)

Identification of new promising materials with a given property requirement

MULTI-OBJECTIVE, COMBINATORIAL OPTIMIZATION

4
Supplementary Movie in Ikebata et al., J Compt Aided Mol Des, 2017

BAYESIAN INFERENCE
Transforming forward models into the backward prediction

Given data

FORWARD PREDICTIONUnknown

BACKWARD PREDICTION

Posterior - backward prediction

Backward

ForwardPrior

Likeliness of S with respect to observed Y

Likelihood - forward prediction
Goodness-of-fit of S with respect to Y

Prior distribution

Inversion

Prior knowledge on S5

BAYESIAN INVERSE MATERIALS DESIGN
Ikebata et al., J Compt Aided Mol Des, 2017

backward Y → S

forward S → Y

prior

R package iqspr v2.3 @CRAN
https://cran.r-project.org/web/packages/iqspr/index.htm

R package iqspr v2.3 @GitHub
https://github.com/GLambard/inverse-moleculardesign6高分子材料ビッグデータに基づく材料探索を中心に

ベイズ推論×7

PolyInfo (@NIMS) PROJECTMorikawa (Tokyo Tech)
Xu, Kuwajima, Kondo (NIMS)
Hongo (JAIST)
Yamada, Lambard (ISM)

Wu S(ISM)WORLD LARGEST POLYMER DATABASE × STATE-OF-THE-ART MACHINE LEARNING
Target: higher thermal conductivity polymers

Structure (S)Descriptor (φ)
01000100000011010011

S = molecular structure of repeat unit (monomer)(and/or process/other parameters)

Properties (Y)λ: thermal conductivity
Tg: glass transition temp.
Tm: melting points
and so on

8Quantitative Structure-Property Relationship

順方向の物性予測モデルを作る

9

FORWARD PREDICTION
Quantitative structure-property relationship analysis (QSPR)Structure (S)Descriptor (φ)

Properties (Y)Physiological activity
ToxicityBand gap
Energy01000100000011010011

Deep Learning
Elastic Net
Random Forest
Gradient Boosting
Support Vector Machineetc

Molecular fingerprint
PubChem
MDL MACCS keys
ECFP6
Kolekota-Rothetce.g. molecular fingerprint( 1,0,1,1,1,)
10

R PACKAGES: DESCRIPTOR & ML ALGORITHM
Regression and classification

Descriptors in the rcdk package

MethodPackage

Name

Description

Elastic-net

glmnetstandard

Paths of a given length

Deep Learningh2o, mxnet

extended

standard + ring + atomic property

Support Vector Machine

kernlabmaccs

166 bits MDL MACCS keys

Gaussian Process regression

GPfit, laGPcircular

ECFP6Boosting

Boostpubchem

881 bits PubChem fingerprint

Bagging

Ipred

graph

standard + connectivity

Random Forest

randomForest,
ranger, Rboristkr

4860 bits Kolekota-Rothhybridization

standard + Info. on hybridization

shortestpath

shortest paths between atoms

signaturecount type of fingerprint

Gradient Boosting

xgboostHyperparameter optimization
MethodPackage

Bayesian OptimizationrBayesianOptimization+ more than 200 physical and chemical descriptors

11

POLYMER PROPERTY
Glass transition temperature
Correlation between observed and predicted Tg with different descriptors and ML models
circularhybridization
kr

standard extendedgraph

Densityshortestpath

pubchemmaccs
estateCorrelation between observed and predicted Ds with different descriptors and ML models
hybridization

standard extended

graph

kr

estatemaccs

circularpubchemshortestpath
12

One of the grand challenges in Materials Informatics

HIGH-THROUGHPUT SCREENING
高品質・大量の仮想物質を作製・保有することが，莫大な価値を生み出す
Millions of virtualmaterials libraryMaterial property
Descriptor space

Machine Learning based HTS (w/wo BayesianOptimization) for materials property prediction
Gómez-Bombarelli et al., Nat Mater, 15, 1120-1127, 2016
Seko et al., Phys Rev B, 89, 054303, 2014
Seko et al., Phys Rev Lett, 115, 205901, 2015
Balachandran et al., Sci Rep, 6, 19660, 2016
Kiyohara et al., Sci Adv, 2, e1600746, 2016
Rupp et al., Phys Rev Lett, 108, 058301, 2012
Pilania et al., Sci Rep, 3, 2810, 2013

Deep Learning for solving Schrödinger equation
Mills et al., Deep learning and the Schrödinger equation,arXiv:1702.01361, 2017
Carleo et al., Solving the quantum many-body problem withartificial neural networks, Science, 355(6325), 602-606, 2017

入力はポテンシャル画像，出力は波動関数

13Inverse Quantitative Structure-Property Relationship

逆方向の計算で埋蔵物質を発掘

14

BACKWARD PREDICTION= Low hit rate - when the input space is exceedingly highP(Y|S) Limitation
dimensional, the exhaustive search is impossible to perform
Virtual materials

Properties
Descriptor spaceP(S|Y) ∝ P(Y|S)P(S)

Explore promising materials intensively aroundhigh probability regions of the posterior

Virtual materials

PropertiesDescriptor space

15

BACKWARD PREDICTIONMONTE CARLO STRUCTURE GENERATOR

Creation of promising hypothetical materials (e.g. molecules) with the posterior
probability distribution
Backward prediction

Forward prediction

Prior※ Prior: “ポリマーらしさ” の学習済モデル

16

CHEMICALLY FAVORABLE STRUCTURE?
Molecular graphp(S)prior knowledge

General graph

Geometry
Exclusion of chemically unfavorable bonds, e.g. F-N, C=O=C
Chemical stability
Drug-likeliness
Synthesizability
Liquid crystalline likeliness17

p(S) CHEMICAL LANGUAGE MODEL
Ikebata et al., J Compt Aided Mol Des, 2017

 実在の化合物セットから ‘化合物らしさ’ を学習•

化学構造を文字列で表現 (SMILES)•

言語モデルを用いて文字のパターンや文法を学習

vanillin C8H8O3
O=Cc1ccc(O)c(OC)c1環の始点と終点は同じ数字
側鎖は括弧 訓練された言語モデル：‘化合物らしい構造’ の生成器として使用• Prior dist.
• Proposal distribution in SMC or MCMC

18

TWO TECHNICAL ISSUES Grammar error: Avoid the occurrence of unclosed ring or side-chain indicators
S[1:i-1] = CCCCCC(CCCC(OC(O)C
Two unclosed parentheses ‘(’The Markov probability should be biased such that two closing letters ‘)’
appear somewhere in the rightward Long term dependency: ‘Neighbors’ in a molecular graph are not neighboring inthe string representation

CN1CCC[C@H]1c2cccnc2

19NEW FUNCTIONAL POLYMERSHigher glass transition temperatures and higher melting points

Property refinement process

Structure modification

Orange: existing polymers with experimentally measured Tg and Tm
Blue: polymers computationally created by iqspr (circle size: the magnitude of likelihood)
20

NEW FUNCTIONAL POLYMERSHigher glass transition temperatures and higher melting points

Further expansion of the materials landscape (Tg and Tm)
Orange: polymers with observed Tg and Tm
Green: HTS of polymers with unobserved Tg and Tm
Blue: newly created polymers of iqspr

Result & Conclusion
Our algorithm has predicted the existence of yet-undiscovered many (9,685) polymers with Tg and Tmmuch higher than those of the state-of-the-art
Target region

observed < HTS < iqspr

21

LOW BAND GAP MOLECULES
Application of iqspr on novel monomer discovery

Property requirements
Band gap in [0, 3] eV
MW unconstrainedBand gap vs molecular weight(vertical, horizontal axes)

Orange: existing molecules (PubChem)
Blue: molecules created by iqspr
22

R PACKAGE iqspr v2.3
CRAN: https://cran.r-project.org/web/packages/iqspr/index.htm
GitHub: https://github.com/GLambard/inverse-molecular-designLambard, G(ISM) Version 1 released: 2016/05/22
 Number of downloads: 2,805 (2017/08/31)
 Version 2.3 released: 2017/07/27

Forward prediction methods•

Deep Learning (mxnet)•

Gradient Boosting (xgboost)•User-plugin models•

van Krevelen models

Other additional features•

Synthetic Accessibility scoring•

A set of molecular filters•

Richer descriptor set23

GO BEYOND INTERPOLATIVE PREDICTION

内挿的予測の限界を超える

24

計算機実験による検証
FIRST PRINCIPLES CALCULATION (DFT) Identification of promising materials based on DFT calculation (filtering) Poor prediction of the machine learning models in regions with few data points
DFT scheme
B3LYP functional, 6-31G basis set
Orange: existing molecules (DFT)
Blue: hypothetical (ML prediction)
Green: hypothetical (DFT)25

VALIDATION (DFT)HOMO-LUMO gap [eV]Black: existing compounds
Green: 10 hypothetical compounds with theproperties calculated by GAUSSIAN 09 (DFTB3LYP/6-31+G(d))

Desired property region

Internal energy [kcal/mol]26

VALIDATION (DFT)HOMO-LUMO gap [eV]Black: existing compounds
Green: 10 hypothetical compounds with theproperties calculated by GAUSSIAN 09 (DFTB3LYP/6-31+G(d))

Desired property region

Internal energy [kcal/mol]27

VALIDATION (DFT)HOMO-LUMO gap [eV]Black: existing compounds
Green: 10 hypothetical compounds with theproperties calculated by GAUSSIAN 09 (DFTB3LYP/6-31+G(d))

Desired property region

Internal energy [kcal/mol]28

VALIDATION (DFT)HOMO-LUMO gap [eV]Black: existing compounds
Green: 10 hypothetical compounds with theproperties calculated by GAUSSIAN 09 (DFTB3LYP/6-31+G(d))

Desired property region

Internal energy [kcal/mol]29

VALIDATION (DFT)HOMO-LUMO gap [eV]Black: existing compounds
Green: 10 hypothetical compounds with theproperties calculated by GAUSSIAN 09 (DFTB3LYP/6-31+G(d))

Desired property region

Internal energy [kcal/mol]30

VALIDATION (DFT)HOMO-LUMO gap [eV]Black: existing compounds
Green: 10 hypothetical compounds with theproperties calculated by GAUSSIAN 09 (DFTB3LYP/6-31+G(d))

Desired property region

Internal energy [kcal/mol]

31

VALIDATION (DFT)HOMO-LUMO gap [eV]Black: existing compounds
Green: 10 hypothetical compounds with theproperties calculated by GAUSSIAN 09 (DFTB3LYP/6-31+G(d))

Desired property region

Internal energy [kcal/mol]

32

GO BEYOND INTERPOLATIVE PREDICTION
 Limitation of iqspr: unreachable to regions where data are sparsely populated (i.e.,interpolative prediction)
 Extrapolative prediction by the integration of iqspr and DFT (computer experiments)

Vertical axis: HOMO-LUMO gap
Horizontal axis: internal energy
Gray dot: existing molecules
Black dot: newly created molecules

33

SPACIER GO BEYOND INTERPOLATIVE PREDICTION

Lambard, GISM

34

SPACIER GO BEYOND INTERPOLATIVE PREDICTION

Vertical axis: HOMO-LUMO gap
Horizontal axis: internal energy
Gray dot: existing molecules
Black dot: newly created molecules

Higher HOMO-LUMO gap
HOMO-LUMO gap (eV)

物質空間のフロンティアを発見

Internal Energy (kcal/mol)

HOMO-LUMO gap (eV)

Lower HOMO-LUMO gap

Internal Energy (kcal/mol)

35

SPACIER 低バンドギャップモノマーの発掘
学術的意義：物質・材料科学の事前知識を全く活用しない完全なデータ科学駆動型手法でここまで到達できた．

データが存在しない低バンドギャップ領域に埋蔵する大量の新物質が発掘された．

Orange: DFT properties of existing materials
Blue: newly discovered materials (SPACIER)

8,601 hypotheticalmaterials

Synthesis
Experimental validationin the next 1-2 years

36

材料研究と実験計画法
 材料研究では，実験（実空間・仮想空間）とデータ科学を連動させることが極めて重要 目的：① 実験コストの抑制，②外挿こそが究極の目標

実験が済んだあとに統計学者に意見を求めても、大抵の場合、それは単に終わったことを議論しようと言っているようなものだ。もしかしたら統計学者は、失敗の原因を語り出すかもしれない。
Fisher R.A. (1938) Presidential Address, Sankhyā, 4:14-17.

R. A. フィッシャー (1890-1962)
現代の統計学，集団遺伝学の創始者．ロザムステッド農事試験場における穀物量の変動に関する研究を通して，実験計画法や分散分析等，数々の統計理論を確立．
37

物性計算（第一原理計算など）の試行回数を削減

物質・材料科学におけるベイズ最適化の活用
Seko et al., Phys Rev B, 2014
Property: melting point
Input: crystal structure

Seko et al., Phys Rev Lett, 2015
Property: LTC (lattice thermal conductivity)
Input: crystal structure
Balachandran et al., Sci Rep, 2016
Property: elastic properties [bulk, shear, and Young’s modulus]
Input: crystal structure
Kiyohara et al., Sci Adv, 2016
Property: grain boundary energy
Input: interface structure (grain boundary)Ju et al., arXiv, arXiv:1609.04972
Property: interfacial thermal conductivity
Input: interface structure[SOFTWARE] Ueno et al., COMBO: an efficient Bayesian Optimization library for materials science,
Mater Dis, 201638

ベイズ最適化に基づく実空間の合成・機能計測の循環

4×9回の実験サイクル 新規形状記憶合金を同定

39

外挿アルゴリズムの到来：何が変わるか
著作権・知財の在り方 アルゴリズム（& データ）さえあれば，誰もが新材料を発掘できる時代？
 遥かに高い生産性を有する（弱い）AI による著作物の数が爆発的に増加
 独占の不可：全候補物質の合成は不可能．同じ計算をすれば，第三者も他の物質を発掘できる．データやアルゴリズムのオープン化は危険かもしれない．
 特許の網目を抜ける技術としても（e.g. A社とB社の特許化合物のハイブリッド）

例：SPACIERによる低バンドギャップポリマーの探索．
データが存在しない低バンドギャップ領域に埋蔵する大量の新物質が発掘された．

Orange: DFT properties of existing materials
Blue: newly discovered materials (SPACIER)
40