!    
 
"$#1

6:966

777%

1250:2RD )B•••M:ET13•B• ( .

%29G!

3

-"3A;@!?/%*',&
:59 +086
#>")= 71.4(<2$45•6"!"• !"!#!$!=!%!& ('#' = '$'%
 )7

:()!#!
!= $!%!&(#( = ($(%'⋮) 

*# = + ,#! + .#*$ = +(,$*# + .$)⋮( = ,2*3 + .2,.
+8

/,

0$'.&%+/(!"#)/) /!))-(A9/) /!)*190•1 !
(#$ , &$ )min ∑$ #$ − .+ &$+/11•••B))((13)G neffbkCC /9(BlPo0B:1( (U8S.8 3:1:(231)))%((((.3 44 8%.3 44 8/419 17

419 1795

241 17

16 17

///1
:/
18

2()

6)

6./

2

•
•(((321R 2017C
2

220%

3

72/

2

1+ .+(()(3( 3.)/)

Li+ (2018)3)/

/3Li+ (2018)

// .

()

/3.//03.
2.

3

3

.
1

12.

1

1111122

.
23.


•(1).
24

BSkIee.0 .

&, .1

1

21&(F7 9i
21 &fa
(25.

.
.

.
.



.

1

1

126

27
2

2

2

2

22.

2

R)




≈(CC

≈!#2VR.

!"!", !#V

.
()

(2  

a!#aAA⋮!"b0851. 7

,. 20(hNA+Bl.

)

Ae30

.

1

1.

1

3

1

1

111P.
•33-O32.

•aa

33-OaPaAllen-Zhu+ (2019), Liang+(2018), Kawaguchi+ (2019)

a.

a

.
3OO3))O322(01O34•535336?2)>A)> +#%3.9:52;=)>*)> (8@&$B

.9:52;-<!2;-"C72;-• 'D2;-.91/%40(862;-/,('+'• +,• 92' $)8+

1756!#/• &.- *• 4" 0

%3(

1()0•2•39G) (9• F072 21

40

41• • Stone, C. J. (1980). Optimal rates of convergence fornonparametric estimators. The annals of Statistics.
• Cybenko, G. (1989). Approximation by superpositions ofa sigmoidal function. Mathematics of control, signals
and systems• Safran, I., & Shamir, O. (2017). Depth-width tradeoffs inapproximating natural functions with neural networks.
International Conference on Machine Learning.
• Li, H., Xu, Z., Taylor, G., Studer, C., & Goldstein, T. (2018).
Visualizing the loss landscape of neural nets. Advancesin Neural Information Processing Systems.
42
• Imaizumi, M., & Fukumizu, K. (2019). Deep NeuralNetworks Learn Non-Smooth FuncEons EﬀecEvely.
ArEﬁcial Intelligence and StaEsEcs.
• Suzuki, T. (2018). AdapEvity of deep ReLU network forlearning in Besov and mixed smooth Besov spaces:
InternaEonal Conference on Learning RepresentaEons.
• Neyshabur, B., Tomioka, R., & Srebro, N. (2015). Normbased capacity control in neural networks. In Conference
on Learning Theory.
• BartleY, P. L., Foster, D. J., & Telgarsky, M. J. (2017).
Spectrally-normalized margin bounds for neuralnetworks. Advances in Neural InformaEon Processing
Systems.
43
• Arora, S., Ge, R., Neyshabur, B., & Zhang, Y. (2018).
Stronger generalizaBon bounds for deep nets via acompression approach. InternaBonal Conference onMachine Learning.
• Allen-Zhu, Z., Li, Y., & Song, Z. (2018). A convergence
theory for deep learning via over-parameterizaBon.
InternaBonal Conference on Machine Learning.
• Liang, S., Sun, R., Lee, J. D., & Srikant, R. (2018). Adding
one neuron can eliminate all bad local minima. Advancesin Neural InformaBon Processing Systems.
• Kawaguchi, K., & Kaelbling, L. P. (2019). EliminaBon of allbad local minima in deep learning. arXiv preprint.
44••• imaizumi@ism.ac.jp

45