ニューラルネットワークと深層学習

What this book is about

On the exercises and problems

ニューラルネットワークを用いた手書き文字認識

• Perceptrons
• Sigmoid neurons
• The architecture of neural networks
• A simple network to classify handwritten digits
• Learning with gradient descent
• Implementing our network to classify digits
• Toward deep learning

逆伝播の仕組み

• Warm up: a fast matrix-based approach to computing the output from a neural network
• The two assumptions we need about the cost function
• The Hadamard product, $s \odot t$
• The four fundamental equations behind backpropagation
• Proof of the four fundamental equations (optional)
• The backpropagation algorithm
• The code for backpropagation
• In what sense is backpropagation a fast algorithm?
• Backpropagation: the big picture

ニューラルネットワークの学習の改善

• The cross-entropy cost function
• Overfitting and regularization
• Weight initialization
• Handwriting recognition revisited: the code
• How to choose a neural network's hyper-parameters?
• Other techniques

ニューラルネットワークが任意の関数を表現できることの視覚的証明

• Two caveats
• Universality with one input and one output
• Many input variables
• Extension beyond sigmoid neurons
• Fixing up the step functions
• Conclusion

ニューラルネットワークを訓練するのはなぜ難しいのか

• The vanishing gradient problem
• What's causing the vanishing gradient problem? Unstable gradients in deep neural nets
• Unstable gradients in more complex networks
• Other obstacles to deep learning

深層学習

• Introducing convolutional networks
• Convolutional neural networks in practice
• The code for our convolutional networks
• Recent progress in image recognition
• Other approaches to deep neural nets
• On the future of neural networks

Appendix: 知性のある シンプルな アルゴリズムはあるか?

Acknowledgements

Frequently Asked Questions

---

Sponsors

著者と共にこの本を作り出してくださったサポーターの皆様に感謝いたします。 また、バグ発見者の殿堂に名を連ねる皆様にも感謝いたします。 また、日本語版の出版にあたっては、翻訳者の皆様に深く感謝いたします。

この本は目下のところベータ版で、開発続行中です。 エラーレポートは mn@michaelnielsen.org まで、日本語版に関する質問は muranushi@gmail.com までお送りください。 その他の質問については、まずはFAQをごらんください。

---

Resources

Code repository

Mailing list for book announcements

Michael Nielsen's project announcement mailing list

---

著：Michael Nielsen / 2014年9月-12月
訳：「ニューラルネットワークと深層学習」翻訳プロジェクト

This work is dedicated to my wife, Jen Dodd, whose love, integrity, and support made the book (and all else I do) possible.

The book grew out of a set of notes I prepared for an online study group on ニューラルネットワークと深層学習. Many thanks to all the participants in that study group: Paul Bloore, Chris Dawson, Andrew Doherty, Ilya Grigorik, Alex Kosorukoff, Chris Olah, and Rob Spekkens. I learned a lot from all of you. I am particularly grateful to Rob, for providing so many insightful questions and ideas, and to Chris, who has continued to share his rapidly expanding knowledge of neural networks. Thanks also to Yoshua Bengio, who read and provided feedback on a chapter.