Vision and generative models

From the Neocognitron and LeNet to ImageNet, ResNet and generative models (GAN, VAE, diffusion).

Solid lines are confirmed citations (OpenAlex or the citing paper's own reference list); dotted lines are influences stated in textbooks without a confirmed citation. Vertical position is the theme group.

Year	First author	Title	Era
1980	Fukushima	Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position	Ch. 3 Knowledge systems and connectionism 1974–93
1982	Marr	Vision: A Computational Investigation into the Human Representation and Processing of Visual Information	Ch. 3 Knowledge systems and connectionism 1974–93
1989	LeCun et al.	Backpropagation Applied to Handwritten Zip Code Recognition	Ch. 3 Knowledge systems and connectionism 1974–93
1998	LeCun et al.	Gradient-based learning applied to document recognition	Ch. 4 Statistical learning 1990–2011
2009	Deng et al.	ImageNet: A large-scale hierarchical image database	Ch. 4 Statistical learning 1990–2011
2010	Ciresan et al.	Deep, Big, Simple Neural Nets for Handwritten Digit Recognition	Ch. 4 Statistical learning 1990–2011
2012	Ciresan et al.	Multi-column deep neural networks for image classification	Ch. 5 Deep learning 2012–17
2012	Krizhevsky et al.	ImageNet Classification with Deep Convolutional Neural Networks	Ch. 5 Deep learning 2012–17
2013	Graves	Generating Sequences With Recurrent Neural Networks	Ch. 5 Deep learning 2012–17
2013	Girshick et al.	Rich feature hierarchies for accurate object detection and semantic segmentation	Ch. 5 Deep learning 2012–17
2013	Kingma & Welling	Auto-Encoding Variational Bayes	Ch. 5 Deep learning 2012–17
2014	Goodfellow et al.	Generative Adversarial Nets	Ch. 5 Deep learning 2012–17
2014	Simonyan & Zisserman	Very Deep Convolutional Networks for Large-Scale Image Recognition	Ch. 5 Deep learning 2012–17
2014	Szegedy et al.	Going Deeper with Convolutions	Ch. 5 Deep learning 2012–17
2015	Sohl-Dickstein et al.	Deep Unsupervised Learning using Nonequilibrium Thermodynamics	Ch. 5 Deep learning 2012–17
2015	Radford et al.	Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks	Ch. 5 Deep learning 2012–17
2015	He et al.	Deep Residual Learning for Image Recognition	Ch. 5 Deep learning 2012–17
2016	van den Oord et al.	WaveNet: A Generative Model for Raw Audio	Ch. 5 Deep learning 2012–17
2020	Dosovitskiy et al.	An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale	Ch. 6 Pretraining and generation 2017–22
2021	Radford et al.	Learning Transferable Visual Models From Natural Language Supervision	Ch. 6 Pretraining and generation 2017–22