Knowledge systems and the return of connectionism (1974–1993)
Can expert knowledge be written as rules, and can multilayer networks be trained?
Key point
Two streams run side by side: the knowledge stream of frames (1974), scripts (1977), DENDRAL (1978) and the physical symbol system hypothesis (1976), and the connectionist stream of the Neocognitron (1980), the Hopfield network (1982), the Boltzmann machine (1985) and backpropagation (1986).
The backpropagation paper in Nature of 9 October 1986 has four references, one of them Perceptrons. Who wrote backpropagation first is treated in this chapter's box with both sides' bibliographies only.
Temporal-difference learning (1988), the Q-learning thesis (1989), the two universal approximation theorems (1988 and 1989), the forerunner of LeNet (1989) and the HMM tutorial (1989) crowd into two years. The "winter" at the chapter's end, in the early 1990s, is visible in the attendance series on the winters page.
Writing knowledge as structure
Minsky's "A Framework for Representing Knowledge", MIT AI Memo 306 of June 1974, proposed organizing knowledge in frames, remembered data structures for stereotyped situations: "A frame is a data-structure for representing a stereotyped situation, like being in a certain kind of living room, or going to a child's birthday party." Schank and Abelson's Scripts, Plans, Goals and Understanding (1977) developed the same idea as scripts, typical sequences of events; the book could not be read for this version. Buchanan and Feigenbaum's "Dendral and Meta-Dendral" (August 1978, Artificial Intelligence) describes an expert system that infers molecular structure from mass-spectrometry data and a meta-system that learns its rules; it too could not be read. These three are the original texts of the expert-system boom of the 1980s, whose size can be measured on the winters page.
Newell and Simon's "Computer science as empirical inquiry: symbols and search" was delivered as the Turing Award lecture on 20 October 1975 and printed in the March 1976 Communications of the ACM. "A physical symbol system has the necessary and sufficient means for general intelligent action." That sentence has been read as the creed of symbolic AI; the references include McCarthy's LISP and Turing 1950.
Probability and statistics come in
The EM algorithm of Dempster, Laird and Rubin was read before the Royal Statistical Society on 8 December 1976 and printed in JRSS B in 1977. Its iteration of an expectation step and a maximization step for maximum-likelihood estimation from incomplete data is cited by Rabiner in this chapter and by Brown et al. (statistical machine translation) in chapter 4. Geman and Geman's "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images" (November 1984) modeled images as Markov random fields and restored them with a Gibbs sampler combined with annealing; the annealing came from Kirkpatrick et al., "Optimization by Simulated Annealing" (13 May 1983, Science), and both are cited by the Boltzmann machine paper. Pearl's Probabilistic Reasoning in Intelligent Systems (1988) wrote reasoning under uncertainty as belief networks; it is cited by Hinton et al. 2012 in chapter 5 but could not be read here. Valiant's "A theory of the learnable" (1984; STOC preliminary version, then CACM in November) posed the question of which concept classes can be learned in a polynomial number of steps and became the starting point of learning theory; Cybenko 1989 cites it.
The return of connectionism
Fukushima's "Neocognitron" (April 1980, Biological Cybernetics) stacks alternating S-cell and C-cell layers and learns, without a teacher, recognition that is unaffected by shifts in position. LeCun et al. (1989) later in this chapter cite it, the oldest confirmed citation line in the convolutional lineage. Hopfield's "Neural networks and physical systems with emergent collective computational abilities" (April 1982, PNAS) is a model of asynchronously updated units whose collective dynamics flow to stable states and so act as a content-addressable memory; it cites McCulloch–Pitts, Hebb and Perceptrons, and is the work named in the 2024 Nobel Prize in Physics citation. The Boltzmann machine of Ackley, Hinton and Sejnowski (January 1985, Cognitive Science) added stochastic units and annealing to Hopfield's network and derived a rule that learns the constraints of a domain simply from examples.
Rumelhart, Hinton and Williams, "Learning representations by back-propagating errors" appeared in Nature on 9 October 1986: "We describe a new learning procedure, back-propagation, for networks of neurone-like units." Repeatedly adjusting weights to reduce the difference between actual and desired outputs makes hidden units come to represent features of the task. The paper has four references, one of them Perceptrons; it reads as an answer to the limits the 1969 book raised. Among the other papers in this version eight (Waibel 1989, Cortes 1995, Glorot 2010, Hinton 2012, both Mikolov 2013 papers, Sutskever 2014, LeCun 2015) list the Nature paper in their references; several others cite the PDP book chapter of the same year instead, and the cards' notes keep the two apart.
Werbos's "Applications of advances in nonlinear sensitivity analysis" (1982, IFIP proceedings) is known as the application of his 1974 thesis method to neural networks; it is behind a Springer paywall and was not read. The box on the priority of backpropagation follows below.
Reinforcement learning gets its name
Barto, Sutton and Anderson, "Neuronlike adaptive elements" (September 1983) learned to balance a pole on a cart with two elements, an associative search element and an adaptive critic element, from a failure signal alone; its references (Hebb, Widrow–Hoff, Minsky 1961, Samuel, McCulloch–Pitts) tie it most closely to chapters 1 and 2. Sutton's "Learning to predict by the methods of temporal differences" (August 1988, Machine Learning) defined temporal-difference methods, which assign credit by the difference between successive predictions, and proved convergence for special cases. The next year Watkins's Cambridge thesis "Learning from Delayed Rewards" (May 1989) presented a method that learns the optimal values of actions directly without a model of the environment, later called Q-learning. The 2024 Turing Award citation for Barto and Sutton points to Barto et al. 1983, Sutton 1988 and Sutton–Barto 1998 (chapter 4).
The height of symbolic AI, and its critique
Holland's Adaptation in Natural and Artificial Systems (1975, University of Michigan Press) is known as the original text of genetic algorithms but could not be read for this version; Sutton–Barto 1998 and Schmidhuber 2015 cite it. Searle's "Minds, brains, and programs" (September 1980, Behavioral and Brain Sciences) argued through the thought experiment of a person following rules for Chinese symbols without understanding them, the "Chinese room", that "instantiating a computer program is never by itself a sufficient condition of intentionality". Its references cite ELIZA and Schank and Abelson's scripts; it stands opposite the claim of "understanding" that Winograd represented in chapter 2. McDermott's R1 appeared first as a Carnegie Mellon technical report in April 1980 and in Artificial Intelligence in 1982: written in the production-system language OPS4, it configured DEC's VAX orders and was in regular use in a manufacturing organization; the report states that R1 uses match rather than generate-and-test, so little search is required. It is the record of an expert system in commercial use. Marr's Vision (1982) is known for its three levels of computational theory, representation and algorithm, and implementation, but could not be read.
Quinlan's "Induction of decision trees" (March 1986, the first issue of Machine Learning) described ID3, which chooses the attribute at each node by an information-based criterion. Symbolic learning got its journal in the same year as the return of neural networks; C4.5 in chapter 4 is its successor.
1989
Five papers appeared in journals in 1989 (two of them first as reports). Cybenko (December) and Hornik, Stinchcombe and White (a UCSD discussion paper in 1988, then Neural Networks in 1989) each proved that a feedforward network with one hidden layer can approximate any continuous function on the unit hypercube to any accuracy, given enough hidden units (the universal approximation theorems); Cybenko cites Hornik's 1988 preprint. LeCun et al., "Backpropagation Applied to Handwritten Zip Code Recognition" (December, Neural Computation) built the constraints of the task into the network and recognized handwritten U.S. Postal Service zip codes, the direct forerunner of LeNet (1998) in chapter 4. Rabiner's HMM tutorial (February, Proceedings of the IEEE) organized hidden Markov models for speech around three problems, and the time-delay neural network of Waibel et al. appeared first as an ATR technical report in October 1987 and in the journal in 1989; its sentence on 1946 test tokens — TDNN 98.5% against the best HMM's 93.7% — records neural networks beginning to compete with statistical models in speech. Elman's "Finding Structure in Time" (March 1990) closes the chapter with the simple recurrent network that feeds the hidden layer's pattern back as the next step's context.
What was and was not checked
Of 28 papers, 19 were read in the journal printing or an equivalent copy. Schank–Abelson, Buchanan–Feigenbaum, Werbos 1982, Pearl, Holland and Marr were not read; R1 was read in the technical-report version; Minsky 1974 was read in the author's web version and Elman in a retyped copy without page numbers. Candidates of the period that did not pass the rule include MYCIN (Shortliffe 1975), Kohonen's self-organizing map (1982), Brooks's subsumption architecture (1986) and Kowalski's "Algorithm = logic + control" (1979).
Priority: who wrote backpropagation first
This site makes no ruling. It places both sides' bibliographies and what this version could verify side by side.
Work
First public
Verified in this version
Linnainmaa, S., "The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors". Master's thesis, University of Helsinki (in Finnish; English title is a translation)
1970
Bibliography from secondary sources only; no original or OpenAlex record. Did not pass the selection rule
Read. Four references (including Perceptrons); no citation of Werbos or Linnainmaa
Schmidhuber's "Annotated History" (2022, v3 2025) attributes backpropagation to Linnainmaa 1970 and its first neural-network application to Werbos 1982; we present that as one side's claim. The 1986 paper uses the name "back-propagation" and records the observation that hidden units come to represent features.
Dates: the year usually quoted and the date in the original
1988, UCSD discussion paper 88-45 (the paper's self-citation)
Lineage
Solid lines are confirmed citations (OpenAlex or the citing paper's own reference list); dotted lines are influences stated in textbooks without a confirmed citation. Vertical position is the theme group.
Paper list (28)
A Framework for Representing Knowledge
Marvin Minsky · First public 1974-06 (tech report) · Venue: MIT Artificial Intelligence Laboratory, AI Memo 306 · Pages page unknown (HTML version) · Citations (OpenAlex) 4,535 (checked 2026-11-03)
DOI · Reprinted in P. Winston (ed.), The Psychology of Computer Vision (McGraw-Hill, 1975); shorter versions in Haugeland (ed.), Mind Design (1981) and Readings in Cognitive Science (1988, pp. 156-189; manifest DOI 10.1016/b978-1-4832-1446-7.50018-2)
A frame is a data-structure for representing a stereotyped situation, like being in a certain kind of living room, or going to a child's birthday party.page unknown
What changed. Proposes that knowledge be organized in 'frames': remembered data-structures for stereotyped situations whose top levels are fixed and whose lower-level terminals ('slots') are filled with specific instances, linked into frame-systems whose transformations represent actions or changes of viewpoint.
Cites (within this list): Minsky & Papert 1969 Cited by (within this list): none
Adaptation in Natural and Artificial Systems
John H. Holland · First public 1975 (book) · Venue: Ann Arbor: University of Michigan Press, 1975 · Pages viii, 183 pp. (catalog record) · Citations (OpenAlex) 47,262 (checked 2026-11-03)
DOI · MIT Press edition 1992 (IA catalog record adaptationinnatu00holl)
Computer science as empirical inquiry: symbols and search
Allen Newell, Herbert A. Simon · First public 1975-10-20 (conference) · Venue: Communications of the ACM 19(3) · Pages 113–126 · Citations (OpenAlex) 2,354 (checked 2026-11-03)
DOI · Communications of the ACM 19(3), March 1976 (published text of the Turing lecture)
A physical symbol system has the necessary and sufficient means for general intelligent action.116
What changed. States the Physical Symbol System Hypothesis and the Heuristic Search Hypothesis as laws of qualitative structure, framing computer science (and AI) as an empirical inquiry into symbol systems and search.
Maximum Likelihood from Incomplete Data Via the EM Algorithm
A. P. Dempster, Nan M. Laird, Donald B. Rubin · First public 1976-12-08 (conference) · Venue: Journal of the Royal Statistical Society, Series B (Methodological) 39(1) · Pages 1–38 (paper 1–22, discussion and reply 22–38) · Citations (OpenAlex) 50,443 (checked 2026-11-03)
DOI · JRSS Series B 39(1), 1977 (journal publication with discussion)
A broadly applicable algorithm for computing maximum likelihood estimates from incomplete data is presented at various levels of generality.1
What changed. Presents a general iterative method for maximum-likelihood estimation from incomplete data, in which each iteration is an expectation step followed by a maximization step (named the EM algorithm), with theory on monotone likelihood and convergence and examples across missing-data, mixture, variance-component and factor-analysis problems.
Roger C. Schank, Robert P. Abelson · First public 1977 (book) · Venue: Lawrence Erlbaum Associates, Hillsdale, NJ (book, Schank & Abelson) · Citations (OpenAlex) 4,536 (checked 2026-11-03)
DOI · Excerpt reprinted in Readings in Cognitive Science (1988), pp. 190-223 (manifest DOI 10.1016/b978-1-4832-1446-7.50019-4); related earlier paper Schank & Abelson, 'Scripts, Plans and Knowledge', IJCAI 1975
What changed.
Cites (within this list): none Cited by (within this list): Searle 1980
Dendral and Meta-Dendral: Their applications dimension
Bruce G. Buchanan, Edward A. Feigenbaum · First public 1978-08 (journal issue) · Venue: Artificial Intelligence 11(1-2) · Pages 5–24 · Citations (OpenAlex) 471 (checked 2026-11-03)
DOI · Stanford Heuristic Programming Project memo HPP-78-1 / Stanford CS report STAN-CS-78-649 (1978, month unverified); reprinted in Webber & Nilsson (eds.), Readings in Artificial Intelligence (1981), pp. 313-322
What changed.
Cites (within this list): none Cited by (within this list): none
Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position
DOI · Earlier version: Fukushima, 'Self-organization of a neural network which gives position-invariant response', Proc. 6th IJCAI, Tokyo, Aug 20-23 1979, pp. 291-293, and IECE Japan technical papers 1979 (cited in the paper's reference list)
A neural network model for a mechanism of visual pattern recognition is proposed in this paper.193
What changed. Proposes the 'neocognitron', a multilayered network of alternating S-cell and C-cell layers, self-organized by learning without a teacher, whose last-layer responses to stimulus patterns are not affected by the pattern's position or by small changes in shape or size.
John P. McDermott · First public 1980-04 (tech report) · Venue: Carnegie-Mellon University, Department of Computer Science, technical report CMU-CS-80-119; journal version: Artificial Intelligence 19(1) · Pages 56 PDF pp. (technical report scan, text numbered 1-37 plus appendices); journal 39–88 (not read) · Citations (OpenAlex) 951 (checked 2026-11-03)
R1 is implemented as a production system.abstract page, unnumbered (PDF p. 2 of the technical report scan)
What changed. Describes R1, a program written in the production-system language OPS4 that configures VAX-11/780 computer systems from customer orders and was in regular use by Digital Equipment Corporation's manufacturing organization. The report states that R1 uses Match rather than Generate-and-test, so little search is required to configure a system.
Cites (within this list): none Cited by (within this list): none
Minds, brains, and programs
John Rogers Searle · First public 1980-09 (journal issue) · Venue: The Behavioral and Brain Sciences 3(3) · Pages 417–424 (target article); 417–457 (whole treatment incl. Open Peer Commentary, Author's Response and references) · Citations (OpenAlex) 6,955 (checked 2026-11-03)
DOI · Cambridge Core online 2010-02-04 (Crossref published-online)
(2) Instantiating a computer program is never by itself a sufficient condition of intentionality.p. 417
What changed. Argues, through a thought experiment in which a person follows a program for manipulating Chinese symbols without understanding them, that instantiating a computer program is never by itself sufficient for intentionality. Distinguishes 'strong' AI from 'weak' AI and concludes that strong AI has little to tell us about thinking.
Neural networks and physical systems with emergent collective computational abilities
J. J. Hopfield · First public 1982-04 (journal issue) · Venue: Proceedings of the National Academy of Sciences USA 79(8) · Pages 2554–2558 · Citations (OpenAlex) 19,264 (checked 2026-11-03)
DOI · CaltechAUTHORS record gives publication date April 15, 1982
The collective properties of this model produce a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.2554
What changed. Gives a model of interacting neuron-like units, with asynchronous updating, whose collective dynamics flow to stable states and so act as a content-addressable memory, with further collective properties including generalization, familiarity recognition, categorization, error correction and time-sequence retention.
Vision: A Computational Investigation into the Human Representation and Processing of Visual Information
David Marr · First public 1982 (book) · Venue: W. H. Freeman and Company, San Francisco (1982); reissued by The MIT Press (2010) · Citations (OpenAlex) 5,723 (checked 2026-11-03)
URL · MIT Press reissue 2010-07-09 (Books Gateway schema.org datePublished; DOI 10.7551/mitpress/9780262514620.001.0001, ISBN 9780262514620)
Applications of advances in nonlinear sensitivity analysis
Paul J. Werbos · First public 1982 (book) · Venue: R. F. Drenick & F. Kozin (eds.), System Modeling and Optimization, Lecture Notes in Control and Information Sciences 38, Springer · Pages 762–770 · Citations (OpenAlex) 259 (checked 2026-11-03)
DOI · Presented at the 10th IFIP Conference on System Modeling and Optimization, New York, 1981 (from later citations; not verified from the volume)
Scott Kirkpatrick, C. D. Gelatt, Mario P. Vecchi · First public 1983-05-13 (journal issue) · Venue: Science 220(4598) · Pages 671–680 · Citations (OpenAlex) 45,141 (checked 2026-11-03)
DOI · IBM Thomas J. Watson Research Center report of the same title, 1982 (as cited in Geman & Geman 1984, ref. [40]; not seen directly)
A detailed analogy with annealing in solids provides a framework for optimization of the properties of very large and complex systems.671
What changed. Uses the Metropolis algorithm from statistical mechanics, with a slowly lowered temperature ('simulated annealing'), as a general method for combinatorial optimization, applied to partitioning, placement and wiring in computer design and to traveling salesman problems with up to several thousand cities.
Neuronlike adaptive elements that can solve difficult learning control problems
Andrew G. Barto, Richard S. Sutton, Charles W. Anderson · First public 1983-09 (journal issue) · Venue: IEEE Transactions on Systems, Man, and Cybernetics SMC-13(5) · Pages 834–846 · Citations (OpenAlex) 3,237 (checked 2026-11-03)
The learning system consists of a single associative search element (ASE) and a single adaptive critic element (ACE).834
What changed. Shows a system of two neuronlike adaptive elements, an associative search element and an adaptive critic element, learning to balance a pole on a cart when the only feedback is a failure signal, with the critic constructing a more informative evaluation than the reinforcement signal alone.
Leslie Gabriel Valiant · First public 1984 (conference) · Venue: Communications of the ACM 27(11) · Pages 1134–1142 · Citations (OpenAlex) 3,279 (checked 2026-11-03)
DOI · STOC 1984 preliminary version (pp. 436-445; month not checked); Communications of the ACM issue Nov 1984; Received 1/84, revised 6/84, accepted 7/84
We give a precise methodology for studying this phenomenon from a computational viewpoint.1134
What changed. Defines learning as knowledge acquisition without explicit programming and proposes a methodology of choosing a learning protocol and studying which classes of concepts can be learned in a polynomial number of steps. Shows that some nontrivial classes of propositional concepts are learnable in this sense.
Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
Stuart Geman, Donald Geman · First public 1984-11 (journal issue) · Venue: IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-6(6) · Pages 721–741 · Citations (OpenAlex) 17,943 (checked 2026-11-03)
We establish convergence properties of the algorithm and we experiment with some simple pictures, for which good restorations are obtained at low signal-to-noise ratios.721
What changed. Models images with Gibbs distributions (equivalently Markov random fields), including line processes for edges, and gives a parallel stochastic relaxation algorithm with annealing (the Gibbs sampler) that computes the MAP restoration of a degraded image, with convergence results.
David H. Ackley, Geoffrey E. Hinton, Terrence J. Sejnowski A B · First public 1985-01 (journal issue) · Venue: Cognitive Science 9(1) · Pages 147–169 · Citations (OpenAlex) 3,402 (checked 2026-11-03)
DOI · Related earlier tech report cited in the paper: Hinton, Sejnowski & Ackley (1984, May) 'Boltzmann Machines: Constraint satisfaction networks that learn', CMU-CS-84-119; Wiley online 2010-02-11
This paper presents a type of parallel constraint satisfaction network which we call a “Boltzmann Machine” that is capable of learning the underlying constraints that characterize a domain simply by being shown examples from the domain.148
What changed. Describes a parallel search method based on statistical mechanics and derives from it a learning rule for adjusting connection strengths in a 'Boltzmann Machine'. The network builds an internal generative model whose examples match the probability distribution of the examples it is shown.
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail.p. 81
What changed. Describes the TDIDT (Top-Down Induction of Decision Trees) family of learning systems and the ID3 algorithm in detail, using an information-based criterion to choose the attribute tested at each node. Presents extensions for noisy and incomplete data and compares two ways of overcoming a reported shortcoming of the attribute-selection criterion.
Cites (within this list): none Cited by (within this list): none
Learning representations by back-propagating errors
David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams · First public 1986-10-09 (journal issue) · Venue: Nature 323(6088) · Pages 533–536 · Citations (OpenAlex) 31,903 (checked 2026-11-03)
We describe a new learning procedure, back-propagation, for networks of neurone-like units.533
What changed. Describes back-propagation, which repeatedly adjusts connection weights to reduce the difference between actual and desired output vectors. As a result, hidden units come to represent features of the task domain.
Phoneme recognition using time-delay neural networks
Alexander H. Waibel, Toshiyuki Hanazawa, Geoffrey E. Hinton, Kiyohiro Shikano, K.J. Lang · First public 1987-10 (tech report) · Venue: IEEE Transactions on Acoustics, Speech, and Signal Processing 37(3) · Pages 328–339 · Citations (OpenAlex) 2,642 (checked 2026-11-03)
DOI · ATR Technical Report TR-I-0006 (Oct 1987); IEEE TASSP March 1989; manuscript received November 3, 1987; revised January 3, 1988
Performance evaluation over 1946 testing tokens from three speakers showed that the TDNN achieves a recognition rate of 98.5 percent correct while the rate obtained by the best of our HMM's was only 93.7 percent.328
What changed. Presents a Time-Delay Neural Network for phoneme recognition whose time-delay arrangement learns acoustic-phonetic features independent of position in time. On speaker-dependent recognition of B, D and G it is compared with discrete hidden Markov models trained on the same task.
Jeffrey L. Elman · First public 1988 (tech report) · Venue: Cognitive Science 14(2) · Pages 179–211 (journal); copy has 28 pp. with its own pagination · Citations (OpenAlex) 11,132 (checked 2026-11-03)
DOI · Cognitive Science 14(2) March 1990 (journal issue); CRL TR 8801 (1988) known only from a citation
The current report develops a proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory.p. 1 of the retyped copy (journal p. 179 assumed for the abstract, not seen)
What changed. Represents time implicitly through its effect on processing by feeding hidden-unit patterns back to the network as context, giving it a dynamic memory. Simulations from a temporal XOR task to word sequences show that the learned internal representations encode lexical categories and the type/token distinction.
Multilayer feedforward networks are universal approximators
Kurt Hornik, Maxwell B. Stinchcombe, Halbert White · First public 1988 (tech report) · Venue: Neural Networks, 2(5) · Pages 359–366 · Citations (OpenAlex) 21,724 (checked 2026-11-03)
DOI · Neural Networks 2(5), 1989 (journal; Crossref published-print 1989-01, probably year-only); received 16 Sep 1988, revised and accepted 9 Mar 1989 (not public dates)
We show that multilayer feedforward networks with as few as one hidden layer are indeed capable of universal approximation in a very precise and satisfactory sense.p. 359
What changed. Proved, using the Stone-Weierstrass theorem and the cosine squasher of Gallant and White, that standard multilayer feedforward networks with as few as one hidden layer and arbitrary squashing functions can approximate any Borel measurable function between finite-dimensional spaces to any desired accuracy, given sufficiently many hidden units. The results do not address how many hidden units are needed.
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
Judea Pearl · First public 1988 (book) · Venue: Morgan Kaufmann (Morgan Kaufmann Series in Representation and Reasoning) · Citations (OpenAlex) 16,575 (checked 2026-11-03)
URL · Elsevier shop listing gives '1st Edition - September 1, 1988' (shop metadata, not verified against the book); ScienceDirect DOI 10.1016/C2009-0-27609-4
Learning to predict by the methods of temporal differences
Richard S. Sutton · First public 1988-08 (journal issue) · Venue: Machine Learning 3(1) · Pages 9–44 · Citations (OpenAlex) 5,131 (checked 2026-11-03)
DOI · Received April 22, 1987; revised February 4, 1988
Here we prove their convergence and optimality for special cases and relate them to supervised-learning methods.9
What changed. Introduces temporal-difference methods as a class of incremental prediction-learning procedures that assign credit using the difference between temporally successive predictions, and proves convergence and optimality for special cases. Relates these methods formally to supervised-learning procedures.
A tutorial on hidden Markov models and selected applications in speech recognition
L. R. Rabiner · First public 1989-02 (journal issue) · Venue: Proceedings of the IEEE 77(2) · Pages 257–286 · Citations (OpenAlex) 22,745 (checked 2026-11-03)
DOI · Manuscript received January 15, 1988; revised October 4, 1988
In this paper we attempt to carefully and methodically review the theoretical aspects of this type of statistical modeling and show how they have been applied to selected problems in machine recognition of speech.257
What changed. Reviews the theory of hidden Markov models, from discrete Markov chains to variable-duration and continuous-density models, and shows how they apply to problems in speech recognition. The paper frames HMM design around three basic problems.
Chris Watkins · First public 1989-05 (thesis) · Venue: PhD thesis, King's College, University of Cambridge · Pages 234 numbered pp. (241 PDF pages) · Citations (OpenAlex) 5,624 (checked 2026-11-03)
DOI · Apollo repository deposit/issue 2026-07-10; Watkins & Dayan, Q-learning, Machine Learning 1992 (separate paper)
In particular, there is an incremental Monte-Carlo method that enables the optimal values ( or 'canonical costs') of actions to be learned directly, without any requirement for the animal to model its environment, or to remember situations and actions for more than a short period of time.page unknown (unnumbered Summary page; PDF p. 4)
What changed. Treats Markov decision processes as a model of an animal's behavioral choices and describes an incremental Monte-Carlo method that learns the optimal values of actions directly, without a model of the environment. Gives a proof that this learning method works and also covers hierarchical policies.
Approximation by superpositions of a sigmoidal function
George V. Cybenko · First public 1989-12 (journal issue) · Venue: Mathematics of Control, Signals, and Systems 2(4) · Pages 303–314 · Citations (OpenAlex) 13,875 (checked 2026-11-03)
DOI · Date received October 21, 1988; date revised February 17, 1989
In particular, we show that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a single internal, hidden layer and any continuous sigmoidal nonlinearity.303
What changed. Shows that finite linear combinations of compositions of a fixed univariate sigmoidal function with affine functionals are dense in the continuous functions on the unit hypercube. This means networks with a single hidden layer and continuous sigmoidal nonlinearity can approximate any such function and arbitrary decision regions.
Backpropagation Applied to Handwritten Zip Code Recognition
Yann LeCun, Bernhard E. Boser, John S. Denker, D. Henderson, Richard E. Howard, W. Hubbard et al. · First public 1989-12 (journal issue) · Venue: Neural Computation 1(4) · Pages 541–551 · Citations (OpenAlex) 11,882 (checked 2026-11-03)
DOI · Received 7 July 1989; accepted 12 September 1989
This paper demonstrates how such constraints can be integrated into a backpropagation network through the architecture of the network.541
What changed. Builds task-domain constraints into a backpropagation network's architecture and applies it to handwritten zip code digits from the U.S. Postal Service. A single network goes from the normalized character image to the final classification.