논문으로 읽는 인공지능의 역사
English

언어·음성

통계적 언어 모델과 음성 인식에서 임베딩, 어텐션, 트랜스포머, 사전학습 언어 모델까지.

언어·음성 1960 1970 1980 1990 2000 2010 2020
실선은 인용이 확인된 선(OpenAlex 또는 인용한 논문의 참고문헌 목록), 점선은 교과서가 말하는 영향이지만 인용이 확인되지 않은 선. 세로 위치는 주제 묶음.
연도첫 저자제목장
1956ChomskyThree models for the description of language1장 기원 1943–56
1966WeizenbaumELIZA—a computer program for the study of natural language communication between man and machine2장 기호와 퍼셉트론 1956–74
1971WinogradUnderstanding natural language2장 기호와 퍼셉트론 1956–74
1987Waibel et al.Phoneme recognition using time-delay neural networks3장 지식 시스템과 연결주의 1974–93
1989RabinerA tutorial on hidden Markov models and selected applications in speech recognition3장 지식 시스템과 연결주의 1974–93
1990Brown et al.A statistical approach to machine translation4장 통계 학습 1990–2011
2001Lafferty et al.Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data4장 통계 학습 1990–2011
2003Bengio et al.A Neural Probabilistic Language Model4장 통계 학습 1990–2011
2010Mikolov et al.Recurrent neural network based language model4장 통계 학습 1990–2011
2011Collobert et al.Natural Language Processing (almost) from Scratch4장 통계 학습 1990–2011
2012Hinton et al.Deep Neural Networks for Acoustic Modeling in Speech Recognition5장 딥러닝 2012–17
2013Mikolov et al.Efficient Estimation of Word Representations in Vector Space5장 딥러닝 2012–17
2013Mikolov et al.Distributed Representations of Words and Phrases and their Compositionality5장 딥러닝 2012–17
2014Cho et al.Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation5장 딥러닝 2012–17
2014Bahdanau et al.Neural Machine Translation by Jointly Learning to Align and Translate5장 딥러닝 2012–17
2014Sutskever et al.Sequence to Sequence Learning with Neural Networks5장 딥러닝 2012–17
2014Pennington et al.GloVe: Global Vectors for Word Representation5장 딥러닝 2012–17
2015Luong et al.Effective Approaches to Attention-based Neural Machine Translation5장 딥러닝 2012–17
2015Sennrich et al.Neural Machine Translation of Rare Words with Subword Units5장 딥러닝 2012–17
2016Wu et al.Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation5장 딥러닝 2012–17
2017Vaswani et al.Attention Is All You Need5장 딥러닝 2012–17
2018Howard & RuderUniversal Language Model Fine-tuning for Text Classification6장 사전학습·생성 2017–22
2018Peters et al.Deep contextualized word representations6장 사전학습·생성 2017–22
2018Radford et al.Improving Language Understanding by Generative Pre-Training6장 사전학습·생성 2017–22
2018Devlin et al.BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding6장 사전학습·생성 2017–22
2019Radford et al.Language Models are Unsupervised Multitask Learners6장 사전학습·생성 2017–22
2019Raffel et al.Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer6장 사전학습·생성 2017–22
2020Brown et al.Language Models are Few-Shot Learners6장 사전학습·생성 2017–22
2020Lewis et al.Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks6장 사전학습·생성 2017–22
2021Chen et al.Evaluating Large Language Models Trained on Code6장 사전학습·생성 2017–22
2022Wei et al.Chain-of-Thought Prompting Elicits Reasoning in Large Language Models6장 사전학습·생성 2017–22
2022Wang et al.Self-Consistency Improves Chain of Thought Reasoning in Language Models6장 사전학습·생성 2017–22
2022Kojima et al.Large Language Models are Zero-Shot Reasoners6장 사전학습·생성 2017–22
2023Touvron et al.LLaMA: Open and Efficient Foundation Language Models6장 사전학습·생성 2017–22
2023OpenAIGPT-4 Technical Report6장 사전학습·생성 2017–22