Тілдік деректүрлендіру және компьютерлік лингвистика

Field of linguistics and computer sciencenatural language processing done by computers
Natural language processing (NLP) is an interdisciplinary subfield of computer science and information retrieval. It is primarily concerned with giving computers the ability to support and manipulate human language. It involves processing natural language datasets, such as text corpora or speech corpora, using either rule based or probabilistic (i. e. statistical and, most recently, neural network based) machine learning approaches. The goal is a computer capable of "understanding" the contents of documents, including the contextual nuances of the language within them. To this end, natural language processing often borrows ideas from theoretical linguistics. The technology can then accurately extract information and insights contained in the documents as well as categorize and organize the documents themselves. Challenges in natural language processing frequently involve speech recognition, natural language understanding, and natural language generation.

The premise of symbolic NLP is well summarized by John Searle's Chinese room experiment: Given a collection of rules (e. g., a Chinese phrasebook, with questions and matching answers), the computer emulates natural language understanding (or other NLP tasks) by applying those rules to the data it confronts. 1950s: The Georgetown experiment in 1954 involved fully automatic translation of more than sixty Russian sentences into English. The authors claimed that within three or five years, machine translation would be a solved problem. However, real progress was much slower, and after the ALPAC report in 1966, which found that ten year long research had failed to fulfill the expectations, funding for machine translation was dramatically reduced. Little further research in machine translation was conducted in America (though some research continued elsewhere, such as Japan and Europe) until the late 1980s when the first statistical machine translation systems were developed. 1960s: Some notably successful natural language processing systems developed in the 1960s were SHRDLU, a natural language system working in restricted "blocks worlds" with restricted vocabularies, and ELIZA, a simulation of a Rogerian psychotherapist, written by Joseph Weizenbaum between 1964 and 1966. Using almost no information about human thought or emotion, ELIZA sometimes provided a startlingly human like interaction. When the "patient" exceeded the very small knowledge base, ELIZA might provide a generic response, for example, responding to "My head hurts" with "Why do you say your head hurts?". Ross Quillian's successful work on natural language was demonstrated with a vocabulary of only twenty words, because that was all that would fit in a computer memory at the time. 1970s: During the 1970s, many programmers began to write "conceptual ontologies", which structured real world information into computer understandable data. Examples are MARGIE (Schank, 1975), SAM (Cullingford, 1978), PAM (Wilensky, 1978), TaleSpin (Meehan, 1976), QUALM (Lehnert, 1977), Politics (Carbonell, 1979), and Plot Units (Lehnert 1981). During this time, the first chatterbots were written (e. g., PARRY). 1980s: The 1980s and early 1990s mark the heyday of symbolic methods in NLP. Focus areas of the time included research on rule based parsing (e. g., the development of HPSG as a computational operationalization of generative grammar), morphology (e. g., two level morphology), semantics (e. g., Lesk algorithm), reference (e. g., within Centering Theory) and other areas of natural language understanding (e. g., in the Rhetorical Structure Theory). Other lines of research were continued, e. g., the development of chatterbots with Racter and Jabberwacky. An important development (that eventually led to the statistical turn in the 1990s) was the rising importance of quantitative evaluation in this period.

Up until the 1980s, most natural language processing systems were based on complex sets of hand written rules. Starting in the late 1980s, however, there was a revolution in natural language processing with the introduction of machine learning algorithms for language processing. This was due to both the steady increase in computational power (see Moore's law) and the gradual lessening of the dominance of Chomskyan theories of linguistics (e. g. transformational grammar), whose theoretical underpinnings discouraged the sort of corpus linguistics that underlies the machine learning approach to language processing. 1990s: Many of the notable early successes on statistical methods in NLP occurred in the field of machine translation, due especially to work at IBM Research, such as IBM alignment models. These systems were able to take advantage of existing multilingual textual corpora that had been produced by the Parliament of Canada and the European Union as a result of laws calling for the translation of all governmental proceedings into all official languages of the corresponding systems of government. However, most other systems depended on corpora specifically developed for the tasks implemented by these systems, which was (and often continues to be) a major limitation in the success of these systems. As a result, a great deal of research has gone into methods of more effectively learning from limited amounts of data. 2000s: With the growth of the web, increasing amounts of raw (unannotated) language data has become available since the mid 1990s. Research has thus increasingly focused on unsupervised and semi supervised learning algorithms. Such algorithms can learn from data that has not been hand annotated with the desired answers or using a combination of annotated and non annotated data. Generally, this task is much more difficult than supervised learning, and typically produces less accurate results for a given amount of input data. However, there is an enormous amount of non annotated data available (including, among other things, the entire content of the World Wide Web), which can often make up for the inferior results if the algorithm used has a low enough time complexity to be practical.

In 2003, word n gram model, at the time the best statistical algorithm, was overperformed by a multi layer perceptron (with a single hidden layer and context length of several words trained on up to 14 million of words with a CPU cluster in language modelling) by Yoshua Bengio with co authors. In 2010, Tomáš Mikolov (then a PhD student at Brno University of Technology) with co authors applied a simple recurrent neural network with a single hidden layer to language modelling, and in the following years he went on to develop Word2vec. In the 2010s, representation learning and deep neural network style (featuring many hidden layers) machine learning methods became widespread in natural language processing. That popularity was due partly to a flurry of results showing that such techniques can achieve state of the art results in many natural language tasks, e. g., in language modeling and parsing. This is increasingly important in medicine and healthcare, where NLP helps analyze notes and text in electronic health records that would otherwise be inaccessible for study when seeking to improve care or protect patient privacy.

In the late 1980s and mid 1990s, the statistical approach ended a period of AI winter, which was caused by the inefficiencies of the rule based approaches. The earliest decision trees, producing systems of hard if–then rules, were still very similar to the old rule based approaches. Only the introduction of hidden Markov models, applied to part of speech tagging, announced the end of the old rule based approach.

A major drawback of statistical methods is that they require elaborate feature engineering. Since 2015, the statistical approach was replaced by the neural networks approach, using word embeddings to capture semantic properties of words. Intermediate tasks (e. g., part of speech tagging and dependency parsing) have not been needed anymore. Neural machine translation, based on then newly invented sequence to sequence transformations, made obsolete the intermediate steps, such as word alignment, previously necessary for statistical machine translation.

The following is a list of some of the most commonly researched tasks in natural language processing. Some of these tasks have direct real world applications, while others more commonly serve as subtasks that are used to aid in solving larger tasks. Though natural language processing tasks are closely intertwined, they can be subdivided into categories for convenience. A coarse division is given below.