Вычислительный лексикон английского языка: WordNet и его применение.

Computational lexicon of English

WordNet is a lexical database of semantic relations between words that links words into semantic relations including synonyms, hyponyms, and meronyms. The synonyms are grouped into synsets with short definitions and usage examples. It can thus be seen as a combination and extension of a dictionary and thesaurus. While it is accessible to human users via a web browser, its primary use is in automatic text analysis and artificial intelligence applications. It was first created in the English language and the English WordNet database and software tools have been released under a BSD style license and are freely available for download from that WordNet website. There are now WordNets in more than 200 languages.

WordNet was first created in 1985, in English only, in the Cognitive Science Laboratory of Princeton University under the direction of psychology professor George Armitage Miller. It was later directed by Christiane Fellbaum. The project was initially funded by the U. S. Office of Naval Research, and later also by other U. S. government agencies including the DARPA, the National Science Foundation, the Disruptive Technology Office (formerly the Advanced Research and Development Activity) and REFLEX. George Miller and Christiane Fellbaum received the 2006 Antonio Zampolli Prize for their work with WordNet. The Global WordNet Association is a non commercial organization that provides a platform for discussing, sharing and connecting WordNets for all languages in the world. Christiane Fellbaum and Piek Th. J. M. Vossen are its co presidents.

The initial goal of the WordNet project was to build a lexical database that would be consistent with theories of human semantic memory developed in the late 1960s. Psychological experiments indicated that speakers organized their knowledge of concepts in an economic, hierarchical fashion. Retrieval time required to access conceptual knowledge seemed to be directly related to the number of hierarchies the speaker needed to "traverse" to access the knowledge. Thus, speakers could more quickly verify that canaries can sing because a canary is a songbird, but required slightly more time to verify that canaries can fly (where they had to access the concept "bird" on the superordinate level) and even more time to verify canaries have skin (requiring look up across multiple levels of hyponymy, up to "animal"). While such psycholinguistic experiments and the underlying theories have been subject to criticism, some of WordNet's organization is consistent with experimental evidence. For example, anomic aphasia selectively affects speakers' ability to produce words from a specific semantic category, a WordNet hierarchy. Antonymous adjectives (WordNet's central adjectives in the dumbbell structure) are found to co occur far more frequently than chance, a fact that has been found to hold for many languages.

WordNet is sometimes called an ontology, a persistent claim that its creators do not make. The hypernym/hyponym relationships among the noun synsets can be interpreted as specialization relations among conceptual categories. In other words, WordNet can be interpreted and used as a lexical ontology in the computer science sense. However, such an ontology should be corrected before being used, because it contains hundreds of basic semantic inconsistencies; for example there are, (i) common specializations for exclusive categories and (ii) redundancies in the specialization hierarchy. Furthermore, transforming WordNet into a lexical ontology usable for knowledge representation should normally also involve (i) distinguishing the specialization relations into subtypeOf and instanceOf relations, and (ii) associating intuitive unique identifiers to each category. Although such corrections and transformations have been performed and documented as part of the integration of WordNet 1.7 into the cooperatively updatable knowledge base of WebKB 2, most projects claiming to reuse WordNet for knowledge based applications (typically, knowledge oriented information retrieval) simply reuse it directly. WordNet has also been converted to a formal specification, by means of a hybrid bottom up top down methodology to automatically extract association relations from it and interpret these associations in terms of a set of conceptual relations, formally defined in the DOLCE foundational ontology. In most works that claim to have integrated WordNet into ontologies, the content of WordNet has not simply been corrected when it seemed necessary; instead, it has been heavily reinterpreted and updated whenever suitable. This was the case when, for example, the top level ontology of WordNet was restructured according to the OntoClean based approach, or when it was used as a primary source for constructing the lower classes of the SENSUS ontology.

The most widely discussed limitation of WordNet (and related resources like ImageNet) is that some of the semantic relations are more suited to concrete concepts than to abstract concepts. For example, it is easy to create hyponyms/hypernym relationships to capture that a "conifer" is a type of "tree", a "tree" is a type of "plant", and a "plant" is a type of "organism", but it is difficult to classify emotions like "fear" or "happiness" into equally deep and well defined hyponyms/hypernym relationships. Many of the concepts in WordNet are specific to certain languages and the most accurate reported mapping between languages is 94%. Synonyms, hyponyms, meronyms, and antonyms occur in all languages with a WordNet so far, but other semantic relationships are language specific. This limits the interoperability across languages. However, it also makes WordNet a resource for highlighting and studying the differences between languages, so it is not necessarily a limitation for all use cases. WordNet does not include information about the etymology or the pronunciation of words and it contains only limited information about usage. WordNet aims to cover most everyday words and does not include much domain specific terminology. WordNet is the most commonly used computational lexicon of English for word sense disambiguation (WSD), a task aimed at assigning the context appropriate meanings (i. e. synset members) to words in a text. However, it has been argued that WordNet encodes sense distinctions that are too fine grained. This issue prevents WSD systems from achieving a level of performance comparable to that of humans, who do not always agree when confronted with the task of selecting a sense from a dictionary that matches a word in a context. The granularity issue has been tackled by proposing clustering methods that automatically group together similar senses of the same word.

WordNet includes words that can be perceived as pejorative or offensive. The interpretation of a word can change over time and between social groups, so it is not always possible for WordNet to define a word as "pejorative" or "offensive" in isolation. Therefore, people using WordNet must apply their own methods to identify offensive or pejorative words. However, this limitation is true of other lexical resources like dictionaries and thesauruses, which also contain pejorative and offensive words. Some dictionaries indicate words that are pejoratives, but do not include all the contexts in which words might be acceptable or offensive to different social groups. Therefore, people using dictionaries must apply their own methods to identify all offensive words.

Some wordnets were subsequently created for other languages. A 2012 survey lists the wordnets and their availability. In an effort to propagate the usage of WordNets, the Global WordNet community had been slowly re licensing their WordNets to an open domain where researchers and developers can easily access and use WordNets as language resources to provide ontological and lexical knowledge in natural language processing (NLP) tasks. The Open Multilingual WordNet provides access to open licensed wordnets in a variety of languages, all linked to the Princeton Wordnet of English (PWN). The goal is to make it easy to use wordnets in multiple languages.

WordNet has been used for a number of purposes in information systems, including word sense disambiguation, information retrieval, automatic text classification, automatic text summarization, machine translation and even automatic crossword puzzle generation. A common use of WordNet is to determine the similarity between words. Various algorithms have been proposed, including measuring the distance among words and synsets in WordNet's graph structure, such as by counting the number of edges among synsets. The intuition is that the closer two words or synsets are, the closer their meaning. A number of WordNet based word similarity algorithms are implemented in a Perl package called WordNet::Similarity, and in a Python package called NLTK. Other more sophisticated WordNet based similarity techniques include ADW, whose implementation is available in Java. WordNet can also be used to inter link other vocabularies.

Princeton maintains a list of related projects that includes links to some of the widely used application programming interfaces available for accessing WordNet using various programming languages and environments.

WordNet is connected to several databases of the Semantic Web. WordNet is also commonly reused via mappings between the WordNet synsets and the categories from ontologies. Most often, only the top level categories of WordNet are mapped.

The Global WordNet Association (GWA) is a public and non commercial organization that provides a platform for discussing, sharing and connecting wordnets for all languages in the world. The GWA also promotes the standardization of wordnets across languages, to ensure its uniformity in enumerating the synsets in human languages. The GWA keeps a list of wordnets developed around the world.

BabelNet, a very large multilingual semantic network with millions of concepts obtained by integrating WordNet and Wikipedia using an automatic mapping algorithm. The SUMO ontology has a complete manual mapping between all of the WordNet synsets and all of SUMO (including its domain ontologies, when WordNet contains a word sense for a given SUMO term) which is browsable at, for example OpenCyc, an open ontology and knowledge base of everyday common sense knowledge, has 12,000 terms linked to WordNet synonym sets. DOLCE, is the first module of the WonderWeb Foundational Ontologies Library (WFOL). This upper ontology has been developed in light of rigorous ontological principles inspired by the philosophical tradition, with a clear orientation toward language and cognition. OntoWordNet is the result of an experimental alignment of WordNet's upper level with DOLCE. It is suggested that such alignment could lead to an "ontologically sweetened" WordNet, meant to be conceptually more rigorous, cognitively transparent, and efficiently exploitable in several applications. DBpedia, a database of structured information, is linked to WordNet. The eXtended WordNet is a project at the University of Texas at Dallas which aims to improve WordNet by semantically parsing the glosses, thus making the information contained in these definitions available for automatic knowledge processing systems. It is freely available under a license similar to WordNet's. The GCIDE project produced a dictionary by combining a public domain Webster's Dictionary from 1913 with some WordNet definitions and material provided by volunteers. It was released under the copyleft license GPL. ImageNet is an image database organized according to the WordNet hierarchy (currently only the nouns), in which each node of the hierarchy is depicted by millions of images. Currently, it has over 500 images per node on average. BioWordnet, a biomedical extension of wordnet was abandoned due to issues about stability over versions. WikiTax2WordNet, a mapping between WordNet synsets and Wikipedia categories. WordNet++, a resource including over millions of semantic edges harvested from Wikipedia and connecting pairs of WordNet synsets. SentiWordNet, a resource for supporting opinion mining applications obtained by tagging all the WordNet 3.0 synsets according to their estimated degrees of positivity, negativity, and neutrality. ColorDict, is an Android application to mobiles phones that use Wordnet database and others, like Wikipedia. UBY LMF a database of 10 resources including WordNet.

FrameNet is a lexical database that shares some similarities with, and refers to, WordNet. Lexical markup framework (LMF) is an ISO standard specified within ISO/TC37 in order to define a common standardized framework for the construction of lexicons, including WordNet. The subset of LMF for Wordnet is called Wordnet LMF. An instantiation has been made within the KYOTO project. UNL Programme is a project under the auspices of UNO aimed to consolidate lexicosemantic data of many languages to be used in machine translation and information extraction systems. Meaning Monkey is a free online dictionary based on the WordNet database. Dictionary. video is a video dictionary focusing on pronunciations. It's text part is extended from WordNet.