On the emergence of rules in neural networks.

Full Abstract

A simple associationist neural network learns to factor abstract rules (i.e., grammars) from sequences of arbitrary input symbols by inventing abstract representations that accommodate unseen symbol sets as well as unseen but similar grammars. The neural network is shown to have the ability to transfer grammatical knowledge to both new symbol vocabularies and new grammars. Analysis of the state-space shows that the network learns generalized abstract structures of the input and is not simply memorizing the input strings. These representations are context sensitive, hierarchical, and based on the state variable of the finite-state machines that the neural network has learned. Generalization to new symbol sets or grammars arises from the spatial nature of the internal representations used by the network, allowing new symbol sets to be encoded close to symbol sets that have already been learned in the hidden unit space of the network. The results are counter to the arguments that learning algorithms based on weight adaptation after each exemplar presentation (such as the long term potentiation found in the mammalian nervous system) cannot in principle extract symbolic knowledge from positive examples as prescribed by prevailing human linguistic theory and evolutionary psychology.

Learn Faster Today Improve your study skills

Author information

Author/s: Hanson, Stephen José (SJ); Negishi, Michiro (M);

Affiliation: Psychology Department, Rutgers University, Newark, N.J. 07102, USA. jose@tractatus.rutgers.edu

Journal and publication information

Publication Type: Journal Article

Journal: Neural computation (Neural Comput), published in United States. (Language: eng)

Reference: 2002-Sep; vol 14 (issue 9) : pp 2245-68

Dates: Created 2002/08/19; Completed 2002/09/23; Revised 2004/11/17;

PMID: 12184850, status: MEDLINE (last retrieval date: 11/6/2008)

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article (including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.

MeSH headings (categories)

This article was linked to the MESH Headings shown below.

This article has not been indexed for related articles as yet, however you can still use the live related article search links below.

See 100+ related articles.