Dynamical and stationary properties of on-line learning from finite training sets.

Full Abstract

The dynamical and stationary properties of on-line learning from finite training sets are analyzed by using the cavity method. For large input dimensions, we derive equations for the macroscopic parameters, namely, the student-teacher correlation, the student-student autocorrelation and the learning force fluctuation. This enables us to provide analytical solutions to Adaline learning as a benchmark. Theoretical predictions of training errors in transient and stationary states are obtained by a Monte Carlo sampling procedure. Generalization and training errors are found to agree with simulations. The physical origin of the critical learning rate is presented. Comparison with batch learning is discussed throughout the paper.

Learn Faster Today Improve your study skills

Author information

Author/s: Luo, Peixun (P); Wong, K Y Michael (KY);

Affiliation: Department of Physics, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China. physlpx@ust.hk

Journal and publication information

Publication Type: Journal Article; Research Support, Non-U.S. Gov't

Journal: Physical review. E, Statistical, nonlinear, and soft matter physics (Phys Rev E Stat Nonlin Soft Matter Phys), published in United States. (Language: eng)

Reference: 2003-Jan; vol 67 (issue 1 Pt 1) : pp 011906

Dates: Created 2003/03/14; Completed 2003/06/30; Revised 2006/11/15;

PMID: 12636531, status: MEDLINE (last retrieval date: 11/6/2008)

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article (including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.

MeSH headings (categories)

This article was linked to the MESH Headings shown below.

This article has not been indexed for related articles as yet, however you can still use the live related article search links below.

See 100+ related articles.