Multiple model-based reinforcement learning.

Full Abstract

We propose a modular reinforcement learning architecture for nonlinear, nonstationary control tasks, which we call multiple model-based reinforcement learning (MMRL). The basic idea is to decompose a complex task into multiple domains in space and time based on the predictability of the environmental dynamics. The system is composed of multiple modules, each of which consists of a state prediction model and a reinforcement learning controller. The "responsibility signal," which is given by the softmax function of the prediction errors, is used to weight the outputs of multiple modules, as well as to gate the learning of the prediction models and the reinforcement learning controllers. We formulate MMRL for both discrete-time, finite-state case and continuous-time, continuous-state case. The performance of MMRL was demonstrated for discrete case in a nonstationary hunting task in a grid world and for continuous case in a nonlinear, nonstationary control task of swinging up a pendulum with variable physical parameters.

Learn Faster Today Improve your study skills

Author information

Author/s: Doya, Kenji (K); Samejima, Kazuyuki (K); Katagiri, Ken-ichi (K); Kawato, Mitsuo (M);

Affiliation: Human Information Science Laboratories, ATR International, Seika, Soraku, Kyoto 619-0288, Japan. doya(-atsign-)atr.co.jp

Journal and publication information

Publication Type: Journal Article

Journal: Neural computation (Neural Comput), published in United States. (Language: eng)

Reference: 2002-Jun; vol 14 (issue 6) : pp 1347-69

Dates: Created 2002/05/21; Completed 2002/06/06; Revised 2004/11/17;

PMID: 12020450, status: MEDLINE (last retrieval date: 11/6/2008)

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article (including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.

MeSH headings (categories)

This article was linked to the MESH Headings shown below.

This article has not been indexed for related articles as yet, however you can still use the live related article search links below.

See 100+ related articles.