Back
One-shot learning and behavioral eligibility traces in sequential decision making
Journal article

One-shot learning and behavioral eligibility traces in sequential decision making

  • Lehmann, Marco P ORCID School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
  • Xu, He A Laboratory of Psychophysics, School of Life Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
  • Liakoni, Vasiliki ORCID School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
  • Herzog, Michael H Laboratory of Psychophysics, School of Life Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
  • Gerstner, Wulfram School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
  • Preuschoff, Kerstin Swiss Center for Affective Sciences, University of Geneva, Geneva, Switzerland
Show more…
  • 2019-11-11
Published in:
  • eLife. - eLife Sciences Publications, Ltd. - 2019, vol. 8
General Biochemistry, Genetics and Molecular Biology
General Immunology and Microbiology
General Neuroscience
General Medicine
English In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms solving this credit assignment problem: In classic temporal-difference learning, earlier actions receive reward information only after multiple repetitions of the task, whereas models with eligibility traces reinforce entire sequences of actions from a single experience (one-shot). Here, we show one-shot learning of sequences. We developed a novel paradigm to directly observe which actions and states along a multi-step sequence are reinforced after a single reward. By focusing our analysis on those states for which RL with and without eligibility trace make qualitatively distinct predictions, we find direct behavioral (choice probability) and physiological (pupil dilation) signatures of reinforcement learning with eligibility trace across multiple sensory modalities.
Language
  • English
Open access status
gold
Identifiers
Persistent URL
https://folia.unifr.ch/global/documents/252448
Statistics
Document views: 11 File downloads: