André Fiala and Florentin Wörgötter

Reinforcement learning (RL) is a fundamental adaptive process in fruit flies (Drosophila melanogaster), which can learn to avoid punishment by associating an odor (CS) with an electric shock (US), reflected by prolonged dopaminergic (DA) responses. In vertebrates, reinforcement signals ("Critic") and motor signals ("Actor") remain separate in the basal ganglia. Flies, on the other hand, may exhibit a fundamentally different organization of DA-circuitry. Intriguingly, differential Hebbian learning theory (Diff.-Hebb) proposes a link between Actor and Critic. In this project we will experimentally assess as well as model the physiological circuitry of the fly DA-system. To this end, we will design a biophysical implementation of the Diff-Hebb 'Actor-joint-Critic' architecture. This model will be based on prior knowledge of structure and function of the fly DA-system and on general Diff.-Hebb theory. In parallel, experiments start with several testable predictions from the current theory. The specific outcome of these experiments will be used to improve and constrain the model. This study will, thus, be the first to biophysically model the fly sensori-motor & DA system. It will help to differentiate it from that in vertebrates and, thus, potentially identify fundamentally novel learning principles and architectures for reinforcement learning realized by biological neural systems.