Optogenetic modulation of neural circuits that underlie reward seeking

Tools and Strategies for Optogenetic Manipulation of Neural Circuits that Underlie Reward Processing

A number of optogenetic actuators are now available for both excitation and inhibition of neural circuits for use both in vitro and in vivo (1). To date, the most commonly used light-gated proteins for activation of neural tissue are engineered mutants of channelrhodopsin-2 (ChR2) (2). ChR2 mutants are typically maximally activated by 450–500 nm light, which allows for large inward flux of Na⁺ and Ca²⁺ at resting membrane potentials. Brief pulses of light (typically 1–5 ms) result in reliable and repeatable action potential generation in a variety of neuronal subtypes over a large range of firing frequencies (3, 4). Expression of ChR2 in neuronal fibers can also be used to selectively activate pathway-specific neurotransmitter release in brain slices (3, 5, 6) or in behaving animals (3, 7) to study the effects of afferent-specific synaptic transmission. For optogenetic inhibition studies, modified variants of both halorhodopsin (8–10) and archaerhodopsin (11, 12) have been shown to reliably silence neural activity both in vitro and in vivo. Transgenes coding for gated proteins to modulate neural activity are typically introduced into neural tissue via transgenic animals that express these proteins under cell type specific neuronal promoters (13, 14), or, more commonly, by recombinant viral vectors that can be stereotactically delivered to discrete brain nuclei. For delivery of light to neural tissue in vivo, optical fibers coupled with high-powered light sources such as lasers or LEDs are used (15–17), providing a way to restrict light delivery and thus optical modulation to specific brain structures. Combined with genetic targeting approaches to selectively express these light-activated proteins in genetically defined neurons (4), use of these tools allows for selectively modulation of neural circuits that underlie reward-related behaviors.

Neuronal Populations in the VTA and Their Role in Mediating Reward-seeking Behavior

The VTA is a heterogeneous brain structure containing neuronal populations that are essential to the expression of motivated behaviors related to addiction (18, 19). While the VTA is oftentimes treated as a distinct neural structure, few anatomical markers distinguish it from neighboring structures such as the substantia nigra pars compacta (SNc). Immunohistochemical and tracing studies have suggested that SNc is a relatively homogenous population of neurons, the majority of which are dopaminergic (90%) and project to the dorsal striatum (20, 21). The VTA, on the other hand, contains a mixture of DAergic (~65%), GABAergic (~30%), and glutamatergic neurons (~ 5%; 22) that project throughout the forebrain to structures including the amygdala, prefrontal cortex, and NAc (20). Importantly, VTA neurons that project to the NAc are a heterogeneous population of both DAergic and GABAergic neurons (21, 23, 24). Within the VTA, GABAergic neurons also are thought to form inhibitory contacts onto at least some DAergic projection neurons (25). Thus, VTA GABAergic neurons, as well as GABAergic input from the posterior segment of the VTA, also known as the rostromedial tegmental nucleus (RMTg; 26, 27), may play an important role in regulating DAergic neuron function. These diverse neuronal populations in the VTA are likely components of distinct neural circuits incorporating neurons from their afferent and efferent structures, which may act to mediate specific aspects of motivated behavioral processing (28).

The pioneering optogenetic studies in this field introduced the blue light-sensitive cation channel channelrhodopsin-2 (ChR2) exclusively into VTA DAergic neurons using viral delivery methods to examine the role of these specific VTA neurons in reward-seeking behavior (4, 5, 29, 30). While the actions of DA were known to play an important role in reward-related behaviors, it was not possible to study how selective activation of DAergic neurons alone could modulate reward-related behavior. Non-contigent, high-frequency, optical activation of VTA DAergic neurons led to the formation of a conditioned place preference to the associated environment (4). Importantly, DAergic stimulation frequencies that led to the development of a conditioned place preference also resulted in transient surges of DA release in the NAc, suggesting that only stimulation frequencies that lead to detectable changes in DA release can induce associative learning. In addition, it has been demonstrated that direct optical activitation of these neurons can reinforce operant behavioral responding (31) and also facilitate development of positive reinforcment (32). Taken together, these studies demonstrate that direct activation of VTA DAergic neurons alone can promote behavioral conditioning and reinforce behavioral responding in the absence of any additional reward.

Optogenetic strategies have also been employed in brain slice experiments to examine the possibility of neurotransmitter co-release. Neurons in the medial VTA co-express tyrosine hyrodylase and the vesicular glutamate transporter-2 (VGluT2), indicating that they are capable of both synthesizing DA and packaging glutamate into synaptic vesicles (33). Prior to optogenetic manipulations, however, it was not possible selectively stimulate DAergic fibers originating from VTA neurons to determine whether they co-release dopamine and other small-molecule neurotransmitters. Optogenetic stimulation of DAergic terminals in the NAc led to detectable glutamate-mediated excitatory postsynaptic currents (5, 34) that were not present in mice that lack VGluT2 in DAergic neurons (5). Furthermore, glutamate release was not detected in dorsal striatal regions despite optogenetic stimulation of DAergic fibers producing detectable DA release (5). These studies suggest that midbrain DAergic neurons that project the ventral, but not dorsal, striatum can co-release glutamate as a neurotransmitter. Interestingly, a recent electron microscopy study demonstrated that axonal fibers within different striatal subregions do not co-express TH and VGlutT1, VGlutT2, or VglutT3 (35). These results suggest that there may be major species-specific differences in the neurotransmitter content of DAergic neurons (the optogenetic studies were performed in mice, whereas the electron micrscopy study performed was in rats). However, it is worth noting that cultured DAergic neurons from rats also co-release glutamate (36, 37), and that electrical stimulation of the VTA results in glutamate mediated EPSPs in the prefrontal cortex of rats (38), suggesting that DAergic neurons in species other than mice also co-release glutamate. It is possible that different axonal fibers that orginate from a single DAergic neuron in the VTA may release DA or glutmate, but not both. In addition, transcriptional suppression of TH may occur in VTA neurons that coexpress VGluT2. While further studies are required, these ideas could account for the discrepancies between the optogenetic studies demonatrating DA/glutamate co-release and the electron microscopy data showing that axonal fibers in the striatum do not co-express TH and VGlut isoforms.

Excitatory Afferent Projections to the VTA and Their Role in Reward-related Behaviors

Since direct optogenetic stimulation of VTA DAergic neurons has demonstrated unequivocally that activation of these neurons is sufficient to modulate reward-related behaviors, an important line of research that remains largely unexplored is determining how specific VTA afferents modulate the activity of both DAergic and non-DAergic neurons in the VTA. Both excitatory and inhibtory afferents from a number of nuclei (see Figure 1), innervate postsynaptic neurons within the VTA. The heterogeneity of these inputs is such that electrical stimulation cannot be used to activate specific presynaptic fibers. Therefore, future studies will have to rely on afferent-specific optogenetic stimulation to study pathway-specific synaptic function in the VTA. Optogenetic stimulation of presynaptic fibers in other brain regions has already uncovered novel functions of afferent specific inputs (3, 7, 39). Given that exposure to many drugs and natural rewards can alter excitatory synaptic function onto midbrain DAergic neurons (28, 40–46), further identifying the exact synaptic connectivity to different populations of VTA neurons should provide a framework for better understanding the synaptic mechanism by which natural reward and drug abuse alter synaptic function within the VTA.

The lateral hypothalamus (LH) is thought to send the largest subcortical glutamatergic projection to the VTA/Sn (47). Electrical stimulation of the LH predominately increases the firing rates of VTA/Sn neurons that display long-duration action potential waveforms (48). In contrast, neurons that show short-duration waveforms are generally suppressed by LH stimulation (48). While most of the fibers originating from the LH and projecting to the VTA are glutamatergic (47) and may also co-release the neuropeptide orexin (49), whether there is a GABAergic input from the LH to the VTA is unknown. VTA-projecting LH neurons show increased activity as indexed by c-fos following cocaine or morphine CPP (50, 51), and VTA neurons show increases in c-fos expression following LH stimulation (52). In addition, LH self-stimulation leads to large increases in NAc DA release (53), further demonstrating an important role of this pathway in the activation of brain reward circuits as well as reinforcing behavioral responding. Taken together, these studies show that the LH is an important source of excitatory drive to the VTA. Future studies in which afferent-specific optogenetic stimulation of LH afferents to the VTA will illuminate the synaptic connectivity between these regions as well as their role in reward-related behaviors.

Another major source of glutamatergic input to the VTA comes from a long-range projection from the medial prefrontal cortex (mPFC), which is thought to target both DAergic and non-DAergic neurons (47, 54, 55). Stimulation of the mPFC leads to an increase in extracellular glutamate in the VTA (56), activates DAergic and non-DAergic neurons (57–59), and elevates DA release in the forebrain (60, 61). Interestingly, an electron microscopy study by Carr and Sesack (54) showed that mPFC afferents to the VTA form synapses onto mPFC-projecting, but not NAc-projecting, DAergic neurons. mPFC afferents also formed synapses onto VTA GABAergic neurons that project to the NAc, but not those that project to the mPFC. Afferent-specific optogenetic stimulation with postsynaptic recordings from DAergic and non-DAergic neurons in the VTA is needed to corroborate these findings.

The lateral habenula (LHb) is another major excitatory input to the VTA and is an important modulator of DAergic neuronal activity (62-64). Glutamatergic fibers originating in the LHb project directly to the VTA (47, 65) and are thought to form synapses on both DAergic and GABAergic neurons there (66). Interestingly, electrophysiological recordings from LHb neurons in behaving monkeys have demonstrated that these neurons are predominantly inhibited during reward expectation, while DAergic neurons in the midbrain show excitatory responses (63, 64, 67). In addition, LHb neurons are excited by aversive stimuli (63), and stimulation of the LHb leads to reduced DA release in the NAc (68). These studies suggest that LHb neurons send a direct glutamatergic projection to predominantly GABAergic neurons in the VTA/Sn that can inhibit DAergic neuronal activity.

The VTA also receives a mixed glutamatergic/cholinergic projection from the pedunculopontine tegmental nucleus (PPTg) and the laterodorsal tegmental nucleus (LDT) (69–71). Consistent with this, electrical stimulation of PPTg/LDT increases the activity of VTA neurons and increases DA release in the NAc (72). While the PPTg/LDT plays an important role in driving drug-seeking behavior (73), how the synaptic connectivity between the PPTg/LDT and the VTA controls reward-related behaviors remains unknown.

Inhibitory Afferents to the VTA/SNc and Their Role in Reward-related Behaviors

Within the VTA, afferent-specific optogenetic stimulation experiments examining the synaptic connectivity between neurons have demonstrated that distal GABAergic neurons orginating in the NAc form functional inhibitory synaptic contacts onto non-DAergic neurons in the VTA (74). Furthermore, some non-DAergic neurons that receive inhibitory inputs from the NAc were shown to project back to the NAc (74). This study elegantly demonstrated the precise functional inhibitory connectivity between the NAc to VTA GABAergic neurons. Furthermore, other neurons throughout the extended amydala, such as those from the central nucleus of the amygdala (CeA) and the bed nucleus of the stria terminalus (BNST), also project to the VTA (75–77). While electrical stimulation of the BNST produces both excitatory and inhibitory responses in the VTA (76), given the known cellular composition of the extended amygdala, it is likely that these VTA projections are mostly GABAergic (78). Local VTA GABAergic activity is also thought to be a potent modulator of DAergic output. The specific role for this pathway in regulating VTA neural activity in vivo and during behavioral tasks should be explored further.

Neuronal Populations in the NAc and Their Role in Reward-related Behaviors

The NAc integrates reward-related information conveyed by dopamine and GABAergic inputs from the midbrain with glutamatergic inputs from regions throughout the brain. There is a fairly consistent cellular architecture throughout the dorsal and ventral striatum, including the NAc (79, 80). Striatal projection neurons, also known as medium spiny neurons (MSNs), make up more than 90% of the local neurons in the NAc and are GABAergic (81). Striatal interneurons are a mix of both cholinergic and GABAergic neurons (82). MSNs are typically classified into two main groups based on the type of opioid peptides they release (dynorphin or enkephalin) and the dopamine receptors they express (D1 or D2), as well as their projection targets (direct or indirect to the midbrain; see Figure 2; 79, 83). However, this strict division between the two types of MSNs may be less true in the NAc than in the dorsal striatum (84, 85). It is also unclear if this division is as steadfast in species other than mice, where transgenic animals have provided strong evidence for discrete populations of projection neurons (86).

Optogenetic investigations utilizing ChR2 in acute brain slices have demonstrated that direct pathway MSNs in the NAc specifically target GABAergic neurons in the VTA (74; see above). This target specificity has also been observed in direct pathway MSNs of the dorsal striatum, which only innervate GABAergic neurons of the substantia nigra pars reticulata (87). Neither of these optogenetic investigations found evidence for direct innervation of midbrain DA neurons by MSNs. Indirect pathway MSNs from the dorsal and ventral striatum target the external segment of the globus pallidus and the ventral pallidum, respectively (88). As with the direct pathway MSNs, the projections from indirect pathway MSNs may selectively target specific GABAergic neurons (87). In addition, striatal MSNs form inhibitory synapses onto each other, as well as local cholinergic, but not fast-spiking GABAergic interneurons (87). The intricate connectivity within the NAc suggests that multiple genetically distinct neuronal subtypes cooperate to orchestrate motivated behavior.

The distinct projection targets of the two main types of MSNs suggest that the activity of these pathways oppose each other and can produce a bidirectional regulation of behavior (79, 89). Recent optogenetic investigations have provided additional support for this hypothesis. When ChR2 was expressed selectively in either D1R or D2R expressing MSNs in the dorsomedial striatum by viral delivery, basal locomotion could be increased or decreased, respectively (90). While these results support the idea that the dorsal striatum has a critical role in the bidirectional control of motor behavior, selective activation of these distinct neural circuit elements on a physiologically relevant timescale was not possible without optogenetic manipulations.

Optogenetic investigations specifically in the NAc have explored whether the activation or inhibition of specific cell types alters either basal locomotion or cocaine-induced behaviors. In one study in which ChR2 was expressed in D1R- or D2R-expressing MSNs in naïve mice, optogenetic activation of these two neuronal subtypes in the NAc did not influence basal locomotor activity (91). However, when the mice were treated with multiple injections of cocaine that could reliably produce behavioral sensitization, optogenetic activation of NAc D1R-expressing MSNs became sufficient to enhance locomotion (91). This demonstrates that repeated exposure to cocaine might alter the functional activity of NAc D1R-expressing MSNs so that effects on locomotion become evident upon activation of this pathway. This emergent effect may account for the increased locomotion observed in behavioral sensitization, although this idea remains to be tested.

The optical activation of D1R- or D2R-expressing NAc MSNs on its own was ineffective in eliciting a place preference when optically stimulated at 10 Hz (91). However, activation of D1R-expressing NAc MSNs during cocaine exposure enabled a sub-threshold dose of cocaine to produce a conditioned place preference (91). Similar effects were observed with an indiscriminate ChR2-mediated activation of all NAc neurons during cocaine exposure (91). Conversely, activation of D2R-expressing MSNs in the NAc during cocaine exposure diminished the development of a conditioned place preference (91). These opposing effects are another example of how the two types of MSNs can exert bidirectional control over behavior, specifically on the ability of cocaine to induce a conditioned place preference.

The influence of interneuron populations on behavior has been explored by optogenetically silencing or augmenting their activity. Both the light-activated chloride pump halorhodopsin 3.0 and ChR2 have been introduced into NAc cholinergic interneurons by viral methods. Neither activating nor inhibiting these neurons with light produced gross behavioral effects (92). These manipulations in their own right were also ineffective in eliciting a place preference (92). Inhibiting these neurons during cocaine exposure, however, decreased CPP, which is similar in effect to that observed from activating NAc D2R-expressing MSNs during cocaine exposure (91, 92). Interestingly, the inhibition of NAc cholinergic interneurons increased the firing rate in the majority of neighboring cells (92). This observation is inconsistent with the finding that indiscriminate activation of all NAc neurons with ChR2 augments cocaine reward (91), but perhaps cholinergic interneuron inhibition preferentially activates D2R-expressing MSNs.

Taken together, these experiments confirm that the two main populations of MSNs exert opposing effects on behavior. In the NAc, optogenetic manipulations to specific cell types affect the development of cocaine-induced conditioned place preference. While no publications to date have examined the behavioral effects of optogenetic manipulations to NAc GABAergic interneurons, it is likely that they also influence cocaine-related behaviors. Overall, the behaviors examined with optogenetic techniques to date (locomotion, behavioral sensitization, and conditioned place preference) are indirect metrics of reward-seeking. As optogenetic techniques become more widely adopted, it will be interesting to see how specific neural circuit perturbation directly alters reward-seeking behaviors such as drug self-administration.

Inputs to the NAc and Their Role in Reward-related Behaviors

Both cortical and subcortical structures innervate the NAc and form asymmetrical synapses onto spines of MSNs (88). Interestingly, DA release sites often converge onto the necks of these same spines (93). This location of DA synapses is consistent with the neuromodulatory role of DA. That is, the extent to which ongoing glutamatergic activity can influence the firing of MSNs is probably dependent on local DA release (80, 94). Importantly, as described above, VTA DA neurons also release glutamate onto NAc MSNs (5, 34). In addition to the DA and glutamate projections from the VTA, there is evidence of a GABAergic projection from the VTA to the NAc (24, 74). While the optogenetic modulation of VTA DAergic neurons induce behavioral effects, how selective activation of the VTA GABAergic projections can influence behavior is not known.

Glutamatergic inputs to the NAc come primarily from the amygdala, PFC, hippocampus, and thalamus (88). These afferents are heterogeneously distributed, but specific clusters or ensembles of cells tend to have a convergence of afferents from the same nuclei (95-97). There is also evidence that single MSNs receive inputs from several afferent structures, including the amygdala, PFC, and hippocampus (3, 97). This synaptic arrangement is similar to that observed in the dorsal striatum, where single axons innervating the striatum target both D1R- and D2R-expressing MSNs and single MSNs receive inputs from both cortical and thalamic regions (98). This highlights the divergent and convergent nature of the information flow into the ventral striatum.

Optogenetic examinations of afferents to the NAc have explored amygdala and PFC inputs (3). ChR2 was introduced into these regions under the control of the calcium-calmodulin-dependent protein kinase II_ promoter, which is preferentially expressed in glutamatergic projection neurons originating from the BLA or PFC. The ability of these pathways to control reward-seeking behavior was then examined by determining whether mice would engage in self-stimulation behavior to selectively activate either the BLA-to-NAc or mPFC-to-NAc pathways (3). Interestingly, activation of glutamatergic afferents from the BLA readily supported self-stimulation behavior, while the mPFC-to-NAc pathway did not. This finding demonstrates that specific glutamatergic afferents to the NAc can promote reward-seeking behavior. In addition, optogenetically silencing BLA-to-NAc afferents disrupted the formation of cue-reward associations, implying that both DA release from VTA-to-NAc fibers and glutamate release from BLA inputs cooperatively regulate direct-pathway neuronal output to facilitate reward seeking.

Corticostriatal projections, especially those to the NAc, are important for generating and maintaining goal-directed behaviors such as drug self-administration, but also for responding to a changing environment (99). It has been hypothesized that repeated exposure to drugs of abuse results in a down regulation of mPFC-to-NAc glutamate signaling that leads to the inability to control drug-taking behavior (100), although this has not yet been explored with optogenetic manipulations. Future research should address this role of the mPFC inputs and the behavioral impact of activating hippocampal or thalamic inputs to the NAc.

Conclusions and Future Directions

While optogenetic manipulations of brain reward circuitry have already helped establish and refute many hypotheses that were previously untestable with traditional techniques, studies that will expand our understanding of the mechanistic insight into the neural circuits that underlie reward-seeking behavior are yet to come. Most of the studies reviewed have utilized ChR2 for optogenetic stimulation of defined neuronal populations. While these studies have certainly provided groundbreaking findings, many of these early studies failed to take advantage of one of the greatest benefits that optogenetic methodologies allow for—transient activation or inactivation of neural tissue time-locked to discrete and infrequent behaviorally relevant events. In addition, while optogenetic stimulation experiments can demonstrate that synchronous activation of neurons can modulate behaviors related to reward seeking, they do not demonstrate the necessity of specific neural circuit elements in behavior. Loss of function experiments utilizing optogenetic inhibition of neural circuits have lagged behind the adoption of ChR2-mediated stimulation, partially due to technical limitations that have been recently overcome (74). These future studies will likely play a more prominent role in determining the role of specific connections between neurons in mediating motivated behavioral responses. Finally, a plethora of in vivo electrophysiological and neurochemical studies have elegantly shown correlations between neural activity and reward-seeking behavior. As optogenetic strategies for neural circuit manipulation are refined, these techniques to monitor neural activity will certainly become fully integrated with optogenetic circuit manipulation to assess how neuronal actuation can alter neurophysiological and neurochemical responses that are observed during reward-seeking behavior. Taken together, optogenetic strategies for perturbation of neural function, used in conjunction with sophisticated behavioral paradigms to study reward seeking, will likely greatly enhance our understanding of the circuitry that mediates behavior.