How Can the Brain Efficiently Build an Understanding of the Natural World
Date Posted:
February 16, 2018
Date Recorded:
February 16, 2018
Speaker(s):
Ann M. Hermundstad
All Captioned Videos CBMM Special Seminars
Description:
Ann M. Hermundstad, PhD, Janelia Research Campus
Abstract: The brain exploits the statistical regularities of the natural world. In the visual system, an efficient representation of light intensity begins in retina, where statistical redundancies are removed via spatiotemporal decorrelation. Much less is known, however, about the efficient representation of complex features in higher visual areas. I will discuss how the central visual system, operating with different goals and under different constraints, makes efficient use of resources to extract meaningful features from complex visual stimuli. I will then highlight how these same principles can be generalized to dynamic situations, where both the environment and the goals of the system are in flux. Together, these principles have implications for understanding a broad range of phenomena across animals and sensory modalities.
VICTOR Młynarski: My name is Victor Młynarski. I am a post-doc in Josh McDermott's lab for computational [INAUDIBLE]. And I would like to welcome you all to the CBMM special seminar organized by our trainee council, or if you don't know what that means, by the Council of Post-Docs and Graduate Students of the CBMM.
It is truly my great pleasure to introduce today's guest speaker, Dr. Ann Hermundstad, from Jenelia Research Campus. Ann was trained as a physicist firstly in the School of Mines in her native Colorado. And later during her PhD, in the University of California Santa Barbara. And after completing her doctorate she moved on to University of Pennsylvania, UPenn, to work with Vijay Balasubramanian during her post-doc. She also collaborated with Jonathan Victor at Cornell and Olivia [? Marr ?] in Paris.
Since 2016 she's a group leader in theoretical and computational neuroscience at Janelia Research Campus. And to say a few more words about her research interest, let me tell that Ann is a true theorist in that she focuses her work on fundamental laws or principles of information processing by the brain. And she applies that perspective to a very broad range of systems and model organisms. To name a few she worked on the retina, on the olfactory system, on human visual perception, and on motion processing the flies.
I guess today we are going to see a cross-section for at least some part of that vast space. To introduce our speaker further, let me tell you that in addition to her academic talents, Ann is an accomplished tap dancer. And if you look at her academic web page, you can find photographic evidence of that. Ladies and gentlemen, please join in welcoming Dr. Ann Hermundstad.
DR. ANN HERMUNDSTAD: Aw. Thank you, Victor, for the amazing introduction. It's a pleasure to be here. I really enjoyed my visit here today so far. And I'm excited to get to share some of the work that we've been doing with you. So I'm going to talk today about work that we've been doing to understand how the brain can build efficient representations of the natural world that can be used to guide behavior. I'm going to frame much of this work in the context of the visual system. But many of the concepts and frameworks that we're developing we think can be applied much more broadly.
Now, we know that in higher order correlations that involve multiple points in space, or in time, are processed higher in the visual pathway. And we know that these higher order correlations contain a lot of information that we can use to parse texture and shape and movement. And we can see that very directly by isolating these types of higher order correlations in natural movies.
So does anyone yet know what it is that we're looking at?
Koala.
DR. ANN HERMUNDSTAD: Yeah. So some people thought this might be Einstein climbing a tree. I can tell you of the maker of the movie this is a koala. And we can tell that it's a koala despite the fact that I've removed a significant amount of structure from this video. So I've removed information about color, about luminance. So every frame of this video is in black and white and there's an equal number of black and white pixels in every frame. I've also removed expected, or average, pair correlations from every frame of this image, which is something that is characteristic of our natural visual world. And what's left over are higher order correlations that we can use to make sense of the fur of the koala and its shape.
And so we'd like to understand how the brain creates representations of these higher order visual features. And we can begin to gain some intuition about this question by measuring higher order correlations that are found in our natural world. So
If I imagine that I measure a particular three point correlation, so something that involves three pixels separated in space. And everywhere I look in the world I measure exactly the same thing. There's no variability in the measurements that I make. Then any particular measurement doesn't tell me any information about what it is that I'm seeing. But if, on the other hand, I make many, many different measurements and they all have slightly different values, there's a lot of variability in these measurements. Then any particular measurement has the potential to tell me something about what it is that I'm seeing.
And so we would argue that the brain should devote sensitivity to those particular features that are variable in the natural world. Because they have the potential to tell us something about what we're seeing. Now, we can formalize this intuition within the framework of efficient coding, which posits that sensory systems should use limited metabolic resources to maximize the information that they encode about the natural world.
And the prediction that comes out of efficient coding is a restatement of this intuition that I just laid out. Namely, that visual sensitivity to complex image features, like this three point correlation, should be matched to the variability of those same image features in the natural world. All right. So this is a prediction that comes out of efficient coding. And this is a prediction we'd like to test.
Now, testing this involves two very different types of measurements. So on the one hand, I want to say something about visual sensitivity. And so for this I'm going to tell you about a set of human psychophysics experiments that measure visual sensitivity to different textural patterns. And then we'd like to relate this visual sensitivity to some variability of these same patterns in nature. And so for this I'm going to tell you about analyzes that we did to capture higher order statistics in natural scenes.
So I'm going to tell you about these two different sets of analyzes individually and then I'm going to bring them back together and compare them to try and test this prediction, that these two should be matched to one another. Now, this prediction and the efficient coding framework that we use to formulate it provide a very powerful approach for understanding representations of sensory stimuli in terms of the prevalence of these stimuli in the natural visual world.
But it doesn't tell us how this representation should change over time depending on behavioral context. So in the second part of the talk, I'm going to come back and revisit some of these ideas. And try to push these into a new domain, where we can think about the relevance the difference stimuli have for behavior. And how that might evolve over time.
All right. But for the time being, I'm going to start by explaining to you what I mean by complex image features. So I'm going to focus on image features that involve correlations between pixels arranged in a two by two square. All right. So you can think of this as a window that you can use to look at the world. I mean, imagine that the world exists in black and white. We can relax this assumption later. But for now, we live in a black and white world. And we can only look through this two by two window at that black and white world.
Now, there are 16 possible things that we could see when we look through this window. And these involve different spatial arrangements of black and white pixels. And each of these patterns is something that we could see or count or measure in a patch of an image. But these patterns are not independent. They're correlated. And so we can reduce these to a set of 10 independent coordinates that capture first through fourth order correlations.
OK. So I'm going to walk you through these. We have one first order coordinate that describes overall luminance in an image, in terms of the probability that we would see a white pixel relative to the probability that we'd see a black pixel. So this tells us about overall brightness. We have four second order coordinates that involve the captured second order correlations between two pixels arranged horizontally, vertically, or in the two diagonal directions. We have four third order coordinates that describe three point correlations between pixels arranged into different triangular configurations. And we have one single fourth order coordinate that describes the four point correlation.
Now, each of these coordinates is defined in terms of the probability of measuring particular colorings, or particular patterns here, in a spatial region of an image. So this means, if you give me an image, I can return you a set of 10 values that capture these local correlations in the image.
So this is how we do that. So we work with a set of natural images that were collected in Botswana. We've tested this against other natural image databases. But these images were taken in Botswana. This is actually a close up of a baboon. This is the eye here. And then we block average those pixels, or block average the images, which means we coarse grain the pixels. We then divide these images up into patches and we whiten these patches. This removes average, or expected, pair correlations. And then we binarize each patch at its pixel intensity median. Right?
So this means there's an equal number of black and white pixels in every image patch. And this means that that luminance coordinate that I told you about on the last slide, that measures the probability of light versus dark, is always fixed to be zero. So I've effectively reduced my set of coordinates from 10 down to 9. Then I can take any one of these image patches and now I'd like to build up a probability distribution, a histogram, of all of these different colorings that I could measure.
All right. So I do that by scanning my two by two window across this image. And then I can count the number of times I see each of these different colorings. And this histogram allows me to compute values for each of these coordinates that I described. So this is how I can assign a particular two point correlation to a patch of an image, because I counted the probability of measuring each of these different colorings in the image. And I can do this for all of my nine coordinates.
All right. So this then places this particular image patch at a location. If I'm looking at a plane mapped out by two of these coordinates, then this image patch sits at a particular point in that plane. It's characterized by a set of numbers. And if I take another image patch it sits at another point in this plane. And I can do this for many, many image patches and build up a probability distribution over the ensemble of natural images that I'm looking at. Right?
And this distribution is defined in terms of these nine independent coordinates. And I'm just showing you here two dimensional projection. So it's a variability in this distribution that we think is predictive of human visual sensitivity. All right. So I'm arguing to you that if I look along the directions where there's high variability in this distribution, I should be able to predict things that you will see really well. This is what I'm trying to test. OK?
So to test that, we recognize that not only can we extract these coordinates from natural images, but we can use them to generate synthetic textures with prescribed correlational structure. So these are examples of the types of textures that we can generate by manipulating each of these coordinates independently. So if you look along the horizontal axis here, you're looking at white noise. As you scan up or scan down, you're moving towards stronger positive or stronger negative correlations.
And perhaps you can convince yourself that as you visually scan away from the horizontal axis, it becomes easier to get a feel for what the structure is in this particular texture. And this is what we can measure psychophysically. All right. So I'll walk you through these experiments.
These experiments were done by Jonathan Victor and Mary Conti at Cornell. And these are human psychophysics experiments. So people sit in front of a computer screen. And they're first presented with a fixation cross, followed by a brief presentation of a stimulus frame, followed by a white noise mask. Now, the stimulus presentation is 200 milliseconds. So it's not long enough to be able to scan across the image. So this happens very fast.
There is, on this white noise background, there's a correlated strip of texture. This texture is not outlined in red normally. This is just so that we can see it easily. But this texture can appear in one of four locations, at the top, the bottom, the left, or the right of this stimulus. And the goal of the task is to identify the location of this target.
OK. So I think the easiest way to illustrate this, is to have you guys do an example of the task. So I'm going to show you a sequence of three frames. I'll first show you a fixation cross. It will be slow. Then I will show you a brief presentation of a stimulus, followed by a white noise mask. The stimulus will be a white noise background with a correlated strip of texture. That texture will appear in the top, the bottom, the left, or the right. And I'll ask you to tell me which location it appeared. Does everybody understand the task? OK. Is everybody ready? OK.
All right. So how many people saw something in the top? In the bottom? In the left? And in the right? How many people saw nothing at all? And how many people want me to do it again? OK. All right. Everyone close your eyes really quickly. I have to go back in my slides. I'm not that advanced. OK. All right. Ready?
Any better? No better? All right. Top? Bottom? Left? Right? What you were shown was a correlated strip of texture in the right hand side of this-- this frame. So those of you who said right-- if you were a subject in this experiment, this would be trial one of 50,000. So I always-- I always have to make this plug because we're looking for subjects. And if anyone has a couple of years to spend in New York, we would love your help.
But in all seriousness, this is repeated many, many times, varying the location of the target, the particular type of correlation, the strength of correlation, and a specific instantiation. All right? And so in doing this, you can vary for a particular coordinate, vary the strength of this coordinate from zero up to one and down to negative one. And measure the fraction of correct responses. And so we can fit psychometric curves that show that the performance increases from chance when the texture is random, up towards perfect when the texture is highly correlated.
All right. And I happened to show you a very strong value of this third order coordinate. But I didn't give you any time to train on this. So it's-- so we can define a threshold value of criterion performance as a value of this coordinate, for which performance is halfway between chance and perfect. And we define sensitivity as the inverse of this threshold. So this means that if there's a high threshold, we have to really crank up the strength of this correlation before people can distinguish it from noise, which means we're not very sensitive to this particular structure.
Now, we can measure thresholds or sensitivities along single coordinate dimensions. But we can also create textures that involve combinations of these coordinates. So this is an example of the types of textures that we would get by co-varying two of these coordinates. And so here, white noise is in the center. And as you move out radially you're moving towards strong correlations and combinations of these coordinates.
And so we can map out thresholds in different directions in this space. And together this maps out a perceptual threshold, or an ISO discrimination contour. It tells us how far away from the origin we need to move before subjects can reliably distinguish a target from noise. And what you'll notice is that these perceptual threshold-- perceptual thresholds are not circular, or elliptical, and they're not aligned with the cardinal axes. So this means that we're more sensitive here to certain combinations of these coordinates than to others.
I'm going to show you the full data in a minute. But this is our measurement of visual sensitivity to these particular image features that we've isolated in synthetic textures. And now we'd like to relate this to the variability of these same coordinates found a natural scenes. And I told you that these two measurements should match up.
So I'm going to tell you-- I was going to show you two different comparisons. So in one case, I'm going to compare variants along single coordinate dimensions to sensitivity. And so here, again, we would predict that directions of high variance are directions in which we have high sensitivity or a low threshold.
I'm then going to look at the covariance structure and relate the inverse co-variance to these perceptual threshold contours. So this takes into account correlations between these different coordinates. So I'm going to show you these two sets of measurements. I'm going to start by focusing on the variability found in natural images. OK.
So if we look at the variance in natural images across these nine different coordinates, we find that natural images have the most variability in these vertical and horizontal pair correlations, followed by diagonal pair correlations, followed by fourth order, And finally, by third order. And the different points that you're looking at here correspond to different ways-- different sizes for which we defined a pixel in an image, and different sizes that we chopped an image up into patches.
So these are two different scales in our image analysis. And our results are very consistent across these range of scales. And the rank ordering that you see here is preserved within each individual measurement.
So this makes a specific prediction. I'm arguing to you about visual sensitivity. And so if I then show you human visual sensitivity, you're now looking at the results for four different observers. And you see that we are most sensitive to these horizontal and vertical pair correlations and least sensitive to these third order correlations.
So if you noticed, I was a little bit unfair in the example that I showed you. Because I picked this particular coordinate that we're actually really bad at seeing. So-- all right. So we see a very consistent match between the variability of these coordinates and our own sensitivity to them. We can take this further and look at pairwise combinations of these coordinates in all possible pairwise planes.
And so this is that comparison. What you're looking at in each entry of this matrix, there is a single blue ellipse. This is the contour of constant inverse covariance in natural images. And there are four red ellipses that correspond to psychophysical thresholds in these coordinate planes. And so what you'll see across all of these different coordinate planes is a consistent match between the eccentricity and the tilt of these ellipses.
And we didn't fit any parameters to get this match. All right? So this is a parameter free match between variability and sensitivity. So this is super cool. We were really excited about this. We formulated this within the context of efficient coding. We made a prediction that the variability of image features in nature should be matched to our own visual sensitivity. We tested this by doing two independent sets of measurements, which we measured correlations in nature and related them to our own ability to see these types of correlations. And we were able to compare these different sets of measurements because we work with a common coordinate system for parameterizing these features in natural scenes.
Now, in parallel, Jonathan Victor has been doing single unit recordings in anesthetized macaque to identify the location of this sensitivity in the visual system using the same types of visual textures that I've been showing you. And these are representative responses from a single neuron in V1-- or a single unit and V1 and V2. And they were interested in asking the same question as to what extent one could discriminate the responses to these higher order correlated textures from noise. And so this is a discriminability index that they measure using the responses in V1 and V2.
And they find that the sensitivity, the ability to discriminate these textures from noise increases significantly in V2 relative to V1. And I encourage you to read it. It's a beautiful paper that goes into much more detail about the locus of this sensitivity. So this gives us a place to start building more mechanistic models of how this might be arising.
So stepping back. I think this provides us, again, with a very powerful framework for understanding organizational principles of sensory systems, in terms of the specific types of signals that these systems have to process. But this doesn't tell us how to deal with the fact that different stimuli can have different relevance for behavior and that relevance can change over time.
So as a simple example, if I show you a collection of image patches, the local textural details that I've been telling you about could be very important for helping you to determine that we're looking at a tiger and not looking at grass. Well, once we know this is a tiger, I don't think many of us would care about the details of the tiger's fur or the patterns in the grass.
We don't know where this tiger is and where it's going next. And the details of the stimulus that might be important for inferring the location and the speed of the tiger could be very different than the details of the stimulus that are relevant for identifying its presence in the first place.
And so together with Victor we've been developing a theoretical framework to account for the dynamic interplay between the cost of encoding sensory stimuli and the relevance that these stimuli might have for behavior. So I'm going to sketch out at a high level this theoretical framework. And then I'm going to speculate about places that we might be able to test this framework in behaving animals. OK.
So we're trying to develop a framework to understand encoding of sensory stimuli in neural responses. And so these stimuli could be image patches, patterns of light, acoustic signals, chemical signals. And these neural responses could be firing rates, or spiking patterns of individual cells, or something more complex like a population activity pattern. And what I'm calling this encoding step is something that maps a stimulus onto a response.
Now, a very common form of this encoding is something like a linear/ non-linear model, which maps, convolves, an incoming stimulus with a linear filter, either a temporal filter or a spatial filter, like a receptive field. The outputs of this filter is then fed through a nonlinearity and used to generate spikes via Poisson process. So this is a standard linear/ non-linear model for encoding stimuli and neural responses. And we could use already this framework to ask questions about how one might build this encoding step in order to give the system robustness, or efficiency, or maximize information about the stimulus.
So for example, we could ask how should we tune the parameters of these filters in order to maximize the information that this neural response conveys about these image patches. And this would be something akin to the efficient coding that I described to you at the beginning. Well, we'd like to push this further. And we'd like to ask how can we build [AUDIO OUT] step that preserves those details of the stimulus that are relevant for some downstream computations or some bigger task that the organism might need to solve.
So in this case, we want to know what details of the stimulus should we preserve in order to infer the location or the speed of the tiger. And because we think the relevance of different stimuli for inferring these properties can change over time, then we allow the system to adapt this encoding step based on an internal model of the environment, which itself is changing over time. OK.
So I'm going to sketch out how we go about doing this [AUDIO OUT] slightly simpler scenario. Instead of thinking about a tiger, I'm going to imagine that we live in a much simpler world that consists of just bright and dark rooms. So the light level in the room can change from bright here at the top to dark over time. And we'd like to infer whether we're in bright room or a dark room. And we do that using incoming sensory stimuli. These could be photons. So we have high photon count and low photon count. These photons are encoded in neural spikes and then used to make inferences about the light level in the room.
And instead of dealing with this full linear/ non-linear encoding model, I'm going to think of something much simpler. So just use a linear temporal filter. And I'm going to imagine that the system can adapt the width of this filter in time. It can make it wide or it can make it narrow. If it makes the filter narrow, it means the system is resolving individual stimuli. If it makes it wide, it means that it's averaging stimuli in time.
So we can now ask how should this simple system optimally tune the width of this filter, if [AUDIO OUT] is just to track the underlying light level in the room, to infer brightness or darkness. All right. So if it's trying to track this square wave signal, then what we would predict is that the optimal filter should change it's width over time. It can afford to be very wide at times when the light level in the room is [AUDIO OUT]
But at times when the light level is changing the system should decrease the width of its filter to resolve the stimuli that are indicative of that change. And this is very different from the strategy that would be optimal if the system instead were to try to track these local fluctuations in light. So here we would predict the optimal system should maintain a filter with a constant width. So that width should stay fixed over time and should be relatively narrow to allow the system to track these local fluctuations.
So we think that this framework as a whole provides a way of understanding different types of adaptive coding strategies that could be used to support different types of downstream computations. And we're excited to try and relate this type of framework to the responses of neurons that are adapting to different stimulus conditions, or that are involved in different behavioral contexts.
We also think this framework could relate to behavior. And just to give you a very simplified example, here at times when we would predict that the filter could be very wide, this means that the system is averaging over many stimuli in time, which means that many stimuli are getting mapped onto the same neural response. And we think that this could relate to the perception of metamers, or physically different stimuli, that are perceived to be the same. And in some conditions we would predict that the width of this filter should indeed be changing over time. And therefore, the perception of metamers should also be changing in time, depending on whether the system is averaging over stimuli or resolving individual stimuli.
So we think that we can connect this framework to both physiology and behavior in a range of different systems. We think that in humans, we can connect this to percept-- [AUDIO OUT] using, for example, some of the visual psychophysics tasks that I sketched out at the beginning, or some of the auditory tasks that Victor is doing with Josh. We think we can push this a bit further in model organisms, like the mouse and the fruit fly, where we might be able to uncover underlying synaptic, cellular, and circuit level mechanisms, that could be implementing these adaptive representations of sensory information.
And so I'm going to switch gears a little bit, yet again, to tell you about some exciting experiments that are going on at Janelia. And the fruit fly I think provides an exciting testbed for some of these-- for some of these ideas. Now, over the past several years it's become apparent that the fruit fly has a much richer behavioral repertoire than was originally thought. So flies can engage in a range of different visually guided navigation behaviors, like place [AUDIO OUT] which is beautifully exemplified in this analog to the Morse Watermay's experiments, where instead of a platform submerged in a pool of opaque liquid there's a cool spot in an otherwise inhospitably hot arena.
The walls of this arena are patterned with different visual stimuli. And these visual stimuli are locked to the location of this cool spot. So in principle it could be possible for the flies to associate these visual patterns with the location of this cool spot. So this is what this experiment was trying to explore. So this is an overhead view of the arena. This dotted square here shows the outline of the cool spot. And each of these colored dots is a fly. This is a sped up movie.
So what you'll see is that initially some of the flies pass right through the cool spot. But then they begin to congregate and they stay put. You'll see in a minute that this cool spot actually [AUDIO OUT] and the visual environment will rotate with it. And so if flies can use these visual cues to identify the location of this cool spot, they'll find it more quickly. And this is exactly what we see. This is-- becomes even more pronounced on the third trial when some of those flies go straight [AUDIO OUT] So OK, the red fly.
Now I won't show you the control data but Michael Riser and colleagues did a set of-- did controls in which they-- there was no cool spot here. After the flies had learned they still congregate in the same location based on visual cues where they expected the cool spot to be. Now, in mammals this type of behavior is thought to be-- thought to involve internal representations in place cells and grid cells in regions like the hippocampus. In flies behavioral genetics experiments have identified a subset of neurons in the central complex who's silencing led to deficits in these types of behaviors.
So this is a subset of these neurons. These neurons innervate a region in the center of the fly brain called the ellipsoid body. The ellipsoid body is a donut shaped structure that's about [AUDIO OUT] across. And it's innervated by neurons whose dendrites arborize in these wedge-like tiles. And these wedges cover this whole donut.
Now we can get a sense for the response properties of these neurons in mammalian experiments this would typically be done in freely moving animals. The fly is very small. So this is difficult to do. So the typical setup is a tethered walking set up in virtual reality. So this is a schematic of that set up. The fly here, that's right here, it's head fixed, but it's able to walk on this freely rotating ball. The rotation of this ball it's coupled to this visual environment. So this is a LED screen that wraps 270 degrees around the fly.
And so when the fly turned left or right on the ball [AUDIO OUT] visual patterns displayed on the screen will shift in a compensatory way. All right. So we can unfold this visual arena and look at the patterns that the fly is actually seeing. Again, this is wrapped around the fly, but I've laid it out here for convenience. And then we can watch-- here's the back end of the fly. And this is calcium imaging from this ellipsoid body.
So what you see is is the fly starts walking there's a local bomb of activity here that tethers to a landmark in this visual environment. So as the environment moves left and right so too does this [AUDIO OUT] activity. So we can think of this as a compass in the center of the fly brain that maintains an internal representation of the fly's heading relative to this visual environment.
And so it is clear from these experiments that this bump of activity was tethering to landmarks in the visual scene. But it is not clear exactly which landmarks it was tethering to and why. So why does the bump fix to what it does? So to get at this question we've been mapping out some of the visual inputs to the compass like neurons.
And so these experiments-- I'm going to show you some experiments that were done by Yi Sun, who's a research scientist at Janelia. And Yi found a feedforward pathway from the eyes to this ellipsoid body, this internal compass, in the center of the brain. This feedforward pathway was bilaterally symmetric. And we were interested in understanding the visual response properties of these neurons. And so Yi focused on a region one synapse up from the ellipsoid body, in the bulb, a structure called the bulb. And what you're looking at here are microglomariali. [AUDIO OUT] local regions where axons and dendrites intermingle.
So Yi expressed two different calcium indicators, one green and one red pre and post synaptically to look at the transformation of visual information across synapses int this structure. Found consistent with previous studies is that these neurons have a simple cell like feature selectivity. But he also discovered a broad field of contralateral inhibition that we think arises from a pooling of these feature vectors on the contralateral side that's been fed over to inhibit this feedforward pathway on the ipsolateral side.
And we went on to characterize the temporal properties of these responses. And we found that they depended on past history. So together, this gives us a picture of the spaciotemporal response properties of the visual inputs to this navigational center. And we think that this particular combination of ipsolateral feature selectivity, contralateral inhibition, and history dependence could provide a way [AUDIO OUT] one stimulus from among many in both space and in time.
And so we're now working with Sung Su Kim who is a senior post-doc in the [INAUDIBLE] lab at Janelia to understand this feature selectivity at a population level within a single fly. And relate this to the theoretical framework that I outlined earlier.
Moving forward in the longer term, we're excited to think about this future selectivity in the context of behavior. So this particular set of spatial temporal receptive fields that I'm showing you here, we can think of as a form of an encoding that I was mentioning earlier, a way of mapping a visual stimulus onto a response. And so we're interested in understanding the nature of this feature selectivity in terms of the particular tasks that the fly might need to solve.
So for example, if the fly is trying to navigate towards a cool spot in a hot environment, then certain visual stimuli will be more relevant for this task than others. And we have a framework for trying to understand how that relevance might impact the encoding or this feature selectivity here. And this controls, again, the inputs to this internal representation of the external world that's used to guide these navigational behaviors.
So we think that this is a really exciting place to potentially put together this theoretical framework that we've been developing with neural responses in the context of behavior. And again, this is one of several different directions that we think we could go moving forward. We'd be very excited to talk to you about ways to develop this framework more fully and to test it more directly in specific systems.
But [AUDIO OUT] I just want to step back and thank the people that have been involved. All of the work that I told you about at the beginning, in terms of efficient coding and natural scene statistics, was done by John Bergoglio, who's now at Janelia, Vijay Balasubramanian at UPenn, Gaspar Tkacik at IST Austria, and Jonathan Victor at Cornell.
Both Jonathan and Mary did all of the psychophysics experiments that I told you about. Victor has been an incredible collaborator, developing this theoretical framework. And I'm very excited to find out where it takes us. And the experimental work that I showed you about in the [? fly ?] was done in Vivek Jayaramn's lab at Janelia with Yi Sun and Sung Su Kim. On so with that, I would like to thank you. And I would love to take any questions that you might have.
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
