Foundational architecture of human language comprehension, production, and acquisition (1:13:01)
Date Posted:
August 1, 2018
Date Recorded:
August 1, 2018
Speaker(s):
Roger Levy
All Captioned Videos CBMM Summer Lecture Series
Description:
Roger Levy, Professor of Brain and Cognitive Sciences at MIT, describes his research on human language understanding, which integrates linguistic theory, computational models, psychological experimentation using measurements such as eye tracking while reading, and the use of language datasets such as Google Books. Humans use language to communicate with extraordinary flexibility in spite of challenges such as ambiguity, the presence of environmental noise, memory limitations, and incomplete knowledge of their interlocutors. Dr. Levy illustrates some examples of complex language understanding and leads a Q&A session with students.
Additional Resources:
ROGER LEVY: I run the computational psycholinguistics laboratory here in BCS. And computational psycholinguistics sits at the intersection of three fields, computer science, in particular, artificial intelligence, psychology, and linguistics. And so that's a perfect combination for the sub-part of this department that I'm in, the cognitive area. So this is very much cognitive science, which is an intrinsically interdisciplinary endeavor.
And I'm giving you sort of a nutshell summary, but I also want to give a couple of really specific instances of the kind of research that we do as well. So I'm going to start from the high level and give you a broad overview. And then what I want to do is try to flexibly dive into a few different things as case studies.
And you can interrupt me and ask questions and so forth, just should be totally flexible. And if you have questions, also, I can talk about my career trajectory as well, and I can intersperse that with all these different things. And I can give you some stories about, how did this project go, how did it work out, and so forth.
Actually, before I start into the details, let me tell you a little bit about my personal background as well, because it's-- people take different trajectories in their academic careers. And some people take very-- it's very direct, that you know exactly what you want to do from early on. Oh, that's fantastic. Thanks so much. Sometimes the trajectory is very direct, and sometimes it's not.
So in my case, I-- so I actually did my-- I grew up in Arizona, in Tucson. And I grew up in a household-- I was very lucky from the point of view of somebody who is going to go into academia. I grew up in an academic household, which offers a lot of advantages. I think the context of your upbringing also creates, in a sense, sort of like-- it can also constrain the imagination.
So for example, so I did my undergraduate degree at the University of Arizona in mathematics, like a number of you are studying mathematics. And I decided late on-- I spent a lot of time just having fun and doing math and learning lots of things. And near the end of undergrad, the question is, what do you do next? And I was also very lucky that there was-- at the University Arizona, it's a very large public university. So we don't get the kind of personalized advice-- I mean, when I moved here, this is the first time that-- at two and a half years ago, this is the first time that I've been a faculty member at a private university.
And I was stunned that every undergraduate has their own advisor, for example. At the University of Arizona, not every undergraduate has their own advisor. I was at UC San Diego before moving here, and not everybody had their own advisor.
So anyway, but I was very fortunate that at the University of Arizona, I was in-- there was an Honors College, and I was part of that. And so I did get to have the opportunities to get a little bit more personalization of guidance from faculty who were specializing in-- part of their job was to work with Honors College students. And I remember one of the main Honors College advisors asked me, so what are you going to do afterwards? And I said, well, I'll probably go to grad school.
The response was, wow, it really requires a lot of dedication and perseverance to commit to wanting to go to grad school. And I said, well, in my case, it's probably more a lack of imagination, because that always seemed like an inevitable available thing. And that's obviously not available and inevitable and as accessible-seeming a thing as it is for people who grew up in my environment. And so one thing I'm very glad about, I think-- programs like this and more broadly, we should be moving toward a society in which that's seen as something that's accessible to everyone.
At any rate, I then was working in math, and I got very interested in evolutionary biology. There's a lot of wonderful evolutionary biology you can do with genetics, and many of you are working on genes. And there's a whole field of population genetics. It's a wonderful field, super interesting theoretical questions, super interesting practical things you can do.
But then I, very serendipitously, wound up studying abroad. So many people study abroad during their undergraduate career. And I very serendipitously went to Singapore for a year. And in Singapore, I started studying Mandarin, and I completely fell in love with learning Mandarin.
And my classmates were largely Japanese, because this was a time when the Japanese economy was not doing very well. And so it was a very popular thing to do in Japan to, once you finish your college degree-- since it's harder to get a job than it used to be, you could go abroad for a year. If you go to Singapore, it's a pretty similar-- it's also a very industrialized country so very easy to adapt to. Everybody speaks English. And then if you're learning Mandarin, you're learning two languages at once. So lots of Japanese classmates.
So I completely fell in love with learning Mandarin. I also got very interested in Japanese and just, more generally, that activated an interest in the differences in languages, the differences in cultures around the world. And I spent a couple of years, between undergraduate and graduate school, living in East Asia and studying languages intensively. So I now know Mandarin fluently, and I know Japanese. My Japanese is very rusty, but I was quite fluent by the time I left.
And that was an amazing experience, but also it transformed my intellectual interests, which I was already interested in the distribution-- like variations across space and time in organisms. But I got extremely interested in human variation in space and time, and in particular, the role of culture and language. And that was incredibly eye opening to me, so that really transformed my life.
And then I decided to start a PhD program instead of in evolutionary biology, which I probably would have done otherwise. I thought, if I do that and I want to work on language and culture, people are going to ask you, what are you doing wasting your time on these things when you should be working on biology? And so I decided to start a PhD program in anthropology instead, where I thought I could merge my interest in evolution and language and culture, and it seemed like it would all work out together well.
So I started a PhD program at Stanford. And when I got there, I had never taken a class on language, besides language study classes. I had never taken a linguistics course. I had never taken a course on the cognitive science of language or on artificial intelligence of language. None.
But when I got to grad school, I thought, I should probably learn a little bit. And so I took a couple of courses in the linguistics department and it just so happened-- and I did not know this when I decided to do it at Stanford, it was very serendipitous, again-- that Stanford was one of the top programs in linguistics in the world, like MIT is in linguistics, one of the top linguistics programs in the world. So I was very fortunate. I got this immediate entry, just really fortuitously.
And I started studying linguistics. And I loved it, because I thought, wow, this is-- I was interested-- so language-- so humans are-- we're a unique species in a lot of different ways. Language, of course, is a massively prominent signature unique thing that we have.
But more generally, humans are-- we are-- maybe some of you have read the book, The Symbolic Species by Terrence Deacon. So we are symbolic organisms. We generate symbolic thought, so our internal mental lives are symbolic, largely, maybe not exclusively. We also have non-symbolic mental imagery and so forth.
But symbolic thought is a very important part. Reasoning, creation of content, those are all symbolic activities. And we also externalize symbolic content, and that ranges from gestures, which many either are cross-cultural differences in gestures, of course, in terms of the meanings of the various gestures. And that shows that they have some kind of symbolic-like content.
Also, for example, just think about the non-verbal street sign repertoire. So there's symbolic content or the representations of-- if you think about religious ceremonies, all of these things invoke symbolic content. Symbolic mental activity, both internal and external, is central to what we are as a species.
And this is very different from what we see in any other species, really, where there may be tiny vestiges, vestigial-- vestigial is the wrong word, because that's sounds like leftover-- like embryonic form of such type of activity. But it doesn't dominate life for any other organism the way it does for us.
Because of this confluence of what it is to be human and the fact that also, of course, we are the-- in terms of our ability to reason in sophisticated ways about the world, we can engage in intelligent activity. We don't always engage in intelligent activity, but we have the capacity for intelligent activity, both as individuals and as groups, that is really very different, and in some ways superior, but not necessarily in all ways superior to other organisms.
And so the characteristics of our minds are very important to understand. And language, to me, seems like a uniquely powerful window to investigate that, because it is the most ubiquitous form of our externalized symbolic activity. It's the most measurable. It's the most recordable. We have it in abundance in our daily life, in our environment.
And it's different. It's not universal. There are differences in the forms of languages. So it's also something that's both unique to us but also in very important characteristics, it's learned. And so to me, language is just an absolutely wonderful place to be, both because I love language and because I think it's actually a great inroad to study in cognitive science, studying the human mind.
And of course, as a consequence of studying the human mind, gaining insight into minds more generally, both the minds of other organisms and the space of possible minds, minds that may evolve in the future through biological change, or perhaps more likely at this point due to the relative speeds of change of different types of processes, that we may invent or cause the conditions for the natural emergence of other kinds of intelligence in the form of artificial intelligence. There's all sorts of things. So there's enhancement of human intelligence through all sorts of technologies. There's also the emergence of artificial intelligence.
So that's the backdrop to how I got here. So I finished my PhD. I changed my PhD from anthropology to linguistics. So I got a master's degree in anthropology. I changed into linguistics. I studied linguistics, and then I learned about artificial intelligence and experimental psychology in the context of doing a PhD in linguistics. And as a faculty member, I brought all those things together, and that's what I try to do.
So anyway, foundational architecture of human language comprehension production and acquisition. So my research is dedicated to the following problems. So one is, how do humans use natural language to communicate with such extraordinary flexibility? So every day, you hear hundreds or thousands of sentences that you've never heard before, and they occur in a variety of novel contexts.
And although sometimes, occasionally, you may be a little uncertain as to what somebody meant, by and large, you successfully understand what you hear and read. Not only do you do that, you produce sentences to express what you mean, and the people that you talk to or write to, by and large, understand what you mean. And that's despite the fact that, actually, there are all sorts of reasons to think why this just should not be possible. It should be too hard.
And just to give you examples of that-- so for example, language is ambiguous. So here's an example sentence, very, very simple example. The women discussed the dogs on the beach. So let's take a poll. So first of all, let's start for a moment. This is an ambiguous sentence. Do people see how it's ambiguous? How is it ambiguous?
AUDIENCE: [INAUDIBLE].
ROGER LEVY: Who's on the beach, right? It's probably ambiguous in other ways, but it's definitely ambiguous in, who's on the beach? So maybe the women are on the beach discussing the dogs. Or the women are not necessarily on the beach, and they're discussing the dogs, and the dogs are on the beach.
Now, it could be, of course, that both the women and the dogs are on the beach. And that's actually distinct from the question of the ambiguity in the sentence. So the ambiguity in the sentence is an ambiguity in what the speaker is committing to.
So there is a version of interpretation of this sentence in which the speaker is committing to the women being on the beach, and there is a version of this sentence in which-- interpretation of the sentence in which the speaker is committing to the dogs being on the beach. If you think about it for a moment, there's no version of this sentence where the speaker is committing to both. So take that for a moment. Hopefully, that seems intuitively right.
Now, let's take a poll. What do you think the speaker is likely to mean? So raise your hand if you think the speaker means that it's the dogs that are on the beach. One, two, three, four, five, so it's about six.
So linguistic theory, which is a combination of the empirical study of language with a branch of mathematics which sort of grew out of work here at MIT in the mid-20th century, that was actually pioneered by Noam Chomsky-- has given us the means to mathematically describe this distinction. This bottom interpretation here is the one where the dogs are on the beach. And the way you can see that is that I have trees hanging on top of or resting beneath the word string.
And in one of the trees, in the green tree, there is a node of the tree that the dogs on the beach is entirely inside, and nothing else is inside. And that says that the dogs on the beach is a constituent. It's a unit. It's a syntactically coherent meaning-bearing unit. And if you do a full or semantic analysis combined with a syntax, it turns out that that corresponds to the meaning where it's the dogs that are on the beach.
And now, raise your hand if you had the other interpretation, is the women that are on the beach is what the speaker meant. Am I getting this right in terms of which-- the first one I asked about was the dogs, right? And so about equal numbers think-- so this is actually a pretty balanced sentence. And people recognize the ambiguity, right? So this is the interpretation, where it's the dogs that are on the beach-- sorry, it's the women that are on the beach.
And notice that that interpretation is not just-- it's not just that the women are on the beach now. But it's actually, they were on the beach at the time of discussing the dogs. And so you can actually see that. And I won't go into full detail, but this analysis implicitly represents that by the fact that, on the beach-- and discussed are sisters in this level of the tree. Whereas, they're not in this level of the tree.
A wide variety of facts like that are representable formally in a way that is predictive about what sentences are possible and what sentences are interpretable in what way. So for example, what are impossible sentences in a language? So for example, dog the barked is not a sentence of English.
Now, there are languages in which you can take the words dog, the definite determiner, or something like that, and bark, and you could put them in that order, and it would be an OK sentence of English. But one property of English grammar-- and this is a-- now, not a normative, people should talk this way. This is a descriptive, people do talk this way, and people do have intuitions about this. This is not a normative endeavor. It's a descriptive endeavor.
Descriptively speaking, there are variations across languages in word order. So the constraints on word order are an important part of the grammar of a language. And to describe a fragment of language, we would write down a set of rules that would say that these structures are OK, but then it wouldn't allow you to generate other structures. And that actually constitutes a predictably powerful hypothesis about the structure of a language.
One reason why you might expect that communication doesn't work so well in language is that language is so ambiguous. And so that ambiguity could get in the way. Well, how can you guarantee if like one half of the people in the room might have thought-- might have used this sentence to mean that it's the women who are on the beach, but some people on the other half of the room might have misunderstood it and said, no, no, no, it's the dogs on the beach.
That's an opportunity for misunderstanding, and I'm just giving you one example. But most sentences are, in many ways, ambiguous. And so how is it the case that we're able to converge on the right agreed upon meaning when sentences could have so many meanings?
Here's another reason why sentences might be-- why language might be difficult to understand. Environmental noise, so we very rarely speak to each other in a soundproof booth where one person is absolutely silent while the other person is speaking. A lot of language and communication looks much more like this, a party where people are talking to each other and there are multiple voices going on. There's other environmental noise. Even in the present situation, there's some environmental noise going on.
In fact, even figuring out what words are being said is an inferential problem. We have to infer what words are being said from the ambiguous spoken language, the acoustic input for spoken languages, or the signed input, the manual input for sign languages. And actually, even just figuring out what the words are is an inferential problem.
Here's a famous example in the speech recognition literature. It's not easy to recognize speech. So what did I say? Want me to say it again? It's not easy to recognize speech. Somebody raise their hand and say what they thought they heard. Yeah.
AUDIENCE: It's not easy to recognize speech?
ROGER LEVY: Who thought they heard that, it's not easy to recognize speech? Anybody heard anything else? Yeah.
AUDIENCE: [INAUDIBLE].
ROGER LEVY: Actually, what I said, it's not easy to wreck a nice beach. But that sounds very similar, and that's a much less plausible thing to say, especially in the context. And so that's a demonstration that actually, even just understanding what words are being said is an inferential problem. And that might go wrong. It's yet another way that language understanding could go wrong.
Here's a third, memory limitations. So let me give you an example. As I'm sure you all are aware, you can only hold so many things and operate on them at once in your memory. And when you try to hold too many things at once, it can cause you to make improper calculations, make mistakes, forget things, et cetera.
Let me give you an example of how that works in language. So I'll give you a sentence, and I'll see what you think. It's another very famous sentence in the study of language. The dog that the cat that the rat chased killed ate the malt. Raise your hand if that was like, of course, no problem. It's pretty hard. I'll just say it again. The dog that the rat that the cat chased killed ate the malt.
[INTERPOSING VOICES]
ROGER LEVY: I'm going to make it easier for you. Let me rearrange the words. It's the same meaning. The dog that was chased by the rat that was killed by the cat ate the malt. Sorry, I got that wrong.
[LAUGHTER]
Sorry, I got it wrong. The first sentence, let me change it around. The rat that the cat that the dog chased killed ate the malt. That's better. It's still hard, right? The rat that the cat that the dog chased killed ate the malt. Easy now, right?
AUDIENCE: That's too fast.
ROGER LEVY: Sorry. Sorry. The rat that the cat that the dog chased killed ate the malt. Is that easier?
AUDIENCE: [INAUDIBLE].
ROGER LEVY: No, it's still hard, right? It's not a speed problem. It's a memory problem. So let me reorganize the sentence. The rat that was chased by the cat that was killed by the dog ate the malt.
AUDIENCE: That's better.
ROGER LEVY: Better, right? They actually mean the same thing. And just to demonstrate that, so once again, it was, the rat that the cat that the dog chased killed ate the malt. The rat that the cat that the dog chased killed ate the malt. The cat that the dog chased, fine, right? The rat that the cat killed, fine, right? So the rat that the cat that the dog chased killed, fine.
You see, there's two patterns. I'm just sticking them together. But it's hard, because you have to remember all those nouns at the beginning-- dog, cat-- rat, cat, and dog-- and then associate them with the right verbs in order. You just have to hold them all at once and then discharge them one by one as you associate them with nouns.
Whereas, if I would turn it around and I say, the rat that was killed by the cat that was chased by the dog ate the malt, you get to do those associations one by one. And it's still not that easy, but it's vastly easier, right? And you can sort of feel like you have some hope of understanding what's being talked about.
Once again, those two sentences basically mean the same thing. It's just active versus passive voice. In that context, it completely changes the ease of understanding. In the general case, of course, that's in a totally-- you're focused on the one sentence. But actually, I'm sure in most situations, you're doing other things at the same time as you're doing language understanding.
You're also trying to keep in mind somebody's phone number that they told you or where you're walking to. They're all sorts of other things you're doing the same time. And so your memory resources that are relevant for all sorts of cognitive activity are actually implicated in language processing. An overload of that is entirely possible, so that's another reason why language understanding may fail.
And finally, incomplete knowledge of one's interlocutors. What I mean by that is that, you don't know everything about me, and I don't know everything about you. And when I say everything, I mean everything, every aspect of your knowledge, mental state, intentions, beliefs, and so forth, goals.
In fact, this is a necessary condition for it to be even meaningful to talk about communication. Because if you knew everything about me, including all of my plans, beliefs, goals, and knowledge, then you could predict everything I would say. And I might as well not say it. In fact, I could not provide you any information by saying it, because you would already know.
So in fact, it's actually this discrepancy in knowledge states, incompleteness of knowledge about each other, that makes communication meaningful. But it also creates a challenge. If we have no overlap in knowledge, then for example, you don't know, when I use the word cat, what it means, right? So there are conventionalized features of language that make it necessary to-- that are relevant for understanding, but there's also contextual features.
So for example, if I say, the projector looks like it's getting hot, in order to figure out what I'm referring to by the projector, you will take into account the fact that we're in the same room, and that we can see the same things in our visual field and so forth. And so that partial shared knowledge is going to help you interpret what I mean in context. And so the less complete our knowledge is of each other, the more opportunity there is for the synergy ending.
So for example, if I refer to the third student to join my lab since I moved to MIT, you probably don't know who I'm talking about. Because that's a case where I have knowledge of the domain, and you don't. But I don't know exactly. Maybe one of you does know who that is. So I'm not sure, but I can make a guess. And this uncertainty about what knowledge we share and what knowledge we don't share both governs what's worth talking about, but also, it governs whether we're able to understand what each other means.
So these are the central questions about, how do we actually succeed so well at this use of the language for communication in the face of all these challenges? And the other half of this is, how do we acquire the knowledge that allows this communication to be possible? That is, how do we go from knowing nothing, knowing no words, not knowing the order of syntactic constructions, not knowing these things about the particular language environment that we're going to knowing it?
Do we start off at zero? We probably start off at a lot more than zero. With the more than zero we start off with is that we have human minds. We don't have math book minds or rat minds or paramecium minds, and so forth. We have human minds. And that's a massive starting point, but there's still a lot to learn in any particular setting.
And languages change over time. So it's not even the case that there's a fixed inventory of languages around the world, and it's just, guess which language you're in. It's actually, construct the language out of the infinitely possible set-- infer what the language as a whole is out of the infinitely sized set of possible languages.
So this is what I work on. And I work on it by trying to bring together a number of key things. So I talk to you about the mathematical descriptions of language structure, which I studied, I was absolutely energized by when I first encountered them in Stanford and I continue to use them today.
Linguistics departments, in particular, the very highly active area of research in linguistics departments called generative linguistics, generative grammar-- gives us mathematically precise descriptions of language knowledge that predict what forms will be possible in a given language, what meanings they might have. Each mathematical description itself is a scientific hypothesis about a language that's being studied. And there are people in linguistics departments around the country and around the world developing this theory, both to explain individual languages-- so there's a lot of work for example, on English-- but also trying to study all the languages around the world.
The vast majority of languages have been studied far less than they should be. Does anybody know about how many languages there are in the world? You can just take a guess for a moment.
AUDIENCE: [INAUDIBLE].
ROGER LEVY: It's about-- sorry?
AUDIENCE: 6,000?
ROGER LEVY: There's 6,000 to 7,000 languages in the world. And the vast majority of them have been studied far too little, and we know far too little about them. They are the repository of knowledge information for us as scientists and as members of our species in order to understand what the variety of ways it is possible to be human includes.
And so there are people around the world who are engaging in developing this theory, both to describe specific languages and to understand, also, what kinds of linguistic structures we do and don't see around the world, because there's some kinds of things we never see. There are some kinds of things we see rarely. There are some kinds of things we occasionally see. But if we see that, we have to see something else as well in the language. So there's an intra-intensive, rich, articulated structure of possibilities that we're still in the process of understanding, and that plays a central role in my work.
I also combine that with computational models. So these are models that draw upon tools from artificial intelligence, machine learning, statistics, computer science, more broadly, to try to build explicit hypotheses that are both fittable to data and interpretable as cognitive theories. I combine that with psychological experimentation, where we design experiments, usually behavior experiments-- although, I'm just starting-- I'm very excited to start to do experiments that use brain measurements, in particular, for example, using EEG or MEG to look at the rapid, real-time development of the brain response, moment by moment, as we hear or read language.
But most of the work is behavioral. So for example, I'll bring people into the lab and set them up in an eye tracker. So this is an infrared camera that's recording eye movements. And then I'll give them a text, and we'll record the rich, rapid set of eye movements that they make as they're reading the text. And we extract information from that.
So for example, we have a project in my lab that's ongoing that demonstrates that, for example, if you're a non-native speaker, the eye movements that you make reading English carry information about what your native language is. And they also carry information about your proficiency in English. For example, we're about 2/3 of the way to the quality of a TOEFL exam in figuring out how good you are at English, if you're non-native speaker, just by looking at your eye movements in reading.
But more generally, things like your eye movement patterns or also, in a visual context, if I'm looking at a display and I'm hearing somebody speak, what I look at and its relationship to what I'm hearing, all those things carry a rich amount of information, a huge amount of information about how language processing is unfolding in real time. So psychological experimentation plays a major role in my work.
And finally, language data sets. This is a really amazing, unique era. So who's used the Google Books data set before? Does anybody know about the Google Books data set? I'll show you guys in a moment. That'll be on our agenda for the rest of the time.
This is an amazing and fun data set. It includes something like 5% to 15% of all the books that have ever been published. And they're all scanned, and you can search them.
For most books, you can't-- because of copyright, you can't get the full texts. But what you can do is you can look at multi-word strings and how common they are and how they rise and fall in frequency and when new words appear and so forth. And we try to pull all these things together. And we've drawn all of these things in our lab to understand this foundational architecture of human language comprehension production and acquisition.
So just a couple of very brief examples of what it might look like is, we might use a grammatical description from theoretical linguistics. And we might annotate. We might add on. We might make it probabilistic. So we put probabilities on different pieces of language structure, because some constructions are really rare.
So for example, if I say, an excited poodle pranced into the room versus into the room pranced an excited poodle, which one of those-- does one of those seem a little more surprising than the other?
AUDIENCE: The second?
ROGER LEVY: The second?
AUDIENCE: [INAUDIBLE].
ROGER LEVY: It's true. That was what I did, is I swapped the positions of the subject and the locative phrase, the into the room phrase. And even in that case, I actually set that up to be one of the most invertible examples. But even in that case, it's a pretty surprising thing. So we might say that that inverted word order is a low probability word order.
And those probabilities will help us disambiguate. It helps us with the disambiguation problem, both figuring out, what were the words that are likely to have been said? So for example, a priori, you expect to hear recognize speech as a word sequence versus wreck a nice beach. And because of that prior expectation difference, that pushes you toward one interpretation, even though what I said was different. It was acoustically similar to recognize speech, but it was different.
So that's one thing we do. Here's another example that we can think about. So here's a kind of sentence that we study a lot in my group. The woman brought the sandwich from the kitchen tripped. So is that a weird sentence?
AUDIENCE: Yes.
ROGER LEVY: It's not the first weird sentence I've given you, and it won't be the last. So share for a moment your experiences about-- introspect for a moment about your experiences reading and hearing that sentence. What was weird or hard about it? Where did it happen? Yeah.
AUDIENCE: I still don't understand the sentence. I feel like there should be a who, the woman who brought the sandwich.
ROGER LEVY: The woman who brought the sandwich from the kitchen tripped. Very good. Raise your hand if that seems right to you. Good. Any others? Any other intuitions that are different from that? Yeah.
AUDIENCE: Could you say, the woman brought the sandwich from the kitchen and tripped?
ROGER LEVY: Yes, very good. Anybody else have that intuition, the woman brought the sandwich from the kitchen and tripped? So that's great. So those are your intuitions about globally what's wrong with the sentence. Now, does anybody have any temporally localizable intuitions about, when did something seem to start to be off about the sentence? Yeah.
AUDIENCE: So up until the word tripped, it's a complete sentence [INAUDIBLE] the woman dropped the sandwich from the kitchen. And then adding that is another idea that's attached on without any transition.
ROGER LEVY: Yeah, exactly.
AUDIENCE: It's changes the [INAUDIBLE] speech that the first part of the sentence is from the sentence to a descriptor.
ROGER LEVY: That's right. So the difficulty is the word tripped, right? So this word, tripped-- basically, it's like, this does not fit in, right? That's the experience that you have? And that's true. So we can actually record that. If I measure your eye movements during reading, what we'll see is we'll see signs of disruption of this word, tripped. You'll slow down. You might move your eyes backward.
Well, there's actually nothing wrong with this sentence. This is a perfectly legitimate sentence of English. Just like the rat that the cat that the dog chased killed ate the malt, it's also a perfectly legitimate sentence of English. Those are legitimate from the grammatical principles of English, but they're hard. They're hard for actually importantly different reasons.
So the first sentence example that I gave you, the rat, cat, dog example, it's hard because of memory overload. This is hard for a different reason. And to explain that, I want to point out that what this sentence should mean and should be equivalent to is, the woman who was brought the sandwich from the kitchen tripped. Is that OK for people? Anybody have a hard time with that sentence now?
AUDIENCE: The woman who was--
ROGER LEVY: The woman who was brought the sandwich from the kitchen. So somebody brought the woman a sandwich from the kitchen. And we're talking about that woman, and that woman tripped. A little weird, especially since you'd be thinking about another interpretation of that sentence. So who got that interpretation on first reading? Nobody, right? Oh, you did?
AUDIENCE: Yeah, after the first reading of adding [? the two ?] words together.
ROGER LEVY: After the first reading of the sentence about the who was, who got the same meaning as when I add the who was? Some people may still be a little skeptical that this sentence can mean something like what I just described. But I'm actually going to change it one more time.
Now, let's just substitute the word brought with the word given. The woman who was given the sandwich from the kitchen tripped. Anybody have a problem with that? Any ambiguity about what that means? The woman's getting the sandwich, right? The woman who was given the sandwich from the kitchen tripped. So we're all on board that that is OK, right?
Now, let's take that sentence, and let's take the who was away from it. The woman given the sandwich from the kitchen tripped. Is that OK? It's OK, right? Raise your hand if you don't like that. It's a little surprising, maybe, but it's OK, right, meaning-wise? The woman given the sandwich from the kitchen tripped.
So what I've just demonstrated to you is that in this kind of case, where somebody is-- this is a passive-- given is a passive form, right? So the woman was given the sandwich. The woman who was given the sandwich. The woman given the sandwich. In that situation, I can drop the words, who was.
So now, let's go back to the woman who was brought the sandwich from the kitchen tripped. So bring, unlike give, has the property that the past tense form and the passive participle form are the same. So I brought, I was brought versus I gave, I was given.
So the woman who was brought, this brought form is a passive form. And as I said before, there's a rule in English that says, in that situation, where you have this relative clause structure-- that's what it's called-- that begins with who was and then it continues as a passive, you can just drop the words, who was. And so the woman brought the sandwich from the kitchen is perfectly fine.
But what's confusing about it is that there's another much more appealing way to look at that part of the sentence, which is that the woman is doing the bringing, not getting brought the sandwich. From a structure probability perspective, it's a much higher probability thing to happen, for the same reason that, into the room pranced an excited poodle is an unusual structure-- just an unusual structure.
But figuring out what the sentences are-- what the sentence structure is, that's an inferential problem. So what's happening is that up to this point, there's a very strong plausible inference that is actually-- turns out to be the wrong inference about the sentence structure. And we can model that formally.
So here's a model from a paper that I did, which what I'm doing is I'm showing you the analysis of this sentence in syntactic terms unfolding word by word, starting with the woman, and now, I'm just going to add some words, some of the time. And the sizes of these trees are basically the probabilities given the available information, conditioning on structural frequencies and so forth of each of the interpretations.
So the top analysis is going to be the one where the woman is doing the bringing. The bottom one is the one where the woman is getting brought the sandwich. And so it starts off already favoring the woman doing the bringing, even though the word hasn't arrived yet. So in fact, actually this model predicts that if I get a verb mix, it's probably going to be a verb that the subject is-- that the first noun is the agent of, the one doing. And so it actually is a predictive model in that sense.
And it continues on. And the additional information, brought the sandwich, makes you even more confident about that. And from the kitchen, even more confident about that. And you can see, this is a very, very marginal dispreferred interpretation. But actually, this is the interpretation that's required. This interpretation allows you to accommodate tripped. And this is the same structure, just without a few of the words, as the woman who was brought the sandwich from the kitchen tripped. This structure has no way of accommodating that word.
And so we can use these models to account for and actually make predictions about the kinds of difficulty patterns that people will have in interpreting language in real time, both in reading and in speech. We can also couple this with, and we do things like these kinds of controlled experiments, where we look carefully and deviously construct sentences so as to fool the reader or the listener. But we also look at broad coverage, so we look at like when you're reading natural text. And we do things like ask, well, you remember that predictive component I was talking about? It turns out that we're actually doing prediction all the time in language, and I'll give you an example of this.
So I'm going to give you the beginning of a sentence, and you will find-- this is not saying you have to do-- that it's quite likely that a word will just pop into your head as the next word in the sentence. So I just want you to hold on to that word and think about it, and I will try to guess it on the basis of experimental data that I've collected in the past.
So here's the first example. My brother came inside to-- everybody got a word? So I will try to guess it. These are the kinds of things people guess. Chat?
AUDIENCE: Oh, my god.
ROGER LEVY: Yeah? Good. But anybody else have chat? Better than zero, but only one. Get warm? Probably not this time of year. Wash? Let me try it with another example.
The children went outside to-- now, you're all laughing. And just think about for a moment, you're all laughing, but what is the fact that you're laughing tell you about cognitive state and about theory of mind? So A, you're much more confident than you were before about what the next word was. B, you have introspective- so you guessed a particular word, but you actually have introspective awareness of your confidence level.
And even beyond that, you have awareness or belief, which is confirmed by external events like people giggling around the room, which suggests that you actually expect that everybody else will have the same expectations that you do as to how the word continues-- as a how the sentence continues. Just in one sentence example, we've indicated prediction in language, varying degrees of confidence, and actually, a theory of-- not only a theory of an individual mind of an individual person, but a theory of a distribution of types of minds around the room. So all those things are just encapsulated in that.
So the word was play. Now, raise your hand if you thought of the word play in the first example. My brother came inside to-- did anybody think of play? Yeah, a couple of people. Usually, a couple people think of play. And there's nothing weird about the word play in that context, right? So it is not that high probability of a word. So play is a much lower probability word in the first context than in the second context.
But it's a perfectly reasonable, plausible word. There's nothing anomalous about it in either case. So prediction and probability cannot be reduced entirely to some kind of plausibility or lack of anomaly.
Now, it turns out empirically that when a word is predictable, not even just a word-- when any kind of grammatical thing, any kind of language thing is predictable at a particular moment, your behavioral and neural responses are measurably different. So we can detect distinctive neural signature responses to a predictable word, which is an unpredictable word.
And in behavior, there are also differences. In particular, when you're reading, you're more likely to skip a word that you can predict already. And if you don't skip it, you're more likely to read it quickly. That is, you're using prediction in the sense to optimize your reading-- to not spend time on stuff you don't need to spend on because you already know what it will be likely to be.
And so we can do things like use large-scale statistical analysis with models of language that approximate those predictions from large linguistic data sets, using computational models, to try to understand the relationship, the quantitative relationship, between how predictable a word is in context and how much longer you'll spend reading it than you would otherwise. And it turns out that that actually has a very law-like relationship.
So this is two different methods that we use empirically of studying reading. This is one where you just press a button over and over again to reveal words sequentially. And this is one where it's just-- we're tracking your eye movements, and it's naturalistic reading.
And what I want you to get out of this is on the x-axis is log probabilities. So these are basically like bits. It's just a different base. So it's base 10, but they're basically-- think of this as bits. So it's an information theoretic quantity. How probable, how log probable, how many bits of information did the next word give you above and beyond what you could already expect about the next word-- about the sentence?
And then the y-axis is the contribution of the word's probability above and beyond a bunch of other factors that affect reading times, the contribution of that word's probability to the total amount of time spent reading that word and slowdown in the immediate vicinity. The thing you should get out of this is that these curves are straight lines, basically.
That means that basically, the way prediction and processing load work is that an interpretation of this is that words are information theoretically interpretable pieces of information that come to you. So a word presents you with a certain number of bits or information. That presents you with a certain amount of work that you have to do to incorporate it into your representation of the language, understanding of what you're reading.
You spend that much time. You budget that much time in your reading, and then you continue to read. So it turns out that there's a straightforward information theoretic characterization of how language processing and prediction work.
And we can even do things like embed these kinds of models in a reinforcement learning environment. So we have an agent that moves its eyes to try to optimally gather information about the text. And it turns out that we can then predict-- well, in fact, we can derive from simple principles of optimal action in an uncertain environment, patterns of eye movement behavior that are signature patterns in humans and are reflected automatically in our model, just by building this all up together. So that was a fairly detailed example. [? Mandona, ?] how long are we going again?
AUDIENCE: We have until 1:00, but it can be extended by [? vote. ?]
ROGER LEVY: I guess I want to stop now, and I just want to ask people if they have questions about anything, or if there are particular things that they'd like to hear about. I can talk about other things too, but I want to give you a chance to give feedback.
AUDIENCE: I'm curious about the [INAUDIBLE] grammar, so there is no grammar in that sentence. But when we're given like a [INAUDIBLE] sentence, could one hypothesize that if there were a comma after the woman and then a comma after kitchen, to segment if off [INAUDIBLE] something about the woman. Would that really change predictability?
ROGER LEVY: So that's a great question. There's a whole other part of my research program in modeling real-time language understanding that you're connecting to, which is that-- so the kinds of models that I described basically say-- the problem specification is, I get a sequence of words. I know what those words are. And I have to figure out what the grammatical structure is that underlies those words in order to interpret the sentence.
But as you know from the recognize speech example, that's an oversimplification. In fact, you don't know what the word-- you don't have [? veritical ?] access to the word string that you're trying to understand. That itself is inferential. The problem with figuring out the words is inferential.
We can imagine that there are some different kinds of architectural designs for how the language processing system might be organized. So it might make sense to have two modules, a get the words from acoustic input module and then a analyze the words once you've got them module. But actually, this level has a lot of information about what the words are.
And you can see that, in fact, your reactions to this sentence exemplify that, because the anomaly-- I mean, once again, it's a perfectly well-formed sentence, but the surprisingness of the word tripped, given the context, was enough to make you all think of ways your response was like-- it wasn't that, well, that was hard, but I got it. It was like, I think something's missing.
And you can think of that itself as an inferential problem and process. And it falls into the category of what's often called noisy channel inference. So the idea is, we're not going to treat the word string as [? veritical. ?] We're going to treat the word string as sort of a set of evidence around which our beliefs about what the actual word string was should be probably centered.
And so what that means is that if you can find a word string that has a much more natural, higher probability analysis and interpretation than the actual word string, then you might come to believe, maybe it should have been that string instead. And how could it have been another string instead? So it might be that I misremembered some of the previous words that I read. It might be that the speaker made a mistake. If you're thinking about a child learning language, that might be a situation where the child says, maybe my thoughts about how the grammar works are wrong.
So there are all sorts of degrees of freedom that can be pushed around in this kind of noisy channel inferential problem. And so we actually do a lot of that kind of noisy channel modeling, where we-- and in fact, I think that that is going on all the time. And one thing you'll notice is that if you actually go and look at transcribed speech, it's much more errorful than anybody ever remembers.
And so my view is that the kinds of effects like, oh, the word should've been there, there, there, that you have reading sentences like this, are a consequence of an error correction mechanism that is always happening in comprehension. And that is part of the story of robust language processing. That's part of the solution to how do people do this so well-- is that we have these error correction mechanisms. But when we get something weird that's carefully designed, it can force a hypothesized error to pop out, even when it's not actually necessary by the grammar of the language. So that's great. Yeah.
AUDIENCE: So if you do a study in one language, what requirements are there before you study to generalize it to other languages? Or if you do a study in one language, than you really only conclude things about that language, people who speak that language.
ROGER LEVY: That's a great question. That's a really deep and far-reaching question. It goes to the heart of a bunch of major theoretical issues in the field. There's a lot of things to say about that.
So one is that, of course, we all start off as essentially the same when we're infants, regardless of what language or context were exposed to. So you can take an infant from anywhere in the world and put them anywhere else in the world. And as long as they're taking care of and spoken around, then they will learn the language in their environment.
And there's no genetic variability. There are some disorders that are genetic disorders that affect language. But setting that aside-- and they affect all languages not just one particular language. But there's no certain genetic predetermination or predisposition to certain languages over others, as far as we know. There are some very corner cases, which are people have hypotheses about, but to first order approximation, there's no variability that way. But there's massive variability, of course, in the actual languages.
Now, if we go one step further, a natural hypothesis-- so that's about the possible language structures that can be learned. All humans have the potentiality to learn all languages. But there's a slightly different question, which is, what about the relationship between the language structure and the architecture of understanding, the understanding mechanisms? And of course, once again, from an ontogenetic point of view, we all have the capacity to become proficient under standards of any language. But that's a different question than, is the gross architecture of language understanding describable in common terms for different languages?
So let me give an example of where some people propose that it might be really different. So in languages, there's a lot of action that involves verbs. And of course, verbs describe actions. But actually, what I really mean is that verbs have a lot of syntactic content to them.
Some verbs don't want objects. Some do. Some verbs want multiple objects, like the word give, like give somebody a book. Many verbs don't. There's all sorts of other things to say about verbs. Some introduce complement clauses, like I think that. But you can't say, I break that. I think that it's raining, you can't say, I broke that it's raining. So verbs have a lot of stuff going on there.
That means that when you encounter a verb, in some sense, it gives you access to a rich range of information sources about what else you haven't seen in the sentence yet. But languages vary dramatically in where in the sentence verbs occur. So the most common word order is actually for the verb to occur at the end of the sentence.
The second most common order is for the verb to occur in the middle of the sentence, like it does in English. And then a non-trivial number of languages, but a minority, maybe 15% of languages in the world, have verbs at the beginning of the sentence. For example, Irish is an example of that. Classical Arabic is an example of that.
One might imagine that languages with verbs in different places have very different processing architectures. Alternatively, one might imagine, no, the processing architecture is actually best described as highly uniform across all the languages of the world. And what need to do, from a theory development point of view and from a computational instantiation point of view, is to work out, here is what the architecture of language processing looks like for any language, sans anything that we say about the particulars of the language. And then if we can figure out what that's like and we can say, give me the grammatical description for any language, I can plug it into that architecture and out will pop a set of predictions about how understanding unfolds in real time.
So that's the space of possible hypotheses. I tend to think that the latter-- the, there is a universal architecture of language processing and that then, that interfaces with the-- basically the particularities of the language. My bets are on that. But it's really open in a lot of ways.
And just to give you an example of the kinds of challenges for that kind of proposal, a language with verbs at the end-- so Germany is an example of this that's been studied a lot. Germany has verbs at the end. And famously, and Mark Twain wrote about this, it's painful to read a long German sentence, because you're just having to remember all that stuff in memory. You get everything in the sentence except what happened, because the verb is at the end. And you have to remember all that stuff and then, finally, integrate it with a verb at the very end.
But that actually, it turns out empirically, the German speakers are actually better at some of these. The dog that the cat that the rat-- or the rat that the cat that the dog examples, German speakers are actually better at those examples than English speakers. And maybe it has to do with the fact that they have more practice in maintaining those relationships.
So should we think about that as a language-dependent architecture for understanding? Or should we think about that as an architecture where there's some-- memory itself is shaped. There's a universal way to describe the way memory works in language processing. But the structure of memory is then shaped in a contingent way by the structure of the language and the kinds of experiences the language user has.
And so actually, we're working on a kind of theory like that these days. In fact, the relationship between these kind of probabilistic effects and these kind of memory effects, in my view, is one of the major frontiers for our field. A great question. Yeah.
AUDIENCE: I have a question referring to the eye tracking. [INAUDIBLE] because I'm not sure if all the sentences have that type of certain level of ambiguity on them. But I also want to know, have you also seen different things like-- because you mentioned that you can detect by the pattern of your gaze if they're fluid or non-fluid [INAUDIBLE] you focus on certain words. But other than that, the particular words you like them to focus on, have you seen words that involve emotion or harm or something like that make people stop and go back?
ROGER LEVY: That's a good question. So I haven't worked on that myself, but I have a couple of data points about that. So one interesting thing, I've a former colleague who works on-- he's done work on reading of politically-oriented language. So he has self-identified Democrats and self-identified Republicans read stereotypically liberal leaning versus stereotypically conservative leaning passages of argumentation. And so what do you think my colleague finds? What kind of pattern do you think?
AUDIENCE: [INAUDIBLE] very frustrating or something [INAUDIBLE].
ROGER LEVY: It's like an experiment. It's factorially designed. So sometimes, you're randomly assigned-- you come in as-- let's say you're a self-identified Republican, you're randomly assigned to read either liberal-leaning passages or conservative-leaning passages.
AUDIENCE: [INAUDIBLE].
ROGER LEVY: The first order description of the result is that basically people spend less time reading stuff that they disagree with. They just skip over it. So that's a little related to that stuff you're asking about. Emotionally-charged language, I haven't ever worked on. But there is one other thing I wanted to mention, which is-- oh, I'm forgetting now, sorry.
But people are interested in those questions. Also, another thing that happens when you read is your pupil changes size. And when you're surprised, or just very generally, arousal will generally lead to enlargement of the pupil. So that's something that people might look up. But I'm not sure if people look [INAUDIBLE] particular. There's another question over here. Yeah, Joey.
AUDIENCE: I think you mentioned that for measuring how predictive the next word in a sentence will be, that eye tracking is a measurement and also neural [INAUDIBLE]. If that's the case, then what kind of questions are MEG and EEG helpful for answering that can't be actually answered using eye tracking?
ROGER LEVY: That's a great question. Why are we using neural measures? So there are different views about that. So there are several different things that studying the brain-- language in the brain rather than studying language and behavior can give you, potentially.
So one, of course, is spatial localization. Conceptually, the thing to start with is that behavior is a very low dimensional signal when we're looking at something like reading, say, or even just like where you're looking on a screen as you're listening to a text. So reading tells you, it may just be as low dimensional as, how long did you spend on each word?
But it might be something richer, which is like, well, you get something like-- in fact, you get a data set which looks like this for each sentence. So this is actually a very high dimensional rich signal, but there's a lot of randomness in what people do, even in just like-- even given what you want your eyes to do, the motor command is noisy. And so there's a lot of randomness in this whole thing.
And so really, we use a very low dimensional reduction, traditionally, to analyze. And we're sort of interested in, what can we pull out of this beyond a low dimensional reduction? But you only get a few dimensions, like for example, how likely are you to skip the word altogether without looking at it? If you looked at it, how long do you spend reading it the first time you look at it?
How likely are you to move your eyes backward rather than forward after you look at it for the first time? It's those three things plus a couple of other things. There's a few other things, but it's a low dimensional signal.
Scanning the brain is a insanely high dimensional signal. It gives you many, many dimensions, as many dimensions as you have electrodes. If you're doing an EEG or MEG with an MRI, you get as many dimensions as you have voxels. And that's in each time slice. So it's a very different kind of data.
So there are two advantages, potentially. So one is spatial localization. You can ideally figure out where in the brain things are happening, not just-- with behavior, you can see what's easy and what's hard, what people are tending to at a particular time. You can get spatial localization. You can also get temporal localization about what's going on internally, not just about when people respond behaviorally. And the temporal, that's with fast times slices with EEG and MEG.
So those where and when questions can be important. But there's another thing that you can use the high dimensionality of the data for, which is to try to figure out which language processing phenomena are more like each other and more unlike each other. Under the logic that if a brain responds in a grossly similar way to two different things, maybe it's treating them in similar ways. Once again, those are plausible inferences. We don't know with logical certainty, but that can be very useful.
So for example, broadly speaking, EEG data, in particular-- in the past almost 40 years now, there's been a long history of research on EEG responses in sentence processing. And one major result out of that is that the brain, in general terms, distinguishes as distinct patterns of response to encountering a semantic anomaly to a grammatical anomaly. So if you read, Julie and Sarah walks to the store, that gives you a very different pattern of anomaly relative to walk to the store than I like my coffee with sugar and sock gives relative to sugar and milk. So those are things that you can use neural methods for. Yeah.
AUDIENCE: [INAUDIBLE]. Do the subjects expect a questions afterwards or would that harm their free, casual reading?
ROGER LEVY: It depends on what we're doing. But in a lot of cases, yeah. We ask questions in a lot of our studies for a couple of reasons. One is, it's actually just to maintain attentiveness so that people can't just zone out, because then-- so in fact, actually, here's another cool thing you can do with eye tracking, is that you can actually tell when people's minds are wondering.
It's like you have the experience of your mind wandering while you're reading something boring. So that's detectable. So you can do that. But in general, we don't want to assume too much about the eye movement signal. The ideal situation is when we're doing an experiment, people are attentive all the time, unless we're, in particular, interested in inattentiveness.
AUDIENCE: Have you seen a difference between people who are just reading versus people that are reading because they are [INAUDIBLE] pressure?
ROGER LEVY: Yes. So in terms of the work that I've been involved in, the clearest example of that is probably-- we have people read in different ways, and we see how it affects their eye movement. So for example, we ask people to proofread versus regular read.
And there are different kinds of proofreading. So there's proofreading of looking for typos that create words that-- just non-words. But then there's also proofreading for typos that a spell checker couldn't catch. Normal reading in those two kinds of proofreading, all three of those things elicit different eye movement behavior.
But sometimes, of course, we're also interested in the relationship between people's eye movements and what they did and didn't figure out about the sentence. We're particularly interested in those cases and looking at the relationship between the eye movements and the question answering behavior. Yeah.
AUDIENCE: So what about people who are bi- or trilingual, when even if they aren't proficient in English but they're proficient [INAUDIBLE], is it different when they read?
ROGER LEVY: That's such a great question. So the answer is yes. So we definitely know that. I think the more proficient you become in a-- there's two parts of that. One part of that is, just overall, how good are you-- are you close-- are you at the level where you can comfortably read the text at the difficulty that you're reading? There's that.
So we get signal on that. Signature effects from your native language do show up. So for example, languages vary in whether they have determiners, also called articles, like a and the. English has them. Western European languages generally have them. Eastern European languages don't. East Asian languages generally don't. There's a lot of variation.
Native speakers of languages that don't have determiners tend not to look much at determiners in English until they're quite proficient, which makes a lot of sense. Unless you've gained a high level of proficiency, you're not-- the meaning contribution of [INAUDIBLE] is that it's quite subtle. And it's not like you can't point to it.
It's a contextual property. It's a property of the relationship between the thing that you're talking about and the broader context. It's a very subtle thing. And so it takes a while to master that, and people seem to ignore it if you don't already have a high level of command.
So that's part of it. But there's another question and let me do one last poll. So I'm going to give you another ambiguity, and I'm going to get your intuitions about it. So this is a real experiment. I don't know how this is going to turn out. This will be interesting.
Let me give you a phrase. The daughter of the colonel who was on the balcony. That's ambiguous. Who was on the balcony? So just think about which sounds more likely to be the case. Was the daughter on the balcony or the colonel on the balcony? So raise your hand if you think it was the daughter. Raise your hand if you think it was the colonel.
Now, most people thought it was the colonel. Raise your hand if you thought it was the daughter. So I know people vary a lot here in native language. So what are your native languages?
AUDIENCE: Nepalese.
ROGER LEVY: Nepalese, oh that's so cool. Oh, that's awesome.
AUDIENCE: Arabic.
ROGER LEVY: Arabic? Arabic.
AUDIENCE: English.
ROGER LEVY: English. That didn't work very well.
[LAUGHTER]
Do we have any native Spanish speakers here who are-- OK, great. So what about native, other romance language speakers? Any other native romance language speakers? So turns out, let's do it now. What I want you to do is if you fluently speak another language in your head, just translate that sentence word for word into your native language. Think about it.
AUDIENCE: [INAUDIBLE].
ROGER LEVY: Now, ask who's on the balcony in your native language. So raise your hand if it's the daughter. Did anybody switch when they did that? Nobody did, darn. Because it turns out, if you do this in Spanish with native Spanish speakers, in English with native English speakers, you get different preferences.
So in English, the preference is that it's the colonel who's on the balcony. In Spanish and in, actually, many other languages-- Russian, German, Dutch, many languages that have been studied-- it's the daughter who is on the balcony. And I don't know actually, how do you say that in Nepalese?
AUDIENCE: How do I say that?
ROGER LEVY: Yeah.
AUDIENCE: [INAUDIBLE].
ROGER LEVY: And could you provide us a word for word translation of that? Is the order the same as English?
AUDIENCE: I mean, I thought the daughter is in balcony, so I said, colonel's daughter is in the balcony.
AUDIENCE: Yeah, that's how I did mine, too.
ROGER LEVY: Did you use a who is kind of concern, or did you just say, the daughter's colonel is on the balcony?
AUDIENCE: I said, daughter of colonel is in the balcony.
ROGER LEVY: Oh, I see. But you were actually using a complete sentence, whereas--
AUDIENCE: [INAUDIBLE].
ROGER LEVY: Because the form that I used in English was not a complete sentence. It was just a noun phrase. So well, we'd have to take this online and do it a little more extensive, like writing out the examples in full. But there's real cross linguistic variation in this. So in many languages, if I do the daughter of the colonel who was on the balcony, the-- although in English, the preference tends to be that who was on the balcony is describing the second noun. In many languages, including, actually, most of the European languages that have been studied, the preference is the opposite.
And it has been hypothesized, although not borne out by our very informal sample, that actually that native language preference will actually influence your interpretive preference in English. So those are open research questions. These are great questions.
Well, I expect you all to now join Cognitive Science of Language graduate programs. Anyway, this has been a real pleasure. Thanks.
[APPLAUSE]
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
LEARNING