The following content is provided
under a Creative Commons license.
Your support will help MIT OpenCourseWare continue
to offer high-quality educational resources for free.
To make a donation or to view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: All right.
Good afternoon, everyone.
Today, we are going to have two topics.
In the first half we are going to discuss visual illusions,
and in the second half we are going
to talk about prosthetic devices for the blind.
What I would like to start with is, again,
to send around the attendance sheet,
so please put your name to it.
And then, in addition, I'm going to send
around some handouts, one for each person.
I'm going to hand them out in both directions.
Thank you.
The handouts we're going to look at,
I'll refer to them specifically as we go around.
And I also will show pretty much the same handouts, figures
in the handouts, on the screen.
All right.
So for the first half of the presentation today
we are going to talk about visual illusions.
Now, the main emphasis of what I'm going to talk about,
it'll be an attempt to convey to you what
kinds of procedures one can use to get a better sense of why
we see in illusions what we see.
Some of these approaches are strictly behavioral.
And some of them have involved experiments carried out
on a more thorough basis using electrophysiological procedures
and so on.
But those I'm not going to talk too much about.
But I will relate what we can figure out
through how the visual system works.
Now, because of that, there are hundreds of illusions,
and I will emphasize those that allow
us to think about why we see what we see.
Now, one of these illusory effects
is the so-called Hermann grid illusion,
and that's the one I'm going to start
with because it's a fascinating illusion.
If you look at this, what you see
is that at the intersections you see sort of smudges.
And so the big question arose as to why we see these smudges.
Now, does everybody see those smudges?
If you look at the handouts, I also
show you-- pay attention only to the front page,
the one on the left, which says the Hermann grid illusion.
So if you look at that either on your handout
or you look at it on the screen here,
you see these smudges at the intersections, not where you're
looking at, but in the smudges other than where
you're fixating.
Does everybody see those smudges?
OK.
So the question arose why we have these smudges.
And back in the 1960s, an investigator
came up with a rather clever idea
of why we see these smudges.
And the person who came up with this theory
was a fellow called Baumgartner.
And so what he proposed, that this illusory effect
is not directly attributable to the center-surround
organization or the receptive fields or retinal ganglion
cells.
So what he proposed is that when a retinal ganglion cell falls
in the center like this, there's more inhibition in the surround
than when it falls along not the intersections,
but along the straight lines.
And because of that, there's more inhibition here than here,
and that's why we see the smudges.
Now, that was a very attractive theory.
And in fact, it has appeared and still exists in many textbooks
that you read about illusions and about vision in general.
Now, OK, so here is the exact description
that I just made to tell you this.
And of course, you would expect that if this is the case,
you should also get the smudges if you'd reverse the contrast.
And in fact, that is true.
So here, once again, you see the smudges, maybe not quite
as sharp.
But in this case, it's whiter smudges,
whereas in the previous case, this one,
you have darker smudges.
So that kind of fits with that basic idea.
But then, subsequently, people began
to play around with various conditions to see when you do
and when you don't get the smudge effect
and to see whether that would fit with this theory.
Now, that's the basic tenet in just about everything
that psychologists and neuroscientists
working in the visual system do.
They try to critically examine various hypotheses.
And doesn't have to be just illusions,
it can be a whole bunch of other things.
And they do this then by systematically manipulating
things in experiments that can present these things just
under free viewing conditions, as I'm doing it
in terms of these handouts, or you
can do it under more controlled conditions.
The subjects can look through a tachistoscope or something
like that.
All right.
So anyway, let's now look at this in a bit more detail
and see what's wrong with this theory.
Well, first of all, if you look at this figure,
what you can see here that you can see the smudges
irrespective of the size of the overall display.
Now, that's a problem because, of course,
the size of the receptive fields in the retina,
in their center-surround arrangement, is constant.
So if it were the sharp arrangement
that we described about the way the center and the surround
are activated by the intersections
and the non-intersections in the display, that should not
apply to different sizes.
It should only work then for a particular size
where you approximate that.
Now, I will elaborate on that in more detail in just a minute.
Now, another bug that arose here is
that if you now turn this display just 45 degrees,
all right, so that it's a diamond rather than a square,
the illusory effect has declined considerably.
And then to make this even more problematic,
and this appears in your handout on that backside
of the first page, what you can see here
is that when you make the lines arranged
in such a fashion at slight angles to each other, then
you don't see the illusory effect at all.
Whereas here, we also see it a little bit less but still
slightly visible.
But here it's not visible at all.
And you can compare that even better on the handout there.
So the question is, why would that be?
Well, we give you another example here.
If you make serrated edges-- I don't
know if the people in the back can see the serrations.
Those of you in front can see the serrations.
And when you do that, also you fail to see it.
And the problem here, of course, is
that if you make these slight changes in the angles
of the intersecting lines or you make them serrated,
the center-surround antagonism should be the same.
And yet such a small change causes the illusory effect
to disappear.
So that then, in itself, questions the Baumgartner
theory.
Now, there are several other factors that enter into this.
This is a very subtle one.
If you look at this side, OK, we have different shades
for the lines.
OK?
On this side, we have the darkened lines, OK, in front.
And in this case, they have them in the back.
And here you see the illusory effect, and here you don't.
And once again, how can that be if the illusory effect is
due to the center-surround antagonism?
Even more dramatic is when you do this with colors.
Here you have, again, the colored lines in front,
and here you have them in the back.
And here you can see homonymous shades, all right?
This is darker yellow and so on.
But it's the same color but a little bit darker
at the intersections.
So here you get a pretty good illusory effect.
But once those white lines are in front,
we get very little effect at all.
So that then denotes a number of problems with the theory.
And then one more fact is that when you present the stimuli
at isoluminance, you don't get an effect at all.
So that then raised some serious questions
about the illusory effect.
And then a quantitative study was
undertaken to see just what is the layout of the receptive
fields in this display.
And so here what we have is a display.
This square here is exactly five degrees from your fixation, OK,
and you can calculate the number of midget and parasol cells
that are activated by that square area.
And you can see in total there are more than 360 cells that
are being activated in that little region.
And here is a display, in this case, of only the ON cells,
and you can double that by having the OFF
cells in the same location.
Saying that there's just no good correspondence
whatsoever in the number of ganglion cells
that are being activated and how the center-surround is truly
laid out at a point where you get a very good smudge
effect with this illusion.
So this then, taken all together,
raises the issue then if indeed there's a retina phenomenon,
why is there no fit here?
And so it does look like that it's unlikely
that this illusion can be explained
by the center-surround antagonism theory.
So then another theory was advanced
claiming that this doesn't take place in the retina,
but instead that it takes place in the cortex.
And so another theory that was advanced
is saying that this happens in the cortex due to the fact
that you have these simple cells-- we talked
about those before-- that had these elongated inhibitory,
supposedly inhibitory, surrounds.
And if one calculates this, that can explain the illusion
a little bit better, especially since these cells,
even at any given point in the visual cortex,
come in several different sizes.
So that's an alternate theory.
But still, further research needs
to be done to be able to verify whether this theory is more
valid than the center-surround theory in the retina, which
certainly is totally wrong.
So I think maybe in another 100 years,
they're going to drop the Baumgartner theory out
from textbooks and general reviews of illusions
and vision.
OK.
So now I want to show you one more illusory effect, which
is a variant of the illusion that I have just shown you.
And this one is something that can drive you nuts, I suppose.
And that appears on the second front page, the second sheet,
I should say, of the handout.
You can look at it there, or you can look at it here.
And you can see this scintillating effect,
and therefore it has been called the scintillating grid
illusion.
It was invented by fellow called Lingelbach.
All right?
That's a very strong illusory effect, very dramatic.
And whether this can be explained
by any particular theory about the visual cortex
still remains to be settled.
Right now I don't think there's a specific theory that
can explain this.
Now, I will now move on and talk about yet
another illusory effect, or actually a set of them,
that has a much better explanation.
This one we don't have a handout for.
If you look at this display here,
you'll agree that the left and right bars here are identical,
right, going in shade from light to dark.
All right?
So now what I'm going to do here-- what I want to do
is to present these two images here.
I want you to fixate in the fixation spot,
and we'll count to about, I don't know, 20, say.
Keep fixating.
1, 2, 3, 4, 5, 6, 7, 8, 9, 10.
1, 2, 3, 4, 5, 6, 7, 8, 9, 20.
And then I'm going to go back here,
and you can see that this side is much darker than that one.
Everybody got that effect?
AUDIENCE: Yeah.
PROFESSOR: I'll do it once more.
Pretty dramatic, huh?
OK.
Now I'll tell you what I'm going to do.
What I want you to do is to look at this with one eye.
OK?
Don't look at it yet.
Wait a while.
Just let your eye get settled.
You're going to look at it with one eye.
And then when I switch over, I want you to switch eyes.
OK?
All right, ready?
Go ahead now and fixate, and we'll count to 20 again.
5, 6, 7, 8, 9, 10.
1, 2, 3, 4, 5, 6, 7, 8, 9, 20.
Switch.
And if you do that, you don't have an effect.
OK?
So what does that prove?
Well, that obviously proves-- that's why
it's nice to do these little, itty-bitty experiments one can
do in minutes-- it proves that this is a retinal effect.
And the effect is actually due to adaptation
that we have talked about, when was it, a couple of times ago.
Now, I want to show you yet another effect
that has a similar arrangement.
What I'm going to do here is I'm going to put this display on.
And what I'm going to do here is I'm
going to turn these off successively
as we go around the clock.
OK?
And what I want you to do, again,
is to fixate here on the cross, OK,
and then I'm going to start running it.
And what are you going to see?
A pretty amazing thing you're going to see.
What do you see?
AUDIENCE: [INAUDIBLE].
PROFESSOR: Yeah?
You see a green spot going around?
Everybody see that?
OK.
So now the big question comes up, why on earth
do we see this green spot?
What is the explanation?
Well, this again is a retinal effect.
And to test this in more detail, we
have to go back to what I told you about color vision.
And when I told you about color vision--
if you remember we did that experiment with Gaussians--
that if you look at the color circle, all right, and then
you adapt to a stimulus here, which is pretty similar to what
was on the screen, then the afterimage
is on the opposite side cutting across the color circle.
So the afterimages that you see that are in color
can be perfectly explained by the color circle.
If you adapt to this, this is the afterimage.
You adapt to this, this is the afterimage.
It's always straight across the center line, center point,
I should say, where you have the white center, which perfectly
explains your afterimage.
So what I'll do again, I'll show this to you just once more.
OK, here we go, ready?
You can experience it again just to sort of get a sense.
It's amazing how the green circle
becomes more and more and more and more vivid.
Now, one more thing I'm going to tell
you is when I turn this off-- keep fixating--
and then you're going to see a green afterimage, right?
So it proves that, indeed, it's the afterimage that
appears over a period of time with that rotating circle,
and it's fully explained by the color circle.
All right.
So now we're going to move on, and you
look at yet another illusory effect.
If you look at this display, what do you see here?
You see a light square and a dark square, right?
Everybody see that?
Now, what if I told you that what you're seeing
is a total misconception?
So I'll tell you what I'm going to do.
I'm going to leave these two squares exactly as they are,
but I'm going to change the background.
Ready?
What's the difference here?
The difference is that now the background is uniform, right?
And you can see these two squares are indeed
identical in contrast, right?
But now if I go back to the previous one,
if you look at it more carefully you
can see this goes from light to dark.
Of course, the projector makes this look bluish.
It shouldn't be, but that's OK.
And it's a gradual shift in the illumination level
that fools you to think that one is light, the other is dark.
And that has to do heavily with the fact
that the receptive fields of cells in the retina
have center-surround organization,
and they're sensitive to local contrast differences.
That's their make-up.
And so until I point it out to you,
can't even tell that this is a gradual shift.
And if it did not change in color,
it would be harder to tell.
And it has to do with the fact that you're
making local comparisons.
And because of that, to make a distant comparison
you are inaccurate about it.
So this illusory effect, again, can be readily explained
on the basis of what we know about the center-surround
organization of receptive fields in the retina.
OK.
Now, if you look at the same thing in color, which
doesn't come up too well here because of the projector,
but still you can tell that these are not the same color,
OK, again because of the gradual shift,
color shift, in the background.
But exactly identical they are, as they had been before.
So this works not only for contrast,
but it also works for color.
Now, some illusory effects don't work for color and some do.
All right.
So now yet another related effect
that I'm sure some of you have seen
before is if you look at this circle,
we agree that it's uniform in color.
Now, if I cover half of it with dark and the other with light,
it still looks pretty the same.
Maybe it's a little darker here than here.
But then if I put a partition here, as you can see here,
then this side definitely looks darker than that side.
OK?
So this, again, looks at contrast effects,
and we'll talk about these contrasts in just a bit
with more detail.
Next, let me turn to another illusory effect
that you can see in many, many textbooks, which
is called the wallpaper illusion.
If you look at this-- I'll come to it
in a second in your display, where it will appear
on the back of the second page-- what you see here,
it looks like that these are not parallel lines here.
I shouldn't say lines, but parallel
black and white squares.
It looks like that they're going like back and forth like that,
OK, and it's quite dramatic.
And so now we can ask the question, how can we see this?
What is this due to?
It could be due to the way they're aligned.
It could be due to the fact that there
is space between them, that gray space.
And so one can do experiments like the one
you have on your next page there.
Let me first-- I will prove that, indeed, we
have parallel lines here.
These are fully parallel lines, and they go back and forth.
To convince you, you can see that the lines are identical,
and yet they do not look parallel
at all because of this illusory effect.
OK.
So now what we can do is we can fiddle around with this a bit.
Now, that's the kind of thing you guys would probably
enjoy doing yourselves-- look at various illusions
and start playing around with them.
This is an example.
This is a basic wallpaper illusion.
And now you turn this into an equal checkerboard, OK?
And if you keep the lines in between,
and you get no effect whatsoever.
And then here we eliminate those gray bars,
and the illusory effect is greatly reduced.
So consequently, the illusory effect
has to do with both the fact that you
have space between the alternating black and white
bars, meaning these gray lines here,
and that you have them offset.
So when it's not offset, you don't get it.
And when you don't have the bars, you don't get it.
So these two factors play an important role
in giving rise to the illusory effect.
Now, what we also can examine is what happens at isoluminance.
And in this case, once again, what happens
is that the illusory effect is still there,
but it is reduced under isoluminant conditions.
And this is not the isoluminant actually
because of the projector.
But when it's perfectly isoluminant,
then the illusory effect is largely lost.
All right.
So now we have a variant of this illusion which is fascinating.
And this one I will show you on the next page, OK?
And if you look at just this display,
and you are convinced that we have lines that are alternating
that are filtered back and forth, right?
Well, is that really true?
Well, it turns out if you take a bunch of red lines here
that are perfectly parallel, OK, and then you
superimpose those on the display,
you can see that actually the display is parallel.
OK?
The reason it doesn't look parallel
is due to the way these individual little pieces
that you're not even conscious of are arranged.
So let's look at that in more detail.
Let me add here one more thing.
At isoluminance, you get practically no effect.
So these illusions, this one and the previous one,
heavily depend on contrast differences.
OK.
So here we go.
Here's the basic illusory effect.
And the way this was constructed is shown enlarged on top.
Alternate lines have 1, 2, 1, 2, 1, 2, 1.
OK?
What you have in the top line, and the third, and so on,
you have it in this orientation.
Here you have them reversed.
And that vague cue is what somehow
provides the illusory effect for you.
Now, if you make all the lines similar,
OK, in other words don't reverse the alternate lines,
then what you see here is actually
the bars are all tilted slightly upward.
OK?
And because of that, the illusory effect
is not as dramatic as this one, but you still have it.
All right?
So what then you can do with this
is to enlarge on this effect as shown here.
It looks like that you have somehow a very unequal set
of lines, as a result of which you do not
have a bunch of clear-cut squares.
And if you, again, superimpose true parallel lines through it,
you see that they are actually parallel.
All right.
So that then explains that illusory effect
in terms of how we somehow integrate information of shape
that we are not even aware of to have
a perception of a whole thing.
Now, that's not much of an explanation.
It doesn't tell you what neurons are involved
or anything like that.
But it gives you a general concept
of how that illusory effect is created.
Now we come to yet another fascinating illusory effect,
and that one is in the next few pages in the handout.
So if you take the backside of page 3,
you will see this illusory effect here.
Does everybody see that?
Now, this illusory effect is one which gives you a sense
that you have a whole bunch of swirls.
All right?
Does everybody see those swirls?
Instead of seeing 1, 2, 3, 4 circles, you see some swirls.
And you say what on earth is giving us that sensation?
So what you want to do then is to look at it analytically.
And what you see here is you have
alternating black and white squares, first of all.
And you say, well, is alternating
black and white squares the central point?
But then if you look at it more closely,
you can see this is-- I don't know how they came up
with this one-- what you see here, if you look at each
of these, that there's a gradual shift as you go
around the circle in the orientation of the square.
So this square becomes progressively,
very slowly, more rotated.
OK?
And that goes around the clock, and that's
true for every one of those circles.
And you can say, well, is it due to the black and white squares?
Is it due to the change in the orientation of the squares
that you're not even aware of?
Or what?
So what kind of experiment do you
do to check this out, to come up with some understanding?
What kind of game can you play?
Well, the first game you can play,
so you can say, well, let's eliminate
these slight changes in orientation.
OK?
And so what you can do instead, as shown on the next page,
is that you have a bunch of circles,
black and white circles, which are all the same.
And as you can see here, there's no swirls.
You have four circles, very clear.
So therefore, obviously, the squares,
changing their orientation in a subtle fashion going around
in the clock, is very important in introducing
this illusory effect.
Well, then you can say, OK, but is that all of it?
And so what we can do here, as shown on the next page,
is we can maintain the orientation shift,
but we can make them all the same contrast, as we have here.
And when you do that, you have have a rather weak illusory
effect.
There's still a little bit left maybe, but it's kind of weak.
And then if you instead make it isoluminant,
you get some in-between effect.
So on the basis of that, we can conclude
that very important in this display
is that you have a gradual shift in the orientations
of these squares.
And that, in turn, suggests that somehow the way these squares
activate orientation selective cells in V1
plays an important role in giving rise
to this strange illusory effect.
OK.
So now what we are going to do is
to look at yet another illusory effect, which
has a different source, and that one
is the famous Muller-Lyer illusion.
The fact is that these two bars here
are identical in size-- do I have the picture-- like that.
OK?
But when you put these inducing elements, as we will call them,
OK, on there, you'll swear that this one is longer
than this one is.
OK?
So now we can ask the question, what kind of manipulations
can we undertake to see what that is all about?
And let me come back to that in a minute.
So what you can do here is to make
the inducing elements and the bars different in color
or in contrast, and you still get a strong illusory effect.
OK?
So making them isoluminant or making them
opposite in contrast does not eliminate this illusion.
So the theory that somebody has come up with
is shown here, which I'm not sure I buy,
but that's nevertheless a theory,
it claims that the brain interprets
these two displays as differences in depth,
as if here we had a protruding edge in a building inside
or a receding edge as shown here.
And that difference that we almost unconsciously,
if you will, interpret as being in different depths
is what gives rise to the illusion
according to this theory.
Of course, there's a simpler theory simply saying
that we just integrate these two bits of information.
All right.
So now we can ask the question, if you have illusory effects
of this sort, is this something that really takes place higher
up in the brain, or does it take place in the retina where we
cannot separate these inducing and testing elements?
Well, the way you can do this-- I'm
going to show you another illusion which
is similar to this.
Here we have the so-called Ebbinghaus illusion.
This is a very similar kind of illusion.
This is a test element and an inducing element.
The test element here are these two circles,
and you swear that this one is a smaller
circle than that one is, right?
And we have a bunch of small circles,
and here we have bunch of large circles.
So now we can ask, well, is this something
that takes place in the retina or what?
So what kind of experiment can you do?
Well, the experiment that you can do
is to present these displays interocularly.
What does that mean?
What that means is that you present this
to the left eye and this to the right eye.
And you present this device that's
used by psychologists for this kind of thing.
It's called a tachistoscope.
You look through it.
It can present things separately through the two eyes,
and you can flash them on briefly
so you don't get any binocular rivalry.
And when you do that, then you can systematically
vary the size of these to make sure
that you can get a quantitative measure of it.
And when you do that, you find that the illusory effect
is just as good under interocular conditions
as it is under binocular viewing conditions.
Because of this then-- this has been
done with many kinds of illusions
including the Muller-Lyer one.
And when you do that, you show, in essence,
that this is an effect that takes place
most likely in the cortex, certainly
not in the retina or the lateral geniculate nucleus.
It takes place someplace where the images
from the left and two eyes converge.
All right.
So now yet another set of illusions
that I want to show you.
I think that these illusory effects are done.
You can take this home and enjoy it or show it to your friends.
Now I'm going to show you a set of just two
kinds of motion illusions.
OK, here's the first one.
I want you to look at the screen,
and I'm going to flash something on briefly,
and I'm going to do it repeatedly.
Ready?
What do you see?
Do you see something moving?
A very brief sense of motion that you see.
This twirls a little bit.
And this is called the snake illusion,
which is really curious.
This has, again, appeared in quite a number of textbooks,
and we do not have a really good explanation for it.
But we have another one, which I think is related.
OK.
So what I'm going to do here, this
is called the waterfall illusion.
So what we are going to do here is
I'm going to set this in motion.
In this case, I don't want you to fixate.
What I want you to do is just keep
looking at it in various places.
You can look here, here, here, and here, and so on.
OK?
And I'm going to let it run for a while,
and then I'm going to stop it.
OK?
And I want you to tell me what is your aftereffect.
Ready?
OK.
I'm about to stop it.
Are you ready?
Pay close attention to what you're
going to perceive once it stops.
Anybody see what happened?
What did you see?
AUDIENCE: [INAUDIBLE].
PROFESSOR: You see the water running up here, right?
OK.
I'll do it once more so you can appreciate it.
Pretty nice, huh?
OK.
So this illusory effect is one that also occurs interocularly.
And it has been shown that when you record from area MT,
we talked about the fact that area MT is a region where cells
respond selectively to direction of motion.
Well, people have shown that if you,
for an extended period of time, a minute maybe or whatever,
you stimulate a cell in MT with this kind of a display,
then after you stop it, OK, the cells with the opposite sense
of motion will discharge some, which
then is attributable to the upward sensation that you have.
So it's clear enough that this is something
that does take place predominantly by virtue
of direction-selective cells and, probably,
by direction-selective cells in area MT.
All right.
So then what we are going to do here
is we're going to look at some other limitations
and ambiguities in perception.
And in particular, we're going to examine
some figure/ground relationships.
The first one I'm going to show you is this one.
If you look at these two diagonals, OK, a and b,
they certainly look very, very different, don't they?
OK?
One looks shorter and a different angle.
But now if I eliminate the trapezoid in the surround,
you can see that they're actually identical.
OK?
I'll show it to you again.
Pretty dramatic.
So that's an interesting illusory effect.
And it's clear enough that this is something
that takes place in higher cortical areas.
Yet another one here that I'm going to show you also
has something to do with figure/ground relationships,
which has been extensively studied,
especially by Gestalt psychologists.
And I spoke about this a little bit the last time.
What you see here is a bunch of vases, so to speak, right?
That's good enough.
So what I'm going to do now, I'm going
to show you another set of vases.
Are you ready?
There is another set of vases.
Everybody see the vases?
Now, what if I tell you that there's
much more to this than just vases?
So if you look really closely, you're
going to see something else there.
What else do you see there?
AUDIENCE: Faces.
PROFESSOR: Oh, OK, two faces, two guys facing each other.
Now, if I change the relative number of elements in here,
which changes the figure/ground relationships,
you get a different effect.
Here you have-- predominantly you
see the faces rather than the vases.
OK?
And that's a cute way of putting it.
I see the faces, not the vases.
And then you switch it, and you say,
I see the vases, not the faces.
All right?
So that then is a way you can play around, again,
with various illusory effects by fiddling around
on your computer and creating things like that.
OK.
Now, yet another strange effect is
where you can have difficulty telling what's
what is this kind of a display.
You say, well, what's happening here?
Are arrows pointing to the right,
or are they pointing to the left?
And that reminds one of a fellow called Yogi Berra.
How many of you have ever heard of Yogi Berra?
Oh, my goodness, very good.
OK.
So he was a famous catcher for the New York Yankees, right?
Well, he was a very interesting guy,
had all kinds of interesting statements.
And one of those many curious statements, one of them
is-- it doesn't relate to this.
It just comes into my head.
"It isn't over until it's over."
That's one of his, OK?
This one here that's relevant is,
"When you get to a fork in the road, take it."
Yeah?
And that's a Yogi Berra quote.
And so what we have here is a sign that you
come to a fork in the road, and you have to decide to take it.
What are you going to do?
Are you going to go this way, or are you going to go that way?
So that's a kind of game you can play
with this kind of strange effect that you can create
that is confusing your ability to organize
your visual percepts.
Another famous one, maybe not that famous, actually.
Maybe some of you have seen this.
This, again, has appeared many, many years ago in The New.
Yorker.
It says, "I'm turning into my mother."
Now, keep looking at it and tell me,
why do you think that that comment is being made?
What we have here, a young woman looking at another young woman,
right?
But now if you keep looking at this display, look.
This is a nose here, and this is a mouth.
You see an old lady now?
OK.
You played around with this, it gives you a confusing percept.
And ultimately, you can see the young person
looking that way or an old lady looking the other way.
I mean looking not backwards but across.
And that's why this statement says,
"I'm turning into my mother."
So the essence here is that playing around
with these kinds of illusory effect is a lot of fun,
and we all very much enjoy looking at illusions.
And some of us like to play around with them
to try to figure out what their existence is for the reasons.
Now, I want to show you one more,
which I think is very appropriate at this time.
This is the famous Greek key motif.
Everybody's seen this, I presume.
Does everybody know where this is displayed, actually,
at several places?
In the Senate.
OK?
Whenever you have a picture of the Senate Building,
you will see this.
All right?
Now, that's a little bit amusing, actually,
if you think about it.
So this has become symbolic of democracy.
And considering just what happened most recently,
you could almost say that, perhaps,
what's happening here is that you cannot tell whether we see
this guy or this guy here going around.
OK?
And one of them you could say, oh, these are the Democrats.
These are the Republicans.
And you have sort of a conflict as to which ones you see.
All right?
And that is, in a way, symbolic of what's going on today.
But to educate you a little bit, which
I suspect probably most of you know already anyway,
this Greek key motif actually has its source
in the so-called myth of the Labyrinth
that had imprisoned the Minotaur.
Who knows who the Minotaur is?
Oh, very good.
Many of you do.
All right.
So the Minotaur was a creature that
was half bull and half human.
All right?
And what happened, according to this myth-- and let me tell you
it's a myth, and it's a bit of risque thing.
There was a king called King Minos
whose wife Pasiphae-- Pasiphae, I'm not sure
how you pronounce it properly-- is the one who had intercourse
with a bull and gave birth-- sorry,
had intercourse with a bull.
And as a result of that, gave birth
to the Minotaur, which was a half-bull, half-human monster,
if you will.
All right?
And that became so threatening and so irritating,
I guess, to King Minos that he asked
one of his subjugates, Daedalus, to create a maze, a huge maze,
and he put the Minotaur in it.
So it was such a maze that the Minotaur just
couldn't get out of it.
All right?
But then things got progressively worse,
and King Minos decided that he had better
get this Minotaur killed.
And so he asked a person whose name is Theseus, OK,
to go into the maze and kill the Minotaur.
Now, the king also had another offspring
whose name is Ariadne-- nice name, Ariadne.
And she said to Theseus that if you take a ball of thread
and you walk through the maze, once you
do what you have to do, you can follow the thread back,
and that way you can get out of the maze.
So that's what Theseus did, and then he
went on and killed the Minotaur.
Now, the story didn't end there because somehow,
for reasons I'm not sure I know the details anymore,
King Minos had a fall out with Daedalus,
and he jailed Daedalus and his son Icarus.
And while they were in jail, they
tried to figure out how they could escape.
And so Daedalus fashioned a pair of wings
made out of bird's feathers and wax, and then put it on Icarus.
And then Icarus, by doing this, he could fly.
OK?
And then he flew and escaped from the prison.
But unfortunately, he flew fairly high up
where the sun was very hot.
That melted the wax, and Icarus, sadly, fell to his death
as a result.
So that's quite the story.
And that's something that obviously some of you
already know.
And those of you who don't, you will probably
remember, as so many Greek myths are just fascinating,
ancient, old stories.
All right.
So then here is actually a reconstruction
of the kill of the Minotaur by Theseus.
And this is actually a sculpture that exists in Greece.
Pretty nice, huh?
OK.
Now, the last confusing image that I wanted to show you
is this one that had appeared-- I think this appeared also
in The New Yorker, but I'm not 100% sure.
What you have here is a confusion--
do we have three or two prongs here?
And the reason I show this sometimes in class
is because in our department when it was originally
formed by Hans-Lukas Teuber, he created
what is called three prongs.
OK?
And then it became sort of an issue as to, well,
what are the three prongs?
Are there two prongs?
Are there three prongs?
Who belongs into which prong?
And so somebody came up with this picture
that was put up as a joke in the department when
the department was still in one of the old buildings that
doesn't even exist anymore.
All right.
So that then brings me to the end of the illusory effects
that I thought we'd talk about briefly.
And I will now move on and talk about visual prosthesis.
All right.
Now, as far as prosthetics are concerned,
you're going to hear quite a bit about that from Chris Brown
because in audition, an auditory prosthetic device, the cochlear
implant, has become an incredible success.
There are well over 50,000 people in the country now
who have a prosthetic device like that
that enables them to hear, to converse.
It's really incredible.
Unfortunately, when it comes to visual prostheses,
we have nothing comparable at this stage.
It's a much needed device that will take many, many, many more
years to create.
But I thought I'd give you sort of a crude sense of how
that kind of work progresses because there are
many laboratories that are trying to come up
with something that will provide an ability for people to see.
Now, I can tell you that in the world
there are more than 40 million people who are blind.
In the United States, there are more than a million individuals
who are blind.
So because of that-- that's a huge number--
it, of course, is extremely desirable to come up
with some device to do this for you.
Now, this is a very, very long history,
and people have tried all sorts of things.
One of those is to try to induce the ability to see something
in the world that you normally see with your eye
by putting it through some other sense.
Now, a simple example of that is Braille.
All right?
Everybody knows what Braille is, right?
That's you can use your hand to feel something that
has protruding elements in it that
can be shown in terms of letters.
And so as you get into an elevator and you can't see,
you can feel it, and then you can push the right button
to get to the right floor.
OK?
Now, that's one way it has been done.
Another approach had been to present stimulation
to your somatosensory system, either by putting something
on your back or putting something on your fingertip,
that an image would be converted into actually activating
your somatosensory system that would
be equivalent to something that you would perceive if you could
see.
So that's the basic thing.
And a tremendous amount of effort
has been going on to try to come up with something to create
a workable prosthetic device for the blind.
Now, there's a lot of disagreement
as to where such a device should be placed in the body
and also a lot of debate of what kind of procedures
should be used.
Now, some people advocate that it
should be done on the retina.
Some people advocate it should be
done in the lateral geniculate nucleus.
Actually, some work is being done
at Harvard trying to come up with something
for the lateral geniculate nucleus.
And there's some people who think
it would be best to do it in the visual cortex.
So let me sort of go into this in a bit more detail.
And let me, first of all, raise the question of,
what kinds of devices, and what kinds of necessities
do we have for converting vision into something
that enables blind people to see?
Well, first of all, it's very important
to be able to see some basic patterns, very important to see
motion, and very important, often ignored,
is that you got to be able to see something
in the third dimension, because a person who is blind,
he should be able to walk in the world.
And if you're going to walk around,
you should be able to see where things are in depth.
So those are the basic requirements.
You're certainly not required to process color information.
So that's not important.
And there are several other things that are not important.
But these are the three most important ones
that you would like to have if you are going
to create a prosthetics device for the blind.
Now, then the next question is, how are we
going to deal with this, and what kinds
of issues and problems are we going to deal with?
First of all, the big issue is what kind of prosthetic device
should we use.
One of those that I've already mentioned
is to try to convert visual impressions
into other modalities such as your somatosensory system.
All right?
The other is to create some sort of stimulating device,
which there are various kinds.
But one of those would be, for example,
to put a bunch of electrodes into the brain,
or into the retina, or whatever, and then selectively stimulate
through this series of electrodes
to mimic the visual field.
Now, the other one, and it is a big issue,
what brain areas should be considered?
And because we're at an early stage in this,
I think it is desirable for people
to pursue just about any area they think might work.
Now, the problem with a retinal implant--
there are two problems with it.
One is the retina is extremely small, and very confined,
and has these millions of receptors and ganglion cells,
and how can you selectively activate them?
Now, the biggest problem, though,
is that in most blind people, the retina
becomes non-functional.
And even worse, it degenerates in a relatively short time
so that it no longer is useful for electrical stimulation
or for whatever kind of stimulation
you wish to engage in.
So because of that, some people have
moved on to the lateral geniculate nucleus.
That area has some problems because it's deep in the brain,
and it's quite small.
OK?
And also, it tends to degenerate over time,
although it takes it a lot longer than the degeneration
of the retina.
Thirdly, people have considered stimulating the visual cortex,
and that's what I'm going to talk
about in a bit more detail just now.
But the last thing that's very important here-- I think it's
the last thing.
Yeah, just let me go back.
Ah, sorry.
The third big issue is, can you create
a device that has longevity?
As soon as you put something into the brain
and into the eye, there's going to be
some sort of adverse reaction that
may kill the cells at the tip of the electrodes
or something like that.
So all kinds of approaches have been
created to try to increase the longevity of the device.
Now, I'm going to next look at what
happens in the visual cortex, which
is a promising, at least I think it is, a promising site
where you could electrically stimulate and use
that to create a visual impression.
All right.
If you do that-- here is a monkey brain.
The monkey is very well suited for this
because, as I have mentioned before, the posterior
part of the cortex-- see, this is area V1-- is lissencephalic.
And so it's relatively easy to place accurately electrodes
into it.
OK?
So then if what you do here-- here, just a reminder,
is the central sulcus.
Here's the lunate sulcus.
And here, of course, is area V1.
If you now take the contralateral hemifield,
that projects into this region.
OK?
This is shown out to about five degrees.
That will go to around here like that.
OK?
So now the next big question that, initially,
had been done actually in humans,
what do you see when you insert a microelectrode
into the brain-- as I say, this was done on humans initially--
and you electrically stimulate?
What do you see?
And this work was done by Brindley many years ago.
And he discovered that when you electrically
stimulate in the human primary visual cortex, what you get
is a small, star-like image.
All right?
So that's what he found.
And so that then triggered a great deal of research,
including work in monkeys, to determine
what could be done to create a prosthetic device that
might just work.
So the first step then was to try
to determine what happens when you
put a microelectrode in here.
This is, by the way, the layout of the visual field
in the posterior cortex with much more area devoted
to the fovea than into the periphery because, of course,
there's a much higher packing density of receptor cells
as well as retinal ganglion cells in the fovea
representation than further out.
In the cortex, the thickness of the gray matter is constant.
It's about roughly 2 millimeters thick.
And the frequency, spatial frequency of the cells,
is pretty constant as well.
So therefore, you need to allocate more space
to the incoming huge number of retinal ganglion
cells projected to the geniculate
and then to the cortex than for the periphery.
And that's why you have this kind
of a layout of much more space allocated
to the central than to peripheral vision.
So that's a very important factor
in considering how to create a prosthetic device.
So now the first step in doing this kind of stuff,
if you're going to do serious experiments on monkeys,
is to determine, what does the monkey
see when you electrically stimulate the visual cortex?
And so the experiment that's been done here
is one in which a monkey has been trained
to make an eye movement to the bigger of two visual stimuli
or the brighter of two stimuli.
In this case, we have two visual targets.
By the way, one of these is going
to appear where the receptive field is of the cells
that you're going to stimulate in the visual cortex.
So then you may sometimes make this one brighter, this one
brighter.
And the monkey makes a saccade to it,
and he gets a drop of apple juice for reward.
And then you can systematically vary the relative size
of the two stimuli to see what kind of a function you get.
And the same thing you can do for the relative brightness
of it.
And then on some trials interspersed with those trials,
you present [INAUDIBLE] stimulation
with the visual target and see which
one does the monkey go to.
And then you systematically vary,
as I said, the brightness or the size of those stimuli.
OK?
We vary the contrast in this case, and in this case,
we vary the size.
OK?
So that's the basic experiment.
And if you do that systematically,
you can tell exactly what the size and the contrast
is of the visual stimulus created
by electrical stimulation.
If you do that-- here's an example.
When you present two visual stimuli
and you vary the percent contrast difference, all right,
of the two stimuli, you can see that it crosses over, indeed,
when they're identical.
So that tells you that this system works.
It says these are identical.
And the same thing is true for the size series.
So this way then, if you do the same experiment
with electrical stimulation, you can find out
where this 50% crossover point is.
This is for contrast, and this is for size.
So then you can establish this with slightly different
locations in the visual field.
And if you do that overall, you come up
with a summary statement, which says
that when you use currents between 20 and 120 microamps,
OK, at eccentricities between 2 and 1/2 and 3 and 1/2 degrees,
the contrast of the visual percept created in monkeys
is 6% to 12%, and the size is between 15 and 20 minutes
of visual angle.
Now, that's quite small fortunately.
So it's a local, little spot, like a star-like image.
Now, what do we mean by 6% to 12% contrast?
So let me tell you this, that that's
a useful thing for you to memorize.
We talk about percent contrast.
What you do is you measure, for example,
in candelas per square meter, what the contrast is
of this and what is the contest of the background.
Then you subtract one from the other, divide it
by the sum of the two, and you multiply by 100.
So when you do that, this is roughly 8% contrast.
This is roughly 75% contrast.
So it gives you a sense of what contrast means.
All right?
So what that means, in essence, that the visual stimulus
created by the electrical stimulation
is fairly low contrast.
All right.
So now the next important factor,
once one knows what the visual percept is
that is created by electrical stimulation in the monkey,
the next thing is to ask the question, well,
what kind of electrode array should one put together?
All right.
Well, the big issue here has to do
with the so-called magnification factor
that I just talked about.
So let's look at this analytically.
First of all, I already mentioned this.
We have a very high packing density of photoreceptors
in the fovea and much lower density further out.
And so what you have is that this plate
goes through the optic nerve up to the visual cortex
to the lateral geniculate nucleus.
So that's the basic arrangement, meaning
that there are many, many more retinal ganglion
cells per unit area that impinge on the visual cortex
near the fovea than in the periphery.
So then to examine this, here we have, again, another picture.
It's the geniculate again.
Here's the fact that the cortex is constant in thickness.
And here's the layout of the visual field on the posterior
cortex with more area allocated to the fovea
than to-- excuse me, than to periphery.
All right.
So now I've shown you a variant of this before.
If you present an arrow across the visual field here,
this is the impression made with much more activation
in central vision than in peripheral vision.
I mean, you don't have to worry about that.
That simply tells you, as we had discussed before,
that the images created depend heavily
on where you stimulate in the visual field.
OK.
So now to look at this quantitatively,
so we put a bunch of dots up in the visual field.
This is in the central four degrees,
and this is the actual activation
in the visual cortex.
OK?
Meaning that in the fovea, we have two elements here,
this one and this one, here and here, that they take up
a lot of space, whereas in the periphery,
OK, they're very close to each other.
That's due to the so-called magnification factor.
And so one of the basic rules is that we've
got to be able to stimulate the visual cortex taking
into account the magnification factor.
So if you have this kind of spatial arrangement,
if we then were to put a bunch of electrodes
in like this, which we will call proportional electrodes,
then if you stimulated all those,
you would get a similar square with one proviso,
namely that the little dots would be smaller in the fovea,
obviously, than in the periphery.
So then if you have this kind of arrangement, what you can do
is you can take a camera that has a fixed number of elements,
and each of those elements you hook up
to one of the electrodes.
If you put the word "fiat lux" there,
this is what you hypothetically would create, like that.
OK?
That's what it would look like.
So in contrast with this arrangement,
if we were to use an electrode array,
which you can readily buy-- you cannot buy proportional arrays.
You have to somehow create them yourself.
But if you buy a fixed array, which the elements are
equidistant, then if you stimulated all those
you would get a butterfly image, butterfly image.
I will show a movie of that as well.
And that means that if you do that same conversion as before,
this is what you would see in contrast to this.
So in other words, it's a greatly distorted vision
because you didn't take into account the magnification
factor.
All right.
So then what you can do, actually, you
can also increase the number of electrodes in the center here.
OK?
And that would mean-- in this case
we have 428 elements-- that you can recreate fiat lux pretty
well with this increased number of elements,
where with the original set you couldn't make anything out.
Now, the alternate to this would be
that you could take a whole bunch
of equally-spaced electrodes, which you can commercially buy,
and put them in.
But then to get the same acuity, you would need, in this case,
something like over 600 electrodes,
whereas in this case you would only need about 420.
So because of that, obviously we want
to minimize the number of things you put into the brain,
it's desirable to use a proportional device.
So that's the first big step.
Now, let's assume that you have created a proportional device.
The next thing would be to see, how do you stimulate these
through a camera?
So what you do then is an experiment
in which you take a human subject,
and you can put a camera on a helmet,
and also, the person looks at a monitor.
And otherwise, the rest of the world
is excluded by having this bezel here.
And then the camera looks at something,
and then it can display the image here.
Well, one thing it can do, it can display the image as it is.
But the other thing that's much more important,
it can display that image as if electrical stimulation
had taken place.
Now, this becomes complicated because you
have to do a lot of conversion of the input from the camera
to the computer that's going to then drive the electrodes.
Now, the first important point, I mentioned this before,
is that if you present a small spot-- this is actual real data
here-- you present a small spot in the center of a small array
of cells in the visual cortex, you get a vigorous response,
whether a light increment or light decrement.
But if you use a larger spot, you get no response,
meaning that the cortical cells look at local differences.
That has a lot to do with what we talked about
with the illusions.
And so what you have to create is
an algorithm that mimics that.
And so the way you do that is you
have these individual elements for each of the electrodes.
And if that individual element in the computer that
gets the input from the camera, if the whole element gets
activated, there's no real activation of the electrode.
But if an edge appears like here,
here, or here, then the electrode is activated.
OK?
So you create local differences by this conversion system,
all right, by this algorithm that then,
when the camera looks at the world,
will respond whenever there's an edge, an illumination
difference.
OK.
So now the way this looks is like this.
Again, as I've told you, whatever is in the fovea,
the dots are smaller and further out, but that's OK.
So that's the basic activation system.
And so now what we are going to do,
having created such an algorithm,
we're going to examine what the person who
carries this camera on his head can actually see.
Now, the way this can be done, you can either physically
move the stimuli in the world, or the person looks at it
and can slowly move his head around.
OK?
So let's first look at what would
happen if, instead of a proportional display,
we used an equally-spaced display like that butterfly
display.
OK?
So what I'm going to show you here on the left
is what the camera is viewing.
And here I'm going to show you what is actually
perceived by this algorithm, by the person, where this image is
converted into what he's looking at through that bezel.
OK, you ready?
I mean, you don't see a square at all, right?
What you see is, because you're not taking into account
the magnification factor, you get a huge distortion.
So now let's go move on and ask the question,
well, what happens when you are using a proportional display?
All right?
And now we get a rather nice story.
Again, the visual display's on the left,
and the image created is on the right.
Ready?
So the rule, therefore, is that you
must use a proportional image, proportional display, that
can be done either by having it truly proportional as you put
it into the visual cortex or by having an equally-spaced array,
and the computer computes the proportional arrangement.
But the disadvantage of the latter
is that you need many, many more electrodes.
All right.
Now, what I'm going to do is I'm going
to show you whether you can read something
when you use a proportional display like that.
And in this case, we're going to have a bunch of big letters,
and either the letters are going to move slowly
across the visual field, or your head
is going to move slowly with the camera like that
to view that display that's up on the wall.
OK?
So if we do that-- are you ready?
See if you can read it.
OK.
Everybody able to read that one?
OK.
Who remembers what "fiat lux" means?
I've only shown it about four or five times.
Let there be light.
OK.
So now, just to be [INAUDIBLE] attractive about it,
I'm going to show you one more display.
And this one is a display, again,
a series of letters and words, is one
that John F. Kennedy has used in a very, very famous statement,
which I think most of you will remember.
And eventually, as you see the whole thing,
you will be able to put it together.
Are you ready?
OK.
Everybody's seen that, right?
People nowadays, because of what's going on,
are less inclined to take his advice.
All right.
So that is then the essence of what
I was going to cover today.
And so let me now provide you with a brief summary.
I will start with the prosthetics then.
OK?
Research on visual prosthetics is in its infancy,
as I've noted.
A great deal of basic research is
needed before such a device can become effective.
That's very unlike what you're going
to hear about the auditory system.
The brain area that holds considerable promise
for the prosthetic device based on electrical stimulation
is area V1.
Now, a prosthetic device for electrical stimulation of V1
must take into account the magnification factor.
That's absolutely essential, and too many
people are ignoring it.
Now, in a different color, we're going
to talk about the illusions.
There is no unitary explanation for
the great many visual illusions extant.
I pointed out to you that different illusions
have different sources and different explanations.
The most popular theory explaining the Hermann grid
illusion based on the center/surround organization
of retinal ganglion cells is incorrect.
That's a polite way of saying it's all wrong.
A more likely theory is the one that
assumes that area V1 cells are involved.
Retinal adaptation processes can explain illusions
based on aftereffects, and you've
seen quite a few of those.
And many illusions disappear under isoluminant conditions.
And lastly, there are no viable theories that explain illusions
based on figure/ground relationships.
We know what they are, but we don't
know how the brain resolves figure and ground effects.
So that then is the end of what I wanted to cover today.
And next time, we are going to talk about eye movement.
And I hope that you will enjoy that as well.