In this course, you will learn to design the computer architecture of complex modern microprocessors. All the features of this course are available for free. It does not offer a certificate upon completion.
Simultaneous Multithreading
Loading...
From the course by Princeton University
Computer Architecture
352 ratings
Explore Related Videos
At Coursera, you will find the best lectures in the world. Here are some of our personalized recommendations for you
From the lesson
Multithreading
This lecture covers different types of multithreading.
Meet the Instructors
David WentzlaffAssociate Professor
Electrical Engineering
Okay so, those core screen multi-turning.
We're going to move onto figuring out how to mix instructions of different threads
in the pipeline at the same time, and mix them in the same, and issue two different
threads instructions at the same time to different pipelines.
So, the idea here, this is called simultaneous multi-threading, and
actually, let me start of with a, a, a picture of what this looks like.
Let's work backwards in the slide deck here for a second.
So we have a four issue processor. This is our issue with.
And this is time increasing going down. Each of the different patterns represents
a different thread. And the idea here, in simultaneous
multi-threading is you can execute instructions from different threads into
different pipelines simultaneously. Now, this gets quite a bit harder.
Then this basic design here, because now we actually have to read from the
register file four different threads simultaneously, and we have to fetch
different code from different program counters simultaneously in this
processor. So simultaneous multi-threading where it
came out of was, people were building complex out of order superscalars.
And these complex out-of-order superscalars had all this logic,
to be able to track different dependencies between different
instructions, and to basically restart sub portions of instruction sequences.
So this is like, you take a branch of mispredict.
You had to kill all the instructions that were dependent on the branch mispredict
and leave other ones that were not there alone.
And, when you have this out of order mechanism you have all this extra logic
there to figure that out. Dean Tolson, Susan Eggers, Jim Levy, came
up with this idea that, well,
what if we try to utilize all the dead slots in our out of order superscalar,
but intermix different threads in there simultaneously to fill the time?
So they did this study back from ISCA in 95,' and read a bunch of different
applications, and this right most bar here, is our composite or our, our
average here. And what you should figure out from this
is this black bar on the bottom here is how long the processor is busy, actually
doing work and the rest of this is different reasons the processor was
stalled. So we were stalled on instruction task
misses, branch mispredictions, load delays, just pipeline interlocking,
memory conflicts, other, other sorts of things, and we're only using this
processor less than 20% of the time. So, to show this a different way.
We have our multi issue processor. And we have time here.
We have all these purple boxes which are just dead, dead time.
And we might be able to use subsets, you know.
This is very good. We actually issued four instructions in
this cycle. But here, we only issued two.
Here, we issued one. Here, we issued two to these two
pipelines in the middle. Maybe these are the two ALU pipes.
And there's, like, a load pipe here and a branch pipe there, or something like
that. This is,
this is a kind of a disaster from an IPC perspective.
Can we try to re-use that hardware? And we talked about our core screen
multithreading, which was effectively temporally slicing up the cycles here.
So you run one thread, switch to a different thread, run a different thread,
and you can temporally switch between the threads.
But what would happen in, in another approach, and this is actually done in
the failed Sun Millenium processor, what was going to be Ultra Spark 5.
The, they have this idea that they actually had a clustered VLIW, or excuse
me, a clustered superscalar, and they had a mode that you could flip a
bit, and you could basically cut the two clusters separately and run them as two
separate processors, or two different threads.
So you can actually decrease, let's say you had this width four processor, and
instead you split it in half, and you have two functional units running one
thread, two functional units running another
thread. Well, this has, this has some good
effects. You don't actually have to have really
high IPC, or really high instructions per clock.
You don't ever have to reach four in this design because we've narrowed the two
pipeline widths. And you could think about trying to put
them together. This is what the Millennium processor,
the UUltra-Spark 5 tried to do, is you can actually switch between this mode and
this mode. But you still have a lot of open slots
here that you can't go use. So, it still leaves some vertical waste.
Another downside to this was you can't have one thread using all of the
resources very easily here in this mode. You can't have, let's say, thread one use
this functional unit over here in this very static design.
So, this brings us to full simultaneous multithreading, or SMT.
And in SMT we were able to mix and match all these different instructions, but
this is going to change our proster pipeline quite a bit.
Okay, so let's, let's look at what this does to a processor pipeline.
Well. All of a sudden we need to be fetching
multiple instructions at a time. That's, that's definitely, a hard, harder
thing to do here from different threads. Conveniently, we can actually use our
instruction queue, or our reservation stations, if you will, to go find
different instructions to go execute at the issue stage of our out of order
processor. So what we can do is we can tag the
different threads independently, and have that be part of the instruction,
issue logic, such that you can't issue or, or, you know that thread one,
register one is different than thread two, register one.
And then you use the same logic that we've had in our out of order
superscalars, in our issue queue or our to go find different instructions from
multiple threads to go stick down the pipe, or the respective multiple pipes.
And conveniently we have most of this logic already.
And what's also nice about this is, if you're only running one thread at a time.
When you go to look in your instruction queue, or your issue window,
that only that one thread will show up. So you can know that you don't need to go
and look around the other threads, and you can use the full machine.
And this allows you to use the same machine such that if you have lots of
parallelism, you can different threads and fill all
different slots. And then, if you don't have the
parallelism, and you want to exploit instructionable parallelism, you can use
the slots in different allocation and just have the dead slots, the purple
slots. And you can see here, here we are issuing
three instructions from the checkered board blue, and here we are issuing four.
Well, let's say it's the exact same program, executing on the same cycle.
You could actually have the machine such that it'll issue different amounts of
parallelism, of instructionable parallelism, from a thread,
depending on if there's multiple threads running.
And this is what these truly multi-threading machines, simultaneous
multi-threading machines, attempt to do.
.
See, there's one other point I want to make here.
you have to worry when you're doing this simultaneous multi-threading here, about
priorities between threads, because you want to make sure they, you
have some sort of equal progress guarantee, or some,
and round robin's probably not good enough.
So you need to somehow figure out how to not have one thread hog the machine, and
have some, some, fairness between different threads.
So I wanted to show a few examples here. So here we have on the top, we have the
Power 4 IBM Power 4 processor pipeline. And this was a non-multi-thread machine.
And the Power 5 architecture actually looks very similar to the Power 4
architecture, except they added a second hardware thread.
So what they had to do here is they actually had to add more fetch bandwidth,
and then they had it, this notion of group formation.
And this group formation was the picking, out of the two threads.
So they could figure out what they could actually execute simultaneously at the
same time, and they, effectively had extra pipe
stages out in front to go do that. And at the end here you have to commit to
a, different program counters at the same time.
You can see there is definitely complexity in building this.
That all of the sudden, you have to basically into reorder buffer or kill a
subthread when it branch mispredicts, but not the rest of it.
And we already talked about how to do that in a super scale where we can kill a
subsequent a subsection of instructions, and not have to kill the whole
instruction sequence.
.
Here's a different view of the, the Power 4 in a, in a, physical chip view, here.
And one of the interesting things to, to see is that they went with two threads,
because if they went to four threads, they figured out that they would actually
be using the resources too much and be bottlenecking in different locations in
the pipe. It was basically,
they didn't have enough resources, sort of, over here to have it be filled.
it would be filled if you had more than two threads executing on this anyway,
so there was no, so sort of diminishing returns past that
point. We're almost done, but I think we're out
of time. Let's,
actually, before we do that, let me skip forward here and just summarize, with all
the different types of multi-threading. You have your superscalar processor.
You have your, very, very fine-grain multi-threading,
where you're doing it on each cycle, you have coarse grain, coarser grain where
you sort of do it every few cycles. You can think about cutting the processor
in half, and using some of the function units for one thread, and some of it for
another thread, and then we have our full simultaneous multi-threading.
So we'll pick up on this next time and, and, finish up about some of the
implementations of the Pentium 4, and how they did multi-threading.
But we'll stop here for today.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2019 Coursera Inc. All rights reserved.
