Lecture 1: Course Introduction and Overview

Lecture 1: Course Introduction and Overview

COMP4211 05s1 Seminar 4: Branch Prediction Slides due to David A. Patterson, 2001 W04S1 Review Tomasulo Reservations stations: implicit register renaming to larger set of registers + buffering source operands Prevents registers as bottleneck Avoids WAR, WAW hazards of Scoreboard Allows loop unrolling in HW Not limited to basic blocks (integer units gets ahead, beyond branches) Today, helps cache misses as well Dont stall for L1 Data cache miss (insufficient ILP for L2 miss?) Lasting Contributions Dynamic scheduling Register renaming

Load/store disambiguation 360/91 descendants are Pentium III; PowerPC 604; MIPS R10000; HP-PA 8000; Alpha 21264 W04S2 Tomasulo Algorithm and Branch Prediction 360/91 predicted branches, but did not speculate: pipeline stopped until the branch was resolved No speculation; only instructions that can complete Speculation with Reorder Buffer allows execution past branch, and then discard if branch fails just need to hold instructions in buffer until branch can commit W04S3 Case for Branch Prediction when

Issue N instructions per clock cycle 1. Branches will arrive up to n times faster in an n-issue processor 2. Amdahls Law => relative impact of the control stalls will be larger with the lower potential CPI in an n-issue processor W04S4 7 Branch Prediction Schemes 1. 2. 3. 4. 5. 6. 7. 1-bit Branch-Prediction Buffer 2-bit Branch-Prediction Buffer Correlating Branch Prediction Buffer

Tournament Branch Predictor Branch Target Buffer Integrated Instruction Fetch Units Return Address Predictors W04S5 Dynamic Branch Prediction Performance = (accuracy, cost of misprediction) Branch History Table: Lower bits of PC address index table of 1-bit values Says whether or not branch taken last time No address check (saves HW, but may not be right branch) Problem: in a loop, 1-bit BHT will cause 2 mispredictions (avg is 9 iterations before exit): End of loop case, when it exits instead of looping as before First time through loop on next time through code, when it predicts exit instead of looping Only 80% accuracy even if loop 90% of the time W04S6

Dynamic Branch Prediction (Jim Smith, 1981) Solution: 2-bit scheme where change prediction only if get misprediction twice: (Figure 3.7, p. 198) T Predict Taken NT T T NT NT Predict Not Taken Predict Taken T

Predict Not Taken NT Red: stop, not taken Green: go, taken Adds hysteresis to decision making process W04S7 Prediction accuracy: 4K-entry 2bit table vs infinite table size W04S8 Correlating Predictors 2-bit prediction uses a small amount of (hopefully) local information to predict behaviour Sometimes behaviour is correlated, and we can do better by keeping track of direction of related branches, for example consider the following code: if (d==0) d = 1;

if (d==1) { If the first branch is not taken, neither is the second. Predictors that use the behaviour of other branches to make a prediction are called correlating predictors or two-level predictors W04S9 Correlating Branches Idea: taken/not taken of recently executed branches is related to behavior of next branch (as well as the history of that branch behavior) Branch address (4 bits) 2-bits per branch local predictors Then behavior of recent branches selects between, say, 4 predictions of next branch, updating just that

prediction Prediction Prediction (2,2) predictor: 2-bit global, 2-bit local 2-bit global branch history (01 = not taken then taken) W04S10 Accuracy of Different Schemes (Figure 3.15, p. 206) 20% Frequency of Mispredictions 18% 4096 Entries 2-bit BHT Unlimited Entries 2-bit BHT 1024 Entries (2,2) BHT

18% 16% 14% 12% 11% 10% 8% 6% 6% 6% 5% 4% 4% 2% 0% 0%

6% 5% 1% 1% 0% 4,096 entries: 2-bits per entry Unlimited entries: 2-bits/entry 1,024 entries (2,2) W04S11 Re-evaluating Correlation Several of the SPEC benchmarks have less than a dozen branches responsible for 90% of taken branches: program branch % static# = 90% compress

14%236 13 eqntott 25%494 5 gcc 15%9531 2020 mpeg 10%5598 532 real gcc 13%17361 3214 Real programs + OS more like gcc Small benefits beyond benchmarks for correlation? problems with branch aliases? W04S12 BHT Accuracy Mispredict because either: Wrong guess for that branch Got branch history of wrong branch when index

the table 4096 entry table programs vary from 1% misprediction (nasa7, tomcatv) to 18% (eqntott), with spice at 9% and gcc at 12% For SPEC92, 4096 about as good as infinite table W04S13 Tournament Predictors Motivation for correlating branch predictors is 2-bit predictor failed on important branches; by adding global information, performance improved Tournament predictors: use 2 predictors,

1 based on global information and 1 based on local information, and combine with a selector Hopes to select right predictor for right branch W04S14 Tournament Predictor in Alpha 21264 4K 2-bit counters to choose from among a global predictor and a local predictor Global predictor also has 4K entries and is indexed by the history of the last 12 branches; each entry

in the global predictor is a standard 2-bit predictor 12-bit pattern: ith bit 0 => ith prior branch not taken; ith bit 1 => ith prior branch taken; Local predictor consists of a 2-level predictor: Top level a local history table consisting of 1024 10bit entries; each 10-bit entry corresponds to the most recent 10 branch outcomes for the entry. 10-bit history allows patterns 10 branches to be discovered and predicted. Next level Selected entry from the local history table is used to index a table of 1K entries consisting a 3bit saturating counters, which provide the local prediction Total size: 4K*2 + 4K*2 + 1K*10 + 1K*3 = 29K bits! W04S15 (~180,000 transistors) % of predictions from local predictor in Tournament Prediction Scheme 0% 20%

40% 60% 80% 98% 100% 94% 90% nasa7 matrix300 tomcatv doduc 55% spice 76% 72% 63%

fpppp gcc espresso eqntott li 100% 37% 69% W04S16 Accuracy v. Size (SPEC89) Conditional branch misprediction rate 10% 9% 8% Local 7% 6%

5% Correlating 4% 3% Tournament 2% 1% 0% 0 8 16 24 32 40

48 56 64 72 80 88 96 Total predictor size (Kbits) 104 112 120 128 W04S17 Pitfall: Sometimes bigger and dumber is better 21264 uses tournament predictor (29 Kbits) Earlier 21164 uses a simple 2-bit predictor

with 2K entries (or a total of 4 Kbits) SPEC95 benchmarks, 21264 outperforms 21264 avg. 11.5 mispredictions per 1000 instructions 21164 avg. 16.5 mispredictions per 1000 instructions Reversed for transaction processing (TP) ! 21264 avg. 17 mispredictions per 1000 instructions 21164 avg. 15 mispredictions per 1000 instructions TP code much larger & 21164 hold 2X branch predictions based on local behavior (2K vs. 1K local predictor in the 21264) W04S18 Need Address at Same Time as Prediction Branch Target Buffer (BTB): Address of branch index to get prediction AND branch address (if taken) Note: must check for branch match now, since cant use wrong branch address (Figure 3.19, p. 210) PC of instruction FETCH

Branch PC =? No: branch not predicted, proceed normally (Next PC = PC+4) Predicted PC Extra Yes: prediction state instruction is bits branch and use predicted PC as next PC W04S19 W04S20 Predicated Execution

Avoid branch prediction by turning branches into conditionally executed instructions: if (x) then A = B op C else NOP If false, then neither store result nor cause exception Expanded ISA of Alpha, MIPS, PowerPC, SPARC have conditional move; PA-RISC can annul any following instr. IA-64: 64 1-bit condition fields selected so conditional execution of any instruction This transformation is called if-conversion x A= B op C Drawbacks to conditional instructions Still takes a clock even if annulled Stall if condition evaluated late Complex conditions reduce effectiveness; condition becomes known late in pipeline W04S21

Special Case Return Addresses Register Indirect branch hard to predict address SPEC89 85% such branches for procedure return Since stack discipline for procedures, save return address in small buffer that acts like a stack: 8 to 16 entries has small miss rate W04S22 Dynamic Branch Prediction Summary Prediction becoming important part of scalar execution Branch History Table: 2 bits for loop accuracy Correlation: Recently executed branches correlated with next branch. Either different branches Or different executions of same branches Tournament Predictor: more resources to

competitive solutions and pick between them Branch Target Buffer: include branch address & prediction Predicated Execution can reduce number of branches, number of mispredicted branches Return address stack for prediction of indirect jump W04S23

Recently Viewed Presentations

  • Utilizing NeSSI for Analytical Applications Dave Veltkamp* Brian

    Utilizing NeSSI for Analytical Applications Dave Veltkamp* Brian

    Sample delivery to an adsorbent trap for concentration Desorbtion and delivery to twin capillary columns Temperature programmed column elution Detection by simultaneous flame ionization detectors (FID). Trace levels down to low parts per billion can be measured.
  • Presentazione di PowerPoint

    Presentazione di PowerPoint

    L'elettrone sull'eme bL vicino al lato citosolico della membrana è quindi trasferito ad una molecola di UQ su un secondo legante chinoni QN convertendo questo UQ a UQ.-. UQ.- rimane saldamente legato al sito QN . Si completa la prima...
  • Loyola Emergency Planning Discussion

    Loyola Emergency Planning Discussion

    ICS (Incident Command System) The NIMS model for incident management. A standardized on-scene emergency management organization designed to aid in the management of resources during incidents. Intended to organize field-level incident-management operations.
  • Writing the Literary Essay

    Writing the Literary Essay

    The first element in a conclusion is traditionally a restatement of the thesis. You never copy the thesis exactly as it appears in the introduction, but you do want to remind your reader of the overall point of your essay....
  • The Relevance of Time Series Analysis

    The Relevance of Time Series Analysis

    epn cegrn grenerczy, kenyeres, iugg2003 campbell, 2003 wtzr:=0 the time series sp project performs: careful time series inspection, site-by-site reliable offset estimation externally checked velocity estimation products: outlier table offset table etrf2000 velocities and plots product update: 5 weeks, available...
  • IENG 471 Lecture 16: Warehouse Operations

    IENG 471 Lecture 16: Warehouse Operations

    * IENG 471 Facilities Planning * Warehousing Terms SKU - Stock Keeping Unit Product in (packaged) form for warehouse operations Value-Added A modification to the product to obtain business (a product enhancement from the customer's perspective or an enhancement to...
  • Standard Introduction to CUDA C Programming

    Standard Introduction to CUDA C Programming

    Vector Addition on the Device. With add()running in parallel we can do vector addition. Terminology: each parallel invocation of . add() is referred to as a block. The set of blocks is referred to as a grid
  • Espaol I http://www.youtube.com/watch?v=6WNaHuAHKFU Captulo 1: Leccin A El

    Espaol I http://www.youtube.com/watch?v=6WNaHuAHKFU Captulo 1: Leccin A El

    Capítulo 1: Lección A Argentina España Paraguay Bolivia Estados Unidos Perú Chile Guatemala Puerto Rico Colombia Guinea Ecuatorial Republica Domicana Costa Rica Honduras Uruguay Cuba México Venezuela Ecuador Nicaragua El Salvador Panamá Capítulo 1: Lección A 0 [zero] — cero...