1 # Variable-width Variable-packed SIMD / Simple-V / Parallelism Extension Proposal
3 Key insight: Simple-V is intended as an abstraction layer to provide
4 a consistent "API" to parallelisation of existing *and future* operations.
5 *Actual* internal hardware-level parallelism is *not* required, such
6 that Simple-V may be viewed as providing a "compact" or "consolidated"
7 means of issuing multiple near-identical arithmetic instructions to an
8 instruction queue (FIFO), pending execution.
10 *Actual* parallelism, if added independently of Simple-V in the form
11 of Out-of-order restructuring (including parallel ALU lanes) or VLIW
12 implementations, or SIMD, or anything else, would then benefit *if*
13 Simple-V was added on top.
19 This proposal exists so as to be able to satisfy several disparate
20 requirements: power-conscious, area-conscious, and performance-conscious
21 designs all pull an ISA and its implementation in different conflicting
22 directions, as do the specific intended uses for any given implementation.
24 The existing P (SIMD) proposal and the V (Vector) proposals,
25 whilst each extremely powerful in their own right and clearly desirable,
28 * Clearly independent in their origins (Cray and AndesStar v3 respectively)
29 so need work to adapt to the RISC-V ethos and paradigm
30 * Are sufficiently large so as to make adoption (and exploration for
31 analysis and review purposes) prohibitively expensive
32 * Both contain partial duplication of pre-existing RISC-V instructions
33 (an undesirable characteristic)
34 * Both have independent, incompatible and disparate methods for introducing
35 parallelism at the instruction level
36 * Both require that their respective parallelism paradigm be implemented
37 along-side and integral to their respective functionality *or not at all*.
38 * Both independently have methods for introducing parallelism that
39 could, if separated, benefit
40 *other areas of RISC-V not just DSP or Floating-point respectively*.
42 There are also key differences between Vectorisation and SIMD (full
43 details outlined in the Appendix), the key points being:
45 * SIMD has an extremely seductively compelling ease of implementation argument:
46 each operation is passed to the ALU, which is where the parallelism
47 lies. There is *negligeable* (if any) impact on the rest of the core
48 (with life instead being made hell for compiler writers and applications
49 writers due to extreme ISA proliferation).
50 * By contrast, Vectorisation has quite some complexity (for considerable
51 flexibility, reduction in opcode proliferation and much more).
52 * Vectorisation typically includes much more comprehensive memory load
53 and store schemes (unit stride, constant-stride and indexed), which
54 in turn have ramifications: virtual memory misses (TLB cache misses)
55 and even multiple page-faults... all caused by a *single instruction*,
56 yet with a clear benefit that the regularisation of LOAD/STOREs can
57 be optimised for minimal impact on caches and maximised throughput.
58 * By contrast, SIMD can use "standard" memory load/stores (32-bit aligned
59 to pages), and these load/stores have absolutely nothing to do with the
60 SIMD / ALU engine, no matter how wide the operand. Simplicity but with
61 more impact on instruction and data caches.
63 Overall it makes a huge amount of sense to have a means and method
64 of introducing instruction parallelism in a flexible way that provides
65 implementors with the option to choose exactly where they wish to offer
66 performance improvements and where they wish to optimise for power
67 and/or area (and if that can be offered even on a per-operation basis that
68 would provide even more flexibility).
70 Additionally it makes sense to *split out* the parallelism inherent within
71 each of P and V, and to see if each of P and V then, in *combination* with
72 a "best-of-both" parallelism extension, could be added on *on top* of
73 this proposal, to topologically provide the exact same functionality of
74 each of P and V. Each of P and V then can focus on providing the best
75 operations possible for their respective target areas, without being
76 hugely concerned about the actual parallelism.
78 Furthermore, an additional goal of this proposal is to reduce the number
79 of opcodes utilised by each of P and V as they currently stand, leveraging
80 existing RISC-V opcodes where possible, and also potentially allowing
81 P and V to make use of Compressed Instructions as a result.
83 # Analysis and discussion of Vector vs SIMD
85 There are six combined areas between the two proposals that help with
86 parallelism (increased performance, reduced power / area) without
87 over-burdening the ISA with a huge proliferation of
90 * Fixed vs variable parallelism (fixed or variable "M" in SIMD)
91 * Implicit vs fixed instruction bit-width (integral to instruction or not)
92 * Implicit vs explicit type-conversion (compounded on bit-width)
93 * Implicit vs explicit inner loops.
94 * Single-instruction LOAD/STORE.
95 * Masks / tagging (selecting/preventing certain indexed elements from execution)
97 The pros and cons of each are discussed and analysed below.
99 ## Fixed vs variable parallelism length
101 In David Patterson and Andrew Waterman's analysis of SIMD and Vector
102 ISAs, the analysis comes out clearly in favour of (effectively) variable
103 length SIMD. As SIMD is a fixed width, typically 4, 8 or in extreme cases
104 16 or 32 simultaneous operations, the setup, teardown and corner-cases of SIMD
105 are extremely burdensome except for applications whose requirements
106 *specifically* match the *precise and exact* depth of the SIMD engine.
108 Thus, SIMD, no matter what width is chosen, is never going to be acceptable
109 for general-purpose computation, and in the context of developing a
110 general-purpose ISA, is never going to satisfy 100 percent of implementors.
112 To explain this further: for increased workloads over time, as the
113 performance requirements increase for new target markets, implementors
114 choose to extend the SIMD width (so as to again avoid mixing parallelism
115 into the instruction issue phases: the primary "simplicity" benefit of
116 SIMD in the first place), with the result that the entire opcode space
117 effectively doubles with each new SIMD width that's added to the ISA.
119 That basically leaves "variable-length vector" as the clear *general-purpose*
120 winner, at least in terms of greatly simplifying the instruction set,
121 reducing the number of instructions required for any given task, and thus
122 reducing power consumption for the same.
124 ## Implicit vs fixed instruction bit-width
126 SIMD again has a severe disadvantage here, over Vector: huge proliferation
127 of specialist instructions that target 8-bit, 16-bit, 32-bit, 64-bit, and
128 have to then have operations *for each and between each*. It gets very
131 The V-Extension on the other hand proposes to set the bit-width of
132 future instructions on a per-register basis, such that subsequent instructions
133 involving that register are *implicitly* of that particular bit-width until
134 otherwise changed or reset.
136 This has some extremely useful properties, without being particularly
137 burdensome to implementations, given that instruction decode already has
138 to direct the operation to a correctly-sized width ALU engine, anyway.
140 Not least: in places where an ISA was previously constrained (due for
141 whatever reason, including limitations of the available operand space),
142 implicit bit-width allows the meaning of certain operations to be
143 type-overloaded *without* pollution or alteration of frozen and immutable
144 instructions, in a fully backwards-compatible fashion.
146 ## Implicit and explicit type-conversion
148 The Draft 2.3 V-extension proposal has (deprecated) polymorphism to help
149 deal with over-population of instructions, such that type-casting from
150 integer (and floating point) of various sizes is automatically inferred
151 due to "type tagging" that is set with a special instruction. A register
152 will be *specifically* marked as "16-bit Floating-Point" and, if added
153 to an operand that is specifically tagged as "32-bit Integer" an implicit
154 type-conversion will take place *without* requiring that type-conversion
155 to be explicitly done with its own separate instruction.
157 However, implicit type-conversion is not only quite burdensome to
158 implement (explosion of inferred type-to-type conversion) but also is
159 never really going to be complete. It gets even worse when bit-widths
160 also have to be taken into consideration. Each new type results in
161 an increased O(N^2) conversion space that, as anyone who has examined
162 python's source code (which has built-in polymorphic type-conversion),
163 knows that the task is more complex than it first seems.
165 Overall, type-conversion is generally best to leave to explicit
166 type-conversion instructions, or in definite specific use-cases left to
167 be part of an actual instruction (DSP or FP)
169 ## Zero-overhead loops vs explicit loops
171 The initial Draft P-SIMD Proposal by Chuanhua Chang of Andes Technology
172 contains an extremely interesting feature: zero-overhead loops. This
173 proposal would basically allow an inner loop of instructions to be
174 repeated indefinitely, a fixed number of times.
176 Its specific advantage over explicit loops is that the pipeline in a DSP
177 can potentially be kept completely full *even in an in-order single-issue
178 implementation*. Normally, it requires a superscalar architecture and
179 out-of-order execution capabilities to "pre-process" instructions in
180 order to keep ALU pipelines 100% occupied.
182 By bringing that capability in, this proposal could offer a way to increase
183 pipeline activity even in simpler implementations in the one key area
184 which really matters: the inner loop.
186 However when looking at much more comprehensive schemes
187 "A portable specification of zero-overhead loop control hardware
188 applied to embedded processors" (ZOLC), optimising only the single
189 inner loop seems inadequate, tending to suggest that ZOLC may be
190 better off being proposed as an entirely separate Extension.
192 ## Single-instruction LOAD/STORE
194 In traditional Vector Architectures there are instructions which
195 result in multiple register-memory transfer operations resulting
196 from a single instruction. They're complicated to implement in hardware,
197 yet the benefits are a huge consistent regularisation of memory accesses
198 that can be highly optimised with respect to both actual memory and any
199 L1, L2 or other caches. In Hwacha EECS-2015-263 it is explicitly made
200 clear the consequences of getting this architecturally wrong:
201 L2 cache-thrashing at the very least.
203 Complications arise when Virtual Memory is involved: TLB cache misses
204 need to be dealt with, as do page faults. Some of the tradeoffs are
205 discussed in <http://people.eecs.berkeley.edu/~krste/thesis.pdf>, Section
206 4.6, and an article by Jeff Bush when faced with some of these issues
207 is particularly enlightening
208 <https://jbush001.github.io/2015/11/03/lost-in-translation.html>
210 Interestingly, none of this complexity is faced in SIMD architectures...
211 but then they do not get the opportunity to optimise for highly-streamlined
212 memory accesses either.
214 With the "bang-per-buck" ratio being so high and the indirect improvement
215 in L1 Instruction Cache usage (reduced instruction count), as well as
216 the opportunity to optimise L1 and L2 cache usage, the case for including
217 Vector LOAD/STORE is compelling.
219 ## Mask and Tagging (Predication)
221 Tagging (aka Masks aka Predication) is a pseudo-method of implementing
222 simplistic branching in a parallel fashion, by allowing execution on
223 elements of a vector to be switched on or off depending on the results
224 of prior operations in the same array position.
226 The reason for considering this is simple: by *definition* it
227 is not possible to perform individual parallel branches in a SIMD
228 (Single-Instruction, **Multiple**-Data) context. Branches (modifying
229 of the Program Counter) will result in *all* parallel data having
230 a different instruction executed on it: that's just the definition of
231 SIMD, and it is simply unavoidable.
233 So these are the ways in which conditional execution may be implemented:
235 * explicit compare and branch: BNE x, y -> offs would jump offs
236 instructions if x was not equal to y
237 * explicit store of tag condition: CMP x, y -> tagbit
238 * implicit (condition-code) such as ADD results in a carry, carry bit
239 implicitly (or sometimes explicitly) goes into a "tag" (mask) register
241 The first of these is a "normal" branch method, which is flat-out impossible
242 to parallelise without look-ahead and effectively rewriting instructions.
243 This would defeat the purpose of RISC.
245 The latter two are where parallelism becomes easy to do without complexity:
246 every operation is modified to be "conditionally executed" (in an explicit
247 way directly in the instruction format *or* implicitly).
249 RVV (Vector-Extension) proposes to have *explicit* storing of the compare
250 in a tag/mask register, and to *explicitly* have every vector operation
251 *require* that its operation be "predicated" on the bits within an
252 explicitly-named tag/mask register.
254 SIMD (P-Extension) has not yet published precise documentation on what its
255 schema is to be: there is however verbal indication at the time of writing
258 > The "compare" instructions in the DSP/SIMD ISA proposed by Andes will
259 > be executed using the same compare ALU logic for the base ISA with some
260 > minor modifications to handle smaller data types. The function will not
263 This is an *implicit* form of predication as the base RV ISA does not have
264 condition-codes or predication. By adding a CSR it becomes possible
265 to also tag certain registers as "predicated if referenced as a destination".
268 // in future operations from now on, if r0 is the destination use r5 as
269 // the PREDICATION register
270 SET_IMPLICIT_CSRPREDICATE r0, r5
271 // store the compares in r5 as the PREDICATION register
273 // r0 is used here. ah ha! that means it's predicated using r5!
276 With enough registers (and in RISC-V there are enough registers) some fairly
277 complex predication can be set up and yet still execute without significant
278 stalling, even in a simple non-superscalar architecture.
280 (For details on how Branch Instructions would be retro-fitted to indirectly
281 predicated equivalents, see Appendix)
285 In the above sections the five different ways where parallel instruction
286 execution has closely and loosely inter-related implications for the ISA and
287 for implementors, were outlined. The pluses and minuses came out as
290 * Fixed vs variable parallelism: <b>variable</b>
291 * Implicit (indirect) vs fixed (integral) instruction bit-width: <b>indirect</b>
292 * Implicit vs explicit type-conversion: <b>explicit</b>
293 * Implicit vs explicit inner loops: <b>implicit but best done separately</b>
294 * Single-instruction Vector LOAD/STORE: <b>Complex but highly beneficial</b>
295 * Tag or no-tag: <b>Complex but highly beneficial</b>
299 * variable-length vectors came out on top because of the high setup, teardown
300 and corner-cases associated with the fixed width of SIMD.
301 * Implicit bit-width helps to extend the ISA to escape from
302 former limitations and restrictions (in a backwards-compatible fashion),
303 whilst also leaving implementors free to simmplify implementations
304 by using actual explicit internal parallelism.
305 * Implicit (zero-overhead) loops provide a means to keep pipelines
306 potentially 100% occupied in a single-issue in-order implementation
307 i.e. *without* requiring a super-scalar or out-of-order architecture,
308 but doing a proper, full job (ZOLC) is an entirely different matter.
310 Constructing a SIMD/Simple-Vector proposal based around four of these six
311 requirements would therefore seem to be a logical thing to do.
313 # Note on implementation of parallelism
315 One extremely important aspect of this proposal is to respect and support
316 implementors desire to focus on power, area or performance. In that regard,
317 it is proposed that implementors be free to choose whether to implement
318 the Vector (or variable-width SIMD) parallelism as sequential operations
319 with a single ALU, fully parallel (if practical) with multiple ALUs, or
320 a hybrid combination of both.
322 In Broadcom's Videocore-IV, they chose hybrid, and called it "Virtual
323 Parallelism". They achieve a 16-way SIMD at an **instruction** level
324 by providing a combination of a 4-way parallel ALU *and* an externally
325 transparent loop that feeds 4 sequential sets of data into each of the
328 Also in the same core, it is worth noting that particularly uncommon
329 but essential operations (Reciprocal-Square-Root for example) are
330 *not* part of the 4-way parallel ALU but instead have a *single* ALU.
331 Under the proposed Vector (varible-width SIMD) implementors would
332 be free to do precisely that: i.e. free to choose *on a per operation
333 basis* whether and how much "Virtual Parallelism" to deploy.
335 It is absolutely critical to note that it is proposed that such choices MUST
336 be **entirely transparent** to the end-user and the compiler. Whilst
337 a Vector (varible-width SIM) may not precisely match the width of the
338 parallelism within the implementation, the end-user **should not care**
339 and in this way the performance benefits are gained but the ISA remains
340 straightforward. All that happens at the end of an instruction run is: some
341 parallel units (if there are any) would remain offline, completely
342 transparently to the ISA, the program, and the compiler.
344 The "SIMD considered harmful" trap of having huge complexity and extra
345 instructions to deal with corner-cases is thus avoided, and implementors
346 get to choose precisely where to focus and target the benefits of their
347 implementation efforts, without "extra baggage".
351 By being a topological remap of RVV concepts, the following RVV instructions
352 remain exactly the same: VMPOP, VMFIRST, VEXTRACT, VINSERT, VMERGE, VSELECT,
353 VSLIDE, VCLASS and VPOPC. Two instructions, VCLIP and VCLIPI, do not
354 have RV Standard equivalents, so are left out of Simple-V.
355 All other instructions from RVV are topologically re-mapped and retain
356 their complete functionality, intact.
358 ## Instruction Format
360 The instruction format for Simple-V does not actually have *any* explicit
361 compare operations, *any* arithmetic, floating point or *any*
363 Instead it *overloads* pre-existing branch operations into predicated
364 variants, and implicitly overloads arithmetic operations and LOAD/STORE
365 depending on CSR configurations for vector length, bitwidth and
366 predication. *This includes Compressed instructions* as well as any
367 future instructions and Custom Extensions.
369 * For analysis of RVV see [[v_comparative_analysis]] which begins to
370 outline topologically-equivalent mappings of instructions
371 * Also see Appendix "Retro-fitting Predication into branch-explicit ISA"
372 for format of Branch opcodes.
374 **TODO**: *analyse and decide whether the implicit nature of predication
375 as proposed is or is not a lot of hassle, and if explicit prefixes are
376 a better idea instead. Parallelism therefore effectively may end up
377 as always being 64-bit opcodes (32 for the prefix, 32 for the instruction)
378 with some opportunities for to use Compressed bringing it down to 48.
379 Also to consider is whether one or both of the last two remaining Compressed
380 instruction codes in Quadrant 1 could be used as a parallelism prefix,
381 bringing parallelised opcodes down to 32-bit (when combined with C)
382 and having the benefit of being explicit.*
384 ## Branch Instruction:
386 This is the overloaded table for Integer-base Branch operations. Opcode
387 (bits 6..0) is set in all cases to 1100011.
390 31 .. 25 |24 ... 20 | 19 15 | 14 12 | 11 .. 8 | 7 | 6 ... 0 |
391 imm[12,10:5]| rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
392 7 | 5 | 5 | 3 | 4 | 1 | 7 |
393 reserved | src2 | src1 | BPR | predicate rs3 || BRANCH |
394 reserved | src2 | src1 | 000 | predicate rs3 || BEQ |
395 reserved | src2 | src1 | 001 | predicate rs3 || BNE |
396 reserved | src2 | src1 | 010 | predicate rs3 || rsvd |
397 reserved | src2 | src1 | 011 | predicate rs3 || rsvd |
398 reserved | src2 | src1 | 100 | predicate rs3 || BLE |
399 reserved | src2 | src1 | 101 | predicate rs3 || BGE |
400 reserved | src2 | src1 | 110 | predicate rs3 || BLTU |
401 reserved | src2 | src1 | 111 | predicate rs3 || BGEU |
404 Below is the overloaded table for Floating-point Predication operations.
405 Interestingly no change is needed to the instruction format because
406 FP Compare already stores a 1 or a zero in its "rd" integer register
407 target, i.e. it's not actually a Branch at all: it's a compare.
408 The target needs to simply change to be a predication bitfield (done
412 Standard RVF/D/Q, Opcode (bits 6..0) is set in all cases to 1010011.
413 Likewise Single-precision, fmt bits 26..25) is still set to 00.
414 Double-precision is still set to 01, whilst Quad-precision
415 appears not to have a definition in V2.3-Draft (but should be unaffected).
417 It is however noted that an entry "FNE" (the opposite of FEQ) is missing,
418 and whilst in ordinary branch code this is fine because the standard
419 RVF compare can always be followed up with an integer BEQ or a BNE (or
420 a compressed comparison to zero or non-zero), in predication terms that
421 becomes more of an impact as an explicit (scalar) instruction is needed
422 to invert the predicate bitmask. An additional encoding funct3=011 is
423 therefore proposed to cater for this.
426 31 .. 27| 26 .. 25 |24 ... 20 | 19 15 | 14 12 | 11 .. 7 | 6 ... 0 |
427 funct5 | fmt | rs2 | rs1 | funct3 | rd | opcode |
428 5 | 2 | 5 | 5 | 3 | 4 | 7 |
429 10100 | 00/01/11 | src2 | src1 | 010 | pred rs3 | FEQ |
430 10100 | 00/01/11 | src2 | src1 | **011**| pred rs3 | FNE |
431 10100 | 00/01/11 | src2 | src1 | 001 | pred rs3 | FLT |
432 10100 | 00/01/11 | src2 | src1 | 000 | pred rs3 | FLE |
435 Note (**TBD**): floating-point exceptions will need to be extended
436 to cater for multiple exceptions (and statuses of the same). The
437 usual approach is to have an array of status codes and bit-fields,
438 and one exception, rather than throw separate exceptions for each
441 In Hwacha EECS-2015-262 Section 6.7.2 the following pseudocode is given
442 for predicated compare operations of function "cmp":
444 for (int i=0; i<vl; ++i)
446 preg[pd][i] = cmp(s1 ? vreg[rs1][i] : sreg[rs1],
447 s2 ? vreg[rs2][i] : sreg[rs2]);
449 With associated predication, vector-length adjustments and so on,
450 and temporarily ignoring bitwidth (which makes the comparisons more
451 complex), this becomes:
453 if I/F == INT: # integer type cmp
454 pred_enabled = int_pred_enabled # TODO: exception if not set!
455 preg = int_pred_reg[rd]
458 pred_enabled = fp_pred_enabled # TODO: exception if not set!
459 preg = fp_pred_reg[rd]
462 s1 = CSRvectorlen[src1] > 1;
463 s2 = CSRvectorlen[src2] > 1;
464 for (int i=0; i<vl; ++i)
465 preg[rs3][i] = cmp(s1 ? reg[src1+i] : reg[src1],
466 s2 ? reg[src2+i] : reg[src2]);
470 * Predicated SIMD comparisons would break src1 and src2 further down
471 into bitwidth-sized chunks (see Appendix "Bitwidth Virtual Register
472 Reordering") setting Vector-Length times (number of SIMD elements) bits
473 in Predicate Register rs3 as opposed to just Vector-Length bits.
474 * Predicated Branches do not actually have an adjustment to the Program
475 Counter, so all of bits 25 through 30 in every case are not needed.
476 * There are plenty of reserved opcodes for which bits 25 through 30 could
477 be put to good use if there is a suitable use-case.
478 * FEQ and FNE (and BEQ and BNE) are included in order to save one
479 instruction having to invert the resultant predicate bitfield.
480 FLT and FLE may be inverted to FGT and FGE if needed by swapping
481 src1 and src2 (likewise the integer counterparts).
483 ## Compressed Branch Instruction:
486 15..13 | 12...10 | 9..7 | 6..5 | 4..2 | 1..0 | name |
487 funct3 | imm | rs10 | imm | | op | |
488 3 | 3 | 3 | 2 | 3 | 2 | |
489 C.BPR | pred rs3 | src1 | I/F B | src2 | C1 | |
490 110 | pred rs3 | src1 | I/F 0 | src2 | C1 | P.EQ |
491 111 | pred rs3 | src1 | I/F 0 | src2 | C1 | P.NE |
492 110 | pred rs3 | src1 | I/F 1 | src2 | C1 | P.LT |
493 111 | pred rs3 | src1 | I/F 1 | src2 | C1 | P.LE |
498 * Bits 5 13 14 and 15 make up the comparator type
499 * Bit 6 indicates whether to use integer or floating-point comparisons
500 * In both floating-point and integer cases there are four predication
501 comparators: EQ/NEQ/LT/LE (with GT and GE being synthesised by inverting
504 ## LOAD / STORE Instructions
506 For full analysis of topological adaptation of RVV LOAD/STORE
507 see [[v_comparative_analysis]]. All three types (LD, LD.S and LD.X)
508 may be implicitly overloaded into the one base RV LOAD instruction,
509 and likewise for STORE.
514 31 | 30 | 29 25 | 24 20 | 19 15 | 14 12 | 11 7 | 6 0 |
515 imm[11:0] |||| rs1 | funct3 | rd | opcode |
516 1 | 1 | 5 | 5 | 5 | 3 | 5 | 7 |
517 ? | s | rs2 | imm[4:0] | base | width | dest | LOAD |
520 The exact same corresponding adaptation is also carried out on the single,
521 double and quad precision floating-point LOAD-FP and STORE-FP operations,
522 which fit the exact same instruction format. Thus all three types
523 (unit, stride and indexed) may be fitted into FLW, FLD and FLQ,
524 as well as FSW, FSD and FSQ.
528 * LOAD remains functionally (topologically) identical to RVV LOAD
529 (for both integer and floating-point variants).
530 * Predication CSR-marking register is not explicitly shown in instruction, it's
531 implicit based on the CSR predicate state for the rd (destination) register
532 * rs2, the source, may *also be marked as a vector*, which implicitly
533 is taken to indicate "Indexed Load" (LD.X)
534 * Bit 30 indicates "element stride" or "constant-stride" (LD or LD.S)
535 * Bit 31 is reserved (ideas under consideration: auto-increment)
536 * **TODO**: include CSR SIMD bitwidth in the pseudo-code below.
537 * **TODO**: clarify where width maps to elsize
539 Pseudo-code (excludes CSR SIMD bitwidth for simplicity):
541 if (unit-strided) stride = elsize;
542 else stride = areg[as2]; // constant-strided
544 pred_enabled = int_pred_enabled
545 preg = int_pred_reg[rd]
547 for (int i=0; i<vl; ++i)
548 if (preg_enabled[rd] && [!]preg[i])
549 for (int j=0; j<seglen+1; j++)
551 if CSRvectorised[rs2])
554 offs = i*(seglen+1)*stride;
555 vreg[rd+j][i] = mem[sreg[base] + offs + j*stride];
558 Taking CSR (SIMD) bitwidth into account involves using the vector
559 length and register encoding according to the "Bitwidth Virtual Register
560 Reordering" scheme shown in the Appendix (see function "regoffs").
562 A similar instruction exists for STORE, with identical topological
563 translation of all features. **TODO**
565 ## Compressed LOAD / STORE Instructions
567 Compressed LOAD and STORE are of the same format, where bits 2-4 are
568 a src register instead of dest:
571 15 13 | 12 10 | 9 7 | 6 5 | 4 2 | 1 0 |
572 funct3 | imm | rs10 | imm | rd0 | op |
573 3 | 3 | 3 | 2 | 3 | 2 |
574 C.LW | offset[5:3] | base | offset[2|6] | dest | C0 |
577 Unfortunately it is not possible to fit the full functionality
578 of vectorised LOAD / STORE into C.LD / C.ST: the "X" variants (Indexed)
579 require another operand (rs2) in addition to the operand width
580 (which is also missing), offset, base, and src/dest.
582 However a close approximation may be achieved by taking the top bit
583 of the offset in each of the five types of LD (and ST), reducing the
584 offset to 4 bits and utilising the 5th bit to indicate whether "stride"
585 is to be enabled. In this way it is at least possible to introduce
588 (**TODO**: *assess whether the loss of one bit from offset is worth having
589 "stride" capability.*)
591 We also assume (including for the "stride" variant) that the "width"
592 parameter, which is missing, is derived and implicit, just as it is
593 with the standard Compressed LOAD/STORE instructions. For C.LW, C.LD
594 and C.LQ, the width is implicitly 4, 8 and 16 respectively, whilst for
595 C.FLW and C.FLD the width is implicitly 4 and 8 respectively.
597 Interestingly we note that the Vectorised Simple-V variant of
598 LOAD/STORE (Compressed and otherwise), due to it effectively using the
599 standard register file(s), is the direct functional equivalent of
600 standard load-multiple and store-multiple instructions found in other
603 In Section 12.3 riscv-isa manual V2.3-draft it is noted the comments on
604 page 76, "For virtual memory systems some data accesses could be resident
605 in physical memory and some not". The interesting question then arises:
606 how does RVV deal with the exact same scenario?
607 Expired U.S. Patent 5895501 (Filing Date Sep 3 1996) describes a method
608 of detecting early page / segmentation faults and adjusting the TLB
609 in advance, accordingly: other strategies are explored in the Appendix
610 Section "Virtual Memory Page Faults".
612 # CSRs <a name="csrs"></a>
614 There are a number of CSRs needed, which are used at the instruction
615 decode phase to re-interpret standard RV opcodes (a practice that has
616 precedent in the setting of MISA to enable / disable extensions).
618 * Integer Register N is Vector of length M: r(N) -> r(N..N+M-1)
619 * Integer Register N is of implicit bitwidth M (M=default,8,16,32,64)
620 * Floating-point Register N is Vector of length M: r(N) -> r(N..N+M-1)
621 * Floating-point Register N is of implicit bitwidth M (M=default,8,16,32,64)
622 * Integer Register N is a Predication Register (note: a key-value store)
623 * Vector Length CSR (VSETVL, VGETVL)
627 * for the purposes of LOAD / STORE, Integer Registers which are
628 marked as a Vector will result in a Vector LOAD / STORE.
629 * Vector Lengths are *not* the same as vsetl but are an integral part
631 * Actual vector length is *multipled* by how many blocks of length
632 "bitwidth" may fit into an XLEN-sized register file.
633 * Predication is a key-value store due to the implicit referencing,
634 as opposed to having the predicate register explicitly in the instruction.
638 The Predication CSR is a key-value store indicating whether, if a given
639 destination register (integer or floating-point) is referred to in an
640 instruction, it is to be predicated. The first entry is whether predication
641 is enabled. The second entry is whether the register index refers to a
642 floating-point or an integer register. The third entry is the index
643 of that register which is to be predicated (if referred to). The fourth entry
644 is the integer register that is treated as a bitfield, indexable by the
645 vector element index.
647 | RegNo | 6 | 5 | (4..0) | (4..0) |
648 | ----- | - | - | ------- | ------- |
649 | r0 | pren0 | i/f | regidx | predidx |
650 | r1 | pren1 | i/f | regidx | predidx |
651 | .. | pren.. | i/f | regidx | predidx |
652 | r15 | pren15 | i/f | regidx | predidx |
654 The Predication CSR Table is a key-value store, so implementation-wise
655 it will be faster to turn the table around (maintain topologically
659 int_pred_enabled[32];
660 for (i = 0; i < 16; i++)
662 idx = CSRpred[i].regidx
663 predidx = CSRpred[i].predidx
664 if CSRpred[i].type == 0: # integer
665 int_pred_enabled[idx] = 1
666 int_pred_reg[idx] = predidx
668 fp_pred_enabled[idx] = 1
669 fp_pred_reg[idx] = predidx
671 So when an operation is to be predicated, it is the internal state that
672 is used. In Section 6.4.2 of Hwacha's Manual (EECS-2015-262) the following
673 pseudo-code for operations is given, where p is the explicit (direct)
674 reference to the predication register to be used:
676 for (int i=0; i<vl; ++i)
678 (d ? vreg[rd][i] : sreg[rd]) =
679 iop(s1 ? vreg[rs1][i] : sreg[rs1],
680 s2 ? vreg[rs2][i] : sreg[rs2]); // for insts with 2 inputs
682 This instead becomes an *indirect* reference using the *internal* state
683 table generated from the Predication CSR key-value store:
686 pred_enabled = int_pred_enabled
687 preg = int_pred_reg[rd]
689 pred_enabled = fp_pred_enabled
690 preg = fp_pred_reg[rd]
692 for (int i=0; i<vl; ++i)
693 if (preg_enabled[rd] && [!]preg[i])
694 (d ? vreg[rd][i] : sreg[rd]) =
695 iop(s1 ? vreg[rs1][i] : sreg[rs1],
696 s2 ? vreg[rs2][i] : sreg[rs2]); // for insts with 2 inputs
700 MAXVECTORDEPTH is the same concept as MVL in RVV. However in Simple-V,
701 given that its primary (base, unextended) purpose is for 3D, Video and
702 other purposes (not requiring supercomputing capability), it makes sense
703 to limit MAXVECTORDEPTH to the regfile bitwidth (32 for RV32, 64 for RV64
706 The reason for setting this limit is so that predication registers, when
707 marked as such, may fit into a single register as opposed to fanning out
708 over several registers. This keeps the implementation a little simpler.
709 Note that RVV on top of Simple-V may choose to over-ride this decision.
711 ## Vector-length CSRs
713 Vector lengths are interpreted as meaning "any instruction referring to
714 r(N) generates implicit identical instructions referring to registers
715 r(N+M-1) where M is the Vector Length". Vector Lengths may be set to
716 use up to 16 registers in the register file.
718 One separate CSR table is needed for each of the integer and floating-point
728 An array of 32 4-bit CSRs is needed (4 bits per register) to indicate
729 whether a register was, if referred to in any standard instructions,
730 implicitly to be treated as a vector. A vector length of 1 indicates
731 that it is to be treated as a scalar. Vector lengths of 0 are reserved.
733 Internally, implementations may choose to use the non-zero vector length
734 to set a bit-field per register, to be used in the instruction decode phase.
735 In this way any standard (current or future) operation involving
736 register operands may detect if the operation is to be vector-vector,
737 vector-scalar or scalar-scalar (standard) simply through a single
740 Note that when using the "vsetl rs1, rs2" instruction (caveat: when the
741 bitwidth is specifically not set) it becomes:
743 CSRvlength = MIN(MIN(CSRvectorlen[rs1], MAXVECTORDEPTH), rs2)
745 This is in contrast to RVV:
747 CSRvlength = MIN(MIN(rs1, MAXVECTORDEPTH), rs2)
749 ## Element (SIMD) bitwidth CSRs
751 Element bitwidths may be specified with a per-register CSR, and indicate
752 how a register (integer or floating-point) is to be subdivided.
761 vew may be one of the following (giving a table "bytestable", used below):
774 Extending this table (with extra bits) is covered in the section
775 "Implementing RVV on top of Simple-V".
777 Note that when using the "vsetl rs1, rs2" instruction, taking bitwidth
778 into account, it becomes:
780 vew = CSRbitwidth[rs1]
782 bytesperreg = (XLEN/8) # or FLEN as appropriate
784 bytesperreg = bytestable[vew] # 1 2 4 8 16
785 simdmult = (XLEN/8) / bytesperreg # or FLEN as appropriate
786 vlen = CSRvectorlen[rs1] * simdmult
787 CSRvlength = MIN(MIN(vlen, MAXVECTORDEPTH), rs2)
789 The reason for multiplying the vector length by the number of SIMD elements
790 (in each individual register) is so that each SIMD element may optionally be
793 An example of how to subdivide the register file when bitwidth != default
794 is given in the section "Bitwidth Virtual Register Reordering".
798 > What does an ADD of two different-sized vectors do in simple-V?
800 * if the two source operands are not the same, throw an exception.
801 * if the destination operand is also a vector, and the source is longer
802 than the destination, throw an exception.
804 > And what about instructions like JALR?
805 > What does jumping to a vector do?
807 * Throw an exception. Whether that actually results in spawning threads
808 as part of the trap-handling remains to be seen.
810 # Impementing V on top of Simple-V
812 With Simple-V converting the original RVV draft concept-for-concept
813 from explicit opcodes to implicit overloading of existing RV Standard
814 Extensions, certain features were (deliberately) excluded that need
815 to be added back in for RVV to reach its full potential. This is
816 made slightly complicated by the fact that RVV itself has two
817 levels: Base and reserved future functionality.
819 * Representation Encoding is entirely left out of Simple-V in favour of
820 implicitly taking the exact (explicit) meaning from RV Standard Extensions.
821 * VCLIP and VCLIPI do not have corresponding RV Standard Extension
822 opcodes (and are the only such operations).
823 * Extended Element bitwidths (1 through to 24576 bits) were left out
824 of Simple-V as, again, there is no corresponding RV Standard Extension
825 that covers anything even below 32-bit operands.
826 * Polymorphism was entirely left out of Simple-V due to the inherent
827 complexity of automatic type-conversion.
828 * Vector Register files were specifically left out of Simple-V in favour
829 of fitting on top of the integer and floating-point files. An
830 "RVV re-retro-fit" needs to be able to mark (implicitly marked)
831 registers as being actually in a separate *vector* register file.
832 * Fortunately in RVV (Draft 0.4, V2.3-Draft), the "base" vector
833 register file size is 5 bits (32 registers), whilst the "Extended"
834 variant of RVV specifies 8 bits (256 registers) and has yet to
836 * One big difference: Sections 17.12 and 17.17, there are only two possible
837 predication registers in RVV "Base". Through the "indirect" method,
838 Simple-V provides a key-value CSR table that allows (arbitrarily)
839 up to 16 (TBD) of either the floating-point or integer registers to
840 be marked as "predicated" (key), and if so, which integer register to
841 use as the predication mask (value).
845 # Implementing P (renamed to DSP) on top of Simple-V
847 * Implementors indicate chosen bitwidth support in Vector-bitwidth CSR
848 (caveat: anything not specified drops through to software-emulation / traps)
853 ## V-Extension to Simple-V Comparative Analysis
855 This section has been moved to its own page [[v_comparative_analysis]]
859 This section has been moved to its own page [[p_comparative_analysis]]
861 ## Comparison of "Traditional" SIMD, Alt-RVP, Simple-V and RVV Proposals <a name="parallelism_comparisons"></a>
863 This section compares the various parallelism proposals as they stand,
864 including traditional SIMD, in terms of features, ease of implementation,
865 complexity, flexibility, and die area.
869 Primary benefit of Alt-RVP is the simplicity with which parallelism
870 may be introduced (effective multiplication of regfiles and associated ALUs).
872 * plus: the simplicity of the lanes (combined with the regularity of
873 allocating identical opcodes multiple independent registers) meaning
874 that SRAM or 2R1W can be used for entire regfile (potentially).
875 * minus: a more complex instruction set where the parallelism is much
876 more explicitly directly specified in the instruction and
877 * minus: if you *don't* have an explicit instruction (opcode) and you
878 need one, the only place it can be added is... in the vector unit and
879 * minus: opcode functions (and associated ALUs) duplicated in Alt-RVP are
880 not useable or accessible in other Extensions.
881 * plus-and-minus: Lanes may be utilised for high-speed context-switching
882 but with the down-side that they're an all-or-nothing part of the Extension.
883 No Alt-RVP: no fast register-bank switching.
884 * plus: Lane-switching would mean that complex operations not suited to
885 parallelisation can be carried out, followed by further parallel Lane-based
886 work, without moving register contents down to memory (and back)
887 * minus: Access to registers across multiple lanes is challenging. "Solution"
888 is to drop data into memory and immediately back in again (like MMX).
892 Primary benefit of Simple-V is the OO abstraction of parallel principles
893 from actual (internal) parallel hardware. It's an API in effect that's
894 designed to be slotted in to an existing implementation (just after
895 instruction decode) with minimum disruption and effort.
897 * minus: the complexity of having to use register renames, OoO, VLIW,
898 register file cacheing, all of which has been done before but is a
900 * plus: transparent re-use of existing opcodes as-is just indirectly
901 saying "this register's now a vector" which
902 * plus: means that future instructions also get to be inherently
903 parallelised because there's no "separate vector opcodes"
904 * plus: Compressed instructions may also be (indirectly) parallelised
905 * minus: the indirect nature of Simple-V means that setup (setting
906 a CSR register to indicate vector length, a separate one to indicate
907 that it is a predicate register and so on) means a little more setup
908 time than Alt-RVP or RVV's "direct and within the (longer) instruction"
910 * plus: shared register file meaning that, like Alt-RVP, complex
911 operations not suited to parallelisation may be carried out interleaved
912 between parallelised instructions *without* requiring data to be dropped
913 down to memory and back (into a separate vectorised register engine).
914 * plus-and-maybe-minus: re-use of integer and floating-point 32-wide register
915 files means that huge parallel workloads would use up considerable
916 chunks of the register file. However in the case of RV64 and 32-bit
917 operations, that effectively means 64 slots are available for parallel
919 * plus: inherent parallelism (actual parallel ALUs) doesn't actually need to
920 be added, yet the instruction opcodes remain unchanged (and still appear
921 to be parallel). consistent "API" regardless of actual internal parallelism:
922 even an in-order single-issue implementation with a single ALU would still
923 appear to have parallel vectoristion.
924 * hard-to-judge: if actual inherent underlying ALU parallelism is added it's
925 hard to say if there would be pluses or minuses (on die area). At worse it
926 would be "no worse" than existing register renaming, OoO, VLIW and register
927 file cacheing schemes.
929 ### RVV (as it stands, Draft 0.4 Section 17, RISC-V ISA V2.3-Draft)
931 RVV is extremely well-designed and has some amazing features, including
932 2D reorganisation of memory through LOAD/STORE "strides".
934 * plus: regular predictable workload means that implementations may
935 streamline effects on L1/L2 Cache.
936 * plus: regular and clear parallel workload also means that lanes
937 (similar to Alt-RVP) may be used as an implementation detail,
938 using either SRAM or 2R1W registers.
939 * plus: separate engine with no impact on the rest of an implementation
940 * minus: separate *complex* engine with no RTL (ALUs, Pipeline stages) reuse
942 * minus: no ISA abstraction or re-use either: additions to other Extensions
943 do not gain parallelism, resulting in prolific duplication of functionality
944 inside RVV *and out*.
945 * minus: when operations require a different approach (scalar operations
946 using the standard integer or FP regfile) an entire vector must be
947 transferred out to memory, into standard regfiles, then back to memory,
948 then back to the vector unit, this to occur potentially multiple times.
949 * minus: will never fit into Compressed instruction space (as-is. May
950 be able to do so if "indirect" features of Simple-V are partially adopted).
951 * plus-and-slight-minus: extended variants may address up to 256
952 vectorised registers (requires 48/64-bit opcodes to do it).
953 * minus-and-partial-plus: separate engine plus complexity increases
954 implementation time and die area, meaning that adoption is likely only
955 to be in high-performance specialist supercomputing (where it will
956 be absolutely superb).
960 The only really good things about SIMD are how easy it is to implement and
961 get good performance. Unfortunately that makes it quite seductive...
963 * plus: really straightforward, ALU basically does several packed operations
964 at once. Parallelism is inherent at the ALU, making the addition of
965 SIMD-style parallelism an easy decision that has zero significant impact
966 on the rest of any given architectural design and layout.
967 * plus (continuation): SIMD in simple in-order single-issue designs can
968 therefore result in superb throughput, easily achieved even with a very
969 simple execution model.
970 * minus: ridiculously complex setup and corner-cases that disproportionately
971 increase instruction count on what would otherwise be a "simple loop",
972 should the number of elements in an array not happen to exactly match
973 the SIMD group width.
974 * minus: getting data usefully out of registers (if separate regfiles
975 are used) means outputting to memory and back.
976 * minus: quite a lot of supplementary instructions for bit-level manipulation
977 are needed in order to efficiently extract (or prepare) SIMD operands.
978 * minus: MASSIVE proliferation of ISA both in terms of opcodes in one
979 dimension and parallelism (width): an at least O(N^2) and quite probably
980 O(N^3) ISA proliferation that often results in several thousand
981 separate instructions. all requiring separate and distinct corner-case
983 * minus: EVEN BIGGER proliferation of SIMD ISA if the functionality of
984 8, 16, 32 or 64-bit reordering is built-in to the SIMD instruction.
985 For example: add (high|low) 16-bits of r1 to (low|high) of r2 requires
986 four separate and distinct instructions: one for (r1:low r2:high),
987 one for (r1:high r2:low), one for (r1:high r2:high) and one for
988 (r1:low r2:low) *per function*.
989 * minus: EVEN BIGGER proliferation of SIMD ISA if there is a mismatch
990 between operand and result bit-widths. In combination with high/low
991 proliferation the situation is made even worse.
992 * minor-saving-grace: some implementations *may* have predication masks
993 that allow control over individual elements within the SIMD block.
995 ## Comparison *to* Traditional SIMD: Alt-RVP, Simple-V and RVV Proposals <a name="simd_comparison"></a>
997 This section compares the various parallelism proposals as they stand,
998 *against* traditional SIMD as opposed to *alongside* SIMD. In other words,
999 the question is asked "How can each of the proposals effectively implement
1000 (or replace) SIMD, and how effective would they be"?
1004 * Alt-RVP would not actually replace SIMD but would augment it: just as with
1005 a SIMD architecture where the ALU becomes responsible for the parallelism,
1006 Alt-RVP ALUs would likewise be so responsible... with *additional*
1007 (lane-based) parallelism on top.
1008 * Thus at least some of the downsides of SIMD ISA O(N^3) proliferation by
1009 at least one dimension are avoided (architectural upgrades introducing
1010 128-bit then 256-bit then 512-bit variants of the exact same 64-bit
1012 * Thus, unfortunately, Alt-RVP would suffer the same inherent proliferation
1013 of instructions as SIMD, albeit not quite as badly (due to Lanes).
1014 * In the same discussion for Alt-RVP, an additional proposal was made to
1015 be able to subdivide the bits of each register lane (columns) down into
1016 arbitrary bit-lengths (RGB 565 for example).
1017 * A recommendation was given instead to make the subdivisions down to 32-bit,
1018 16-bit or even 8-bit, effectively dividing the registerfile into
1019 Lane0(H), Lane0(L), Lane1(H) ... LaneN(L) or further. If inter-lane
1020 "swapping" instructions were then introduced, some of the disadvantages
1021 of SIMD could be mitigated.
1025 * RVV is designed to replace SIMD with a better paradigm: arbitrary-length
1027 * However whilst SIMD is usually designed for single-issue in-order simple
1028 DSPs with a focus on Multimedia (Audio, Video and Image processing),
1029 RVV's primary focus appears to be on Supercomputing: optimisation of
1030 mathematical operations that fit into the OpenCL space.
1031 * Adding functions (operations) that would normally fit (in parallel)
1032 into a SIMD instruction requires an equivalent to be added to the
1033 RVV Extension, if one does not exist. Given the specialist nature of
1034 some SIMD instructions (8-bit or 16-bit saturated or halving add),
1035 this possibility seems extremely unlikely to occur, even if the
1036 implementation overhead of RVV were acceptable (compared to
1037 normal SIMD/DSP-style single-issue in-order simplicity).
1041 * Simple-V borrows hugely from RVV as it is intended to be easy to
1042 topologically transplant every single instruction from RVV (as
1043 designed) into Simple-V equivalents, with *zero loss of functionality
1045 * With the "parallelism" abstracted out, a hypothetical SIMD-less "DSP"
1046 Extension which contained the basic primitives (non-parallelised
1047 8, 16 or 32-bit SIMD operations) inherently *become* parallel,
1049 * Additionally, standard operations (ADD, MUL) that would normally have
1050 to have special SIMD-parallel opcodes added need no longer have *any*
1051 of the length-dependent variants (2of 32-bit ADDs in a 64-bit register,
1052 4of 32-bit ADDs in a 128-bit register) because Simple-V takes the
1053 *standard* RV opcodes (present and future) and automatically parallelises
1055 * By inheriting the RVV feature of arbitrary vector-length, then just as
1056 with RVV the corner-cases and ISA proliferation of SIMD is avoided.
1057 * Whilst not entirely finalised, registers are expected to be
1058 capable of being subdivided down to an implementor-chosen bitwidth
1059 in the underlying hardware (r1 becomes r1[31..24] r1[23..16] r1[15..8]
1060 and r1[7..0], or just r1[31..16] r1[15..0]) where implementors can
1061 choose to have separate independent 8-bit ALUs or dual-SIMD 16-bit
1062 ALUs that perform twin 8-bit operations as they see fit, or anything
1063 else including no subdivisions at all.
1064 * Even though implementors have that choice even to have full 64-bit
1065 (with RV64) SIMD, they *must* provide predication that transparently
1066 switches off appropriate units on the last loop, thus neatly fitting
1067 underlying SIMD ALU implementations *into* the arbitrary vector-length
1068 RVV paradigm, keeping the uniform consistent API that is a key strategic
1069 feature of Simple-V.
1070 * With Simple-V fitting into the standard register files, certain classes
1071 of SIMD operations such as High/Low arithmetic (r1[31..16] + r2[15..0])
1072 can be done by applying *Parallelised* Bit-manipulation operations
1073 followed by parallelised *straight* versions of element-to-element
1074 arithmetic operations, even if the bit-manipulation operations require
1075 changing the bitwidth of the "vectors" to do so. Predication can
1076 be utilised to skip high words (or low words) in source or destination.
1077 * In essence, the key downside of SIMD - massive duplication of
1078 identical functions over time as an architecture evolves from 32-bit
1079 wide SIMD all the way up to 512-bit, is avoided with Simple-V, through
1080 vector-style parallelism being dropped on top of 8-bit or 16-bit
1081 operations, all the while keeping a consistent ISA-level "API" irrespective
1082 of implementor design choices (or indeed actual implementations).
1084 ### Example Instruction translation: <a name="example_translation"></a>
1086 Instructions "ADD r2 r4 r4" would result in three instructions being
1087 generated and placed into the FIFO:
1093 ## Example of vector / vector, vector / scalar, scalar / scalar => vector add
1095 register CSRvectorlen[XLEN][4]; # not quite decided yet about this one...
1096 register CSRpredicate[XLEN][4]; # 2^4 is max vector length
1097 register CSRreg_is_vectorised[XLEN]; # just for fun support scalars as well
1098 register x[32][XLEN];
1100 function op_add(rd, rs1, rs2, predr)
1102 /* note that this is ADD, not PADD */
1103 int i, id, irs1, irs2;
1104 # checks CSRvectorlen[rd] == CSRvectorlen[rs] etc. ignored
1105 # also destination makes no sense as a scalar but what the hell...
1106 for (i = 0, id=0, irs1=0, irs2=0; i<CSRvectorlen[rd]; i++)
1107 if (CSRpredicate[predr][i]) # i *think* this is right...
1108 x[rd+id] <= x[rs1+irs1] + x[rs2+irs2];
1109 # now increment the idxs
1110 if (CSRreg_is_vectorised[rd]) # bitfield check rd, scalar/vector?
1112 if (CSRreg_is_vectorised[rs1]) # bitfield check rs1, scalar/vector?
1114 if (CSRreg_is_vectorised[rs2]) # bitfield check rs2, scalar/vector?
1118 ## Retro-fitting Predication into branch-explicit ISA <a name="predication_retrofit"></a>
1120 One of the goals of this parallelism proposal is to avoid instruction
1121 duplication. However, with the base ISA having been designed explictly
1122 to *avoid* condition-codes entirely, shoe-horning predication into it
1123 bcomes quite challenging.
1125 However what if all branch instructions, if referencing a vectorised
1126 register, were instead given *completely new analogous meanings* that
1127 resulted in a parallel bit-wise predication register being set? This
1128 would have to be done for both C.BEQZ and C.BNEZ, as well as BEQ, BNE,
1131 We might imagine that FEQ, FLT and FLT would also need to be converted,
1132 however these are effectively *already* in the precise form needed and
1133 do not need to be converted *at all*! The difference is that FEQ, FLT
1134 and FLE *specifically* write a 1 to an integer register if the condition
1135 holds, and 0 if not. All that needs to be done here is to say, "if
1136 the integer register is tagged with a bit that says it is a predication
1137 register, the **bit** in the integer register is set based on the
1138 current vector index" instead.
1140 There is, in the standard Conditional Branch instruction, more than
1141 adequate space to interpret it in a similar fashion:
1144 31 |30 ..... 25 |24 ... 20 | 19 ... 15 | 14 ...... 12 | 11 ....... 8 | 7 | 6 ....... 0 |
1145 imm[12] | imm[10:5] | rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
1146 1 | 6 | 5 | 5 | 3 | 4 | 1 | 7 |
1147 offset[12,10:5] || src2 | src1 | BEQ | offset[11,4:1] || BRANCH |
1153 31 | 30 .. 25 |24 ... 20 | 19 15 | 14 12 | 11 .. 8 | 7 | 6 ... 0 |
1154 imm[12] | imm[10:5]| rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
1155 1 | 6 | 5 | 5 | 3 | 4 | 1 | 7 |
1156 reserved || src2 | src1 | BEQ | predicate rs3 || BRANCH |
1159 Similarly the C.BEQZ and C.BNEZ instruction format may be retro-fitted,
1160 with the interesting side-effect that there is space within what is presently
1161 the "immediate offset" field to reinterpret that to add in not only a bit
1162 field to distinguish between floating-point compare and integer compare,
1163 not only to add in a second source register, but also use some of the bits as
1164 a predication target as well.
1167 15 ...... 13 | 12 ........... 10 | 9..... 7 | 6 ................. 2 | 1 .. 0 |
1168 funct3 | imm | rs10 | imm | op |
1170 C.BEQZ | offset[8,4:3] | src | offset[7:6,2:1,5] | C1 |
1173 Now uses the CS format:
1176 15 ...... 13 | 12 ........... 10 | 9..... 7 | 6 .. 5 | 4......... 2 | 1 .. 0 |
1177 funct3 | imm | rs10 | imm | | op |
1178 3 | 3 | 3 | 2 | 3 | 2 |
1179 C.BEQZ | predicate rs3 | src1 | I/F B | src2 | C1 |
1182 Bit 6 would be decoded as "operation refers to Integer or Float" including
1183 interpreting src1 and src2 accordingly as outlined in Table 12.2 of the
1184 "C" Standard, version 2.0,
1185 whilst Bit 5 would allow the operation to be extended, in combination with
1186 funct3 = 110 or 111: a combination of four distinct (predicated) comparison
1187 operators. In both floating-point and integer cases those could be
1188 EQ/NEQ/LT/LE (with GT and GE being synthesised by inverting src1 and src2).
1190 ## Register reordering <a name="register_reordering"></a>
1209 May not be an actual CSR: may be generated from Vector Length CSR:
1210 single-bit is less burdensome on instruction decode phase.
1212 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
1213 | - | - | - | - | - | - | - | - |
1214 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
1216 ### Vector Length CSR
1218 | Reg Num | (3..0) |
1229 ### Virtual Register Reordering
1231 This example assumes the above Vector Length CSR table
1233 | Reg Num | Bits (0) | Bits (1) | Bits (2) |
1234 | ------- | -------- | -------- | -------- |
1235 | r0 | (32..0) | (32..0) |
1238 | r4 | (32..0) | (32..0) | (32..0) |
1241 ### Bitwidth Virtual Register Reordering
1243 This example goes a little further and illustrates the effect that a
1244 bitwidth CSR has been set on a register. Preconditions:
1247 * CSRintbitwidth[2] = 010 # integer r2 is 16-bit
1248 * CSRintvlength[2] = 3 # integer r2 is a vector of length 3
1249 * vsetl rs1, 5 # set the vector length to 5
1251 This is interpreted as follows:
1253 * Given that the context is RV32, ELEN=32.
1254 * With ELEN=32 and bitwidth=16, the number of SIMD elements is 2
1255 * Therefore the actual vector length is up to *six* elements
1256 * However vsetl sets a length 5 therefore the last "element" is skipped
1258 So when using an operation that uses r2 as a source (or destination)
1259 the operation is carried out as follows:
1261 * 16-bit operation on r2(15..0) - vector element index 0
1262 * 16-bit operation on r2(31..16) - vector element index 1
1263 * 16-bit operation on r3(15..0) - vector element index 2
1264 * 16-bit operation on r3(31..16) - vector element index 3
1265 * 16-bit operation on r4(15..0) - vector element index 4
1266 * 16-bit operation on r4(31..16) **NOT** carried out due to length being 5
1268 Predication has been left out of the above example for simplicity, however
1269 predication is ANDed with the latter stages (vsetl not equal to maximum
1272 Note also that it is entirely an implementor's choice as to whether to have
1273 actual separate ALUs down to the minimum bitwidth, or whether to have something
1274 more akin to traditional SIMD (at any level of subdivision: 8-bit SIMD
1275 operations carried out 32-bits at a time is perfectly acceptable, as is
1276 8-bit SIMD operations carried out 16-bits at a time requiring two ALUs).
1277 Regardless of the internal parallelism choice, *predication must
1278 still be respected*, making Simple-V in effect the "consistent public API".
1280 vew may be one of the following (giving a table "bytestable", used below):
1293 Pseudocode for vector length taking CSR SIMD-bitwidth into account:
1295 vew = CSRbitwidth[rs1]
1297 bytesperreg = (XLEN/8) # or FLEN as appropriate
1299 bytesperreg = bytestable[vew] # 1 2 4 8 16
1300 simdmult = (XLEN/8) / bytesperreg # or FLEN as appropriate
1301 vlen = CSRvectorlen[rs1] * simdmult
1303 To index an element in a register rnum where the vector element index is i:
1305 function regoffs(rnum, i):
1306 regidx = floor(i / simdmult) # integer-div rounded down
1307 byteidx = i % simdmult # integer-remainder
1308 return rnum + regidx, # actual real register
1310 byteidx * 8 + (vew-1), # high
1314 SIMD register file splitting still to consider. For RV64, benefits of doubling
1315 (quadrupling in the case of Half-Precision IEEE754 FP) the apparent
1316 size of the floating point register file to 64 (128 in the case of HP)
1317 seem pretty clear and worth the complexity.
1319 64 virtual 32-bit F.P. registers and given that 32-bit FP operations are
1320 done on 64-bit registers it's not so conceptually difficult. May even
1321 be achieved by *actually* splitting the regfile into 64 virtual 32-bit
1322 registers such that a 64-bit FP scalar operation is dropped into (r0.H
1323 r0.L) tuples. Implementation therefore hidden through register renaming.
1325 Implementations intending to introduce VLIW, OoO and parallelism
1326 (even without Simple-V) would then find that the instructions are
1327 generated quicker (or in a more compact fashion that is less heavy
1328 on caches). Interestingly we observe then that Simple-V is about
1329 "consolidation of instruction generation", where actual parallelism
1330 of underlying hardware is an implementor-choice that could just as
1331 equally be applied *without* Simple-V even being implemented.
1333 ## Analysis of CSR decoding on latency <a name="csr_decoding_analysis"></a>
1335 It could indeed have been logically deduced (or expected), that there
1336 would be additional decode latency in this proposal, because if
1337 overloading the opcodes to have different meanings, there is guaranteed
1338 to be some state, some-where, directly related to registers.
1340 There are several cases:
1342 * All operands vector-length=1 (scalars), all operands
1343 packed-bitwidth="default": instructions are passed through direct as if
1344 Simple-V did not exist. Simple-V is, in effect, completely disabled.
1345 * At least one operand vector-length > 1, all operands
1346 packed-bitwidth="default": any parallel vector ALUs placed on "alert",
1347 virtual parallelism looping may be activated.
1348 * All operands vector-length=1 (scalars), at least one
1349 operand packed-bitwidth != default: degenerate case of SIMD,
1350 implementation-specific complexity here (packed decode before ALUs or
1352 * At least one operand vector-length > 1, at least one operand
1353 packed-bitwidth != default: parallel vector ALUs (if any)
1354 placed on "alert", virtual parallelsim looping may be activated,
1355 implementation-specific SIMD complexity kicks in (packed decode before
1358 Bear in mind that the proposal includes that the decision whether
1359 to parallelise in hardware or whether to virtual-parallelise (to
1360 dramatically simplify compilers and also not to run into the SIMD
1361 instruction proliferation nightmare) *or* a transprent combination
1362 of both, be done on a *per-operand basis*, so that implementors can
1363 specifically choose to create an application-optimised implementation
1364 that they believe (or know) will sell extremely well, without having
1365 "Extra Standards-Mandated Baggage" that would otherwise blow their area
1366 or power budget completely out the window.
1368 Additionally, two possible CSR schemes have been proposed, in order to
1369 greatly reduce CSR space:
1371 * per-register CSRs (vector-length and packed-bitwidth)
1372 * a smaller number of CSRs with the same information but with an *INDEX*
1373 specifying WHICH register in one of three regfiles (vector, fp, int)
1374 the length and bitwidth applies to.
1376 (See "CSR vector-length and CSR SIMD packed-bitwidth" section for details)
1378 In addition, LOAD/STORE has its own associated proposed CSRs that
1379 mirror the STRIDE (but not yet STRIDE-SEGMENT?) functionality of
1382 Also bear in mind that, for reasons of simplicity for implementors,
1383 I was coming round to the idea of permitting implementors to choose
1384 exactly which bitwidths they would like to support in hardware and which
1385 to allow to fall through to software-trap emulation.
1387 So the question boils down to:
1389 * whether either (or both) of those two CSR schemes have significant
1390 latency that could even potentially require an extra pipeline decode stage
1391 * whether there are implementations that can be thought of which do *not*
1392 introduce significant latency
1393 * whether it is possible to explicitly (through quite simply
1394 disabling Simple-V-Ext) or implicitly (detect the case all-vlens=1,
1395 all-simd-bitwidths=default) switch OFF any decoding, perhaps even to
1396 the extreme of skipping an entire pipeline stage (if one is needed)
1397 * whether packed bitwidth and associated regfile splitting is so complex
1398 that it should definitely, definitely be made mandatory that implementors
1399 move regfile splitting into the ALU, and what are the implications of that
1400 * whether even if that *is* made mandatory, is software-trapped
1401 "unsupported bitwidths" still desirable, on the basis that SIMD is such
1402 a complete nightmare that *even* having a software implementation is
1403 better, making Simple-V have more in common with a software API than
1406 Whilst the above may seem to be severe minuses, there are some strong
1409 * Significant reduction of V's opcode space: over 85%.
1410 * Smaller reduction of P's opcode space: around 10%.
1411 * The potential to use Compressed instructions in both Vector and SIMD
1412 due to the overloading of register meaning (implicit vectorisation,
1414 * Not only present but also future extensions automatically gain parallelism.
1415 * Already mentioned but worth emphasising: the simplification to compiler
1416 writers and assembly-level writers of having the same consistent ISA
1417 regardless of whether the internal level of parallelism (number of
1418 parallel ALUs) is only equal to one ("virtual" parallelism), or is
1419 greater than one, should not be underestimated.
1421 ## Reducing Register Bank porting
1423 This looks quite reasonable.
1424 <https://www.princeton.edu/~rblee/ELE572Papers/MultiBankRegFile_ISCA2000.pdf>
1426 The main details are outlined on page 4. They propose a 2-level register
1427 cache hierarchy, note that registers are typically only read once, that
1428 you never write back from upper to lower cache level but always go in a
1429 cycle lower -> upper -> ALU -> lower, and at the top of page 5 propose
1430 a scheme where you look ahead by only 2 instructions to determine which
1431 registers to bring into the cache.
1433 The nice thing about a vector architecture is that you *know* that
1434 *even more* registers are going to be pulled in: Hwacha uses this fact
1435 to optimise L1/L2 cache-line usage (avoid thrashing), strangely enough
1436 by *introducing* deliberate latency into the execution phase.
1438 ## Overflow registers in combination with predication
1440 **TODO**: propose overflow registers be actually one of the integer regs
1441 (flowing to multiple regs).
1443 **TODO**: propose "mask" (predication) registers likewise. combination with
1444 standard RV instructions and overflow registers extremely powerful, see
1447 When integer overflow is stored in an easily-accessible bit (or another
1448 register), parallelisation turns this into a group of bits which can
1449 potentially be interacted with in predication, in interesting and powerful
1450 ways. For example, by taking the integer-overflow result as a predication
1451 field and shifting it by one, a predicated vectorised "add one" can emulate
1452 "carry" on arbitrary (unlimited) length addition.
1454 However despite RVV having made room for floating-point exceptions, neither
1455 RVV nor base RV have taken integer-overflow (carry) into account, which
1456 makes proposing it quite challenging given that the relevant (Base) RV
1457 sections are frozen. Consequently it makes sense to forgo this feature.
1459 ## Virtual Memory page-faults on LOAD/STORE
1462 ### Notes from conversations
1464 > I was going through the C.LOAD / C.STORE section 12.3 of V2.3-Draft
1465 > riscv-isa-manual in order to work out how to re-map RVV onto the standard
1466 > ISA, and came across an interesting comments at the bottom of pages 75
1469 > " A common mechanism used in other ISAs to further reduce save/restore
1470 > code size is load- multiple and store-multiple instructions. "
1472 > Fascinatingly, due to Simple-V proposing to use the *standard* register
1473 > file, both C.LOAD / C.STORE *and* LOAD / STORE would in effect be exactly
1474 > that: load-multiple and store-multiple instructions. Which brings us
1475 > on to this comment:
1477 > "For virtual memory systems, some data accesses could be resident in
1478 > physical memory and
1479 > some could not, which requires a new restart mechanism for partially
1480 > executed instructions."
1482 > Which then of course brings us to the interesting question: how does RVV
1483 > cope with the scenario when, particularly with LD.X (Indexed / indirect
1484 > loads), part-way through the loading a page fault occurs?
1486 > Has this been noted or discussed before?
1488 For applications-class platforms, the RVV exception model is
1489 element-precise (that is, if an exception occurs on element j of a
1490 vector instruction, elements 0..j-1 have completed execution and elements
1491 j+1..vl-1 have not executed).
1493 Certain classes of embedded platforms where exceptions are always fatal
1494 might choose to offer resumable/swappable interrupts but not precise
1498 > Is RVV designed in any way to be re-entrant?
1503 > What would the implications be for instructions that were in a FIFO at
1504 > the time, in out-of-order and VLIW implementations, where partial decode
1507 The usual bag of tricks for maintaining precise exceptions applies to
1508 vector machines as well. Register renaming makes the job easier, and
1509 it's relatively cheaper for vectors, since the control cost is amortized
1510 over longer registers.
1513 > Would it be reasonable at least to say *bypass* (and freeze) the
1514 > instruction FIFO (drop down to a single-issue execution model temporarily)
1515 > for the purposes of executing the instructions in the interrupt (whilst
1516 > setting up the VM page), then re-continue the instruction with all
1519 This approach has been done successfully, but it's desirable to be
1520 able to swap out the vector unit state to support context switches on
1521 exceptions that result in long-latency I/O.
1524 > Or would it be better to switch to an entirely separate secondary
1525 > hyperthread context?
1527 > Does anyone have any ideas or know if there is any academic literature
1528 > on solutions to this problem?
1530 The Vector VAX offered imprecise but restartable and swappable exceptions:
1531 http://mprc.pku.edu.cn/~liuxianhua/chn/corpus/Notes/articles/isca/1990/VAX%20vector%20architecture.pdf
1533 Sec. 4.6 of Krste's dissertation assesses some of
1534 the tradeoffs and references a bunch of related work:
1535 http://people.eecs.berkeley.edu/~krste/thesis.pdf
1540 Started reading section 4.6 of Krste's thesis, noted the "IEE85 F.P
1541 exceptions" and thought, "hmmm that could go into a CSR, must re-read
1542 the section on FP state CSRs in RVV 0.4-Draft again" then i suddenly
1543 thought, "ah ha! what if the memory exceptions were, instead of having
1544 an immediate exception thrown, were simply stored in a type of predication
1545 bit-field with a flag "error this element failed"?
1547 Then, *after* the vector load (or store, or even operation) was
1548 performed, you could *then* raise an exception, at which point it
1549 would be possible (yes in software... I know....) to go "hmmm, these
1550 indexed operations didn't work, let's get them into memory by triggering
1551 page-loads", then *re-run the entire instruction* but this time with a
1552 "memory-predication CSR" that stops the already-performed operations
1553 (whether they be loads, stores or an arithmetic / FP operation) from
1554 being carried out a second time.
1556 This theoretically could end up being done multiple times in an SMP
1557 environment, and also for LD.X there would be the remote outside annoying
1558 possibility that the indexed memory address could end up being modified.
1560 The advantage would be that the order of execution need not be
1561 sequential, which potentially could have some big advantages.
1562 Am still thinking through the implications as any dependent operations
1563 (particularly ones already decoded and moved into the execution FIFO)
1564 would still be there (and stalled). hmmm.
1568 > > # assume internal parallelism of 8 and MAXVECTORLEN of 8
1573 > > x3[1]: exception
1579 > > what happens to result elements 2-7? those may be *big* results
1581 > > or in the RVV-Extended may be arbitrary bit-widths far greater.
1587 discussion then led to the question of OoO architectures
1589 > The costs of the imprecise-exception model are greater than the benefit.
1590 > Software doesn't want to cope with it. It's hard to debug. You can't
1591 > migrate state between different microarchitectures--unless you force all
1592 > implementations to support the same imprecise-exception model, which would
1593 > greatly limit implementation flexibility. (Less important, but still
1594 > relevant, is that the imprecise model increases the size of the context
1595 > structure, as the microarchitectural guts have to be spilled to memory.)
1598 ## Implementation Paradigms
1600 TODO: assess various implementation paradigms. These are listed roughly
1601 in order of simplicity (minimum compliance, for ultra-light-weight
1602 embedded systems or to reduce design complexity and the burden of
1603 design implementation and compliance, in non-critical areas), right the
1604 way to high-performance systems.
1606 * Full (or partial) software-emulated (via traps): full support for CSRs
1607 required, however when a register is used that is detected (in hardware)
1608 to be vectorised, an exception is thrown.
1609 * Single-issue In-order, reduced pipeline depth (traditional SIMD / DSP)
1610 * In-order 5+ stage pipelines with instruction FIFOs and mild register-renaming
1611 * Out-of-order with instruction FIFOs and aggressive register-renaming
1614 Also to be taken into consideration:
1616 * "Virtual" vectorisation: single-issue loop, no internal ALU parallelism
1617 * Comphrensive vectorisation: FIFOs and internal parallelism
1618 * Hybrid Parallelism
1622 > For great floating point DSPs check TI’s C3x, C4X, and C6xx DSPs
1624 Idea: basic simple butterfly swap on a few element indices, primarily targetted
1625 at SIMD / DSP. High-byte low-byte swapping, high-word low-word swapping,
1626 perhaps allow reindexing of permutations up to 4 elements? 8? Reason:
1627 such operations are less costly than a full indexed-shuffle, which requires
1628 a separate instruction cycle.
1630 Predication "all zeros" needs to be "leave alone". Detection of
1631 ADD r1, rs1, rs0 cases result in nop on predication index 0, whereas
1632 ADD r0, rs1, rs2 is actually a desirable copy from r2 into r0.
1633 Destruction of destination indices requires a copy of the entire vector
1634 in advance to avoid.
1638 * SIMD considered harmful <https://www.sigarch.org/simd-instructions-considered-harmful/>
1639 * Link to first proposal <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/GuukrSjgBH8>
1640 * Recommendation by Jacob Bachmeyer to make zero-overhead loop an
1641 "implicit program-counter" <https://groups.google.com/a/groups.riscv.org/d/msg/isa-dev/vYVi95gF2Mo/SHz6a4_lAgAJ>
1642 * Re-continuing P-Extension proposal <https://groups.google.com/a/groups.riscv.org/forum/#!msg/isa-dev/IkLkQn3HvXQ/SEMyC9IlAgAJ>
1643 * First Draft P-SIMD (DSP) proposal <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/vYVi95gF2Mo>
1644 * B-Extension discussion <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/zi_7B15kj6s>
1645 * Broadcom VideoCore-IV <https://docs.broadcom.com/docs/12358545>
1646 Figure 2 P17 and Section 3 on P16.
1647 * Hwacha <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-262.html>
1648 * Hwacha <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-263.html>
1649 * Vector Workshop <http://riscv.org/wp-content/uploads/2015/06/riscv-vector-workshop-june2015.pdf>
1650 * Predication <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/XoP4BfYSLXA>
1651 * Branch Divergence <https://jbush001.github.io/2014/12/07/branch-divergence-in-parallel-kernels.html>
1652 * Life of Triangles (3D) <https://jbush001.github.io/2016/02/27/life-of-triangle.html>
1653 * Videocore-IV <https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV-3d-Graphics-Pipeline>
1654 * Discussion proposing CSRs that change ISA definition
1655 <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/InzQ1wr_3Ak>
1656 * Zero-overhead loops <https://pdfs.semanticscholar.org/dbaa/66985cc730d4b44d79f519e96ec9c43ab5b7.pdf>
1657 * Multi-ported VLIW Register File Implementation <https://ce-publications.et.tudelft.nl/publications/1517_multiple_contexts_in_a_multiported_vliw_register_file_impl.pdf>
1658 * Fast context save/restore proposal <https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/57F823FA.6030701%40gmail.com>
1659 * Register File Bank Cacheing <https://www.princeton.edu/~rblee/ELE572Papers/MultiBankRegFile_ISCA2000.pdf>
1660 * Expired Patent on Vector Virtual Memory solutions
1661 <https://patentimages.storage.googleapis.com/fc/f6/e2/2cbee92fcd8743/US5895501.pdf>
1662 * Discussion on RVV "re-entrant" capabilities allowing operations to be
1663 restarted if an exception occurs (VM page-table miss)
1664 <https://groups.google.com/a/groups.riscv.org/d/msg/isa-dev/IuNFitTw9fM/CCKBUlzsAAAJ>
1665 * Dot Product Vector <https://people.eecs.berkeley.edu/~biancolin/papers/arith17.pdf>