src/add/singlepipe.py

   1 """ Pipeline and BufferedHandshake implementation, conforming to the same API.
   2     For multi-input and multi-output variants, see multipipe.
   3
   4     Associated development bugs:
   5     * http://bugs.libre-riscv.org/show_bug.cgi?id=64
   6     * http://bugs.libre-riscv.org/show_bug.cgi?id=57
   7
   8     eq:
   9     --
  10
  11     a strategically very important function that is identical in function
  12     to nmigen's Signal.eq function, except it may take objects, or a list
  13     of objects, or a tuple of objects, and where objects may also be
  14     Records.
  15
  16     Stage API:
  17     ---------
  18
  19     stage requires compliance with a strict API that may be
  20     implemented in several means, including as a static class.
  21     the methods of a stage instance must be as follows:
  22
  23     * ispec() - Input data format specification
  24                 returns an object or a list or tuple of objects, or
  25                 a Record, each object having an "eq" function which
  26                 takes responsibility for copying by assignment all
  27                 sub-objects
  28     * ospec() - Output data format specification
  29                 requirements as for ospec
  30     * process(m, i) - Processes an ispec-formatted object
  31                 returns a combinatorial block of a result that
  32                 may be assigned to the output, by way of the "eq"
  33                 function
  34     * setup(m, i) - Optional function for setting up submodules
  35                 may be used for more complex stages, to link
  36                 the input (i) to submodules.  must take responsibility
  37                 for adding those submodules to the module (m).
  38                 the submodules must be combinatorial blocks and
  39                 must have their inputs and output linked combinatorially.
  40
  41     Both StageCls (for use with non-static classes) and Stage (for use
  42     by static classes) are abstract classes from which, for convenience
  43     and as a courtesy to other developers, anything conforming to the
  44     Stage API may *choose* to derive.
  45
  46     StageChain:
  47     ----------
  48
  49     A useful combinatorial wrapper around stages that chains them together
  50     and then presents a Stage-API-conformant interface.  By presenting
  51     the same API as the stages it wraps, it can clearly be used recursively.
  52
  53     RecordBasedStage:
  54     ----------------
  55
  56     A convenience class that takes an input shape, output shape, a
  57     "processing" function and an optional "setup" function.  Honestly
  58     though, there's not much more effort to just... create a class
  59     that returns a couple of Records (see ExampleAddRecordStage in
  60     examples).
  61
  62     PassThroughStage:
  63     ----------------
  64
  65     A convenience class that takes a single function as a parameter,
  66     that is chain-called to create the exact same input and output spec.
  67     It has a process() function that simply returns its input.
  68
  69     Instances of this class are completely redundant if handed to
  70     StageChain, however when passed to UnbufferedPipeline they
  71     can be used to introduce a single clock delay.
  72
  73     ControlBase:
  74     -----------
  75
  76     The base class for pipelines.  Contains previous and next ready/valid/data.
  77     Also has an extremely useful "connect" function that can be used to
  78     connect a chain of pipelines and present the exact same prev/next
  79     ready/valid/data API.
  80
  81     UnbufferedPipeline:
  82     ------------------
  83
  84     A simple stalling clock-synchronised pipeline that has no buffering
  85     (unlike BufferedHandshake).  Data flows on *every* clock cycle when
  86     the conditions are right (this is nominally when the input is valid
  87     and the output is ready).
  88
  89     A stall anywhere along the line will result in a stall back-propagating
  90     down the entire chain.  The BufferedHandshake by contrast will buffer
  91     incoming data, allowing previous stages one clock cycle's grace before
  92     also having to stall.
  93
  94     An advantage of the UnbufferedPipeline over the Buffered one is
  95     that the amount of logic needed (number of gates) is greatly
  96     reduced (no second set of buffers basically)
  97
  98     The disadvantage of the UnbufferedPipeline is that the valid/ready
  99     logic, if chained together, is *combinatorial*, resulting in
 100     progressively larger gate delay.
 101
 102     PassThroughHandshake:
 103     ------------------
 104
 105     A Control class that introduces a single clock delay, passing its
 106     data through unaltered.  Unlike RegisterPipeline (which relies
 107     on UnbufferedPipeline and PassThroughStage) it handles ready/valid
 108     itself.
 109
 110     RegisterPipeline:
 111     ----------------
 112
 113     A convenience class that, because UnbufferedPipeline introduces a single
 114     clock delay, when its stage is a PassThroughStage, it results in a Pipeline
 115     stage that, duh, delays its (unmodified) input by one clock cycle.
 116
 117     BufferedHandshake:
 118     ----------------
 119
 120     nmigen implementation of buffered pipeline stage, based on zipcpu:
 121     https://zipcpu.com/blog/2017/08/14/strategies-for-pipelining.html
 122
 123     this module requires quite a bit of thought to understand how it works
 124     (and why it is needed in the first place).  reading the above is
 125     *strongly* recommended.
 126
 127     unlike john dawson's IEEE754 FPU STB/ACK signalling, which requires
 128     the STB / ACK signals to raise and lower (on separate clocks) before
 129     data may proceeed (thus only allowing one piece of data to proceed
 130     on *ALTERNATE* cycles), the signalling here is a true pipeline
 131     where data will flow on *every* clock when the conditions are right.
 132
 133     input acceptance conditions are when:
 134         * incoming previous-stage strobe (p.i_valid) is HIGH
 135         * outgoing previous-stage ready   (p.o_ready) is LOW
 136
 137     output transmission conditions are when:
 138         * outgoing next-stage strobe (n.o_valid) is HIGH
 139         * outgoing next-stage ready   (n.i_ready) is LOW
 140
 141     the tricky bit is when the input has valid data and the output is not
 142     ready to accept it.  if it wasn't for the clock synchronisation, it
 143     would be possible to tell the input "hey don't send that data, we're
 144     not ready".  unfortunately, it's not possible to "change the past":
 145     the previous stage *has no choice* but to pass on its data.
 146
 147     therefore, the incoming data *must* be accepted - and stored: that
 148     is the responsibility / contract that this stage *must* accept.
 149     on the same clock, it's possible to tell the input that it must
 150     not send any more data.  this is the "stall" condition.
 151
 152     we now effectively have *two* possible pieces of data to "choose" from:
 153     the buffered data, and the incoming data.  the decision as to which
 154     to process and output is based on whether we are in "stall" or not.
 155     i.e. when the next stage is no longer ready, the output comes from
 156     the buffer if a stall had previously occurred, otherwise it comes
 157     direct from processing the input.
 158
 159     this allows us to respect a synchronous "travelling STB" with what
 160     dan calls a "buffered handshake".
 161
 162     it's quite a complex state machine!
 163
 164     SimpleHandshake
 165     ---------------
 166
 167     Synchronised pipeline, Based on:
 168     https://github.com/ZipCPU/dbgbus/blob/master/hexbus/rtl/hbdeword.v
 169 """
 170
 171 from nmigen import Signal, Cat, Const, Mux, Module, Value
 172 from nmigen.cli import verilog, rtlil
 173 from nmigen.lib.fifo import SyncFIFO, SyncFIFOBuffered
 174 from nmigen.hdl.ast import ArrayProxy
 175 from nmigen.hdl.rec import Record, Layout
 176
 177 from abc import ABCMeta, abstractmethod
 178 from collections.abc import Sequence
 179 from queue import Queue
 180
 181
 182 class RecordObject(Record):
 183     def __init__(self, layout=None, name=None):
 184         Record.__init__(self, layout=layout or [], name=None)
 185
 186     def __setattr__(self, k, v):
 187         if k in dir(Record) or "fields" not in self.__dict__:
 188             return object.__setattr__(self, k, v)
 189         self.fields[k] = v
 190         if isinstance(v, Record):
 191             newlayout = {k: (k, v.layout)}
 192         else:
 193             newlayout = {k: (k, v.shape())}
 194         self.layout.fields.update(newlayout)
 195
 196     def __iter__(self):
 197         for x in self.fields.values():
 198             yield x
 199
 200
 201 class PrevControl:
 202     """ contains signals that come *from* the previous stage (both in and out)
 203         * i_valid: previous stage indicating all incoming data is valid.
 204                    may be a multi-bit signal, where all bits are required
 205                    to be asserted to indicate "valid".
 206         * o_ready: output to next stage indicating readiness to accept data
 207         * i_data : an input - added by the user of this class
 208     """
 209
 210     def __init__(self, i_width=1, stage_ctl=False):
 211         self.stage_ctl = stage_ctl
 212         self.i_valid = Signal(i_width, name="p_i_valid") # prev   >>in  self
 213         self._o_ready = Signal(name="p_o_ready") # prev   <<out self
 214         self.i_data = None # XXX MUST BE ADDED BY USER
 215         if stage_ctl:
 216             self.s_o_ready = Signal(name="p_s_o_rdy") # prev   <<out self
 217         self.trigger = Signal(reset_less=True)
 218
 219     @property
 220     def o_ready(self):
 221         """ public-facing API: indicates (externally) that stage is ready
 222         """
 223         if self.stage_ctl:
 224             return self.s_o_ready # set dynamically by stage
 225         return self._o_ready      # return this when not under dynamic control
 226
 227     def _connect_in(self, prev, direct=False, fn=None):
 228         """ internal helper function to connect stage to an input source.
 229             do not use to connect stage-to-stage!
 230         """
 231         i_valid = prev.i_valid if direct else prev.i_valid_test
 232         i_data = fn(prev.i_data) if fn is not None else prev.i_data
 233         return [self.i_valid.eq(i_valid),
 234                 prev.o_ready.eq(self.o_ready),
 235                 eq(self.i_data, i_data),
 236                ]
 237
 238     @property
 239     def i_valid_test(self):
 240         vlen = len(self.i_valid)
 241         if vlen > 1:
 242             # multi-bit case: valid only when i_valid is all 1s
 243             all1s = Const(-1, (len(self.i_valid), False))
 244             i_valid = (self.i_valid == all1s)
 245         else:
 246             # single-bit i_valid case
 247             i_valid = self.i_valid
 248
 249         # when stage indicates not ready, incoming data
 250         # must "appear" to be not ready too
 251         if self.stage_ctl:
 252             i_valid = i_valid & self.s_o_ready
 253
 254         return i_valid
 255
 256     def elaborate(self, platform):
 257         m = Module()
 258         m.d.comb += self.trigger.eq(self.i_valid_test & self.o_ready)
 259         return m
 260
 261     def eq(self, i):
 262         return [self.i_data.eq(i.i_data),
 263                 self.o_ready.eq(i.o_ready),
 264                 self.i_valid.eq(i.i_valid)]
 265
 266     def ports(self):
 267         res = [self.i_valid, self.o_ready]
 268         if hasattr(self.i_data, "ports"):
 269             res += self.i_data.ports()
 270         elif isinstance(self.i_data, Sequence):
 271             res += self.i_data
 272         else:
 273             res.append(self.i_data)
 274         return res
 275
 276
 277 class NextControl:
 278     """ contains the signals that go *to* the next stage (both in and out)
 279         * o_valid: output indicating to next stage that data is valid
 280         * i_ready: input from next stage indicating that it can accept data
 281         * o_data : an output - added by the user of this class
 282     """
 283     def __init__(self, stage_ctl=False):
 284         self.stage_ctl = stage_ctl
 285         self.o_valid = Signal(name="n_o_valid") # self out>>  next
 286         self.i_ready = Signal(name="n_i_ready") # self <<in   next
 287         self.o_data = None # XXX MUST BE ADDED BY USER
 288         #if self.stage_ctl:
 289         self.d_valid = Signal(reset=1) # INTERNAL (data valid)
 290         self.trigger = Signal(reset_less=True)
 291
 292     @property
 293     def i_ready_test(self):
 294         if self.stage_ctl:
 295             return self.i_ready & self.d_valid
 296         return self.i_ready
 297
 298     def connect_to_next(self, nxt):
 299         """ helper function to connect to the next stage data/valid/ready.
 300             data/valid is passed *TO* nxt, and ready comes *IN* from nxt.
 301             use this when connecting stage-to-stage
 302         """
 303         return [nxt.i_valid.eq(self.o_valid),
 304                 self.i_ready.eq(nxt.o_ready),
 305                 eq(nxt.i_data, self.o_data),
 306                ]
 307
 308     def _connect_out(self, nxt, direct=False, fn=None):
 309         """ internal helper function to connect stage to an output source.
 310             do not use to connect stage-to-stage!
 311         """
 312         i_ready = nxt.i_ready if direct else nxt.i_ready_test
 313         o_data = fn(nxt.o_data) if fn is not None else nxt.o_data
 314         return [nxt.o_valid.eq(self.o_valid),
 315                 self.i_ready.eq(i_ready),
 316                 eq(o_data, self.o_data),
 317                ]
 318
 319     def elaborate(self, platform):
 320         m = Module()
 321         m.d.comb += self.trigger.eq(self.i_ready_test & self.o_valid)
 322         return m
 323
 324     def ports(self):
 325         res = [self.i_ready, self.o_valid]
 326         if hasattr(self.o_data, "ports"):
 327             res += self.o_data.ports()
 328         elif isinstance(self.o_data, Sequence):
 329             res += self.o_data
 330         else:
 331             res.append(self.o_data)
 332         return res
 333
 334
 335 class Visitor2:
 336     """ a helper class for iterating twin-argument compound data structures.
 337
 338         Record is a special (unusual, recursive) case, where the input may be
 339         specified as a dictionary (which may contain further dictionaries,
 340         recursively), where the field names of the dictionary must match
 341         the Record's field spec.  Alternatively, an object with the same
 342         member names as the Record may be assigned: it does not have to
 343         *be* a Record.
 344
 345         ArrayProxy is also special-cased, it's a bit messy: whilst ArrayProxy
 346         has an eq function, the object being assigned to it (e.g. a python
 347         object) might not.  despite the *input* having an eq function,
 348         that doesn't help us, because it's the *ArrayProxy* that's being
 349         assigned to.  so.... we cheat.  use the ports() function of the
 350         python object, enumerate them, find out the list of Signals that way,
 351         and assign them.
 352     """
 353     def iterator2(self, o, i):
 354         if isinstance(o, dict):
 355             yield from self.dict_iter2(o, i)
 356
 357         if not isinstance(o, Sequence):
 358             o, i = [o], [i]
 359         for (ao, ai) in zip(o, i):
 360             #print ("visit", fn, ao, ai)
 361             if isinstance(ao, Record):
 362                 yield from self.record_iter2(ao, ai)
 363             elif isinstance(ao, ArrayProxy) and not isinstance(ai, Value):
 364                 yield from self.arrayproxy_iter2(ao, ai)
 365             else:
 366                 yield (ao, ai)
 367
 368     def dict_iter2(self, o, i):
 369         for (k, v) in o.items():
 370             print ("d-iter", v, i[k])
 371             yield (v, i[k])
 372         return res
 373
 374     def _not_quite_working_with_all_unit_tests_record_iter2(self, ao, ai):
 375         print ("record_iter2", ao, ai, type(ao), type(ai))
 376         if isinstance(ai, Value):
 377             if isinstance(ao, Sequence):
 378                 ao, ai = [ao], [ai]
 379             for o, i in zip(ao, ai):
 380                 yield (o, i)
 381             return
 382         for idx, (field_name, field_shape, _) in enumerate(ao.layout):
 383             if isinstance(field_shape, Layout):
 384                 val = ai.fields
 385             else:
 386                 val = ai
 387             if hasattr(val, field_name): # check for attribute
 388                 val = getattr(val, field_name)
 389             else:
 390                 val = val[field_name] # dictionary-style specification
 391             yield from self.iterator2(ao.fields[field_name], val)
 392
 393     def record_iter2(self, ao, ai):
 394         for idx, (field_name, field_shape, _) in enumerate(ao.layout):
 395             if isinstance(field_shape, Layout):
 396                 val = ai.fields
 397             else:
 398                 val = ai
 399             if hasattr(val, field_name): # check for attribute
 400                 val = getattr(val, field_name)
 401             else:
 402                 val = val[field_name] # dictionary-style specification
 403             yield from self.iterator2(ao.fields[field_name], val)
 404
 405     def arrayproxy_iter2(self, ao, ai):
 406         for p in ai.ports():
 407             op = getattr(ao, p.name)
 408             print ("arrayproxy - p", p, p.name)
 409             yield from self.iterator2(op, p)
 410
 411
 412 class Visitor:
 413     """ a helper class for iterating single-argument compound data structures.
 414         similar to Visitor2.
 415     """
 416     def iterate(self, i):
 417         """ iterate a compound structure recursively using yield
 418         """
 419         if not isinstance(i, Sequence):
 420             i = [i]
 421         for ai in i:
 422             print ("iterate", ai)
 423             if isinstance(ai, Record):
 424                 print ("record", list(ai.layout))
 425                 yield from self.record_iter(ai)
 426             elif isinstance(ai, ArrayProxy) and not isinstance(ai, Value):
 427                 yield from self.array_iter(ai)
 428             else:
 429                 yield ai
 430
 431     def record_iter(self, ai):
 432         for idx, (field_name, field_shape, _) in enumerate(ai.layout):
 433             if isinstance(field_shape, Layout):
 434                 val = ai.fields
 435             else:
 436                 val = ai
 437             if hasattr(val, field_name): # check for attribute
 438                 val = getattr(val, field_name)
 439             else:
 440                 val = val[field_name] # dictionary-style specification
 441             print ("recidx", idx, field_name, field_shape, val)
 442             yield from self.iterate(val)
 443
 444     def array_iter(self, ai):
 445         for p in ai.ports():
 446             yield from self.iterate(p)
 447
 448
 449 def eq(o, i):
 450     """ makes signals equal: a helper routine which identifies if it is being
 451         passed a list (or tuple) of objects, or signals, or Records, and calls
 452         the objects' eq function.
 453     """
 454     res = []
 455     for (ao, ai) in Visitor2().iterator2(o, i):
 456         rres = ao.eq(ai)
 457         if not isinstance(rres, Sequence):
 458             rres = [rres]
 459         res += rres
 460     return res
 461
 462
 463 def cat(i):
 464     """ flattens a compound structure recursively using Cat
 465     """
 466     from nmigen.tools import flatten
 467     #res = list(flatten(i)) # works (as of nmigen commit f22106e5) HOWEVER...
 468     res = list(Visitor().iterate(i)) # needed because input may be a sequence
 469     return Cat(*res)
 470
 471
 472 class StageCls(metaclass=ABCMeta):
 473     """ Class-based "Stage" API.  requires instantiation (after derivation)
 474
 475         see "Stage API" above..  Note: python does *not* require derivation
 476         from this class.  All that is required is that the pipelines *have*
 477         the functions listed in this class.  Derivation from this class
 478         is therefore merely a "courtesy" to maintainers.
 479     """
 480     @abstractmethod
 481     def ispec(self): pass       # REQUIRED
 482     @abstractmethod
 483     def ospec(self): pass       # REQUIRED
 484     #@abstractmethod
 485     #def setup(self, m, i): pass # OPTIONAL
 486     @abstractmethod
 487     def process(self, i): pass  # REQUIRED
 488
 489
 490 class Stage(metaclass=ABCMeta):
 491     """ Static "Stage" API.  does not require instantiation (after derivation)
 492
 493         see "Stage API" above.  Note: python does *not* require derivation
 494         from this class.  All that is required is that the pipelines *have*
 495         the functions listed in this class.  Derivation from this class
 496         is therefore merely a "courtesy" to maintainers.
 497     """
 498     @staticmethod
 499     @abstractmethod
 500     def ispec(): pass
 501
 502     @staticmethod
 503     @abstractmethod
 504     def ospec(): pass
 505
 506     #@staticmethod
 507     #@abstractmethod
 508     #def setup(m, i): pass
 509
 510     @staticmethod
 511     @abstractmethod
 512     def process(i): pass
 513
 514
 515 class RecordBasedStage(Stage):
 516     """ convenience class which provides a Records-based layout.
 517         honestly it's a lot easier just to create a direct Records-based
 518         class (see ExampleAddRecordStage)
 519     """
 520     def __init__(self, in_shape, out_shape, processfn, setupfn=None):
 521         self.in_shape = in_shape
 522         self.out_shape = out_shape
 523         self.__process = processfn
 524         self.__setup = setupfn
 525     def ispec(self): return Record(self.in_shape)
 526     def ospec(self): return Record(self.out_shape)
 527     def process(seif, i): return self.__process(i)
 528     def setup(seif, m, i): return self.__setup(m, i)
 529
 530
 531 class StageChain(StageCls):
 532     """ pass in a list of stages, and they will automatically be
 533         chained together via their input and output specs into a
 534         combinatorial chain.
 535
 536         the end result basically conforms to the exact same Stage API.
 537
 538         * input to this class will be the input of the first stage
 539         * output of first stage goes into input of second
 540         * output of second goes into input into third (etc. etc.)
 541         * the output of this class will be the output of the last stage
 542     """
 543     def __init__(self, chain, specallocate=False):
 544         self.chain = chain
 545         self.specallocate = specallocate
 546
 547     def ispec(self):
 548         return self.chain[0].ispec()
 549
 550     def ospec(self):
 551         return self.chain[-1].ospec()
 552
 553     def _specallocate_setup(self, m, i):
 554         for (idx, c) in enumerate(self.chain):
 555             if hasattr(c, "setup"):
 556                 c.setup(m, i)               # stage may have some module stuff
 557             o = self.chain[idx].ospec()     # last assignment survives
 558             m.d.comb += eq(o, c.process(i)) # process input into "o"
 559             if idx == len(self.chain)-1:
 560                 break
 561             i = self.chain[idx+1].ispec()   # new input on next loop
 562             m.d.comb += eq(i, o)            # assign to next input
 563         return o                            # last loop is the output
 564
 565     def _noallocate_setup(self, m, i):
 566         for (idx, c) in enumerate(self.chain):
 567             if hasattr(c, "setup"):
 568                 c.setup(m, i)               # stage may have some module stuff
 569             i = o = c.process(i)            # store input into "o"
 570         return o                            # last loop is the output
 571
 572     def setup(self, m, i):
 573         if self.specallocate:
 574             self.o = self._specallocate_setup(m, i)
 575         else:
 576             self.o = self._noallocate_setup(m, i)
 577
 578     def process(self, i):
 579         return self.o # conform to Stage API: return last-loop output
 580
 581
 582 class ControlBase:
 583     """ Common functions for Pipeline API
 584     """
 585     def __init__(self, stage=None, in_multi=None, stage_ctl=False):
 586         """ Base class containing ready/valid/data to previous and next stages
 587
 588             * p: contains ready/valid to the previous stage
 589             * n: contains ready/valid to the next stage
 590
 591             Except when calling Controlbase.connect(), user must also:
 592             * add i_data member to PrevControl (p) and
 593             * add o_data member to NextControl (n)
 594         """
 595         self.stage = stage
 596
 597         # set up input and output IO ACK (prev/next ready/valid)
 598         self.p = PrevControl(in_multi, stage_ctl)
 599         self.n = NextControl(stage_ctl)
 600
 601         # set up the input and output data
 602         if stage is not None:
 603             self.p.i_data = stage.ispec() # input type
 604             self.n.o_data = stage.ospec()
 605
 606     def connect_to_next(self, nxt):
 607         """ helper function to connect to the next stage data/valid/ready.
 608         """
 609         return self.n.connect_to_next(nxt.p)
 610
 611     def _connect_in(self, prev):
 612         """ internal helper function to connect stage to an input source.
 613             do not use to connect stage-to-stage!
 614         """
 615         return self.p._connect_in(prev.p)
 616
 617     def _connect_out(self, nxt):
 618         """ internal helper function to connect stage to an output source.
 619             do not use to connect stage-to-stage!
 620         """
 621         return self.n._connect_out(nxt.n)
 622
 623     def connect(self, pipechain):
 624         """ connects a chain (list) of Pipeline instances together and
 625             links them to this ControlBase instance:
 626
 627                       in <----> self <---> out
 628                        |                   ^
 629                        v                   |
 630                     [pipe1, pipe2, pipe3, pipe4]
 631                        |    ^  |    ^  |     ^
 632                        v    |  v    |  v     |
 633                      out---in out--in out---in
 634
 635             Also takes care of allocating i_data/o_data, by looking up
 636             the data spec for each end of the pipechain.  i.e It is NOT
 637             necessary to allocate self.p.i_data or self.n.o_data manually:
 638             this is handled AUTOMATICALLY, here.
 639
 640             Basically this function is the direct equivalent of StageChain,
 641             except that unlike StageChain, the Pipeline logic is followed.
 642
 643             Just as StageChain presents an object that conforms to the
 644             Stage API from a list of objects that also conform to the
 645             Stage API, an object that calls this Pipeline connect function
 646             has the exact same pipeline API as the list of pipline objects
 647             it is called with.
 648
 649             Thus it becomes possible to build up larger chains recursively.
 650             More complex chains (multi-input, multi-output) will have to be
 651             done manually.
 652         """
 653         eqs = [] # collated list of assignment statements
 654
 655         # connect inter-chain
 656         for i in range(len(pipechain)-1):
 657             pipe1 = pipechain[i]
 658             pipe2 = pipechain[i+1]
 659             eqs += pipe1.connect_to_next(pipe2)
 660
 661         # connect front of chain to ourselves
 662         front = pipechain[0]
 663         self.p.i_data = front.stage.ispec()
 664         eqs += front._connect_in(self)
 665
 666         # connect end of chain to ourselves
 667         end = pipechain[-1]
 668         self.n.o_data = end.stage.ospec()
 669         eqs += end._connect_out(self)
 670
 671         return eqs
 672
 673     def _postprocess(self, i): # XXX DISABLED
 674         return i # RETURNS INPUT
 675         if hasattr(self.stage, "postprocess"):
 676             return self.stage.postprocess(i)
 677         return i
 678
 679     def set_input(self, i):
 680         """ helper function to set the input data
 681         """
 682         return eq(self.p.i_data, i)
 683
 684     def ports(self):
 685         return self.p.ports() + self.n.ports()
 686
 687     def _elaborate(self, platform):
 688         """ handles case where stage has dynamic ready/valid functions
 689         """
 690         m = Module()
 691
 692         if self.stage is not None and hasattr(self.stage, "setup"):
 693             self.stage.setup(m, self.p.i_data)
 694
 695         if not self.p.stage_ctl:
 696             return m
 697
 698         # intercept the previous (outgoing) "ready", combine with stage ready
 699         m.d.comb += self.p.s_o_ready.eq(self.p._o_ready & self.stage.d_ready)
 700
 701         # intercept the next (incoming) "ready" and combine it with data valid
 702         sdv = self.stage.d_valid(self.n.i_ready)
 703         m.d.comb += self.n.d_valid.eq(self.n.i_ready & sdv)
 704
 705         return m
 706
 707
 708 class BufferedHandshake(ControlBase):
 709     """ buffered pipeline stage.  data and strobe signals travel in sync.
 710         if ever the input is ready and the output is not, processed data
 711         is shunted in a temporary register.
 712
 713         Argument: stage.  see Stage API above
 714
 715         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 716         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 717         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 718                               |             |
 719                             process --->----^
 720                               |             |
 721                               +-- r_data ->-+
 722
 723         input data p.i_data is read (only), is processed and goes into an
 724         intermediate result store [process()].  this is updated combinatorially.
 725
 726         in a non-stall condition, the intermediate result will go into the
 727         output (update_output).  however if ever there is a stall, it goes
 728         into r_data instead [update_buffer()].
 729
 730         when the non-stall condition is released, r_data is the first
 731         to be transferred to the output [flush_buffer()], and the stall
 732         condition cleared.
 733
 734         on the next cycle (as long as stall is not raised again) the
 735         input may begin to be processed and transferred directly to output.
 736     """
 737
 738     def elaborate(self, platform):
 739         self.m = ControlBase._elaborate(self, platform)
 740
 741         result = self.stage.ospec()
 742         r_data = self.stage.ospec()
 743
 744         # establish some combinatorial temporaries
 745         o_n_validn = Signal(reset_less=True)
 746         n_i_ready = Signal(reset_less=True, name="n_i_rdy_data")
 747         nir_por = Signal(reset_less=True)
 748         nir_por_n = Signal(reset_less=True)
 749         p_i_valid = Signal(reset_less=True)
 750         nir_novn = Signal(reset_less=True)
 751         nirn_novn = Signal(reset_less=True)
 752         por_pivn = Signal(reset_less=True)
 753         npnn = Signal(reset_less=True)
 754         self.m.d.comb += [p_i_valid.eq(self.p.i_valid_test),
 755                      o_n_validn.eq(~self.n.o_valid),
 756                      n_i_ready.eq(self.n.i_ready_test),
 757                      nir_por.eq(n_i_ready & self.p._o_ready),
 758                      nir_por_n.eq(n_i_ready & ~self.p._o_ready),
 759                      nir_novn.eq(n_i_ready | o_n_validn),
 760                      nirn_novn.eq(~n_i_ready & o_n_validn),
 761                      npnn.eq(nir_por | nirn_novn),
 762                      por_pivn.eq(self.p._o_ready & ~p_i_valid)
 763         ]
 764
 765         # store result of processing in combinatorial temporary
 766         self.m.d.comb += eq(result, self.stage.process(self.p.i_data))
 767
 768         # if not in stall condition, update the temporary register
 769         with self.m.If(self.p.o_ready): # not stalled
 770             self.m.d.sync += eq(r_data, result) # update buffer
 771
 772         # data pass-through conditions
 773         with self.m.If(npnn):
 774             o_data = self._postprocess(result)
 775             self.m.d.sync += [self.n.o_valid.eq(p_i_valid), # valid if p_valid
 776                               eq(self.n.o_data, o_data),    # update output
 777                              ]
 778         # buffer flush conditions (NOTE: can override data passthru conditions)
 779         with self.m.If(nir_por_n): # not stalled
 780             # Flush the [already processed] buffer to the output port.
 781             o_data = self._postprocess(r_data)
 782             self.m.d.sync += [self.n.o_valid.eq(1),  # reg empty
 783                               eq(self.n.o_data, o_data), # flush buffer
 784                              ]
 785         # output ready conditions
 786         self.m.d.sync += self.p._o_ready.eq(nir_novn | por_pivn)
 787
 788         return self.m
 789
 790
 791 class SimpleHandshake(ControlBase):
 792     """ simple handshake control.  data and strobe signals travel in sync.
 793         implements the protocol used by Wishbone and AXI4.
 794
 795         Argument: stage.  see Stage API above
 796
 797         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 798         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 799         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 800                               |             |
 801                               +--process->--^
 802         Truth Table
 803
 804         Inputs   Temporary  Output Data
 805         -------  ---------- -----  ----
 806         P P N N  PiV& ~NiR&  N P
 807         i o i o  PoR  NoV    o o
 808         V R R V              V R
 809
 810         -------   -    -     - -
 811         0 0 0 0   0    0    >0 0    reg
 812         0 0 0 1   0    1    >1 0    reg
 813         0 0 1 0   0    0     0 1    process(i_data)
 814         0 0 1 1   0    0     0 1    process(i_data)
 815         -------   -    -     - -
 816         0 1 0 0   0    0    >0 0    reg
 817         0 1 0 1   0    1    >1 0    reg
 818         0 1 1 0   0    0     0 1    process(i_data)
 819         0 1 1 1   0    0     0 1    process(i_data)
 820         -------   -    -     - -
 821         1 0 0 0   0    0    >0 0    reg
 822         1 0 0 1   0    1    >1 0    reg
 823         1 0 1 0   0    0     0 1    process(i_data)
 824         1 0 1 1   0    0     0 1    process(i_data)
 825         -------   -    -     - -
 826         1 1 0 0   1    0     1 0    process(i_data)
 827         1 1 0 1   1    1     1 0    process(i_data)
 828         1 1 1 0   1    0     1 1    process(i_data)
 829         1 1 1 1   1    0     1 1    process(i_data)
 830         -------   -    -     - -
 831     """
 832
 833     def elaborate(self, platform):
 834         self.m = m = ControlBase._elaborate(self, platform)
 835
 836         r_busy = Signal()
 837         result = self.stage.ospec()
 838
 839         # establish some combinatorial temporaries
 840         n_i_ready = Signal(reset_less=True, name="n_i_rdy_data")
 841         p_i_valid_p_o_ready = Signal(reset_less=True)
 842         p_i_valid = Signal(reset_less=True)
 843         m.d.comb += [p_i_valid.eq(self.p.i_valid_test),
 844                      n_i_ready.eq(self.n.i_ready_test),
 845                      p_i_valid_p_o_ready.eq(p_i_valid & self.p.o_ready),
 846         ]
 847
 848         # store result of processing in combinatorial temporary
 849         m.d.comb += eq(result, self.stage.process(self.p.i_data))
 850
 851         # previous valid and ready
 852         with m.If(p_i_valid_p_o_ready):
 853             o_data = self._postprocess(result)
 854             m.d.sync += [r_busy.eq(1),      # output valid
 855                          eq(self.n.o_data, o_data), # update output
 856                         ]
 857         # previous invalid or not ready, however next is accepting
 858         with m.Elif(n_i_ready):
 859             o_data = self._postprocess(result)
 860             m.d.sync += [eq(self.n.o_data, o_data)]
 861             # TODO: could still send data here (if there was any)
 862             #m.d.sync += self.n.o_valid.eq(0) # ...so set output invalid
 863             m.d.sync += r_busy.eq(0) # ...so set output invalid
 864
 865         m.d.comb += self.n.o_valid.eq(r_busy)
 866         # if next is ready, so is previous
 867         m.d.comb += self.p._o_ready.eq(n_i_ready)
 868
 869         return self.m
 870
 871
 872 class UnbufferedPipeline(ControlBase):
 873     """ A simple pipeline stage with single-clock synchronisation
 874         and two-way valid/ready synchronised signalling.
 875
 876         Note that a stall in one stage will result in the entire pipeline
 877         chain stalling.
 878
 879         Also that unlike BufferedHandshake, the valid/ready signalling does NOT
 880         travel synchronously with the data: the valid/ready signalling
 881         combines in a *combinatorial* fashion.  Therefore, a long pipeline
 882         chain will lengthen propagation delays.
 883
 884         Argument: stage.  see Stage API, above
 885
 886         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 887         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 888         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 889                               |             |
 890                             r_data        result
 891                               |             |
 892                               +--process ->-+
 893
 894         Attributes:
 895         -----------
 896         p.i_data : StageInput, shaped according to ispec
 897             The pipeline input
 898         p.o_data : StageOutput, shaped according to ospec
 899             The pipeline output
 900         r_data : input_shape according to ispec
 901             A temporary (buffered) copy of a prior (valid) input.
 902             This is HELD if the output is not ready.  It is updated
 903             SYNCHRONOUSLY.
 904         result: output_shape according to ospec
 905             The output of the combinatorial logic.  it is updated
 906             COMBINATORIALLY (no clock dependence).
 907
 908         Truth Table
 909
 910         Inputs  Temp  Output  Data
 911         -------   -   -----   ----
 912         P P N N ~NiR&  N P
 913         i o i o  NoV   o o
 914         V R R V        V R
 915
 916         -------   -    - -
 917         0 0 0 0   0    0 1    reg
 918         0 0 0 1   1    1 0    reg
 919         0 0 1 0   0    0 1    reg
 920         0 0 1 1   0    0 1    reg
 921         -------   -    - -
 922         0 1 0 0   0    0 1    reg
 923         0 1 0 1   1    1 0    reg
 924         0 1 1 0   0    0 1    reg
 925         0 1 1 1   0    0 1    reg
 926         -------   -    - -
 927         1 0 0 0   0    1 1    reg
 928         1 0 0 1   1    1 0    reg
 929         1 0 1 0   0    1 1    reg
 930         1 0 1 1   0    1 1    reg
 931         -------   -    - -
 932         1 1 0 0   0    1 1    process(i_data)
 933         1 1 0 1   1    1 0    process(i_data)
 934         1 1 1 0   0    1 1    process(i_data)
 935         1 1 1 1   0    1 1    process(i_data)
 936         -------   -    - -
 937
 938         Note: PoR is *NOT* involved in the above decision-making.
 939     """
 940
 941     def elaborate(self, platform):
 942         self.m = m = ControlBase._elaborate(self, platform)
 943
 944         data_valid = Signal() # is data valid or not
 945         r_data = self.stage.ospec() # output type
 946
 947         # some temporaries
 948         p_i_valid = Signal(reset_less=True)
 949         pv = Signal(reset_less=True)
 950         buf_full = Signal(reset_less=True)
 951         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
 952         m.d.comb += pv.eq(self.p.i_valid & self.p.o_ready)
 953         m.d.comb += buf_full.eq(~self.n.i_ready_test & data_valid)
 954
 955         m.d.comb += self.n.o_valid.eq(data_valid)
 956         m.d.comb += self.p._o_ready.eq(~data_valid | self.n.i_ready_test)
 957         m.d.sync += data_valid.eq(p_i_valid | buf_full)
 958
 959         with m.If(pv):
 960             m.d.sync += eq(r_data, self.stage.process(self.p.i_data))
 961         o_data = self._postprocess(r_data)
 962         m.d.comb += eq(self.n.o_data, o_data)
 963
 964         return self.m
 965
 966 class UnbufferedPipeline2(ControlBase):
 967     """ A simple pipeline stage with single-clock synchronisation
 968         and two-way valid/ready synchronised signalling.
 969
 970         Note that a stall in one stage will result in the entire pipeline
 971         chain stalling.
 972
 973         Also that unlike BufferedHandshake, the valid/ready signalling does NOT
 974         travel synchronously with the data: the valid/ready signalling
 975         combines in a *combinatorial* fashion.  Therefore, a long pipeline
 976         chain will lengthen propagation delays.
 977
 978         Argument: stage.  see Stage API, above
 979
 980         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 981         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 982         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 983                               |             |    |
 984                               +- process-> buf <-+
 985         Attributes:
 986         -----------
 987         p.i_data : StageInput, shaped according to ispec
 988             The pipeline input
 989         p.o_data : StageOutput, shaped according to ospec
 990             The pipeline output
 991         buf : output_shape according to ospec
 992             A temporary (buffered) copy of a valid output
 993             This is HELD if the output is not ready.  It is updated
 994             SYNCHRONOUSLY.
 995
 996         Inputs  Temp  Output Data
 997         -------   -   -----
 998         P P N N ~NiR&  N P   (buf_full)
 999         i o i o  NoV   o o
1000         V R R V        V R
1001
1002         -------   -    - -
1003         0 0 0 0   0    0 1   process(i_data)
1004         0 0 0 1   1    1 0   reg (odata, unchanged)
1005         0 0 1 0   0    0 1   process(i_data)
1006         0 0 1 1   0    0 1   process(i_data)
1007         -------   -    - -
1008         0 1 0 0   0    0 1   process(i_data)
1009         0 1 0 1   1    1 0   reg (odata, unchanged)
1010         0 1 1 0   0    0 1   process(i_data)
1011         0 1 1 1   0    0 1   process(i_data)
1012         -------   -    - -
1013         1 0 0 0   0    1 1   process(i_data)
1014         1 0 0 1   1    1 0   reg (odata, unchanged)
1015         1 0 1 0   0    1 1   process(i_data)
1016         1 0 1 1   0    1 1   process(i_data)
1017         -------   -    - -
1018         1 1 0 0   0    1 1   process(i_data)
1019         1 1 0 1   1    1 0   reg (odata, unchanged)
1020         1 1 1 0   0    1 1   process(i_data)
1021         1 1 1 1   0    1 1   process(i_data)
1022         -------   -    - -
1023
1024         Note: PoR is *NOT* involved in the above decision-making.
1025     """
1026
1027     def elaborate(self, platform):
1028         self.m = m = ControlBase._elaborate(self, platform)
1029
1030         buf_full = Signal() # is data valid or not
1031         buf = self.stage.ospec() # output type
1032
1033         # some temporaries
1034         p_i_valid = Signal(reset_less=True)
1035         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
1036
1037         m.d.comb += self.n.o_valid.eq(buf_full | p_i_valid)
1038         m.d.comb += self.p._o_ready.eq(~buf_full)
1039         m.d.sync += buf_full.eq(~self.n.i_ready_test & self.n.o_valid)
1040
1041         o_data = Mux(buf_full, buf, self.stage.process(self.p.i_data))
1042         o_data = self._postprocess(o_data)
1043         m.d.comb += eq(self.n.o_data, o_data)
1044         m.d.sync += eq(buf, self.n.o_data)
1045
1046         return self.m
1047
1048
1049 class PassThroughStage(StageCls):
1050     """ a pass-through stage which has its input data spec equal to its output,
1051         and "passes through" its data from input to output.
1052     """
1053     def __init__(self, iospecfn):
1054         self.iospecfn = iospecfn
1055     def ispec(self): return self.iospecfn()
1056     def ospec(self): return self.iospecfn()
1057     def process(self, i): return i
1058
1059
1060 class PassThroughHandshake(ControlBase):
1061     """ A control block that delays by one clock cycle.
1062
1063         Inputs   Temporary          Output Data
1064         -------  ------------------  ----- ----
1065         P P N N  PiV& PiV| NiR| pvr   N P  (pvr)
1066         i o i o  PoR  ~PoR ~NoV       o o
1067         V R R V                       V R
1068
1069         -------   -    -    -   -     - -
1070         0 0 0 0   0    1    1   0     1 1   odata (unchanged)
1071         0 0 0 1   0    1    0   0     1 0   odata (unchanged)
1072         0 0 1 0   0    1    1   0     1 1   odata (unchanged)
1073         0 0 1 1   0    1    1   0     1 1   odata (unchanged)
1074         -------   -    -    -   -     - -
1075         0 1 0 0   0    0    1   0     0 1   odata (unchanged)
1076         0 1 0 1   0    0    0   0     0 0   odata (unchanged)
1077         0 1 1 0   0    0    1   0     0 1   odata (unchanged)
1078         0 1 1 1   0    0    1   0     0 1   odata (unchanged)
1079         -------   -    -    -   -     - -
1080         1 0 0 0   0    1    1   1     1 1   process(in)
1081         1 0 0 1   0    1    0   0     1 0   odata (unchanged)
1082         1 0 1 0   0    1    1   1     1 1   process(in)
1083         1 0 1 1   0    1    1   1     1 1   process(in)
1084         -------   -    -    -   -     - -
1085         1 1 0 0   1    1    1   1     1 1   process(in)
1086         1 1 0 1   1    1    0   0     1 0   odata (unchanged)
1087         1 1 1 0   1    1    1   1     1 1   process(in)
1088         1 1 1 1   1    1    1   1     1 1   process(in)
1089         -------   -    -    -   -     - -
1090
1091     """
1092
1093     def elaborate(self, platform):
1094         self.m = m = ControlBase._elaborate(self, platform)
1095
1096         r_data = self.stage.ospec() # output type
1097
1098         # temporaries
1099         p_i_valid = Signal(reset_less=True)
1100         pvr = Signal(reset_less=True)
1101         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
1102         m.d.comb += pvr.eq(p_i_valid & self.p.o_ready)
1103
1104         m.d.comb += self.p.o_ready.eq(~self.n.o_valid |  self.n.i_ready_test)
1105         m.d.sync += self.n.o_valid.eq(p_i_valid       | ~self.p.o_ready)
1106
1107         odata = Mux(pvr, self.stage.process(self.p.i_data), r_data)
1108         m.d.sync += eq(r_data, odata)
1109         r_data = self._postprocess(r_data)
1110         m.d.comb += eq(self.n.o_data, r_data)
1111
1112         return m
1113
1114
1115 class RegisterPipeline(UnbufferedPipeline):
1116     """ A pipeline stage that delays by one clock cycle, creating a
1117         sync'd latch out of o_data and o_valid as an indirect byproduct
1118         of using PassThroughStage
1119     """
1120     def __init__(self, iospecfn):
1121         UnbufferedPipeline.__init__(self, PassThroughStage(iospecfn))
1122
1123
1124 class FIFOControl(ControlBase):
1125     """ FIFO Control.  Uses SyncFIFO to store data, coincidentally
1126         happens to have same valid/ready signalling as Stage API.
1127
1128         i_data -> fifo.din -> FIFO -> fifo.dout -> o_data
1129     """
1130
1131     def __init__(self, depth, stage, in_multi=None, stage_ctl=False,
1132                                      fwft=True, buffered=False, pipe=False):
1133         """ FIFO Control
1134
1135             * depth: number of entries in the FIFO
1136             * stage: data processing block
1137             * fwft : first word fall-thru mode (non-fwft introduces delay)
1138             * buffered: use buffered FIFO (introduces extra cycle delay)
1139
1140             NOTE 1: FPGAs may have trouble with the defaults for SyncFIFO
1141                     (fwft=True, buffered=False)
1142
1143             NOTE 2: i_data *must* have a shape function.  it can therefore
1144                     be a Signal, or a Record, or a RecordObject.
1145
1146             data is processed (and located) as follows:
1147
1148             self.p  self.stage temp    fn temp  fn  temp  fp   self.n
1149             i_data->process()->result->cat->din.FIFO.dout->cat(o_data)
1150
1151             yes, really: cat produces a Cat() which can be assigned to.
1152             this is how the FIFO gets de-catted without needing a de-cat
1153             function
1154         """
1155
1156         assert not (fwft and buffered), "buffered cannot do fwft"
1157         if buffered:
1158             depth += 1
1159         self.fwft = fwft
1160         self.buffered = buffered
1161         self.pipe = pipe
1162         self.fdepth = depth
1163         ControlBase.__init__(self, stage, in_multi, stage_ctl)
1164
1165     def elaborate(self, platform):
1166         self.m = m = ControlBase._elaborate(self, platform)
1167
1168         # make a FIFO with a signal of equal width to the o_data.
1169         (fwidth, _) = self.n.o_data.shape()
1170         if self.buffered:
1171             fifo = SyncFIFOBuffered(fwidth, self.fdepth)
1172         else:
1173             fifo = Queue(fwidth, self.fdepth, fwft=self.fwft, pipe=self.pipe)
1174         m.submodules.fifo = fifo
1175
1176         # store result of processing in combinatorial temporary
1177         result = self.stage.ospec()
1178         m.d.comb += eq(result, self.stage.process(self.p.i_data))
1179
1180         # connect previous rdy/valid/data - do cat on i_data
1181         # NOTE: cannot do the PrevControl-looking trick because
1182         # of need to process the data.  shaaaame....
1183         m.d.comb += [fifo.we.eq(self.p.i_valid_test),
1184                      self.p.o_ready.eq(fifo.writable),
1185                      eq(fifo.din, cat(result)),
1186                    ]
1187
1188         # connect next rdy/valid/data - do cat on o_data
1189         connections = [self.n.o_valid.eq(fifo.readable),
1190                      fifo.re.eq(self.n.i_ready_test),
1191                    ]
1192         if self.fwft or self.buffered:
1193             m.d.comb += connections
1194         else:
1195             m.d.sync += connections # unbuffered fwft mode needs sync
1196         o_data = cat(self.n.o_data).eq(fifo.dout)
1197         o_data = self._postprocess(o_data)
1198         m.d.comb += o_data
1199
1200         return m
1201
1202
1203 # aka "RegStage".
1204 class UnbufferedPipeline(FIFOControl):
1205     def __init__(self, stage, in_multi=None, stage_ctl=False):
1206         FIFOControl.__init__(self, 1, stage, in_multi, stage_ctl,
1207                                    fwft=True, pipe=False)
1208
1209 # aka "BreakReadyStage" XXX had to set fwft=True to get it to work
1210 class PassThroughHandshake(FIFOControl):
1211     def __init__(self, stage, in_multi=None, stage_ctl=False):
1212         FIFOControl.__init__(self, 1, stage, in_multi, stage_ctl,
1213                                    fwft=True, pipe=True)
1214
1215 # this is *probably* BufferedHandshake, although test #997 now succeeds.
1216 class BufferedHandshake(FIFOControl):
1217     def __init__(self, stage, in_multi=None, stage_ctl=False):
1218         FIFOControl.__init__(self, 2, stage, in_multi, stage_ctl,
1219                                    fwft=True, pipe=False)
1220
1221
1222 """
1223 # this is *probably* SimpleHandshake (note: memory cell size=0)
1224 class SimpleHandshake(FIFOControl):
1225     def __init__(self, stage, in_multi=None, stage_ctl=False):
1226         FIFOControl.__init__(self, 0, stage, in_multi, stage_ctl,
1227                                    fwft=True, pipe=False)
1228 """