src/add/singlepipe.py

   1 """ Pipeline and BufferedHandshake implementation, conforming to the same API.
   2     For multi-input and multi-output variants, see multipipe.
   3
   4     Associated development bugs:
   5     * http://bugs.libre-riscv.org/show_bug.cgi?id=64
   6     * http://bugs.libre-riscv.org/show_bug.cgi?id=57
   7
   8     eq:
   9     --
  10
  11     a strategically very important function that is identical in function
  12     to nmigen's Signal.eq function, except it may take objects, or a list
  13     of objects, or a tuple of objects, and where objects may also be
  14     Records.
  15
  16     Stage API:
  17     ---------
  18
  19     stage requires compliance with a strict API that may be
  20     implemented in several means, including as a static class.
  21     the methods of a stage instance must be as follows:
  22
  23     * ispec() - Input data format specification
  24                 returns an object or a list or tuple of objects, or
  25                 a Record, each object having an "eq" function which
  26                 takes responsibility for copying by assignment all
  27                 sub-objects
  28     * ospec() - Output data format specification
  29                 requirements as for ospec
  30     * process(m, i) - Processes an ispec-formatted object
  31                 returns a combinatorial block of a result that
  32                 may be assigned to the output, by way of the "eq"
  33                 function
  34     * setup(m, i) - Optional function for setting up submodules
  35                 may be used for more complex stages, to link
  36                 the input (i) to submodules.  must take responsibility
  37                 for adding those submodules to the module (m).
  38                 the submodules must be combinatorial blocks and
  39                 must have their inputs and output linked combinatorially.
  40
  41     Both StageCls (for use with non-static classes) and Stage (for use
  42     by static classes) are abstract classes from which, for convenience
  43     and as a courtesy to other developers, anything conforming to the
  44     Stage API may *choose* to derive.
  45
  46     StageChain:
  47     ----------
  48
  49     A useful combinatorial wrapper around stages that chains them together
  50     and then presents a Stage-API-conformant interface.  By presenting
  51     the same API as the stages it wraps, it can clearly be used recursively.
  52
  53     RecordBasedStage:
  54     ----------------
  55
  56     A convenience class that takes an input shape, output shape, a
  57     "processing" function and an optional "setup" function.  Honestly
  58     though, there's not much more effort to just... create a class
  59     that returns a couple of Records (see ExampleAddRecordStage in
  60     examples).
  61
  62     PassThroughStage:
  63     ----------------
  64
  65     A convenience class that takes a single function as a parameter,
  66     that is chain-called to create the exact same input and output spec.
  67     It has a process() function that simply returns its input.
  68
  69     Instances of this class are completely redundant if handed to
  70     StageChain, however when passed to UnbufferedPipeline they
  71     can be used to introduce a single clock delay.
  72
  73     ControlBase:
  74     -----------
  75
  76     The base class for pipelines.  Contains previous and next ready/valid/data.
  77     Also has an extremely useful "connect" function that can be used to
  78     connect a chain of pipelines and present the exact same prev/next
  79     ready/valid/data API.
  80
  81     UnbufferedPipeline:
  82     ------------------
  83
  84     A simple stalling clock-synchronised pipeline that has no buffering
  85     (unlike BufferedHandshake).  Data flows on *every* clock cycle when
  86     the conditions are right (this is nominally when the input is valid
  87     and the output is ready).
  88
  89     A stall anywhere along the line will result in a stall back-propagating
  90     down the entire chain.  The BufferedHandshake by contrast will buffer
  91     incoming data, allowing previous stages one clock cycle's grace before
  92     also having to stall.
  93
  94     An advantage of the UnbufferedPipeline over the Buffered one is
  95     that the amount of logic needed (number of gates) is greatly
  96     reduced (no second set of buffers basically)
  97
  98     The disadvantage of the UnbufferedPipeline is that the valid/ready
  99     logic, if chained together, is *combinatorial*, resulting in
 100     progressively larger gate delay.
 101
 102     PassThroughHandshake:
 103     ------------------
 104
 105     A Control class that introduces a single clock delay, passing its
 106     data through unaltered.  Unlike RegisterPipeline (which relies
 107     on UnbufferedPipeline and PassThroughStage) it handles ready/valid
 108     itself.
 109
 110     RegisterPipeline:
 111     ----------------
 112
 113     A convenience class that, because UnbufferedPipeline introduces a single
 114     clock delay, when its stage is a PassThroughStage, it results in a Pipeline
 115     stage that, duh, delays its (unmodified) input by one clock cycle.
 116
 117     BufferedHandshake:
 118     ----------------
 119
 120     nmigen implementation of buffered pipeline stage, based on zipcpu:
 121     https://zipcpu.com/blog/2017/08/14/strategies-for-pipelining.html
 122
 123     this module requires quite a bit of thought to understand how it works
 124     (and why it is needed in the first place).  reading the above is
 125     *strongly* recommended.
 126
 127     unlike john dawson's IEEE754 FPU STB/ACK signalling, which requires
 128     the STB / ACK signals to raise and lower (on separate clocks) before
 129     data may proceeed (thus only allowing one piece of data to proceed
 130     on *ALTERNATE* cycles), the signalling here is a true pipeline
 131     where data will flow on *every* clock when the conditions are right.
 132
 133     input acceptance conditions are when:
 134         * incoming previous-stage strobe (p.i_valid) is HIGH
 135         * outgoing previous-stage ready   (p.o_ready) is LOW
 136
 137     output transmission conditions are when:
 138         * outgoing next-stage strobe (n.o_valid) is HIGH
 139         * outgoing next-stage ready   (n.i_ready) is LOW
 140
 141     the tricky bit is when the input has valid data and the output is not
 142     ready to accept it.  if it wasn't for the clock synchronisation, it
 143     would be possible to tell the input "hey don't send that data, we're
 144     not ready".  unfortunately, it's not possible to "change the past":
 145     the previous stage *has no choice* but to pass on its data.
 146
 147     therefore, the incoming data *must* be accepted - and stored: that
 148     is the responsibility / contract that this stage *must* accept.
 149     on the same clock, it's possible to tell the input that it must
 150     not send any more data.  this is the "stall" condition.
 151
 152     we now effectively have *two* possible pieces of data to "choose" from:
 153     the buffered data, and the incoming data.  the decision as to which
 154     to process and output is based on whether we are in "stall" or not.
 155     i.e. when the next stage is no longer ready, the output comes from
 156     the buffer if a stall had previously occurred, otherwise it comes
 157     direct from processing the input.
 158
 159     this allows us to respect a synchronous "travelling STB" with what
 160     dan calls a "buffered handshake".
 161
 162     it's quite a complex state machine!
 163
 164     SimpleHandshake
 165     ---------------
 166
 167     Synchronised pipeline, Based on:
 168     https://github.com/ZipCPU/dbgbus/blob/master/hexbus/rtl/hbdeword.v
 169 """
 170
 171 from nmigen import Signal, Cat, Const, Mux, Module, Value
 172 from nmigen.cli import verilog, rtlil
 173 from nmigen.lib.fifo import SyncFIFO, SyncFIFOBuffered
 174 from nmigen.hdl.ast import ArrayProxy
 175 from nmigen.hdl.rec import Record, Layout
 176
 177 from abc import ABCMeta, abstractmethod
 178 from collections.abc import Sequence
 179 from queue import Queue
 180
 181
 182 class Object:
 183     def __init__(self):
 184         self.fields = {}
 185
 186     def __setattr__(self, k, v):
 187         print ("kv", k, v)
 188         if (k.startswith('_') or k in ["fields", "name", "src_loc"] or
 189            k in dir(Object) or "fields" not in self.__dict__):
 190             return object.__setattr__(self, k, v)
 191         self.fields[k] = v
 192
 193     def __getattr__(self, k):
 194         if k in self.fields:
 195             return self.fields[k]
 196         return object.__getattr__(self, k)
 197
 198     def __iter__(self):
 199         for x in self.fields.values():
 200             yield x
 201
 202     def eq(self, inp):
 203         res = []
 204         for (o, i) in zip(self, inp):
 205             res.append(eq(o, i))
 206         return res
 207
 208     def ports(self):
 209         return list(self)
 210
 211
 212 class RecordObject(Record):
 213     def __init__(self, layout=None, name=None):
 214         Record.__init__(self, layout=layout or [], name=None)
 215
 216     def __setattr__(self, k, v):
 217         #print (dir(Record))
 218         if (k.startswith('_') or k in ["fields", "name", "src_loc"] or
 219            k in dir(Record) or "fields" not in self.__dict__):
 220             return object.__setattr__(self, k, v)
 221         self.fields[k] = v
 222         #print ("RecordObject setattr", k, v)
 223         if isinstance(v, Record):
 224             newlayout = {k: (k, v.layout)}
 225         elif isinstance(v, Value):
 226             newlayout = {k: (k, v.shape())}
 227         else:
 228             newlayout = {k: (k, shape(v))}
 229         self.layout.fields.update(newlayout)
 230
 231     def __iter__(self):
 232         for x in self.fields.values():
 233             yield x
 234
 235     def ports(self):
 236         return list(self)
 237
 238
 239 class PrevControl:
 240     """ contains signals that come *from* the previous stage (both in and out)
 241         * i_valid: previous stage indicating all incoming data is valid.
 242                    may be a multi-bit signal, where all bits are required
 243                    to be asserted to indicate "valid".
 244         * o_ready: output to next stage indicating readiness to accept data
 245         * i_data : an input - added by the user of this class
 246     """
 247
 248     def __init__(self, i_width=1, stage_ctl=False):
 249         self.stage_ctl = stage_ctl
 250         self.i_valid = Signal(i_width, name="p_i_valid") # prev   >>in  self
 251         self._o_ready = Signal(name="p_o_ready") # prev   <<out self
 252         self.i_data = None # XXX MUST BE ADDED BY USER
 253         if stage_ctl:
 254             self.s_o_ready = Signal(name="p_s_o_rdy") # prev   <<out self
 255         self.trigger = Signal(reset_less=True)
 256
 257     @property
 258     def o_ready(self):
 259         """ public-facing API: indicates (externally) that stage is ready
 260         """
 261         if self.stage_ctl:
 262             return self.s_o_ready # set dynamically by stage
 263         return self._o_ready      # return this when not under dynamic control
 264
 265     def _connect_in(self, prev, direct=False, fn=None):
 266         """ internal helper function to connect stage to an input source.
 267             do not use to connect stage-to-stage!
 268         """
 269         i_valid = prev.i_valid if direct else prev.i_valid_test
 270         i_data = fn(prev.i_data) if fn is not None else prev.i_data
 271         return [self.i_valid.eq(i_valid),
 272                 prev.o_ready.eq(self.o_ready),
 273                 eq(self.i_data, i_data),
 274                ]
 275
 276     @property
 277     def i_valid_test(self):
 278         vlen = len(self.i_valid)
 279         if vlen > 1:
 280             # multi-bit case: valid only when i_valid is all 1s
 281             all1s = Const(-1, (len(self.i_valid), False))
 282             i_valid = (self.i_valid == all1s)
 283         else:
 284             # single-bit i_valid case
 285             i_valid = self.i_valid
 286
 287         # when stage indicates not ready, incoming data
 288         # must "appear" to be not ready too
 289         if self.stage_ctl:
 290             i_valid = i_valid & self.s_o_ready
 291
 292         return i_valid
 293
 294     def elaborate(self, platform):
 295         m = Module()
 296         m.d.comb += self.trigger.eq(self.i_valid_test & self.o_ready)
 297         return m
 298
 299     def eq(self, i):
 300         return [self.i_data.eq(i.i_data),
 301                 self.o_ready.eq(i.o_ready),
 302                 self.i_valid.eq(i.i_valid)]
 303
 304     def __iter__(self):
 305         yield self.i_valid
 306         yield self.o_ready
 307         if hasattr(self.i_data, "ports"):
 308             yield from self.i_data.ports()
 309         elif isinstance(self.i_data, Sequence):
 310             yield from self.i_data
 311         else:
 312             yield self.i_data
 313
 314     def ports(self):
 315         return list(self)
 316
 317
 318 class NextControl:
 319     """ contains the signals that go *to* the next stage (both in and out)
 320         * o_valid: output indicating to next stage that data is valid
 321         * i_ready: input from next stage indicating that it can accept data
 322         * o_data : an output - added by the user of this class
 323     """
 324     def __init__(self, stage_ctl=False):
 325         self.stage_ctl = stage_ctl
 326         self.o_valid = Signal(name="n_o_valid") # self out>>  next
 327         self.i_ready = Signal(name="n_i_ready") # self <<in   next
 328         self.o_data = None # XXX MUST BE ADDED BY USER
 329         #if self.stage_ctl:
 330         self.d_valid = Signal(reset=1) # INTERNAL (data valid)
 331         self.trigger = Signal(reset_less=True)
 332
 333     @property
 334     def i_ready_test(self):
 335         if self.stage_ctl:
 336             return self.i_ready & self.d_valid
 337         return self.i_ready
 338
 339     def connect_to_next(self, nxt):
 340         """ helper function to connect to the next stage data/valid/ready.
 341             data/valid is passed *TO* nxt, and ready comes *IN* from nxt.
 342             use this when connecting stage-to-stage
 343         """
 344         return [nxt.i_valid.eq(self.o_valid),
 345                 self.i_ready.eq(nxt.o_ready),
 346                 eq(nxt.i_data, self.o_data),
 347                ]
 348
 349     def _connect_out(self, nxt, direct=False, fn=None):
 350         """ internal helper function to connect stage to an output source.
 351             do not use to connect stage-to-stage!
 352         """
 353         i_ready = nxt.i_ready if direct else nxt.i_ready_test
 354         o_data = fn(nxt.o_data) if fn is not None else nxt.o_data
 355         return [nxt.o_valid.eq(self.o_valid),
 356                 self.i_ready.eq(i_ready),
 357                 eq(o_data, self.o_data),
 358                ]
 359
 360     def elaborate(self, platform):
 361         m = Module()
 362         m.d.comb += self.trigger.eq(self.i_ready_test & self.o_valid)
 363         return m
 364
 365     def __iter__(self):
 366         yield self.i_ready
 367         yield self.o_valid
 368         if hasattr(self.o_data, "ports"):
 369             yield from self.o_data.ports()
 370         elif isinstance(self.o_data, Sequence):
 371             yield from self.o_data
 372         else:
 373             yield self.o_data
 374
 375     def ports(self):
 376         return list(self)
 377
 378
 379 class Visitor2:
 380     """ a helper class for iterating twin-argument compound data structures.
 381
 382         Record is a special (unusual, recursive) case, where the input may be
 383         specified as a dictionary (which may contain further dictionaries,
 384         recursively), where the field names of the dictionary must match
 385         the Record's field spec.  Alternatively, an object with the same
 386         member names as the Record may be assigned: it does not have to
 387         *be* a Record.
 388
 389         ArrayProxy is also special-cased, it's a bit messy: whilst ArrayProxy
 390         has an eq function, the object being assigned to it (e.g. a python
 391         object) might not.  despite the *input* having an eq function,
 392         that doesn't help us, because it's the *ArrayProxy* that's being
 393         assigned to.  so.... we cheat.  use the ports() function of the
 394         python object, enumerate them, find out the list of Signals that way,
 395         and assign them.
 396     """
 397     def iterator2(self, o, i):
 398         if isinstance(o, dict):
 399             yield from self.dict_iter2(o, i)
 400
 401         if not isinstance(o, Sequence):
 402             o, i = [o], [i]
 403         for (ao, ai) in zip(o, i):
 404             #print ("visit", fn, ao, ai)
 405             if isinstance(ao, Record):
 406                 yield from self.record_iter2(ao, ai)
 407             elif isinstance(ao, ArrayProxy) and not isinstance(ai, Value):
 408                 yield from self.arrayproxy_iter2(ao, ai)
 409             else:
 410                 yield (ao, ai)
 411
 412     def dict_iter2(self, o, i):
 413         for (k, v) in o.items():
 414             print ("d-iter", v, i[k])
 415             yield (v, i[k])
 416         return res
 417
 418     def _not_quite_working_with_all_unit_tests_record_iter2(self, ao, ai):
 419         print ("record_iter2", ao, ai, type(ao), type(ai))
 420         if isinstance(ai, Value):
 421             if isinstance(ao, Sequence):
 422                 ao, ai = [ao], [ai]
 423             for o, i in zip(ao, ai):
 424                 yield (o, i)
 425             return
 426         for idx, (field_name, field_shape, _) in enumerate(ao.layout):
 427             if isinstance(field_shape, Layout):
 428                 val = ai.fields
 429             else:
 430                 val = ai
 431             if hasattr(val, field_name): # check for attribute
 432                 val = getattr(val, field_name)
 433             else:
 434                 val = val[field_name] # dictionary-style specification
 435             yield from self.iterator2(ao.fields[field_name], val)
 436
 437     def record_iter2(self, ao, ai):
 438         for idx, (field_name, field_shape, _) in enumerate(ao.layout):
 439             if isinstance(field_shape, Layout):
 440                 val = ai.fields
 441             else:
 442                 val = ai
 443             if hasattr(val, field_name): # check for attribute
 444                 val = getattr(val, field_name)
 445             else:
 446                 val = val[field_name] # dictionary-style specification
 447             yield from self.iterator2(ao.fields[field_name], val)
 448
 449     def arrayproxy_iter2(self, ao, ai):
 450         for p in ai.ports():
 451             op = getattr(ao, p.name)
 452             print ("arrayproxy - p", p, p.name)
 453             yield from self.iterator2(op, p)
 454
 455
 456 class Visitor:
 457     """ a helper class for iterating single-argument compound data structures.
 458         similar to Visitor2.
 459     """
 460     def iterate(self, i):
 461         """ iterate a compound structure recursively using yield
 462         """
 463         if not isinstance(i, Sequence):
 464             i = [i]
 465         for ai in i:
 466             #print ("iterate", ai)
 467             if isinstance(ai, Record):
 468                 #print ("record", list(ai.layout))
 469                 yield from self.record_iter(ai)
 470             elif isinstance(ai, ArrayProxy) and not isinstance(ai, Value):
 471                 yield from self.array_iter(ai)
 472             else:
 473                 yield ai
 474
 475     def record_iter(self, ai):
 476         for idx, (field_name, field_shape, _) in enumerate(ai.layout):
 477             if isinstance(field_shape, Layout):
 478                 val = ai.fields
 479             else:
 480                 val = ai
 481             if hasattr(val, field_name): # check for attribute
 482                 val = getattr(val, field_name)
 483             else:
 484                 val = val[field_name] # dictionary-style specification
 485             #print ("recidx", idx, field_name, field_shape, val)
 486             yield from self.iterate(val)
 487
 488     def array_iter(self, ai):
 489         for p in ai.ports():
 490             yield from self.iterate(p)
 491
 492
 493 def eq(o, i):
 494     """ makes signals equal: a helper routine which identifies if it is being
 495         passed a list (or tuple) of objects, or signals, or Records, and calls
 496         the objects' eq function.
 497     """
 498     res = []
 499     for (ao, ai) in Visitor2().iterator2(o, i):
 500         rres = ao.eq(ai)
 501         if not isinstance(rres, Sequence):
 502             rres = [rres]
 503         res += rres
 504     return res
 505
 506
 507 def shape(i):
 508     #print ("shape", i)
 509     r = 0
 510     for part in list(i):
 511         #print ("shape?", part)
 512         s, _ = part.shape()
 513         r += s
 514     return r, False
 515
 516
 517 def cat(i):
 518     """ flattens a compound structure recursively using Cat
 519     """
 520     from nmigen.tools import flatten
 521     #res = list(flatten(i)) # works (as of nmigen commit f22106e5) HOWEVER...
 522     res = list(Visitor().iterate(i)) # needed because input may be a sequence
 523     return Cat(*res)
 524
 525
 526 class StageCls(metaclass=ABCMeta):
 527     """ Class-based "Stage" API.  requires instantiation (after derivation)
 528
 529         see "Stage API" above..  Note: python does *not* require derivation
 530         from this class.  All that is required is that the pipelines *have*
 531         the functions listed in this class.  Derivation from this class
 532         is therefore merely a "courtesy" to maintainers.
 533     """
 534     @abstractmethod
 535     def ispec(self): pass       # REQUIRED
 536     @abstractmethod
 537     def ospec(self): pass       # REQUIRED
 538     #@abstractmethod
 539     #def setup(self, m, i): pass # OPTIONAL
 540     @abstractmethod
 541     def process(self, i): pass  # REQUIRED
 542
 543
 544 class Stage(metaclass=ABCMeta):
 545     """ Static "Stage" API.  does not require instantiation (after derivation)
 546
 547         see "Stage API" above.  Note: python does *not* require derivation
 548         from this class.  All that is required is that the pipelines *have*
 549         the functions listed in this class.  Derivation from this class
 550         is therefore merely a "courtesy" to maintainers.
 551     """
 552     @staticmethod
 553     @abstractmethod
 554     def ispec(): pass
 555
 556     @staticmethod
 557     @abstractmethod
 558     def ospec(): pass
 559
 560     #@staticmethod
 561     #@abstractmethod
 562     #def setup(m, i): pass
 563
 564     @staticmethod
 565     @abstractmethod
 566     def process(i): pass
 567
 568
 569 class RecordBasedStage(Stage):
 570     """ convenience class which provides a Records-based layout.
 571         honestly it's a lot easier just to create a direct Records-based
 572         class (see ExampleAddRecordStage)
 573     """
 574     def __init__(self, in_shape, out_shape, processfn, setupfn=None):
 575         self.in_shape = in_shape
 576         self.out_shape = out_shape
 577         self.__process = processfn
 578         self.__setup = setupfn
 579     def ispec(self): return Record(self.in_shape)
 580     def ospec(self): return Record(self.out_shape)
 581     def process(seif, i): return self.__process(i)
 582     def setup(seif, m, i): return self.__setup(m, i)
 583
 584
 585 class StageChain(StageCls):
 586     """ pass in a list of stages, and they will automatically be
 587         chained together via their input and output specs into a
 588         combinatorial chain.
 589
 590         the end result basically conforms to the exact same Stage API.
 591
 592         * input to this class will be the input of the first stage
 593         * output of first stage goes into input of second
 594         * output of second goes into input into third (etc. etc.)
 595         * the output of this class will be the output of the last stage
 596     """
 597     def __init__(self, chain, specallocate=False):
 598         self.chain = chain
 599         self.specallocate = specallocate
 600
 601     def ispec(self):
 602         return self.chain[0].ispec()
 603
 604     def ospec(self):
 605         return self.chain[-1].ospec()
 606
 607     def _specallocate_setup(self, m, i):
 608         for (idx, c) in enumerate(self.chain):
 609             if hasattr(c, "setup"):
 610                 c.setup(m, i)               # stage may have some module stuff
 611             o = self.chain[idx].ospec()     # last assignment survives
 612             m.d.comb += eq(o, c.process(i)) # process input into "o"
 613             if idx == len(self.chain)-1:
 614                 break
 615             i = self.chain[idx+1].ispec()   # new input on next loop
 616             m.d.comb += eq(i, o)            # assign to next input
 617         return o                            # last loop is the output
 618
 619     def _noallocate_setup(self, m, i):
 620         for (idx, c) in enumerate(self.chain):
 621             if hasattr(c, "setup"):
 622                 c.setup(m, i)               # stage may have some module stuff
 623             i = o = c.process(i)            # store input into "o"
 624         return o                            # last loop is the output
 625
 626     def setup(self, m, i):
 627         if self.specallocate:
 628             self.o = self._specallocate_setup(m, i)
 629         else:
 630             self.o = self._noallocate_setup(m, i)
 631
 632     def process(self, i):
 633         return self.o # conform to Stage API: return last-loop output
 634
 635
 636 class ControlBase:
 637     """ Common functions for Pipeline API
 638     """
 639     def __init__(self, stage=None, in_multi=None, stage_ctl=False):
 640         """ Base class containing ready/valid/data to previous and next stages
 641
 642             * p: contains ready/valid to the previous stage
 643             * n: contains ready/valid to the next stage
 644
 645             Except when calling Controlbase.connect(), user must also:
 646             * add i_data member to PrevControl (p) and
 647             * add o_data member to NextControl (n)
 648         """
 649         self.stage = stage
 650
 651         # set up input and output IO ACK (prev/next ready/valid)
 652         self.p = PrevControl(in_multi, stage_ctl)
 653         self.n = NextControl(stage_ctl)
 654
 655         # set up the input and output data
 656         if stage is not None:
 657             self.p.i_data = stage.ispec() # input type
 658             self.n.o_data = stage.ospec()
 659
 660     def connect_to_next(self, nxt):
 661         """ helper function to connect to the next stage data/valid/ready.
 662         """
 663         return self.n.connect_to_next(nxt.p)
 664
 665     def _connect_in(self, prev):
 666         """ internal helper function to connect stage to an input source.
 667             do not use to connect stage-to-stage!
 668         """
 669         return self.p._connect_in(prev.p)
 670
 671     def _connect_out(self, nxt):
 672         """ internal helper function to connect stage to an output source.
 673             do not use to connect stage-to-stage!
 674         """
 675         return self.n._connect_out(nxt.n)
 676
 677     def connect(self, pipechain):
 678         """ connects a chain (list) of Pipeline instances together and
 679             links them to this ControlBase instance:
 680
 681                       in <----> self <---> out
 682                        |                   ^
 683                        v                   |
 684                     [pipe1, pipe2, pipe3, pipe4]
 685                        |    ^  |    ^  |     ^
 686                        v    |  v    |  v     |
 687                      out---in out--in out---in
 688
 689             Also takes care of allocating i_data/o_data, by looking up
 690             the data spec for each end of the pipechain.  i.e It is NOT
 691             necessary to allocate self.p.i_data or self.n.o_data manually:
 692             this is handled AUTOMATICALLY, here.
 693
 694             Basically this function is the direct equivalent of StageChain,
 695             except that unlike StageChain, the Pipeline logic is followed.
 696
 697             Just as StageChain presents an object that conforms to the
 698             Stage API from a list of objects that also conform to the
 699             Stage API, an object that calls this Pipeline connect function
 700             has the exact same pipeline API as the list of pipline objects
 701             it is called with.
 702
 703             Thus it becomes possible to build up larger chains recursively.
 704             More complex chains (multi-input, multi-output) will have to be
 705             done manually.
 706         """
 707         eqs = [] # collated list of assignment statements
 708
 709         # connect inter-chain
 710         for i in range(len(pipechain)-1):
 711             pipe1 = pipechain[i]
 712             pipe2 = pipechain[i+1]
 713             eqs += pipe1.connect_to_next(pipe2)
 714
 715         # connect front of chain to ourselves
 716         front = pipechain[0]
 717         self.p.i_data = front.stage.ispec()
 718         eqs += front._connect_in(self)
 719
 720         # connect end of chain to ourselves
 721         end = pipechain[-1]
 722         self.n.o_data = end.stage.ospec()
 723         eqs += end._connect_out(self)
 724
 725         return eqs
 726
 727     def _postprocess(self, i): # XXX DISABLED
 728         return i # RETURNS INPUT
 729         if hasattr(self.stage, "postprocess"):
 730             return self.stage.postprocess(i)
 731         return i
 732
 733     def set_input(self, i):
 734         """ helper function to set the input data
 735         """
 736         return eq(self.p.i_data, i)
 737
 738     def __iter__(self):
 739         yield from self.p
 740         yield from self.n
 741
 742     def ports(self):
 743         return list(self)
 744
 745     def _elaborate(self, platform):
 746         """ handles case where stage has dynamic ready/valid functions
 747         """
 748         m = Module()
 749         m.submodules.p = self.p
 750         m.submodules.n = self.n
 751
 752         if self.stage is not None and hasattr(self.stage, "setup"):
 753             self.stage.setup(m, self.p.i_data)
 754
 755         if not self.p.stage_ctl:
 756             return m
 757
 758         # intercept the previous (outgoing) "ready", combine with stage ready
 759         m.d.comb += self.p.s_o_ready.eq(self.p._o_ready & self.stage.d_ready)
 760
 761         # intercept the next (incoming) "ready" and combine it with data valid
 762         sdv = self.stage.d_valid(self.n.i_ready)
 763         m.d.comb += self.n.d_valid.eq(self.n.i_ready & sdv)
 764
 765         return m
 766
 767
 768 class BufferedHandshake(ControlBase):
 769     """ buffered pipeline stage.  data and strobe signals travel in sync.
 770         if ever the input is ready and the output is not, processed data
 771         is shunted in a temporary register.
 772
 773         Argument: stage.  see Stage API above
 774
 775         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 776         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 777         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 778                               |             |
 779                             process --->----^
 780                               |             |
 781                               +-- r_data ->-+
 782
 783         input data p.i_data is read (only), is processed and goes into an
 784         intermediate result store [process()].  this is updated combinatorially.
 785
 786         in a non-stall condition, the intermediate result will go into the
 787         output (update_output).  however if ever there is a stall, it goes
 788         into r_data instead [update_buffer()].
 789
 790         when the non-stall condition is released, r_data is the first
 791         to be transferred to the output [flush_buffer()], and the stall
 792         condition cleared.
 793
 794         on the next cycle (as long as stall is not raised again) the
 795         input may begin to be processed and transferred directly to output.
 796     """
 797
 798     def elaborate(self, platform):
 799         self.m = ControlBase._elaborate(self, platform)
 800
 801         result = self.stage.ospec()
 802         r_data = self.stage.ospec()
 803
 804         # establish some combinatorial temporaries
 805         o_n_validn = Signal(reset_less=True)
 806         n_i_ready = Signal(reset_less=True, name="n_i_rdy_data")
 807         nir_por = Signal(reset_less=True)
 808         nir_por_n = Signal(reset_less=True)
 809         p_i_valid = Signal(reset_less=True)
 810         nir_novn = Signal(reset_less=True)
 811         nirn_novn = Signal(reset_less=True)
 812         por_pivn = Signal(reset_less=True)
 813         npnn = Signal(reset_less=True)
 814         self.m.d.comb += [p_i_valid.eq(self.p.i_valid_test),
 815                      o_n_validn.eq(~self.n.o_valid),
 816                      n_i_ready.eq(self.n.i_ready_test),
 817                      nir_por.eq(n_i_ready & self.p._o_ready),
 818                      nir_por_n.eq(n_i_ready & ~self.p._o_ready),
 819                      nir_novn.eq(n_i_ready | o_n_validn),
 820                      nirn_novn.eq(~n_i_ready & o_n_validn),
 821                      npnn.eq(nir_por | nirn_novn),
 822                      por_pivn.eq(self.p._o_ready & ~p_i_valid)
 823         ]
 824
 825         # store result of processing in combinatorial temporary
 826         self.m.d.comb += eq(result, self.stage.process(self.p.i_data))
 827
 828         # if not in stall condition, update the temporary register
 829         with self.m.If(self.p.o_ready): # not stalled
 830             self.m.d.sync += eq(r_data, result) # update buffer
 831
 832         # data pass-through conditions
 833         with self.m.If(npnn):
 834             o_data = self._postprocess(result)
 835             self.m.d.sync += [self.n.o_valid.eq(p_i_valid), # valid if p_valid
 836                               eq(self.n.o_data, o_data),    # update output
 837                              ]
 838         # buffer flush conditions (NOTE: can override data passthru conditions)
 839         with self.m.If(nir_por_n): # not stalled
 840             # Flush the [already processed] buffer to the output port.
 841             o_data = self._postprocess(r_data)
 842             self.m.d.sync += [self.n.o_valid.eq(1),  # reg empty
 843                               eq(self.n.o_data, o_data), # flush buffer
 844                              ]
 845         # output ready conditions
 846         self.m.d.sync += self.p._o_ready.eq(nir_novn | por_pivn)
 847
 848         return self.m
 849
 850
 851 class SimpleHandshake(ControlBase):
 852     """ simple handshake control.  data and strobe signals travel in sync.
 853         implements the protocol used by Wishbone and AXI4.
 854
 855         Argument: stage.  see Stage API above
 856
 857         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 858         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 859         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 860                               |             |
 861                               +--process->--^
 862         Truth Table
 863
 864         Inputs   Temporary  Output Data
 865         -------  ---------- -----  ----
 866         P P N N  PiV& ~NiR&  N P
 867         i o i o  PoR  NoV    o o
 868         V R R V              V R
 869
 870         -------   -    -     - -
 871         0 0 0 0   0    0    >0 0    reg
 872         0 0 0 1   0    1    >1 0    reg
 873         0 0 1 0   0    0     0 1    process(i_data)
 874         0 0 1 1   0    0     0 1    process(i_data)
 875         -------   -    -     - -
 876         0 1 0 0   0    0    >0 0    reg
 877         0 1 0 1   0    1    >1 0    reg
 878         0 1 1 0   0    0     0 1    process(i_data)
 879         0 1 1 1   0    0     0 1    process(i_data)
 880         -------   -    -     - -
 881         1 0 0 0   0    0    >0 0    reg
 882         1 0 0 1   0    1    >1 0    reg
 883         1 0 1 0   0    0     0 1    process(i_data)
 884         1 0 1 1   0    0     0 1    process(i_data)
 885         -------   -    -     - -
 886         1 1 0 0   1    0     1 0    process(i_data)
 887         1 1 0 1   1    1     1 0    process(i_data)
 888         1 1 1 0   1    0     1 1    process(i_data)
 889         1 1 1 1   1    0     1 1    process(i_data)
 890         -------   -    -     - -
 891     """
 892
 893     def elaborate(self, platform):
 894         self.m = m = ControlBase._elaborate(self, platform)
 895
 896         r_busy = Signal()
 897         result = self.stage.ospec()
 898
 899         # establish some combinatorial temporaries
 900         n_i_ready = Signal(reset_less=True, name="n_i_rdy_data")
 901         p_i_valid_p_o_ready = Signal(reset_less=True)
 902         p_i_valid = Signal(reset_less=True)
 903         m.d.comb += [p_i_valid.eq(self.p.i_valid_test),
 904                      n_i_ready.eq(self.n.i_ready_test),
 905                      p_i_valid_p_o_ready.eq(p_i_valid & self.p.o_ready),
 906         ]
 907
 908         # store result of processing in combinatorial temporary
 909         m.d.comb += eq(result, self.stage.process(self.p.i_data))
 910
 911         # previous valid and ready
 912         with m.If(p_i_valid_p_o_ready):
 913             o_data = self._postprocess(result)
 914             m.d.sync += [r_busy.eq(1),      # output valid
 915                          eq(self.n.o_data, o_data), # update output
 916                         ]
 917         # previous invalid or not ready, however next is accepting
 918         with m.Elif(n_i_ready):
 919             o_data = self._postprocess(result)
 920             m.d.sync += [eq(self.n.o_data, o_data)]
 921             # TODO: could still send data here (if there was any)
 922             #m.d.sync += self.n.o_valid.eq(0) # ...so set output invalid
 923             m.d.sync += r_busy.eq(0) # ...so set output invalid
 924
 925         m.d.comb += self.n.o_valid.eq(r_busy)
 926         # if next is ready, so is previous
 927         m.d.comb += self.p._o_ready.eq(n_i_ready)
 928
 929         return self.m
 930
 931
 932 class UnbufferedPipeline(ControlBase):
 933     """ A simple pipeline stage with single-clock synchronisation
 934         and two-way valid/ready synchronised signalling.
 935
 936         Note that a stall in one stage will result in the entire pipeline
 937         chain stalling.
 938
 939         Also that unlike BufferedHandshake, the valid/ready signalling does NOT
 940         travel synchronously with the data: the valid/ready signalling
 941         combines in a *combinatorial* fashion.  Therefore, a long pipeline
 942         chain will lengthen propagation delays.
 943
 944         Argument: stage.  see Stage API, above
 945
 946         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 947         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 948         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 949                               |             |
 950                             r_data        result
 951                               |             |
 952                               +--process ->-+
 953
 954         Attributes:
 955         -----------
 956         p.i_data : StageInput, shaped according to ispec
 957             The pipeline input
 958         p.o_data : StageOutput, shaped according to ospec
 959             The pipeline output
 960         r_data : input_shape according to ispec
 961             A temporary (buffered) copy of a prior (valid) input.
 962             This is HELD if the output is not ready.  It is updated
 963             SYNCHRONOUSLY.
 964         result: output_shape according to ospec
 965             The output of the combinatorial logic.  it is updated
 966             COMBINATORIALLY (no clock dependence).
 967
 968         Truth Table
 969
 970         Inputs  Temp  Output  Data
 971         -------   -   -----   ----
 972         P P N N ~NiR&  N P
 973         i o i o  NoV   o o
 974         V R R V        V R
 975
 976         -------   -    - -
 977         0 0 0 0   0    0 1    reg
 978         0 0 0 1   1    1 0    reg
 979         0 0 1 0   0    0 1    reg
 980         0 0 1 1   0    0 1    reg
 981         -------   -    - -
 982         0 1 0 0   0    0 1    reg
 983         0 1 0 1   1    1 0    reg
 984         0 1 1 0   0    0 1    reg
 985         0 1 1 1   0    0 1    reg
 986         -------   -    - -
 987         1 0 0 0   0    1 1    reg
 988         1 0 0 1   1    1 0    reg
 989         1 0 1 0   0    1 1    reg
 990         1 0 1 1   0    1 1    reg
 991         -------   -    - -
 992         1 1 0 0   0    1 1    process(i_data)
 993         1 1 0 1   1    1 0    process(i_data)
 994         1 1 1 0   0    1 1    process(i_data)
 995         1 1 1 1   0    1 1    process(i_data)
 996         -------   -    - -
 997
 998         Note: PoR is *NOT* involved in the above decision-making.
 999     """
1000
1001     def elaborate(self, platform):
1002         self.m = m = ControlBase._elaborate(self, platform)
1003
1004         data_valid = Signal() # is data valid or not
1005         r_data = self.stage.ospec() # output type
1006
1007         # some temporaries
1008         p_i_valid = Signal(reset_less=True)
1009         pv = Signal(reset_less=True)
1010         buf_full = Signal(reset_less=True)
1011         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
1012         m.d.comb += pv.eq(self.p.i_valid & self.p.o_ready)
1013         m.d.comb += buf_full.eq(~self.n.i_ready_test & data_valid)
1014
1015         m.d.comb += self.n.o_valid.eq(data_valid)
1016         m.d.comb += self.p._o_ready.eq(~data_valid | self.n.i_ready_test)
1017         m.d.sync += data_valid.eq(p_i_valid | buf_full)
1018
1019         with m.If(pv):
1020             m.d.sync += eq(r_data, self.stage.process(self.p.i_data))
1021         o_data = self._postprocess(r_data)
1022         m.d.comb += eq(self.n.o_data, o_data)
1023
1024         return self.m
1025
1026 class UnbufferedPipeline2(ControlBase):
1027     """ A simple pipeline stage with single-clock synchronisation
1028         and two-way valid/ready synchronised signalling.
1029
1030         Note that a stall in one stage will result in the entire pipeline
1031         chain stalling.
1032
1033         Also that unlike BufferedHandshake, the valid/ready signalling does NOT
1034         travel synchronously with the data: the valid/ready signalling
1035         combines in a *combinatorial* fashion.  Therefore, a long pipeline
1036         chain will lengthen propagation delays.
1037
1038         Argument: stage.  see Stage API, above
1039
1040         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
1041         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
1042         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
1043                               |             |    |
1044                               +- process-> buf <-+
1045         Attributes:
1046         -----------
1047         p.i_data : StageInput, shaped according to ispec
1048             The pipeline input
1049         p.o_data : StageOutput, shaped according to ospec
1050             The pipeline output
1051         buf : output_shape according to ospec
1052             A temporary (buffered) copy of a valid output
1053             This is HELD if the output is not ready.  It is updated
1054             SYNCHRONOUSLY.
1055
1056         Inputs  Temp  Output Data
1057         -------   -   -----
1058         P P N N ~NiR&  N P   (buf_full)
1059         i o i o  NoV   o o
1060         V R R V        V R
1061
1062         -------   -    - -
1063         0 0 0 0   0    0 1   process(i_data)
1064         0 0 0 1   1    1 0   reg (odata, unchanged)
1065         0 0 1 0   0    0 1   process(i_data)
1066         0 0 1 1   0    0 1   process(i_data)
1067         -------   -    - -
1068         0 1 0 0   0    0 1   process(i_data)
1069         0 1 0 1   1    1 0   reg (odata, unchanged)
1070         0 1 1 0   0    0 1   process(i_data)
1071         0 1 1 1   0    0 1   process(i_data)
1072         -------   -    - -
1073         1 0 0 0   0    1 1   process(i_data)
1074         1 0 0 1   1    1 0   reg (odata, unchanged)
1075         1 0 1 0   0    1 1   process(i_data)
1076         1 0 1 1   0    1 1   process(i_data)
1077         -------   -    - -
1078         1 1 0 0   0    1 1   process(i_data)
1079         1 1 0 1   1    1 0   reg (odata, unchanged)
1080         1 1 1 0   0    1 1   process(i_data)
1081         1 1 1 1   0    1 1   process(i_data)
1082         -------   -    - -
1083
1084         Note: PoR is *NOT* involved in the above decision-making.
1085     """
1086
1087     def elaborate(self, platform):
1088         self.m = m = ControlBase._elaborate(self, platform)
1089
1090         buf_full = Signal() # is data valid or not
1091         buf = self.stage.ospec() # output type
1092
1093         # some temporaries
1094         p_i_valid = Signal(reset_less=True)
1095         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
1096
1097         m.d.comb += self.n.o_valid.eq(buf_full | p_i_valid)
1098         m.d.comb += self.p._o_ready.eq(~buf_full)
1099         m.d.sync += buf_full.eq(~self.n.i_ready_test & self.n.o_valid)
1100
1101         o_data = Mux(buf_full, buf, self.stage.process(self.p.i_data))
1102         o_data = self._postprocess(o_data)
1103         m.d.comb += eq(self.n.o_data, o_data)
1104         m.d.sync += eq(buf, self.n.o_data)
1105
1106         return self.m
1107
1108
1109 class PassThroughStage(StageCls):
1110     """ a pass-through stage which has its input data spec equal to its output,
1111         and "passes through" its data from input to output.
1112     """
1113     def __init__(self, iospecfn):
1114         self.iospecfn = iospecfn
1115     def ispec(self): return self.iospecfn()
1116     def ospec(self): return self.iospecfn()
1117     def process(self, i): return i
1118
1119
1120 class PassThroughHandshake(ControlBase):
1121     """ A control block that delays by one clock cycle.
1122
1123         Inputs   Temporary          Output Data
1124         -------  ------------------  ----- ----
1125         P P N N  PiV& PiV| NiR| pvr   N P  (pvr)
1126         i o i o  PoR  ~PoR ~NoV       o o
1127         V R R V                       V R
1128
1129         -------   -    -    -   -     - -
1130         0 0 0 0   0    1    1   0     1 1   odata (unchanged)
1131         0 0 0 1   0    1    0   0     1 0   odata (unchanged)
1132         0 0 1 0   0    1    1   0     1 1   odata (unchanged)
1133         0 0 1 1   0    1    1   0     1 1   odata (unchanged)
1134         -------   -    -    -   -     - -
1135         0 1 0 0   0    0    1   0     0 1   odata (unchanged)
1136         0 1 0 1   0    0    0   0     0 0   odata (unchanged)
1137         0 1 1 0   0    0    1   0     0 1   odata (unchanged)
1138         0 1 1 1   0    0    1   0     0 1   odata (unchanged)
1139         -------   -    -    -   -     - -
1140         1 0 0 0   0    1    1   1     1 1   process(in)
1141         1 0 0 1   0    1    0   0     1 0   odata (unchanged)
1142         1 0 1 0   0    1    1   1     1 1   process(in)
1143         1 0 1 1   0    1    1   1     1 1   process(in)
1144         -------   -    -    -   -     - -
1145         1 1 0 0   1    1    1   1     1 1   process(in)
1146         1 1 0 1   1    1    0   0     1 0   odata (unchanged)
1147         1 1 1 0   1    1    1   1     1 1   process(in)
1148         1 1 1 1   1    1    1   1     1 1   process(in)
1149         -------   -    -    -   -     - -
1150
1151     """
1152
1153     def elaborate(self, platform):
1154         self.m = m = ControlBase._elaborate(self, platform)
1155
1156         r_data = self.stage.ospec() # output type
1157
1158         # temporaries
1159         p_i_valid = Signal(reset_less=True)
1160         pvr = Signal(reset_less=True)
1161         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
1162         m.d.comb += pvr.eq(p_i_valid & self.p.o_ready)
1163
1164         m.d.comb += self.p.o_ready.eq(~self.n.o_valid |  self.n.i_ready_test)
1165         m.d.sync += self.n.o_valid.eq(p_i_valid       | ~self.p.o_ready)
1166
1167         odata = Mux(pvr, self.stage.process(self.p.i_data), r_data)
1168         m.d.sync += eq(r_data, odata)
1169         r_data = self._postprocess(r_data)
1170         m.d.comb += eq(self.n.o_data, r_data)
1171
1172         return m
1173
1174
1175 class RegisterPipeline(UnbufferedPipeline):
1176     """ A pipeline stage that delays by one clock cycle, creating a
1177         sync'd latch out of o_data and o_valid as an indirect byproduct
1178         of using PassThroughStage
1179     """
1180     def __init__(self, iospecfn):
1181         UnbufferedPipeline.__init__(self, PassThroughStage(iospecfn))
1182
1183
1184 class FIFOControl(ControlBase):
1185     """ FIFO Control.  Uses SyncFIFO to store data, coincidentally
1186         happens to have same valid/ready signalling as Stage API.
1187
1188         i_data -> fifo.din -> FIFO -> fifo.dout -> o_data
1189     """
1190
1191     def __init__(self, depth, stage, in_multi=None, stage_ctl=False,
1192                                      fwft=True, buffered=False, pipe=False):
1193         """ FIFO Control
1194
1195             * depth: number of entries in the FIFO
1196             * stage: data processing block
1197             * fwft : first word fall-thru mode (non-fwft introduces delay)
1198             * buffered: use buffered FIFO (introduces extra cycle delay)
1199
1200             NOTE 1: FPGAs may have trouble with the defaults for SyncFIFO
1201                     (fwft=True, buffered=False)
1202
1203             NOTE 2: i_data *must* have a shape function.  it can therefore
1204                     be a Signal, or a Record, or a RecordObject.
1205
1206             data is processed (and located) as follows:
1207
1208             self.p  self.stage temp    fn temp  fn  temp  fp   self.n
1209             i_data->process()->result->cat->din.FIFO.dout->cat(o_data)
1210
1211             yes, really: cat produces a Cat() which can be assigned to.
1212             this is how the FIFO gets de-catted without needing a de-cat
1213             function
1214         """
1215
1216         assert not (fwft and buffered), "buffered cannot do fwft"
1217         if buffered:
1218             depth += 1
1219         self.fwft = fwft
1220         self.buffered = buffered
1221         self.pipe = pipe
1222         self.fdepth = depth
1223         ControlBase.__init__(self, stage, in_multi, stage_ctl)
1224
1225     def elaborate(self, platform):
1226         self.m = m = ControlBase._elaborate(self, platform)
1227
1228         # make a FIFO with a signal of equal width to the o_data.
1229         (fwidth, _) = shape(self.n.o_data)
1230         if self.buffered:
1231             fifo = SyncFIFOBuffered(fwidth, self.fdepth)
1232         else:
1233             fifo = Queue(fwidth, self.fdepth, fwft=self.fwft, pipe=self.pipe)
1234         m.submodules.fifo = fifo
1235
1236         # store result of processing in combinatorial temporary
1237         result = self.stage.ospec()
1238         m.d.comb += eq(result, self.stage.process(self.p.i_data))
1239
1240         # connect previous rdy/valid/data - do cat on i_data
1241         # NOTE: cannot do the PrevControl-looking trick because
1242         # of need to process the data.  shaaaame....
1243         m.d.comb += [fifo.we.eq(self.p.i_valid_test),
1244                      self.p.o_ready.eq(fifo.writable),
1245                      eq(fifo.din, cat(result)),
1246                    ]
1247
1248         # connect next rdy/valid/data - do cat on o_data
1249         connections = [self.n.o_valid.eq(fifo.readable),
1250                      fifo.re.eq(self.n.i_ready_test),
1251                    ]
1252         if self.fwft or self.buffered:
1253             m.d.comb += connections
1254         else:
1255             m.d.sync += connections # unbuffered fwft mode needs sync
1256         o_data = cat(self.n.o_data).eq(fifo.dout)
1257         o_data = self._postprocess(o_data)
1258         m.d.comb += o_data
1259
1260         return m
1261
1262
1263 # aka "RegStage".
1264 class UnbufferedPipeline(FIFOControl):
1265     def __init__(self, stage, in_multi=None, stage_ctl=False):
1266         FIFOControl.__init__(self, 1, stage, in_multi, stage_ctl,
1267                                    fwft=True, pipe=False)
1268
1269 # aka "BreakReadyStage" XXX had to set fwft=True to get it to work
1270 class PassThroughHandshake(FIFOControl):
1271     def __init__(self, stage, in_multi=None, stage_ctl=False):
1272         FIFOControl.__init__(self, 1, stage, in_multi, stage_ctl,
1273                                    fwft=True, pipe=True)
1274
1275 # this is *probably* BufferedHandshake, although test #997 now succeeds.
1276 class BufferedHandshake(FIFOControl):
1277     def __init__(self, stage, in_multi=None, stage_ctl=False):
1278         FIFOControl.__init__(self, 2, stage, in_multi, stage_ctl,
1279                                    fwft=True, pipe=False)
1280
1281
1282 """
1283 # this is *probably* SimpleHandshake (note: memory cell size=0)
1284 class SimpleHandshake(FIFOControl):
1285     def __init__(self, stage, in_multi=None, stage_ctl=False):
1286         FIFOControl.__init__(self, 0, stage, in_multi, stage_ctl,
1287                                    fwft=True, pipe=False)
1288 """