src/add/singlepipe.py

   1 """ Pipeline and BufferedHandshake implementation, conforming to the same API.
   2     For multi-input and multi-output variants, see multipipe.
   3
   4     Associated development bugs:
   5     * http://bugs.libre-riscv.org/show_bug.cgi?id=64
   6     * http://bugs.libre-riscv.org/show_bug.cgi?id=57
   7
   8     eq:
   9     --
  10
  11     a strategically very important function that is identical in function
  12     to nmigen's Signal.eq function, except it may take objects, or a list
  13     of objects, or a tuple of objects, and where objects may also be
  14     Records.
  15
  16     Stage API:
  17     ---------
  18
  19     stage requires compliance with a strict API that may be
  20     implemented in several means, including as a static class.
  21     the methods of a stage instance must be as follows:
  22
  23     * ispec() - Input data format specification
  24                 returns an object or a list or tuple of objects, or
  25                 a Record, each object having an "eq" function which
  26                 takes responsibility for copying by assignment all
  27                 sub-objects
  28     * ospec() - Output data format specification
  29                 requirements as for ospec
  30     * process(m, i) - Processes an ispec-formatted object
  31                 returns a combinatorial block of a result that
  32                 may be assigned to the output, by way of the "eq"
  33                 function
  34     * setup(m, i) - Optional function for setting up submodules
  35                 may be used for more complex stages, to link
  36                 the input (i) to submodules.  must take responsibility
  37                 for adding those submodules to the module (m).
  38                 the submodules must be combinatorial blocks and
  39                 must have their inputs and output linked combinatorially.
  40
  41     Both StageCls (for use with non-static classes) and Stage (for use
  42     by static classes) are abstract classes from which, for convenience
  43     and as a courtesy to other developers, anything conforming to the
  44     Stage API may *choose* to derive.
  45
  46     StageChain:
  47     ----------
  48
  49     A useful combinatorial wrapper around stages that chains them together
  50     and then presents a Stage-API-conformant interface.  By presenting
  51     the same API as the stages it wraps, it can clearly be used recursively.
  52
  53     RecordBasedStage:
  54     ----------------
  55
  56     A convenience class that takes an input shape, output shape, a
  57     "processing" function and an optional "setup" function.  Honestly
  58     though, there's not much more effort to just... create a class
  59     that returns a couple of Records (see ExampleAddRecordStage in
  60     examples).
  61
  62     PassThroughStage:
  63     ----------------
  64
  65     A convenience class that takes a single function as a parameter,
  66     that is chain-called to create the exact same input and output spec.
  67     It has a process() function that simply returns its input.
  68
  69     Instances of this class are completely redundant if handed to
  70     StageChain, however when passed to UnbufferedPipeline they
  71     can be used to introduce a single clock delay.
  72
  73     ControlBase:
  74     -----------
  75
  76     The base class for pipelines.  Contains previous and next ready/valid/data.
  77     Also has an extremely useful "connect" function that can be used to
  78     connect a chain of pipelines and present the exact same prev/next
  79     ready/valid/data API.
  80
  81     UnbufferedPipeline:
  82     ------------------
  83
  84     A simple stalling clock-synchronised pipeline that has no buffering
  85     (unlike BufferedHandshake).  Data flows on *every* clock cycle when
  86     the conditions are right (this is nominally when the input is valid
  87     and the output is ready).
  88
  89     A stall anywhere along the line will result in a stall back-propagating
  90     down the entire chain.  The BufferedHandshake by contrast will buffer
  91     incoming data, allowing previous stages one clock cycle's grace before
  92     also having to stall.
  93
  94     An advantage of the UnbufferedPipeline over the Buffered one is
  95     that the amount of logic needed (number of gates) is greatly
  96     reduced (no second set of buffers basically)
  97
  98     The disadvantage of the UnbufferedPipeline is that the valid/ready
  99     logic, if chained together, is *combinatorial*, resulting in
 100     progressively larger gate delay.
 101
 102     PassThroughHandshake:
 103     ------------------
 104
 105     A Control class that introduces a single clock delay, passing its
 106     data through unaltered.  Unlike RegisterPipeline (which relies
 107     on UnbufferedPipeline and PassThroughStage) it handles ready/valid
 108     itself.
 109
 110     RegisterPipeline:
 111     ----------------
 112
 113     A convenience class that, because UnbufferedPipeline introduces a single
 114     clock delay, when its stage is a PassThroughStage, it results in a Pipeline
 115     stage that, duh, delays its (unmodified) input by one clock cycle.
 116
 117     BufferedHandshake:
 118     ----------------
 119
 120     nmigen implementation of buffered pipeline stage, based on zipcpu:
 121     https://zipcpu.com/blog/2017/08/14/strategies-for-pipelining.html
 122
 123     this module requires quite a bit of thought to understand how it works
 124     (and why it is needed in the first place).  reading the above is
 125     *strongly* recommended.
 126
 127     unlike john dawson's IEEE754 FPU STB/ACK signalling, which requires
 128     the STB / ACK signals to raise and lower (on separate clocks) before
 129     data may proceeed (thus only allowing one piece of data to proceed
 130     on *ALTERNATE* cycles), the signalling here is a true pipeline
 131     where data will flow on *every* clock when the conditions are right.
 132
 133     input acceptance conditions are when:
 134         * incoming previous-stage strobe (p.i_valid) is HIGH
 135         * outgoing previous-stage ready   (p.o_ready) is LOW
 136
 137     output transmission conditions are when:
 138         * outgoing next-stage strobe (n.o_valid) is HIGH
 139         * outgoing next-stage ready   (n.i_ready) is LOW
 140
 141     the tricky bit is when the input has valid data and the output is not
 142     ready to accept it.  if it wasn't for the clock synchronisation, it
 143     would be possible to tell the input "hey don't send that data, we're
 144     not ready".  unfortunately, it's not possible to "change the past":
 145     the previous stage *has no choice* but to pass on its data.
 146
 147     therefore, the incoming data *must* be accepted - and stored: that
 148     is the responsibility / contract that this stage *must* accept.
 149     on the same clock, it's possible to tell the input that it must
 150     not send any more data.  this is the "stall" condition.
 151
 152     we now effectively have *two* possible pieces of data to "choose" from:
 153     the buffered data, and the incoming data.  the decision as to which
 154     to process and output is based on whether we are in "stall" or not.
 155     i.e. when the next stage is no longer ready, the output comes from
 156     the buffer if a stall had previously occurred, otherwise it comes
 157     direct from processing the input.
 158
 159     this allows us to respect a synchronous "travelling STB" with what
 160     dan calls a "buffered handshake".
 161
 162     it's quite a complex state machine!
 163
 164     SimpleHandshake
 165     ---------------
 166
 167     Synchronised pipeline, Based on:
 168     https://github.com/ZipCPU/dbgbus/blob/master/hexbus/rtl/hbdeword.v
 169 """
 170
 171 from nmigen import Signal, Cat, Const, Mux, Module, Value
 172 from nmigen.cli import verilog, rtlil
 173 from nmigen.lib.fifo import SyncFIFO, SyncFIFOBuffered
 174 from nmigen.hdl.ast import ArrayProxy
 175 from nmigen.hdl.rec import Record, Layout
 176
 177 from abc import ABCMeta, abstractmethod
 178 from collections.abc import Sequence, Iterable
 179 from collections import OrderedDict
 180 from queue import Queue
 181
 182
 183 class Object:
 184     def __init__(self):
 185         self.fields = OrderedDict()
 186
 187     def __setattr__(self, k, v):
 188         print ("kv", k, v)
 189         if (k.startswith('_') or k in ["fields", "name", "src_loc"] or
 190            k in dir(Object) or "fields" not in self.__dict__):
 191             return object.__setattr__(self, k, v)
 192         self.fields[k] = v
 193
 194     def __getattr__(self, k):
 195         if k in self.__dict__:
 196             return object.__getattr__(self, k)
 197         try:
 198             return self.fields[k]
 199         except KeyError as e:
 200             raise AttributeError(e)
 201
 202     def __iter__(self):
 203         for x in self.fields.values():
 204             if isinstance(x, Iterable):
 205                 yield from x
 206             else:
 207                 yield x
 208
 209     def eq(self, inp):
 210         res = []
 211         for (k, o) in self.fields.items():
 212             i = getattr(inp, k)
 213             print ("eq", o, i)
 214             rres = o.eq(i)
 215             if isinstance(rres, Sequence):
 216                 res += rres
 217             else:
 218                 res.append(rres)
 219         print (res)
 220         return res
 221
 222     def ports(self):
 223         return list(self)
 224
 225
 226 class RecordObject(Record):
 227     def __init__(self, layout=None, name=None):
 228         Record.__init__(self, layout=layout or [], name=None)
 229
 230     def __setattr__(self, k, v):
 231         #print (dir(Record))
 232         if (k.startswith('_') or k in ["fields", "name", "src_loc"] or
 233            k in dir(Record) or "fields" not in self.__dict__):
 234             return object.__setattr__(self, k, v)
 235         self.fields[k] = v
 236         #print ("RecordObject setattr", k, v)
 237         if isinstance(v, Record):
 238             newlayout = {k: (k, v.layout)}
 239         elif isinstance(v, Value):
 240             newlayout = {k: (k, v.shape())}
 241         else:
 242             newlayout = {k: (k, shape(v))}
 243         self.layout.fields.update(newlayout)
 244
 245     def __iter__(self):
 246         for x in self.fields.values():
 247             if isinstance(x, Iterable):
 248                 yield from x
 249             else:
 250                 yield x
 251
 252     def ports(self):
 253         return list(self)
 254
 255
 256 class PrevControl:
 257     """ contains signals that come *from* the previous stage (both in and out)
 258         * i_valid: previous stage indicating all incoming data is valid.
 259                    may be a multi-bit signal, where all bits are required
 260                    to be asserted to indicate "valid".
 261         * o_ready: output to next stage indicating readiness to accept data
 262         * i_data : an input - added by the user of this class
 263     """
 264
 265     def __init__(self, i_width=1, stage_ctl=False):
 266         self.stage_ctl = stage_ctl
 267         self.i_valid = Signal(i_width, name="p_i_valid") # prev   >>in  self
 268         self._o_ready = Signal(name="p_o_ready") # prev   <<out self
 269         self.i_data = None # XXX MUST BE ADDED BY USER
 270         if stage_ctl:
 271             self.s_o_ready = Signal(name="p_s_o_rdy") # prev   <<out self
 272         self.trigger = Signal(reset_less=True)
 273
 274     @property
 275     def o_ready(self):
 276         """ public-facing API: indicates (externally) that stage is ready
 277         """
 278         if self.stage_ctl:
 279             return self.s_o_ready # set dynamically by stage
 280         return self._o_ready      # return this when not under dynamic control
 281
 282     def _connect_in(self, prev, direct=False, fn=None):
 283         """ internal helper function to connect stage to an input source.
 284             do not use to connect stage-to-stage!
 285         """
 286         i_valid = prev.i_valid if direct else prev.i_valid_test
 287         i_data = fn(prev.i_data) if fn is not None else prev.i_data
 288         return [self.i_valid.eq(i_valid),
 289                 prev.o_ready.eq(self.o_ready),
 290                 eq(self.i_data, i_data),
 291                ]
 292
 293     @property
 294     def i_valid_test(self):
 295         vlen = len(self.i_valid)
 296         if vlen > 1:
 297             # multi-bit case: valid only when i_valid is all 1s
 298             all1s = Const(-1, (len(self.i_valid), False))
 299             i_valid = (self.i_valid == all1s)
 300         else:
 301             # single-bit i_valid case
 302             i_valid = self.i_valid
 303
 304         # when stage indicates not ready, incoming data
 305         # must "appear" to be not ready too
 306         if self.stage_ctl:
 307             i_valid = i_valid & self.s_o_ready
 308
 309         return i_valid
 310
 311     def elaborate(self, platform):
 312         m = Module()
 313         m.d.comb += self.trigger.eq(self.i_valid_test & self.o_ready)
 314         return m
 315
 316     def eq(self, i):
 317         return [self.i_data.eq(i.i_data),
 318                 self.o_ready.eq(i.o_ready),
 319                 self.i_valid.eq(i.i_valid)]
 320
 321     def __iter__(self):
 322         yield self.i_valid
 323         yield self.o_ready
 324         if hasattr(self.i_data, "ports"):
 325             yield from self.i_data.ports()
 326         elif isinstance(self.i_data, Sequence):
 327             yield from self.i_data
 328         else:
 329             yield self.i_data
 330
 331     def ports(self):
 332         return list(self)
 333
 334
 335 class NextControl:
 336     """ contains the signals that go *to* the next stage (both in and out)
 337         * o_valid: output indicating to next stage that data is valid
 338         * i_ready: input from next stage indicating that it can accept data
 339         * o_data : an output - added by the user of this class
 340     """
 341     def __init__(self, stage_ctl=False):
 342         self.stage_ctl = stage_ctl
 343         self.o_valid = Signal(name="n_o_valid") # self out>>  next
 344         self.i_ready = Signal(name="n_i_ready") # self <<in   next
 345         self.o_data = None # XXX MUST BE ADDED BY USER
 346         #if self.stage_ctl:
 347         self.d_valid = Signal(reset=1) # INTERNAL (data valid)
 348         self.trigger = Signal(reset_less=True)
 349
 350     @property
 351     def i_ready_test(self):
 352         if self.stage_ctl:
 353             return self.i_ready & self.d_valid
 354         return self.i_ready
 355
 356     def connect_to_next(self, nxt):
 357         """ helper function to connect to the next stage data/valid/ready.
 358             data/valid is passed *TO* nxt, and ready comes *IN* from nxt.
 359             use this when connecting stage-to-stage
 360         """
 361         return [nxt.i_valid.eq(self.o_valid),
 362                 self.i_ready.eq(nxt.o_ready),
 363                 eq(nxt.i_data, self.o_data),
 364                ]
 365
 366     def _connect_out(self, nxt, direct=False, fn=None):
 367         """ internal helper function to connect stage to an output source.
 368             do not use to connect stage-to-stage!
 369         """
 370         i_ready = nxt.i_ready if direct else nxt.i_ready_test
 371         o_data = fn(nxt.o_data) if fn is not None else nxt.o_data
 372         return [nxt.o_valid.eq(self.o_valid),
 373                 self.i_ready.eq(i_ready),
 374                 eq(o_data, self.o_data),
 375                ]
 376
 377     def elaborate(self, platform):
 378         m = Module()
 379         m.d.comb += self.trigger.eq(self.i_ready_test & self.o_valid)
 380         return m
 381
 382     def __iter__(self):
 383         yield self.i_ready
 384         yield self.o_valid
 385         if hasattr(self.o_data, "ports"):
 386             yield from self.o_data.ports()
 387         elif isinstance(self.o_data, Sequence):
 388             yield from self.o_data
 389         else:
 390             yield self.o_data
 391
 392     def ports(self):
 393         return list(self)
 394
 395
 396 class Visitor2:
 397     """ a helper class for iterating twin-argument compound data structures.
 398
 399         Record is a special (unusual, recursive) case, where the input may be
 400         specified as a dictionary (which may contain further dictionaries,
 401         recursively), where the field names of the dictionary must match
 402         the Record's field spec.  Alternatively, an object with the same
 403         member names as the Record may be assigned: it does not have to
 404         *be* a Record.
 405
 406         ArrayProxy is also special-cased, it's a bit messy: whilst ArrayProxy
 407         has an eq function, the object being assigned to it (e.g. a python
 408         object) might not.  despite the *input* having an eq function,
 409         that doesn't help us, because it's the *ArrayProxy* that's being
 410         assigned to.  so.... we cheat.  use the ports() function of the
 411         python object, enumerate them, find out the list of Signals that way,
 412         and assign them.
 413     """
 414     def iterator2(self, o, i):
 415         if isinstance(o, dict):
 416             yield from self.dict_iter2(o, i)
 417
 418         if not isinstance(o, Sequence):
 419             o, i = [o], [i]
 420         for (ao, ai) in zip(o, i):
 421             #print ("visit", fn, ao, ai)
 422             if isinstance(ao, Record):
 423                 yield from self.record_iter2(ao, ai)
 424             elif isinstance(ao, ArrayProxy) and not isinstance(ai, Value):
 425                 yield from self.arrayproxy_iter2(ao, ai)
 426             else:
 427                 yield (ao, ai)
 428
 429     def dict_iter2(self, o, i):
 430         for (k, v) in o.items():
 431             print ("d-iter", v, i[k])
 432             yield (v, i[k])
 433         return res
 434
 435     def _not_quite_working_with_all_unit_tests_record_iter2(self, ao, ai):
 436         print ("record_iter2", ao, ai, type(ao), type(ai))
 437         if isinstance(ai, Value):
 438             if isinstance(ao, Sequence):
 439                 ao, ai = [ao], [ai]
 440             for o, i in zip(ao, ai):
 441                 yield (o, i)
 442             return
 443         for idx, (field_name, field_shape, _) in enumerate(ao.layout):
 444             if isinstance(field_shape, Layout):
 445                 val = ai.fields
 446             else:
 447                 val = ai
 448             if hasattr(val, field_name): # check for attribute
 449                 val = getattr(val, field_name)
 450             else:
 451                 val = val[field_name] # dictionary-style specification
 452             yield from self.iterator2(ao.fields[field_name], val)
 453
 454     def record_iter2(self, ao, ai):
 455         for idx, (field_name, field_shape, _) in enumerate(ao.layout):
 456             if isinstance(field_shape, Layout):
 457                 val = ai.fields
 458             else:
 459                 val = ai
 460             if hasattr(val, field_name): # check for attribute
 461                 val = getattr(val, field_name)
 462             else:
 463                 val = val[field_name] # dictionary-style specification
 464             yield from self.iterator2(ao.fields[field_name], val)
 465
 466     def arrayproxy_iter2(self, ao, ai):
 467         for p in ai.ports():
 468             op = getattr(ao, p.name)
 469             print ("arrayproxy - p", p, p.name)
 470             yield from self.iterator2(op, p)
 471
 472
 473 class Visitor:
 474     """ a helper class for iterating single-argument compound data structures.
 475         similar to Visitor2.
 476     """
 477     def iterate(self, i):
 478         """ iterate a compound structure recursively using yield
 479         """
 480         if not isinstance(i, Sequence):
 481             i = [i]
 482         for ai in i:
 483             #print ("iterate", ai)
 484             if isinstance(ai, Record):
 485                 #print ("record", list(ai.layout))
 486                 yield from self.record_iter(ai)
 487             elif isinstance(ai, ArrayProxy) and not isinstance(ai, Value):
 488                 yield from self.array_iter(ai)
 489             else:
 490                 yield ai
 491
 492     def record_iter(self, ai):
 493         for idx, (field_name, field_shape, _) in enumerate(ai.layout):
 494             if isinstance(field_shape, Layout):
 495                 val = ai.fields
 496             else:
 497                 val = ai
 498             if hasattr(val, field_name): # check for attribute
 499                 val = getattr(val, field_name)
 500             else:
 501                 val = val[field_name] # dictionary-style specification
 502             #print ("recidx", idx, field_name, field_shape, val)
 503             yield from self.iterate(val)
 504
 505     def array_iter(self, ai):
 506         for p in ai.ports():
 507             yield from self.iterate(p)
 508
 509
 510 def eq(o, i):
 511     """ makes signals equal: a helper routine which identifies if it is being
 512         passed a list (or tuple) of objects, or signals, or Records, and calls
 513         the objects' eq function.
 514     """
 515     res = []
 516     for (ao, ai) in Visitor2().iterator2(o, i):
 517         rres = ao.eq(ai)
 518         if not isinstance(rres, Sequence):
 519             rres = [rres]
 520         res += rres
 521     return res
 522
 523
 524 def shape(i):
 525     #print ("shape", i)
 526     r = 0
 527     for part in list(i):
 528         #print ("shape?", part)
 529         s, _ = part.shape()
 530         r += s
 531     return r, False
 532
 533
 534 def cat(i):
 535     """ flattens a compound structure recursively using Cat
 536     """
 537     from nmigen.tools import flatten
 538     #res = list(flatten(i)) # works (as of nmigen commit f22106e5) HOWEVER...
 539     res = list(Visitor().iterate(i)) # needed because input may be a sequence
 540     return Cat(*res)
 541
 542
 543 class StageCls(metaclass=ABCMeta):
 544     """ Class-based "Stage" API.  requires instantiation (after derivation)
 545
 546         see "Stage API" above..  Note: python does *not* require derivation
 547         from this class.  All that is required is that the pipelines *have*
 548         the functions listed in this class.  Derivation from this class
 549         is therefore merely a "courtesy" to maintainers.
 550     """
 551     @abstractmethod
 552     def ispec(self): pass       # REQUIRED
 553     @abstractmethod
 554     def ospec(self): pass       # REQUIRED
 555     #@abstractmethod
 556     #def setup(self, m, i): pass # OPTIONAL
 557     @abstractmethod
 558     def process(self, i): pass  # REQUIRED
 559
 560
 561 class Stage(metaclass=ABCMeta):
 562     """ Static "Stage" API.  does not require instantiation (after derivation)
 563
 564         see "Stage API" above.  Note: python does *not* require derivation
 565         from this class.  All that is required is that the pipelines *have*
 566         the functions listed in this class.  Derivation from this class
 567         is therefore merely a "courtesy" to maintainers.
 568     """
 569     @staticmethod
 570     @abstractmethod
 571     def ispec(): pass
 572
 573     @staticmethod
 574     @abstractmethod
 575     def ospec(): pass
 576
 577     #@staticmethod
 578     #@abstractmethod
 579     #def setup(m, i): pass
 580
 581     @staticmethod
 582     @abstractmethod
 583     def process(i): pass
 584
 585
 586 class RecordBasedStage(Stage):
 587     """ convenience class which provides a Records-based layout.
 588         honestly it's a lot easier just to create a direct Records-based
 589         class (see ExampleAddRecordStage)
 590     """
 591     def __init__(self, in_shape, out_shape, processfn, setupfn=None):
 592         self.in_shape = in_shape
 593         self.out_shape = out_shape
 594         self.__process = processfn
 595         self.__setup = setupfn
 596     def ispec(self): return Record(self.in_shape)
 597     def ospec(self): return Record(self.out_shape)
 598     def process(seif, i): return self.__process(i)
 599     def setup(seif, m, i): return self.__setup(m, i)
 600
 601
 602 class StageChain(StageCls):
 603     """ pass in a list of stages, and they will automatically be
 604         chained together via their input and output specs into a
 605         combinatorial chain.
 606
 607         the end result basically conforms to the exact same Stage API.
 608
 609         * input to this class will be the input of the first stage
 610         * output of first stage goes into input of second
 611         * output of second goes into input into third (etc. etc.)
 612         * the output of this class will be the output of the last stage
 613     """
 614     def __init__(self, chain, specallocate=False):
 615         self.chain = chain
 616         self.specallocate = specallocate
 617
 618     def ispec(self):
 619         return self.chain[0].ispec()
 620
 621     def ospec(self):
 622         return self.chain[-1].ospec()
 623
 624     def _specallocate_setup(self, m, i):
 625         for (idx, c) in enumerate(self.chain):
 626             if hasattr(c, "setup"):
 627                 c.setup(m, i)               # stage may have some module stuff
 628             o = self.chain[idx].ospec()     # last assignment survives
 629             m.d.comb += eq(o, c.process(i)) # process input into "o"
 630             if idx == len(self.chain)-1:
 631                 break
 632             i = self.chain[idx+1].ispec()   # new input on next loop
 633             m.d.comb += eq(i, o)            # assign to next input
 634         return o                            # last loop is the output
 635
 636     def _noallocate_setup(self, m, i):
 637         for (idx, c) in enumerate(self.chain):
 638             if hasattr(c, "setup"):
 639                 c.setup(m, i)               # stage may have some module stuff
 640             i = o = c.process(i)            # store input into "o"
 641         return o                            # last loop is the output
 642
 643     def setup(self, m, i):
 644         if self.specallocate:
 645             self.o = self._specallocate_setup(m, i)
 646         else:
 647             self.o = self._noallocate_setup(m, i)
 648
 649     def process(self, i):
 650         return self.o # conform to Stage API: return last-loop output
 651
 652
 653 class ControlBase:
 654     """ Common functions for Pipeline API
 655     """
 656     def __init__(self, stage=None, in_multi=None, stage_ctl=False):
 657         """ Base class containing ready/valid/data to previous and next stages
 658
 659             * p: contains ready/valid to the previous stage
 660             * n: contains ready/valid to the next stage
 661
 662             Except when calling Controlbase.connect(), user must also:
 663             * add i_data member to PrevControl (p) and
 664             * add o_data member to NextControl (n)
 665         """
 666         self.stage = stage
 667
 668         # set up input and output IO ACK (prev/next ready/valid)
 669         self.p = PrevControl(in_multi, stage_ctl)
 670         self.n = NextControl(stage_ctl)
 671
 672         # set up the input and output data
 673         if stage is not None:
 674             self.p.i_data = stage.ispec() # input type
 675             self.n.o_data = stage.ospec()
 676
 677     def connect_to_next(self, nxt):
 678         """ helper function to connect to the next stage data/valid/ready.
 679         """
 680         return self.n.connect_to_next(nxt.p)
 681
 682     def _connect_in(self, prev):
 683         """ internal helper function to connect stage to an input source.
 684             do not use to connect stage-to-stage!
 685         """
 686         return self.p._connect_in(prev.p)
 687
 688     def _connect_out(self, nxt):
 689         """ internal helper function to connect stage to an output source.
 690             do not use to connect stage-to-stage!
 691         """
 692         return self.n._connect_out(nxt.n)
 693
 694     def connect(self, pipechain):
 695         """ connects a chain (list) of Pipeline instances together and
 696             links them to this ControlBase instance:
 697
 698                       in <----> self <---> out
 699                        |                   ^
 700                        v                   |
 701                     [pipe1, pipe2, pipe3, pipe4]
 702                        |    ^  |    ^  |     ^
 703                        v    |  v    |  v     |
 704                      out---in out--in out---in
 705
 706             Also takes care of allocating i_data/o_data, by looking up
 707             the data spec for each end of the pipechain.  i.e It is NOT
 708             necessary to allocate self.p.i_data or self.n.o_data manually:
 709             this is handled AUTOMATICALLY, here.
 710
 711             Basically this function is the direct equivalent of StageChain,
 712             except that unlike StageChain, the Pipeline logic is followed.
 713
 714             Just as StageChain presents an object that conforms to the
 715             Stage API from a list of objects that also conform to the
 716             Stage API, an object that calls this Pipeline connect function
 717             has the exact same pipeline API as the list of pipline objects
 718             it is called with.
 719
 720             Thus it becomes possible to build up larger chains recursively.
 721             More complex chains (multi-input, multi-output) will have to be
 722             done manually.
 723         """
 724         eqs = [] # collated list of assignment statements
 725
 726         # connect inter-chain
 727         for i in range(len(pipechain)-1):
 728             pipe1 = pipechain[i]
 729             pipe2 = pipechain[i+1]
 730             eqs += pipe1.connect_to_next(pipe2)
 731
 732         # connect front of chain to ourselves
 733         front = pipechain[0]
 734         self.p.i_data = front.stage.ispec()
 735         eqs += front._connect_in(self)
 736
 737         # connect end of chain to ourselves
 738         end = pipechain[-1]
 739         self.n.o_data = end.stage.ospec()
 740         eqs += end._connect_out(self)
 741
 742         return eqs
 743
 744     def _postprocess(self, i): # XXX DISABLED
 745         return i # RETURNS INPUT
 746         if hasattr(self.stage, "postprocess"):
 747             return self.stage.postprocess(i)
 748         return i
 749
 750     def set_input(self, i):
 751         """ helper function to set the input data
 752         """
 753         return eq(self.p.i_data, i)
 754
 755     def __iter__(self):
 756         yield from self.p
 757         yield from self.n
 758
 759     def ports(self):
 760         return list(self)
 761
 762     def _elaborate(self, platform):
 763         """ handles case where stage has dynamic ready/valid functions
 764         """
 765         m = Module()
 766         m.submodules.p = self.p
 767         m.submodules.n = self.n
 768
 769         if self.stage is not None and hasattr(self.stage, "setup"):
 770             self.stage.setup(m, self.p.i_data)
 771
 772         if not self.p.stage_ctl:
 773             return m
 774
 775         # intercept the previous (outgoing) "ready", combine with stage ready
 776         m.d.comb += self.p.s_o_ready.eq(self.p._o_ready & self.stage.d_ready)
 777
 778         # intercept the next (incoming) "ready" and combine it with data valid
 779         sdv = self.stage.d_valid(self.n.i_ready)
 780         m.d.comb += self.n.d_valid.eq(self.n.i_ready & sdv)
 781
 782         return m
 783
 784
 785 class BufferedHandshake(ControlBase):
 786     """ buffered pipeline stage.  data and strobe signals travel in sync.
 787         if ever the input is ready and the output is not, processed data
 788         is shunted in a temporary register.
 789
 790         Argument: stage.  see Stage API above
 791
 792         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 793         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 794         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 795                               |             |
 796                             process --->----^
 797                               |             |
 798                               +-- r_data ->-+
 799
 800         input data p.i_data is read (only), is processed and goes into an
 801         intermediate result store [process()].  this is updated combinatorially.
 802
 803         in a non-stall condition, the intermediate result will go into the
 804         output (update_output).  however if ever there is a stall, it goes
 805         into r_data instead [update_buffer()].
 806
 807         when the non-stall condition is released, r_data is the first
 808         to be transferred to the output [flush_buffer()], and the stall
 809         condition cleared.
 810
 811         on the next cycle (as long as stall is not raised again) the
 812         input may begin to be processed and transferred directly to output.
 813     """
 814
 815     def elaborate(self, platform):
 816         self.m = ControlBase._elaborate(self, platform)
 817
 818         result = self.stage.ospec()
 819         r_data = self.stage.ospec()
 820
 821         # establish some combinatorial temporaries
 822         o_n_validn = Signal(reset_less=True)
 823         n_i_ready = Signal(reset_less=True, name="n_i_rdy_data")
 824         nir_por = Signal(reset_less=True)
 825         nir_por_n = Signal(reset_less=True)
 826         p_i_valid = Signal(reset_less=True)
 827         nir_novn = Signal(reset_less=True)
 828         nirn_novn = Signal(reset_less=True)
 829         por_pivn = Signal(reset_less=True)
 830         npnn = Signal(reset_less=True)
 831         self.m.d.comb += [p_i_valid.eq(self.p.i_valid_test),
 832                      o_n_validn.eq(~self.n.o_valid),
 833                      n_i_ready.eq(self.n.i_ready_test),
 834                      nir_por.eq(n_i_ready & self.p._o_ready),
 835                      nir_por_n.eq(n_i_ready & ~self.p._o_ready),
 836                      nir_novn.eq(n_i_ready | o_n_validn),
 837                      nirn_novn.eq(~n_i_ready & o_n_validn),
 838                      npnn.eq(nir_por | nirn_novn),
 839                      por_pivn.eq(self.p._o_ready & ~p_i_valid)
 840         ]
 841
 842         # store result of processing in combinatorial temporary
 843         self.m.d.comb += eq(result, self.stage.process(self.p.i_data))
 844
 845         # if not in stall condition, update the temporary register
 846         with self.m.If(self.p.o_ready): # not stalled
 847             self.m.d.sync += eq(r_data, result) # update buffer
 848
 849         # data pass-through conditions
 850         with self.m.If(npnn):
 851             o_data = self._postprocess(result)
 852             self.m.d.sync += [self.n.o_valid.eq(p_i_valid), # valid if p_valid
 853                               eq(self.n.o_data, o_data),    # update output
 854                              ]
 855         # buffer flush conditions (NOTE: can override data passthru conditions)
 856         with self.m.If(nir_por_n): # not stalled
 857             # Flush the [already processed] buffer to the output port.
 858             o_data = self._postprocess(r_data)
 859             self.m.d.sync += [self.n.o_valid.eq(1),  # reg empty
 860                               eq(self.n.o_data, o_data), # flush buffer
 861                              ]
 862         # output ready conditions
 863         self.m.d.sync += self.p._o_ready.eq(nir_novn | por_pivn)
 864
 865         return self.m
 866
 867
 868 class SimpleHandshake(ControlBase):
 869     """ simple handshake control.  data and strobe signals travel in sync.
 870         implements the protocol used by Wishbone and AXI4.
 871
 872         Argument: stage.  see Stage API above
 873
 874         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 875         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 876         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 877                               |             |
 878                               +--process->--^
 879         Truth Table
 880
 881         Inputs   Temporary  Output Data
 882         -------  ---------- -----  ----
 883         P P N N  PiV& ~NiR&  N P
 884         i o i o  PoR  NoV    o o
 885         V R R V              V R
 886
 887         -------   -    -     - -
 888         0 0 0 0   0    0    >0 0    reg
 889         0 0 0 1   0    1    >1 0    reg
 890         0 0 1 0   0    0     0 1    process(i_data)
 891         0 0 1 1   0    0     0 1    process(i_data)
 892         -------   -    -     - -
 893         0 1 0 0   0    0    >0 0    reg
 894         0 1 0 1   0    1    >1 0    reg
 895         0 1 1 0   0    0     0 1    process(i_data)
 896         0 1 1 1   0    0     0 1    process(i_data)
 897         -------   -    -     - -
 898         1 0 0 0   0    0    >0 0    reg
 899         1 0 0 1   0    1    >1 0    reg
 900         1 0 1 0   0    0     0 1    process(i_data)
 901         1 0 1 1   0    0     0 1    process(i_data)
 902         -------   -    -     - -
 903         1 1 0 0   1    0     1 0    process(i_data)
 904         1 1 0 1   1    1     1 0    process(i_data)
 905         1 1 1 0   1    0     1 1    process(i_data)
 906         1 1 1 1   1    0     1 1    process(i_data)
 907         -------   -    -     - -
 908     """
 909
 910     def elaborate(self, platform):
 911         self.m = m = ControlBase._elaborate(self, platform)
 912
 913         r_busy = Signal()
 914         result = self.stage.ospec()
 915
 916         # establish some combinatorial temporaries
 917         n_i_ready = Signal(reset_less=True, name="n_i_rdy_data")
 918         p_i_valid_p_o_ready = Signal(reset_less=True)
 919         p_i_valid = Signal(reset_less=True)
 920         m.d.comb += [p_i_valid.eq(self.p.i_valid_test),
 921                      n_i_ready.eq(self.n.i_ready_test),
 922                      p_i_valid_p_o_ready.eq(p_i_valid & self.p.o_ready),
 923         ]
 924
 925         # store result of processing in combinatorial temporary
 926         m.d.comb += eq(result, self.stage.process(self.p.i_data))
 927
 928         # previous valid and ready
 929         with m.If(p_i_valid_p_o_ready):
 930             o_data = self._postprocess(result)
 931             m.d.sync += [r_busy.eq(1),      # output valid
 932                          eq(self.n.o_data, o_data), # update output
 933                         ]
 934         # previous invalid or not ready, however next is accepting
 935         with m.Elif(n_i_ready):
 936             o_data = self._postprocess(result)
 937             m.d.sync += [eq(self.n.o_data, o_data)]
 938             # TODO: could still send data here (if there was any)
 939             #m.d.sync += self.n.o_valid.eq(0) # ...so set output invalid
 940             m.d.sync += r_busy.eq(0) # ...so set output invalid
 941
 942         m.d.comb += self.n.o_valid.eq(r_busy)
 943         # if next is ready, so is previous
 944         m.d.comb += self.p._o_ready.eq(n_i_ready)
 945
 946         return self.m
 947
 948
 949 class UnbufferedPipeline(ControlBase):
 950     """ A simple pipeline stage with single-clock synchronisation
 951         and two-way valid/ready synchronised signalling.
 952
 953         Note that a stall in one stage will result in the entire pipeline
 954         chain stalling.
 955
 956         Also that unlike BufferedHandshake, the valid/ready signalling does NOT
 957         travel synchronously with the data: the valid/ready signalling
 958         combines in a *combinatorial* fashion.  Therefore, a long pipeline
 959         chain will lengthen propagation delays.
 960
 961         Argument: stage.  see Stage API, above
 962
 963         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
 964         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
 965         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
 966                               |             |
 967                             r_data        result
 968                               |             |
 969                               +--process ->-+
 970
 971         Attributes:
 972         -----------
 973         p.i_data : StageInput, shaped according to ispec
 974             The pipeline input
 975         p.o_data : StageOutput, shaped according to ospec
 976             The pipeline output
 977         r_data : input_shape according to ispec
 978             A temporary (buffered) copy of a prior (valid) input.
 979             This is HELD if the output is not ready.  It is updated
 980             SYNCHRONOUSLY.
 981         result: output_shape according to ospec
 982             The output of the combinatorial logic.  it is updated
 983             COMBINATORIALLY (no clock dependence).
 984
 985         Truth Table
 986
 987         Inputs  Temp  Output  Data
 988         -------   -   -----   ----
 989         P P N N ~NiR&  N P
 990         i o i o  NoV   o o
 991         V R R V        V R
 992
 993         -------   -    - -
 994         0 0 0 0   0    0 1    reg
 995         0 0 0 1   1    1 0    reg
 996         0 0 1 0   0    0 1    reg
 997         0 0 1 1   0    0 1    reg
 998         -------   -    - -
 999         0 1 0 0   0    0 1    reg
1000         0 1 0 1   1    1 0    reg
1001         0 1 1 0   0    0 1    reg
1002         0 1 1 1   0    0 1    reg
1003         -------   -    - -
1004         1 0 0 0   0    1 1    reg
1005         1 0 0 1   1    1 0    reg
1006         1 0 1 0   0    1 1    reg
1007         1 0 1 1   0    1 1    reg
1008         -------   -    - -
1009         1 1 0 0   0    1 1    process(i_data)
1010         1 1 0 1   1    1 0    process(i_data)
1011         1 1 1 0   0    1 1    process(i_data)
1012         1 1 1 1   0    1 1    process(i_data)
1013         -------   -    - -
1014
1015         Note: PoR is *NOT* involved in the above decision-making.
1016     """
1017
1018     def elaborate(self, platform):
1019         self.m = m = ControlBase._elaborate(self, platform)
1020
1021         data_valid = Signal() # is data valid or not
1022         r_data = self.stage.ospec() # output type
1023
1024         # some temporaries
1025         p_i_valid = Signal(reset_less=True)
1026         pv = Signal(reset_less=True)
1027         buf_full = Signal(reset_less=True)
1028         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
1029         m.d.comb += pv.eq(self.p.i_valid & self.p.o_ready)
1030         m.d.comb += buf_full.eq(~self.n.i_ready_test & data_valid)
1031
1032         m.d.comb += self.n.o_valid.eq(data_valid)
1033         m.d.comb += self.p._o_ready.eq(~data_valid | self.n.i_ready_test)
1034         m.d.sync += data_valid.eq(p_i_valid | buf_full)
1035
1036         with m.If(pv):
1037             m.d.sync += eq(r_data, self.stage.process(self.p.i_data))
1038         o_data = self._postprocess(r_data)
1039         m.d.comb += eq(self.n.o_data, o_data)
1040
1041         return self.m
1042
1043 class UnbufferedPipeline2(ControlBase):
1044     """ A simple pipeline stage with single-clock synchronisation
1045         and two-way valid/ready synchronised signalling.
1046
1047         Note that a stall in one stage will result in the entire pipeline
1048         chain stalling.
1049
1050         Also that unlike BufferedHandshake, the valid/ready signalling does NOT
1051         travel synchronously with the data: the valid/ready signalling
1052         combines in a *combinatorial* fashion.  Therefore, a long pipeline
1053         chain will lengthen propagation delays.
1054
1055         Argument: stage.  see Stage API, above
1056
1057         stage-1   p.i_valid >>in   stage   n.o_valid out>>   stage+1
1058         stage-1   p.o_ready <<out  stage   n.i_ready <<in    stage+1
1059         stage-1   p.i_data  >>in   stage   n.o_data  out>>   stage+1
1060                               |             |    |
1061                               +- process-> buf <-+
1062         Attributes:
1063         -----------
1064         p.i_data : StageInput, shaped according to ispec
1065             The pipeline input
1066         p.o_data : StageOutput, shaped according to ospec
1067             The pipeline output
1068         buf : output_shape according to ospec
1069             A temporary (buffered) copy of a valid output
1070             This is HELD if the output is not ready.  It is updated
1071             SYNCHRONOUSLY.
1072
1073         Inputs  Temp  Output Data
1074         -------   -   -----
1075         P P N N ~NiR&  N P   (buf_full)
1076         i o i o  NoV   o o
1077         V R R V        V R
1078
1079         -------   -    - -
1080         0 0 0 0   0    0 1   process(i_data)
1081         0 0 0 1   1    1 0   reg (odata, unchanged)
1082         0 0 1 0   0    0 1   process(i_data)
1083         0 0 1 1   0    0 1   process(i_data)
1084         -------   -    - -
1085         0 1 0 0   0    0 1   process(i_data)
1086         0 1 0 1   1    1 0   reg (odata, unchanged)
1087         0 1 1 0   0    0 1   process(i_data)
1088         0 1 1 1   0    0 1   process(i_data)
1089         -------   -    - -
1090         1 0 0 0   0    1 1   process(i_data)
1091         1 0 0 1   1    1 0   reg (odata, unchanged)
1092         1 0 1 0   0    1 1   process(i_data)
1093         1 0 1 1   0    1 1   process(i_data)
1094         -------   -    - -
1095         1 1 0 0   0    1 1   process(i_data)
1096         1 1 0 1   1    1 0   reg (odata, unchanged)
1097         1 1 1 0   0    1 1   process(i_data)
1098         1 1 1 1   0    1 1   process(i_data)
1099         -------   -    - -
1100
1101         Note: PoR is *NOT* involved in the above decision-making.
1102     """
1103
1104     def elaborate(self, platform):
1105         self.m = m = ControlBase._elaborate(self, platform)
1106
1107         buf_full = Signal() # is data valid or not
1108         buf = self.stage.ospec() # output type
1109
1110         # some temporaries
1111         p_i_valid = Signal(reset_less=True)
1112         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
1113
1114         m.d.comb += self.n.o_valid.eq(buf_full | p_i_valid)
1115         m.d.comb += self.p._o_ready.eq(~buf_full)
1116         m.d.sync += buf_full.eq(~self.n.i_ready_test & self.n.o_valid)
1117
1118         o_data = Mux(buf_full, buf, self.stage.process(self.p.i_data))
1119         o_data = self._postprocess(o_data)
1120         m.d.comb += eq(self.n.o_data, o_data)
1121         m.d.sync += eq(buf, self.n.o_data)
1122
1123         return self.m
1124
1125
1126 class PassThroughStage(StageCls):
1127     """ a pass-through stage which has its input data spec equal to its output,
1128         and "passes through" its data from input to output.
1129     """
1130     def __init__(self, iospecfn):
1131         self.iospecfn = iospecfn
1132     def ispec(self): return self.iospecfn()
1133     def ospec(self): return self.iospecfn()
1134     def process(self, i): return i
1135
1136
1137 class PassThroughHandshake(ControlBase):
1138     """ A control block that delays by one clock cycle.
1139
1140         Inputs   Temporary          Output Data
1141         -------  ------------------  ----- ----
1142         P P N N  PiV& PiV| NiR| pvr   N P  (pvr)
1143         i o i o  PoR  ~PoR ~NoV       o o
1144         V R R V                       V R
1145
1146         -------   -    -    -   -     - -
1147         0 0 0 0   0    1    1   0     1 1   odata (unchanged)
1148         0 0 0 1   0    1    0   0     1 0   odata (unchanged)
1149         0 0 1 0   0    1    1   0     1 1   odata (unchanged)
1150         0 0 1 1   0    1    1   0     1 1   odata (unchanged)
1151         -------   -    -    -   -     - -
1152         0 1 0 0   0    0    1   0     0 1   odata (unchanged)
1153         0 1 0 1   0    0    0   0     0 0   odata (unchanged)
1154         0 1 1 0   0    0    1   0     0 1   odata (unchanged)
1155         0 1 1 1   0    0    1   0     0 1   odata (unchanged)
1156         -------   -    -    -   -     - -
1157         1 0 0 0   0    1    1   1     1 1   process(in)
1158         1 0 0 1   0    1    0   0     1 0   odata (unchanged)
1159         1 0 1 0   0    1    1   1     1 1   process(in)
1160         1 0 1 1   0    1    1   1     1 1   process(in)
1161         -------   -    -    -   -     - -
1162         1 1 0 0   1    1    1   1     1 1   process(in)
1163         1 1 0 1   1    1    0   0     1 0   odata (unchanged)
1164         1 1 1 0   1    1    1   1     1 1   process(in)
1165         1 1 1 1   1    1    1   1     1 1   process(in)
1166         -------   -    -    -   -     - -
1167
1168     """
1169
1170     def elaborate(self, platform):
1171         self.m = m = ControlBase._elaborate(self, platform)
1172
1173         r_data = self.stage.ospec() # output type
1174
1175         # temporaries
1176         p_i_valid = Signal(reset_less=True)
1177         pvr = Signal(reset_less=True)
1178         m.d.comb += p_i_valid.eq(self.p.i_valid_test)
1179         m.d.comb += pvr.eq(p_i_valid & self.p.o_ready)
1180
1181         m.d.comb += self.p.o_ready.eq(~self.n.o_valid |  self.n.i_ready_test)
1182         m.d.sync += self.n.o_valid.eq(p_i_valid       | ~self.p.o_ready)
1183
1184         odata = Mux(pvr, self.stage.process(self.p.i_data), r_data)
1185         m.d.sync += eq(r_data, odata)
1186         r_data = self._postprocess(r_data)
1187         m.d.comb += eq(self.n.o_data, r_data)
1188
1189         return m
1190
1191
1192 class RegisterPipeline(UnbufferedPipeline):
1193     """ A pipeline stage that delays by one clock cycle, creating a
1194         sync'd latch out of o_data and o_valid as an indirect byproduct
1195         of using PassThroughStage
1196     """
1197     def __init__(self, iospecfn):
1198         UnbufferedPipeline.__init__(self, PassThroughStage(iospecfn))
1199
1200
1201 class FIFOControl(ControlBase):
1202     """ FIFO Control.  Uses SyncFIFO to store data, coincidentally
1203         happens to have same valid/ready signalling as Stage API.
1204
1205         i_data -> fifo.din -> FIFO -> fifo.dout -> o_data
1206     """
1207
1208     def __init__(self, depth, stage, in_multi=None, stage_ctl=False,
1209                                      fwft=True, buffered=False, pipe=False):
1210         """ FIFO Control
1211
1212             * depth: number of entries in the FIFO
1213             * stage: data processing block
1214             * fwft : first word fall-thru mode (non-fwft introduces delay)
1215             * buffered: use buffered FIFO (introduces extra cycle delay)
1216
1217             NOTE 1: FPGAs may have trouble with the defaults for SyncFIFO
1218                     (fwft=True, buffered=False)
1219
1220             NOTE 2: i_data *must* have a shape function.  it can therefore
1221                     be a Signal, or a Record, or a RecordObject.
1222
1223             data is processed (and located) as follows:
1224
1225             self.p  self.stage temp    fn temp  fn  temp  fp   self.n
1226             i_data->process()->result->cat->din.FIFO.dout->cat(o_data)
1227
1228             yes, really: cat produces a Cat() which can be assigned to.
1229             this is how the FIFO gets de-catted without needing a de-cat
1230             function
1231         """
1232
1233         assert not (fwft and buffered), "buffered cannot do fwft"
1234         if buffered:
1235             depth += 1
1236         self.fwft = fwft
1237         self.buffered = buffered
1238         self.pipe = pipe
1239         self.fdepth = depth
1240         ControlBase.__init__(self, stage, in_multi, stage_ctl)
1241
1242     def elaborate(self, platform):
1243         self.m = m = ControlBase._elaborate(self, platform)
1244
1245         # make a FIFO with a signal of equal width to the o_data.
1246         (fwidth, _) = shape(self.n.o_data)
1247         if self.buffered:
1248             fifo = SyncFIFOBuffered(fwidth, self.fdepth)
1249         else:
1250             fifo = Queue(fwidth, self.fdepth, fwft=self.fwft, pipe=self.pipe)
1251         m.submodules.fifo = fifo
1252
1253         # store result of processing in combinatorial temporary
1254         result = self.stage.ospec()
1255         m.d.comb += eq(result, self.stage.process(self.p.i_data))
1256
1257         # connect previous rdy/valid/data - do cat on i_data
1258         # NOTE: cannot do the PrevControl-looking trick because
1259         # of need to process the data.  shaaaame....
1260         m.d.comb += [fifo.we.eq(self.p.i_valid_test),
1261                      self.p.o_ready.eq(fifo.writable),
1262                      eq(fifo.din, cat(result)),
1263                    ]
1264
1265         # connect next rdy/valid/data - do cat on o_data
1266         connections = [self.n.o_valid.eq(fifo.readable),
1267                      fifo.re.eq(self.n.i_ready_test),
1268                    ]
1269         if self.fwft or self.buffered:
1270             m.d.comb += connections
1271         else:
1272             m.d.sync += connections # unbuffered fwft mode needs sync
1273         o_data = cat(self.n.o_data).eq(fifo.dout)
1274         o_data = self._postprocess(o_data)
1275         m.d.comb += o_data
1276
1277         return m
1278
1279
1280 # aka "RegStage".
1281 class UnbufferedPipeline(FIFOControl):
1282     def __init__(self, stage, in_multi=None, stage_ctl=False):
1283         FIFOControl.__init__(self, 1, stage, in_multi, stage_ctl,
1284                                    fwft=True, pipe=False)
1285
1286 # aka "BreakReadyStage" XXX had to set fwft=True to get it to work
1287 class PassThroughHandshake(FIFOControl):
1288     def __init__(self, stage, in_multi=None, stage_ctl=False):
1289         FIFOControl.__init__(self, 1, stage, in_multi, stage_ctl,
1290                                    fwft=True, pipe=True)
1291
1292 # this is *probably* BufferedHandshake, although test #997 now succeeds.
1293 class BufferedHandshake(FIFOControl):
1294     def __init__(self, stage, in_multi=None, stage_ctl=False):
1295         FIFOControl.__init__(self, 2, stage, in_multi, stage_ctl,
1296                                    fwft=True, pipe=False)
1297
1298
1299 """
1300 # this is *probably* SimpleHandshake (note: memory cell size=0)
1301 class SimpleHandshake(FIFOControl):
1302     def __init__(self, stage, in_multi=None, stage_ctl=False):
1303         FIFOControl.__init__(self, 0, stage, in_multi, stage_ctl,
1304                                    fwft=True, pipe=False)
1305 """