# How to add a new peripheral This document describes the process of adding a new peripheral to the pinmux and auto-generator, through a worked example, adding SDRAM. # Creating the specifications The tool is split into two halves that are separated by tab-separated files. The first step is therefore to add a function that defines the peripheral as a python function. That implies in turn that the pinouts of the peripheral must be known. Looking at the BSV code for the SDRAM peripheral, we find its interface is defined as follows: interface Ifc_sdram_out; (*always_enabled,always_ready*) method Action ipad_sdr_din(Bit#(64) pad_sdr_din); method Bit#(9) sdram_sdio_ctrl(); method Bit#(64) osdr_dout(); method Bit#(8) osdr_den_n(); method Bool osdr_cke(); method Bool osdr_cs_n(); method Bool osdr_ras_n (); method Bool osdr_cas_n (); method Bool osdr_we_n (); method Bit#(8) osdr_dqm (); method Bit#(2) osdr_ba (); method Bit#(13) osdr_addr (); interface Clock sdram_clk; endinterface So now we go to src/spec/pinfunctions.py and add a corresponding function that returns a list of all of the required pin signals. However, we note that it is a huge number of pins so a decision is made to split it into groups: sdram1, sdram2 and sdram3. Firstly, sdram1, covering the base functionality: def sdram1(suffix, bank): buspins = [] inout = [] for i in range(8): pname = "SDRDQM%d*" % i buspins.append(pname) for i in range(8): pname = "SDRD%d*" % i buspins.append(pname) inout.append(pname) for i in range(12): buspins.append("SDRAD%d+" % i) for i in range(2): buspins.append("SDRBA%d+" % i) buspins += ['SDRCKE+', 'SDRRASn+', 'SDRCASn+', 'SDRWEn+', 'SDRCSn0++'] return (buspins, inout) This function, if used on its own, would define an 8-bit SDRAM bus with 12-bit addressing. Checking off the names against the corresponding BSV definition we find that most of them are straightforward. Outputs must have a "+" after the name (in the python representation), inputs must have a "-". However we run smack into an interesting brick-wall with the in/out pins. In/out pins which are routed through the same IO pad need a *triplet* of signals: one input wire, one output wire and *one direction control wire*. Here however we find that the SDRAM controller, which is a wrapper around the opencores SDRAM controller, has a *banked* approach to direction-control that will need to be dealt with, later. So we do *not* make the mistake of adding 8 SDRDENx pins: the BSV code will need to be modified to add 64 one-for-one enabling pins. We do not also make the mistake of adding separate unidirectional "in" and separate unidirectional "out" signals under different names, as the pinmux code is a *PAD* centric tool. The second function extends the 8-bit data bus to 64-bits, and extends the address lines to 13-bit wide: def sdram3(suffix, bank): buspins = [] inout = [] for i in range(12, 13): buspins.append("SDRAD%d+" % i) for i in range(8, 64): pname = "SDRD%d*" % i buspins.append(pname) inout.append(pname) return (buspins, inout) In this way, alternative SDRAM controller implementations can use sdram1 on its own; implementors may add "extenders" (named sdram2, sdram4) that cover extra functionality, and, interestingly, in a pinbank scenario, the number of pins on any given GPIO bank may be kept to a sane level. The next phase is to add the (now supported) peripheral to the list of pinspecs at the bottom of the file, so that it can actually be used: pinspec = (('IIS', i2s), ('MMC', emmc), ('FB', flexbus1), ('FB', flexbus2), ('SDR', sdram1), ('SDR', sdram2), ('SDR', sdram3), <--- ('EINT', eint), ('PWM', pwm), ('GPIO', gpio), ) This gives a declaration that any time the function(s) starting with "sdram" are used to add pins to a pinmux, it will be part of the "SDR" peripheral. Note that flexbus is similarly subdivided into two groups. Note however that due to a naming convention issue, interfaces must be declared with names that are lexicographically unique even in subsets of their names. i.e two interfaces, one named "SD" which is shorthand for SDMMC and another named "SDRAM" may *not* be added: the first has to be the full "SDMMC" or renamed to "MMC". # Adding the peripheral to a chip's pinmux specification Next, we add the peripheral to an actual chip's specification. In this case it is to be added to i\_class, so we open src/spec/i\_class.py. The first thing to do is to add a single-mux (dedicated) bank of 92 pins (!) which covers all of the 64-bit Data lines, 13 addresses and supporting bank-selects and control lines. It is added as Bank "D", the next contiguous bank: def pinspec(): pinbanks = { 'A': (28, 4), 'B': (18, 4), 'C': (24, 1), 'D': (92, 1), <--- } fixedpins = { 'CTRL_SYS': [ This declares the width of the pinmux to one (a dedicated peripheral bank). Note in passing that A and B are both 4-entry. Next, an SDRAM interface is conveniently added to the chip's pinmux with two simple lines of code: ps.gpio("", ('B', 0), 0, 0, 18) ps.flexbus1("", ('B', 0), 1, spec=flexspec) ps.flexbus2("", ('C', 0), 0) ps.sdram1("", ('D', 0), 0) <-- ps.sdram3("", ('D', 35), 0) <-- Note that the first argument is blank, indicating that this is the only SDRAM interface to be added. If more than one SDRAM interface is desired they would be numbered from 0 and identified by their suffix. The second argument is a tuple of (Bank Name, Bank Row Number), and the third argument is the pinmux column (which in this case must be zero). At the top level the following command is then run: $ python src/pinmux_generator.py -o i_class -s i_class The output may be found in the ./i\_class subdirectory, and it is worth examining the i\_class.mdwn file. A table named "Bank D" will have been created and it is worth just showing the first few entries here: | Pin | Mux0 | Mux1 | Mux2 | Mux3 | | --- | ----------- | ----------- | ----------- | ----------- | | 70 | D SDR_SDRDQM0 | | 71 | D SDR_SDRDQM1 | | 72 | D SDR_SDRDQM2 | | 73 | D SDR_SDRDQM3 | | 74 | D SDR_SDRDQM4 | | 75 | D SDR_SDRDQM5 | | 76 | D SDR_SDRDQM6 | | 77 | D SDR_SDRDQM7 | | 78 | D SDR_SDRD0 | | 79 | D SDR_SDRD1 | | 80 | D SDR_SDRD2 | | 81 | D SDR_SDRD3 | | 82 | D SDR_SDRD4 | | 83 | D SDR_SDRD5 | | 84 | D SDR_SDRD6 | | 85 | D SDR_SDRD7 | | 86 | D SDR_SDRAD0 | | 87 | D SDR_SDRAD1 | Returning to the definition of sdram1 and sdram3, this table clearly corresponds to the functions in src/spec/pinfunctions.py which is exactly what we want. It is however extremely important to verify. Lastly, the peripheral is a "fast" peripheral, i.e. it must not be added to the "slow" peripherals AXI4-Lite Bus, so must be added to the list of "fast" peripherals, here: ps = PinSpec(pinbanks, fixedpins, function_names, ['lcd', 'jtag', 'fb', 'sdr']) <-- # Bank A, 0-27 ps.gpio("", ('A', 0), 0, 0, 28) This basically concludes the first stage of adding a peripheral to the pinmux / autogenerator tool. It allows peripherals to be assessed for viability prior to actually committing the engineering resources to their deployment. # Adding the code auto-generators. With the specification now created and well-defined (and now including the SDRAM interface), the next completely separate phase is to auto-generate the code that will drop an SDRAM instance onto the fabric of the SoC. This particular peripheral is classified as a "Fast Bus" peripheral. "Slow" peripherals will need to be the specific topic of an alternative document, however the principles are the same. The first requirement is that the pins from the peripheral side be connected through to IO cells. This can be verified by running the pinmux code generator (to activate "default" behaviour), just to see what happens: $ python src/pinmux_generator.py -o i_class Files are auto-generated in ./i\_class/bsv\_src and it is recommended to examine the pinmux.bsv file in an editor, and search for occurrences of the string "sdrd63". It can clearly be seen that an interface named "PeripheralSideSDR" has been auto-generated: // interface declaration between SDR and pinmux (*always_ready,always_enabled*) interface PeripheralSideSDR; interface Put#(Bit#(1)) sdrdqm0; interface Put#(Bit#(1)) sdrdqm1; interface Put#(Bit#(1)) sdrdqm2; interface Put#(Bit#(1)) sdrdqm3; interface Put#(Bit#(1)) sdrdqm4; interface Put#(Bit#(1)) sdrdqm5; interface Put#(Bit#(1)) sdrdqm6; interface Put#(Bit#(1)) sdrdqm7; interface Put#(Bit#(1)) sdrd0_out; interface Put#(Bit#(1)) sdrd0_outen; interface Get#(Bit#(1)) sdrd0_in; .... .... endinterface Note that for the data lines, that where in the sdram1 specification function the signals were named "SDRDn+, out, out-enable *and* in interfaces/methods have been created, as these will be *directly* connected to the I/O pads. Further down the file we see the *actual* connection to the I/O pad (cell). An example: // -------------------- // ----- cell 161 ----- // output muxer for cell idx 161 cell161_mux_out= wrsdr_sdrd63_out; // outen muxer for cell idx 161 cell161_mux_outen= wrsdr_sdrd63_outen; // bi-directional // priority-in-muxer for cell idx 161 rule assign_wrsdr_sdrd63_in_on_cell161; wrsdr_sdrd63_in<=cell161_mux_in; endrule Here, given that this is a "dedicated" cell (with no muxing), we have *direct* assignment of all three signals (in, out, outen). 2-way, 3-way and 4-way muxing creates the required priority-muxing for inputs and straight-muxing for outputs, however in this instance, a deliberate pragmatic decision is being taken not to put 92 pins of 133mhz+ signalling through muxing. In examining the slow\_peripherals.bsv file, there should at this stage be no sign of an SDRAM peripheral having been added, at all. This is because it is missing from the peripheral\_gen side of the tool. However, as the slow\_peripherals module takes care of the IO cells (because it contains a declared and configured instance of the pinmux package), signals from the pinmux PeripheralSideSDR instance need to be passed *through* the slow peripherals module as an external interface. This will happen automatically once a code-generator class is added. So first, we must identify the nearest similar class. FlexBus looks like a good candidate, so we take a copy of src/bsv/peripheral\_gen/flexbus.py called sdram.py. The simplest next step is to global/search/replace "flexbus" with "sdram", and for peripheral instance declaration replace "fb" with "sdr". At this phase, despite knowing that it will auto-generate the wrong code, we add it as a "supported" peripheral at the bottom of src/bsv/peripheral\_gen/base.py, in the "PFactory" (Peripheral Factory) class: from gpio import gpio from rgbttl import rgbttl from flexbus import flexbus from sdram import sdram <-- for k, v in {'uart': uart, 'rs232': rs232, 'sdr': sdram, 'twi': twi, 'quart': quart, Note that the name "SDR" matches with the prefix used in the pinspec declaration, back in src/spec/pinfunctions.py, except lower-cased. Once this is done, and the auto-generation tool re-run, examining the slow\_peripherals.bsv file again shows the following (correct) and only the following (correct) additions: method Bit#(1) quart0_intr; method Bit#(1) quart1_intr; interface GPIO_config#(28) pad_configa; interface PeripheralSideSDR sdr0; <-- interface PeripheralSideFB fb0; .... .... interface iocell_side=pinmux.iocell_side; interface sdr0 = pinmux.peripheral_side.sdr; <-- interface fb0 = pinmux.peripheral_side.fb; These automatically-generated declarations are sufficient to "pass through" the SDRAM "Peripheral Side", which as we know from examination of the code is directly connected to the relevant IO pad cells, so that the *actual* peripheral may be declared in the "fast" fabric and connected up to the relevant and required "fast" bus. ## Connecting in the fabric Now we can begin the process of systematically inserting the correct "voodoo magic" incantations that, as far as this auto-generator tool is concerned, are just bits of ASCII text. In this particular instance, an SDRAM peripheral happened to already be *in* the SoC's BSV source code, such that the process of adding it to the tool is primarily one of *conversion*. **Please note that it is NOT recommended to do two tasks at once. It is strongly recommended to add any new peripheral to a pre-existing verified project, manually, by hand, and ONLY then to carry out a conversion process to have this tool understand how to auto-generate the fabric** So examining the i\_class socgen.bsv file, we also open up src/bsv/bsv\_lib/soc\_template.bsv in side-by-side windows of maximum 80 characters in width each, and *respect the coding convention for this exact purpose*, can easily fit two such windows side-by-side *as well as* a third containing the source code files that turn that same template into its corresponding output. We can now begin by searching for strings "SDRAM" and "sdr" in both the template and the auto-generated socgen.bsv file. The first such encounter is the import, in the template: `ifdef BOOTROM import BootRom ::*; `endif `ifdef SDRAM <-- xxxx import sdr_top :: *; <-- xxxx `endif <-- xxxx `ifdef BRAM This we can **remove**, and drop the corresponding code-fragment into the sdram slowimport function: class sdram(PBase): def slowimport(self): return "import sdr_top::*;" <-- def num_axi_regs32(self): Now we re-run the auto-generator tool and confirm that, indeed, the ifdef'd code is gone and replaced with an unconditional import: import mqspi :: *; import sdr_top::*; <-- import Uart_bs :: *; import RS232_modified::*; import mspi :: *; Progress! Next, we examine the instance declaration clause. Remember that we cut/paste the flexbus class, so we are expecting to find code that declares the sdr0 instance as a FlexBus peripheral. We are also looking for the hand-created code that is to be *replaced*. Sure enough: AXI4_Slave_to_FlexBus_Master_Xactor_IFC #(`PADDR, `DATA, `USERSPACE) sdr0 <- mkAXI4_Slave_to_FlexBus_Master_Xactor; <-- AXI4_Slave_to_FlexBus_Master_Xactor_IFC #(`PADDR, `DATA, `USERSPACE) fb0 <- mkAXI4_Slave_to_FlexBus_Master_Xactor; ... ... `ifdef BOOTROM BootRom_IFC bootrom <-mkBootRom; `endif `ifdef SDRAM <-- Ifc_sdr_slave sdram<- mksdr_axi4_slave(clk0); <-- `endif <-- So, the mksdr\_axi4\_slave call we *remove* from the template and cut/paste it into the sdram class's mkfast_peripheral function, making sure to substitute the hard-coded instance name "sdram" with a python-formatted template that can insert numerical instance identifiers, should it ever be desired that there be more than one SDRAM peripheral put into a chip: class sdram(PBase): ... ... def mkfast_peripheral(self): return "Ifc_sdr_slave sdr{0} <- mksdr_axi4_slave(clk0);" Re-run the tool and check that the correct-looking code has been created: Ifc_sdr_slave sdr0 <- mksdr_axi4_slave(clk0); <-- AXI4_Slave_to_FlexBus_Master_Xactor_IFC #(`PADDR, `DATA, `USERSPACE) fb0 <- mkAXI4_Slave_to_FlexBus_Master_Xactor; Ifc_rgbttl_dummy lcd0 <- mkrgbttl_dummy(); The next thing to do: searching for the string "sdram\_out" shows that the original hand-generated code contains (contained) a declaration of the SDRAM Interface, presumably to which, when compiling to run on an FPGA, the SDRAM interface would be connected at the top level. Through this interface, connections would be done *by hand* to the IO pads, whereas now they are to be connected *automatically* (on the peripheral side) to the IO pads in the pinmux. However, at the time of writing this is not fully understood by the author, so the fastifdecl and extfastifinstance functions are modified to generate the correct output but the code is *commented out* def extfastifinstance(self, name, count): return "// TODO" + self._extifinstance(name, count, "_out", "", True, ".if_sdram_out") def fastifdecl(self, name, count): return "// (*always_ready*) interface " + \ "Ifc_sdram_out sdr{0}_out;".format(count) Also the corresponding (old) manual declarations of sdram\_out removed from the template: `ifdef SDRAM <-- xxxx (*always_ready*) interface Ifc_sdram_out sdram_out; <-- xxxx `endif <-- xxxx ... ... `ifdef SDRAM <--- xxxx interface sdram_out=sdram.ifc_sdram_out; <--- xxxx `endif <--- xxxx Next, again searching for signs of the "hand-written" code, we encounter the fabric connectivity, which wires the SDRAM to the AXI4. We note however that there is not just one AXI slave device but *two*: one for the SDRAM itself and one for *configuring* the SDRAM. We therefore need to be quite careful about assigning these, as will be subsequently explained. First however, the two AXI4 slave interfaces of this peripheral are declared: class sdram(PBase): ... ... def _mk_connection(self, name=None, count=0): return ["sdr{0}.axi4_slave_sdram", "sdr{0}.axi4_slave_cntrl_reg"] Note that, again, in case multiple instances are ever to be added, the python "format" string "{0}" is inserted so that it can be substituted with the numerical identifier suffix. Also note that the order of declaration of these two AXI4 slave is **important**. Re-running the auto-generator tool, we note the following output has been created, and match it against the corresponding hand-generated (old) code: `ifdef SDRAM mkConnection (fabric.v_to_slaves [fromInteger(valueOf(Sdram_slave_num))], sdram.axi4_slave_sdram); // mkConnection (fabric.v_to_slaves [fromInteger(valueOf(Sdram_cfg_slave_num))], sdram.axi4_slave_cntrl_reg); // `endif // fabric connections mkConnection (fabric.v_to_slaves [fromInteger(valueOf(SDR0_fastslave_num))], sdr0.axi4_slave_sdram); mkConnection (fabric.v_to_slaves [fromInteger(valueOf(SDR0_fastslave_num))], sdr0.axi4_slave_cntrl_reg); Immediately we can spot an issue: whilst the correctly-named slave(s) have been added, they have been added with the *same* fabric slave index. This is unsatisfactory and needs resolving. Here we need to explain a bit more about what is going on. The fabric on an AXI4 Bus is allocated numerical slave numbers, and each slave is also allocated a memory-mapped region that must be resolved in a bi-directional fashion. i.e whenever a particular memory region is accessed, the AXI slave peripheral responsible for dealing with it **must** be correctly identified. So this requires some further crucial information, which is the size of the region that is to be allocated to each slave device. Later this will be extended to being part of the specification, but for now it is auto-allocated based on the size. As a huge hack, it is allocated in 32-bit chunks, as follows: class sdram(PBase): def num_axi_regs32(self): return [0x400000, # defines an entire memory range (hack...) 12] # defines the number of configuration regs So after running the autogenerator again, to confirm that this has generated the correct code, we examine several files, starting with fast\+memory\_map.bsv: /*====== Fast peripherals Memory Map ======= */ `define SDR0_0_Base 'h50000000 `define SDR0_0_End 'h5FFFFFFF // 4194304 32-bit regs `define SDR0_1_Base 'h60000000 `define SDR0_1_End 'h600002FF // 12 32-bit regs This looks slightly awkward (and in need of an external specification section for addresses) but is fine: the range is 1GB for the main map and covers 12 32-bit registers for the SDR Config map. Next we note the slave numbering: typedef 0 SDR0_0__fastslave_num; typedef 1 SDR0_1__fastslave_num; typedef 2 FB0_fastslave_num; typedef 3 LCD0_fastslave_num; typedef 3 LastGen_fastslave_num; typedef TAdd#(LastGen_fastslave_num,1) Sdram_slave_num; typedef TAdd#(Sdram_slave_num ,`ifdef SDRAM 1 `else 0 `endif ) Sdram_cfg_slave_num; Again this looks reasonable and we may subsequently (carefully! noting the use of the TAdd# chain!) remove the #define for Sdram\_cfg\_slave\_num. The next phase is to examine the fn\_addr\_to\_fastslave\_num function, where we note that there were *two* hand-created sections previously, now joined by two *auto-generated* sections: function Tuple2 #(Bool, Bit#(TLog#(Num_Fast_Slaves))) fn_addr_to_fastslave_num (Bit#(`PADDR) addr); if(addr>=`SDRAMMemBase && addr<=`SDRAMMemEnd) return tuple2(True,fromInteger(valueOf(Sdram_slave_num))); <-- else if(addr>=`DebugBase && addr<=`DebugEnd) return tuple2(True,fromInteger(valueOf(Debug_slave_num))); <-- `ifdef SDRAM else if(addr>=`SDRAMCfgBase && addr<=`SDRAMCfgEnd ) return tuple2(True,fromInteger(valueOf(Sdram_cfg_slave_num))); `endif ... ... if(addr>=`SDR0_0_Base && addr<=`SDR0_0_End) <-- return tuple2(True,fromInteger(valueOf(SDR0_0__fastslave_num))); else if(addr>=`SDR0_1_Base && addr<=`SDR0_1_End) <-- return tuple2(True,fromInteger(valueOf(SDR0_1__fastslave_num))); else if(addr>=`FB0Base && addr<=`FB0End) Now, here is where, in a slightly unusual unique set of circumstances, we cannot just remove all instances of this address / typedef from the template code. Looking in the shakti-core repository's src/lib/MemoryMap.bsv file, the SDRAMMemBase macro is utilise in the is\_IO\_Addr function. So as a really bad hack, which will need to be properly resolved, whilst the hand-generated sections from fast\_tuple2\_template.bsv are removed, and the corresponding (now redundant) defines in src/core/core\_parameters.bsv are commented out, some temporary typedefs to deal with the name change are also added: `define SDRAMMemBase SDR0_0_Base `define SDRAMMemEnd SDR0_0_End This needs to be addressed (pun intended) including being able to specify the name(s) of the configuration parameters, as well as specifying which memory map range they must be added to. Now however finally, after carefully comparing the hard-coded fabric connections to what was formerly named sdram, we may remove the mkConnections that drop sdram.axi4\_slave\_sdram and its associated cntrl reg from the soc\_template.bsv file. ## Connecting up the pins We are still not done! It is however worth pointing out that if this peripheral were not wired into the pinmux, we would in fact be finished. However there is a task that (previously having been left to outside tools) now needs to be specified, which is to connect the sdram's pins, declared in this instance in Ifc\_sdram\_out, and the PeripheralSideSDR instance that was kindly / strategically / thoughtfully / absolutely-necessarily exported from slow\_peripherals for exactly this purpose. Recall earlier that we took a cut/paste copy of the flexbus.py code. If we now examine socgen.bsv we find that it contains connections to pins that match the FlexBus specification, not SDRAM. So, returning to the declaration of the Ifc\_sdram\_out interface, we first identify the single-bit output-only pins, and add a mapping table between them: class sdram(PBase): def pinname_out(self, pname): return {'sdrwen': 'ifc_sdram_out.osdr_we_n', 'sdrcsn0': 'ifc_sdram_out.osdr_cs_n', 'sdrcke': 'ifc_sdram_out.osdr_cke', 'sdrrasn': 'ifc_sdram_out.osdr_ras_n', 'sdrcasn': 'ifc_sdram_out.osdr_cas_n', }.get(pname, '') Re-running the tool confirms that the relevant mkConnections are generated: //sdr {'action': True, 'type': 'out', 'name': 'sdrcke'} mkConnection(slow_peripherals.sdr0.sdrcke, sdr0_sdrcke_sync.get); mkConnection(sdr0_sdrcke_sync.put, sdr0.ifc_sdram_out.osdr_cke); //sdr {'action': True, 'type': 'out', 'name': 'sdrrasn'} mkConnection(slow_peripherals.sdr0.sdrrasn, sdr0_sdrrasn_sync.get); mkConnection(sdr0_sdrrasn_sync.put, sdr0.ifc_sdram_out.osdr_ras_n); Next, the multi-