* ftp server (<https://ftp.libre-soc.org/>): large (temporary,
auto-generated) file store.
-We will add an IRC channel at some point when there are enough people
-to warrant having one (and it will be publicly archived)
-
Note also the lack of a "forum" in the above list. this is very
deliberate. forums are a serious distraction when it comes to technical
heavily goal-orientated development. recent internet users may enjoy
## Bugtracker
+* **LibreSOC bug/task process**: [[libresoc_bug_process]]
+
bugzilla. old and highly effective. sign up in the usual way. any
problems, ask on the list.
proprietary nonfree service "because it's soooo convenient", as the
lions are getting wind and gout from overfeeding on that one.
+one.
+
## ikiwiki
Runs the main libre-soc.org site (including this page). effective,
# Software prerequisites<a name="software-prerequisites"></a>
+**Please make sure if you install manually that you install dependencies
+in strict order. Failing to adhere to this will result in pip3 downloading
+unauthorised older software versions. See
+<http://lists.libre-soc.org/pipermail/libre-soc-dev/2021-September/003666.html>**
+
Whilst many resources online advocate "`sudo`" in front of all root-level
commands below, this quickly becomes tiresome. run "`sudo bash`", get a
root prompt, and save yourself some typing.
* sudo bash
* apt-get install vim exuberant-ctags
* apt-get install build-essential
-* apt-get install git python3.7 python3.7-dev python-nosetest3
+* apt-get install git python3.7 python3.7-dev python3-nose
* apt-get install graphviz xdot gtkwave
* apt-get install python3-venv
* apt-get install python-virtualenv # this is an alternative to python3-venv
get "lost" or isolated and out of touch due to major branch diversion,
and that people communicate and coordinate with each other.
+This is not a hard rule: under special cirmstances branches can be useful.
+They should not however be considered "routine".
+
+For guidance on when branches are appropriate,
+see [[HDL_workflow/libresoc_bug_process]].
+
+For advice on commit messages see the Coding section further down on this page.
+
## yosys
Follow the source code (git clone) instructions here, do **not** use
the "stable" version (do not download the tarball):
-<http://www.clifford.at/yosys/download.html>
+<https://github.com/YosysHQ/yosys>
Or, alternatively, use the
-[yosys-et-al](https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=yosys-et-al;hb=HEAD)
+[hdl-tools-yosys](https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=hdl-tools-yosys;hb=HEAD)
script (which also installs symbiyosys and its dependencies)
Do not try to use a fixed revision of yosys (currently 0.9), nmigen is
evolving and frequently interacts with yosys.
-[Yosys](http://www.clifford.at/yosys/) is a framework for Verilog RTL.
+[Yosys](https://github.com/YosysHQ/yosys is a framework for Verilog RTL.
[Verilog](https://en.wikipedia.org/wiki/Verilog) is a hardware description
language.
RTL [Register Transfer
front-end driver program for Yosys-based formal hardware verification
flows.
-## nmigen
+## nmigen (TM)
+
+*nmigen is a registered trademark of M-Labs <https://uspto.report/TM/88980893>*
+
+**PLEASE NOTE: it is critical to install nmigen as the first dependency
+prior to installing any further python-based Libre-SOC HDL repositories.
+If "pip3 list" shows that nmigen has been auto-installed please remove it**
[nmigen](https://m-labs.hk/gateware/nmigen/) may be installed as follows:
* mkdir ~/src
* cd !$
-* git clone https://github.com/nmigen/nmigen.git
+* git clone https://gitlab.com/nmigen/nmigen.git
* cd nmigen
* sudo bash
* python3 setup.py develop
git clone --recursive https://github.com/billzorn/sfpy.git
cd sfpy
+ git apply /path/to/ieee754fpu/sfpy.patch
cd SoftPosit
git apply ../softposit_sfpy_build.patch
git apply /path/to/ieee754fpu/SoftPosit.patch
You can test your installation by doing the following:
python3
- >>> from sfpy import *
+ >>> from sfpy import Posit8
>>> Posit8(1.3)
It should print out `Posit8(1.3125)`
programs. [qemu](https://www.qemu.org/) emulates processors, you can
run programs under qemu.
-## power_instruction_analyzer (pia)
+## power-instruction-analyzer (pia)
-We have a custom tool built in rust by programmerjake to help analyze
-the power instructions execution on *actual* hardware.
+We have a custom tool built in Rust by programmerjake to help analyze
+the OpenPower instructions' execution on *actual* hardware.
-Note: a very recent version of pip3 is required for this to work.
-
-Install rust:
+Install Rust:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
-Make sure we have the correct and up-to-date rust compiler (rustc):
+Make sure we have the correct and up-to-date rust compiler (rustc & cargo):
rustup default stable
rustup update
-Use rust's package manager *cargo* to install the rust-python build
-tool maturin:
-
- cargo install maturin
-
-Install from git source by doing the following:
+Install the Python extension from git source by doing the following:
git clone https://salsa.debian.org/Kazan-team/power-instruction-analyzer.git pia
cd pia
- maturin build --cargo-extra-args=--features=python-extension
- python3 -m pip install --user target/wheels/*.whl
-
-Note: an ongoing bug in maturin interferes with successful installation.
-This can be worked around by explicitly installing only the `.whl`
-files needed rather than installing everything (`\*.whl`).
+ ./libre-soc-install.sh
## Chips4Makers JTAG
A portable FPGA place and route tool.
-See [[HDL_workflow/nextpnr]] page for installation instructions of nextpnr with ECP5 support for Lattice FPGA ECP5 series.
+See [[HDL_workflow/nextpnr]] page for installation instructions of nextpnr with ECP5 support for Lattice FPGA ECP5 series. Also see
+[[HDL_workflow/ECP5_FPGA]] for connecting up to JTAG with a ULX3S
+and the Lattice VERSA_ECP5.
+
+## Nextpnr-xilinx
+
+An open source place and route framework for Xilinx FPGAs using Project Xray. We will use it for Xilinx 7-series FPGAs like Artix-7.
+
+One of the ways to get Arty A7 100t Digilent FPGA board working.
+
+See [[HDL_workflow/nextpnr-xilinx]] for installation instructions and dependencies.
+
## Verilator
See [[HDL_workflow/cocotb]] page for installation instructions.
+## Symbiflow
+
+A fully open source toolchain for the development of FPGAs. Currently it targets Xilinx 7-series, Lattice iCE40 and ECP5, Quicklogic EOS S3.
+
+One way to get the Arty A7 100t Digilent FPGA board working.
+
+See [[HDL_workflow/symbiflow]] for installation instructions
+and dependencies.
+
+## FPGA/Board Boot-Loaders-Programmers
+
+Open source FPGA/Board boot-loaders and programmers for ULX3S, ECP5 and
+OrangeCrab.
+
+Currently these programs dfu-util, openFPGALoader, ujprog, fujprog,
+xc3sprog and ecpprog are going to be used.
+
+See [[HDL_workflow/fpga-boot-loaders-progs]] for installation instructions and dependencies.
+
+## ls2 peripheral fabric
+
+[[HDL_workflow/ls2]]
+
# Registering for git repository access<a name="gitolite3_access"></a>
After going through the onboarding process and having agreed to take
Host git.libre-soc.org
Port 922
+Test that you have access with this command:
+
+ ssh -v -p922 gitolite3@git.libre-soc.org
+
+Please note: **DO NOT TYPE A PASSWORD** - the server gets hit by a lot of
+port-scanning, and detection of password failures are used to instantly
+ban IP addresses.
+
Wait for the Project Admin to confirm that the ssh key has been added
to the required repositories. Once confirmed, you can clone any of the
repos at https://git.libre-soc.org/:
dependencies. This is easiest done with this script
<https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=install-hdl-apt-reqs;hb=HEAD>
+**It is critically important to install these in STRICT order, otherwise
+pip3 interferes and performs unauthorised downloads without informing
+you of what it is doing**.
+
* mkdir ~/src
* cd !$
-* git clone gitolite3@git.libre-soc.org:nmigen.git
+* git clone https://gitlab.com/nmigen/nmigen
+* git clone https://gitlab.com/nmigen/nmigen-boards
+* git clone https://gitlab.com/nmigen/nmigen-soc
+* git clone https://gitlab.com/nmigen/nmigen-stdio
* git clone gitolite3@git.libre-soc.org:c4m-jtag.git
* git clone gitolite3@git.libre-soc.org:nmutil.git
* git clone gitolite3@git.libre-soc.org:openpower-isa.git
* git clone gitolite3@git.libre-soc.org:ieee754fpu.git
-* git clone gitolite3@git.libre-soc.org:nmigen-soc.git
* git clone gitolite3@git.libre-soc.org:soc.git
-In each of these directories, in the order listed, track down the
+In each of these directories, **in the order listed**, track down the
`setup.py` file, then, as root (`sudo bash`), run the following:
* python3 setup.py develop
own machine these days.
The reason for the order is because soc depends on ieee754fpu, and
-ieee754fpu depends on nmutil
+ieee754fpu depends on nmutil. If you do not follow the listed order
+pip3 will go off and download an arbitrary version without your
+consent.
If "`python3 setup.py install`" is used it is a pain: edit, then
install. edit, then install. It gets extremely tedious, hence why
for actual code development
+### Copyright Notices
+
+**All code must have copyright and grant notices (where work was done
+under budget).**
+
+* [Example from soc.git repo](https://git.libre-soc.org/?p=soc.git;a=blob;f=src/soc/fu/div/experiment/goldschmidt_div_sqrt.py;h=3f7c2480742d6913859461da120099385f99d18a;hb=HEAD)
+
+Breakdown of the header in the above example:
+
+- Code was worked on by Jacob Lifshay during 2022.
+- Work was done under LibreSOC's Crypto Router
+[grant](https://libre-soc.org/nlnet_2021_crypto_router/) submitted to NLnet.
+NLnet grant code is `2021-02-052`.
+- The NLnet grant was under the
+[NLnet Assure fund](https://nlnet.nl/assure).
+- Financial support for NGI Assure comes from European Commission's
+[Next Generation Internet](https://ngi.eu/) Programme,
+grant agreement no. 957073.
+
+Template:
+
+```
+# SPDX-License-Identifier: LGPL-3-or-later
+# Copyright 202X [Name] [email]
+#
+# Funded by NLnet [Programme Name] Programme [202X-YY-ZZZ], [NLnet URL] part
+# of [EU Programme Name] 202X EU Programme [Programme Number].
+```
+
### Plan unit tests
* plan in advance to write not just code but a full test suite for
that the commit has not been properly broken down into separate-purpose
commits. ask for advice on-list on how to proceed.
+### *Git commit message format*
+
+* Additional articles on commit messages
+[here](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html)
+and
+[here](https://github.com/torvalds/subsurface-for-dirk/blob/master/README.md#contributing)
+
+LibreSOC message format based on description given in
+[bug #1126#c40](https://bugs.libre-soc.org/show_bug.cgi?id=1126#c40):
+
+1. Every commit MUST start with a short title, up to 50 characters.
+2. The commit title MUST contain either subsystem, or a file path,
+or a subsystem/path, or a subsystem/subsubsystem combination, which got
+modified or introduced, and a short summary. These parts must be separated
+by the colon.
+3. A good rule is to imagine that the short message begins with
+"if this patch is applied, it will". For example, a good title is
+"X: update Y", not "updated Y in X".
+4. After the title, there must be an empty line, which documents the
+changes. The limit is 72 characters per line.
+5. The long description can be omitted if the short description provides
+enough information or if the commit itself is simple enough.
+
+Example:
+
+```
+subsystem/file.py: document usage
+
+Here goes the long description, which explains everything. First of all,
+we stick to limit of 72 characters. Then, perhaps, we'd like to explain
+the rationale in more details.
+```
+
+It is suggested to stick to common sense whenever choosing subsystem names
+or files or long descriptions.
+
+Primary concerns are:
+
+1. short titles
+2. short summaries
+3. wording for the first line
+
+The rest is up for the committers.
+
### Exceptions to small commit: atomic single purpose commit
* if it is essential to commit large amounts of code, ensure that it
commit confuses several unrelated changes, not only the diff is larger
than it should be, the reversion process becomes extremely painful.
+### PHP-style python format-strings
+
+As the name suggests, "PHP-style" is not given as a compliment.
+Format-strings - `f"{variable} {pythoncodefragment}" are a nightmare
+to read. The lesson from PHP, Zope and Plone: when code is embedded,
+the purpose of the formatting - the separation of the format from
+the data to be placed in it - is merged, and consequently become
+unreadable.
+
+By contrast, let us imagine a situation where 12 variables need to
+be inserted into a string, four of which are the same variablename:
+
+ x = "%s %s %s %s %s %s %s %s %s %s %s %s" % (var1, var2, var3,
+ var3, var4, var2,
+ var1, var9, var1,
+ var3, var4, var1)
+
+This is just as unreadable, but for different reasons. Here it *is*
+useful to do this as:
+
+ x = f"{var1} {var2} {var3}" \
+ ...
+ f"{var3} {var4} {var1}"
+
+As a general rule, though, format-specifiers should be strongly
+avoided, given that they mix even variable-names directly inside
+a string.
+
+This additionally gives text editors (and online web syntax
+highlighters) the opportunity to colour syntax-highlight the
+ASCII string (the format) from the variables to be inserted *into*
+that format. gitweb for example (used by this project) cannot
+highlight string-formatted code.
+
+It turns out that colour is processed by the **opposite** hemisphere
+of the brain from written language. Thus, colour-syntax-highlighting
+is not just a "nice-to-have", it's **vital** for easier and faster
+identification of context and an aid to rapid understanding.
+
+Anything that interferes with that - such as python format-strings -
+has to take a back seat, regardless of its perceived benefits.
+
+**If you absolutely must** use python-format-strings, **only** do
+so by restricting to variables. Create temporary variables if you
+have to.
+
+ y = '/'.join(a_list)
+ x = f"{y}"
+
### PEP8 format
* all code needs to conform to pep8. use either pep8checker or better
Really. don't. use. wildcards.
+More about this here:
+
+* <https://www.asmeurer.com/removestar/>
+* <https://rules.sonarsource.com/python/RSPEC-2208>
+
### Keep file and variables short but clear
* try to keep both filenames and variable names short but not ridiculously
preferably with a link to a URL in the [bugtracker](https://bugs.libre-soc.org/)
with further details as to why the unit test should not be run.
+## Liskov Substitution Principle
+
+* [Wikipedia entry](https://en.wikipedia.org/wiki/Liskov_substitution_principle)
+* [Stackoverflow answer on LSP](https://stackoverflow.com/questions/56860/what-is-an-example-of-the-liskov-substitution-principle)
+* [Article on LSP](https://reflectoring.io/lsp-explained/)
+
+Copying from Wikipedia:
+
+```
+The Liskov substitution principle (LSP) is a particular definition of a
+subtyping relation, called strong behavioral subtyping, that was initially
+introduced by Barbara Liskov in a 1987 conference keynote address titled
+Data abstraction and hierarchy. It is based on the concept of
+"substitutability" - a principle in object-oriented programming stating
+that an object (such as a class) may be replaced by a sub-object (such as
+a class that extends the first class) without breaking the program.
+It is a semantic rather than merely syntactic relation, because it intends
+to guarantee semantic interoperability of types in a hierarchy, object
+types in particular.
+```
+
+To paraphrase: an *original object/class may be replaced with another object*
+(whose class extends the first class) *without breaking* the program.
+
+Python is a programming language which makes using LSP pretty straightforward.
+
+In LibreSOC, we aim to follow this principle whenever possible and (bearing
+time and budget constraints).
+
+*(Luke, please include some examples from LibreSOC source here)*
+
+## Principle of Least Astonishment/Surprise (POLA)
+
+* Example shown by Luke on
+[comment #33 bug #1039](https://bugs.libre-soc.org/show_bug.cgi?id=1039#c33)
+* [Wikipedia entry](https://en.wikipedia.org/wiki/Principle_of_least_astonishment)
+* Example answer on
+[software eng. stack exchange](https://softwareengineering.stackexchange.com/questions/187457/what-is-the-principle-of-least-astonishment)
+
+Wikipedia mentions that the origin of the term "Principle of Least
+Astonishment" (or Surprise) comes from a PL/I programming language bulletin
+board from 1957:
+
+```
+For those parts of the system which cannot be adjusted to the peculiarities
+of the user, the designers of a systems programming language should obey
+the "Law of Least Astonishment." In short, this law states that every construct
+in the system should behave exactly as its syntax suggests. Widely accepted
+conventions should be followed whenever possible, and exceptions to previously
+established rules of the language should be minimal.
+```
+
+If a method name has a prefix `test_` it should be a unit test (or some other
+test which is there to check the functionality of a given feature).
+
+A method/function (or attributes/variables, etc.) *shouldn't* be given a name
+which is wildly different from the what the user would expect it to do.
+
+*Andrey*: One example which may meet this rule is `TestIssuer`,
+which is the FSM-based nMigen HDL CPU core used by LibreSOC (so far).
+It is so called because it is used to issue unit tests (while also
+being synthesisable). This name was however was initially confusing
+to me, because my background is in hardware not software engineering.
+
+
+# Task management guidelines
+
+* New guide for RfP submission (in-progress):
+[[HDL_workflow/rfp_submission_guide]]
+
+(This section needs to be compared with [[HDL_workflow/libresoc_bug_process]])
+
+1. Create the task in appropriate "Product" section with appropriate
+ "Component" section. Most code tasks generally use "Libre-SOC's
+ first SOC".
+2. Fill in "Depends on" and "Blocks" section whenever appropriate.
+ Also add as many related ("See Also") links to other bugreports
+ as possible. bugreports are never isolated.
+3. Choose the correct task for a budget allocation. Usually the parent
+ task is used.
+4. Choose the correct NLnet milestone. The best practice is to check
+ the parent task for a correct milestone.
+5. Assign the budget to the task in `"USER=SUM"` form, where "USER"
+ corresponds to your username and "SUM" corresponds to the actual
+ budget in EUR. There may be multiple users.
+6. When the task is completed, you can begin writing an RFP.
+ **DO NOT submit it without explicit authorisation and review**.
+ Leave out your bank and personal address details if you prefer
+ when sending to the Team Manager for review.
+7. Once the RFP is written, notify the Team Manager and obtain their
+ explicit approval to send it.
+8. Once approval is received and the RFP sent, update the `"USER=SUM"`
+ field to include the submitted date:
+ `"USER={amount=SUM, submitted=SDATE}"`. The SDATE is entered in
+ `YYYY-MM-DD` form.
+9. Once the task is paid, again notify the Team Manager (IRC is fine),
+ and update `"USER={amount=SUM, submitted=SDATE}"`
+ to `"USER={amount=SUM, submitted=SDATE, paid=PDATE}"`. The PDATE is
+ entered in `YYYY-MM-DD` form, too.
+
+Throughout all of this you should be using budget-sync to check the
+database consistency
+<https://git.libre-soc.org/?p=utils.git;a=blob;f=README.txt;hb=HEAD>
+
+[[!img bugzilla_RFP_fields.jpg size=640x ]]
+
# TODO Tutorials
Find appropriate tutorials for nmigen and yosys, as well as symbiyosys.
* Although a verilog example this is very useful to do
<https://symbiyosys.readthedocs.io/en/latest/quickstart.html#first-step-a-simple-bmc-example>
* This tutorial looks pretty good and will get you started
- <http://blog.lambdaconcept.com/doku.php?id=nmigen:nmigen_install> and
- walks not just through simulation, it takes you through using gtkwave
- as well.
+ <https://web.archive.org/web/20210123052724/http://blog.lambdaconcept.com/doku.php?id=nmigen:nmigen_install>
+ and walks not just through simulation, it takes you through using
+ gtkwave as well.
* There exist several nmigen examples which are also executable
- <https://github.com/m-labs/nmigen/tree/master/examples/> exactly as
+ <https://gitlab.com/nmigen/nmigen/tree/master/examples/> exactly as
described in the above tutorial (python3 filename.py -h)
-
+* More nmigen tutorials at [[learning_nmigen]]
projects / libreriscv.git / blobdiff