SFFS/gentoo_bootstrap.mdwn

   1 # Tutorial for bootstrapping Gentoo with sffs compliance from source.
   2
   3 **This project is part of [NLnet NGI0 Entrust](https://nlnet.nl/entrust)
   4 and has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101069594.**
   5
   6 Useful Links:
   7
   8 * bugreport: <https://bugs.libre-soc.org/show_bug.cgi?id=1131>
   9 * (External) [Read the Gentoo handbook thoroughly, it's all you need](https://wiki.gentoo.org/wiki/Handbook:PPC64)
  10 * please note this tarball contains executable binaries and it
  11   is 100% your responsibility to check them or take appropriate
  12   measures. we take no responsibility and accept no liability
  13   <https://ftp.libre-soc.org/sffs/stage3-ppc64le-openrc-20230720T024654Z.tar.xz>
  14 * TODO investigate: a mirror of stage3 snapshots is available at archive.org
  15   <https://web.archive.org/web/20221218010734/https://distfiles.gentoo.org/releases/ppc/autobuilds/20221217T033210Z/>
  16   more recent snapshot: all stage3-ppc64le-openrc-20230823T030134Z.tar.xz
  17   files:
  18   <https://web.archive.org/web/20230823160828/https://distfiles.gentoo.org/releases/ppc/autobuilds/20230823T030134Z/>
  19
  20 ## Video Tutorial
  21
  22 [todo](todo)
  23
  24 ## Reasoning:
  25
  26 Libre-SOC builds upon the OpenPOWER ISA v3.0 but does not include many
  27 parts of the architecture typically found on production silicon, like
  28 AltiVec, VSX, and 128-bit hardware floating point. Because of this,
  29 we need (at this time, might change in the future when glibc supports
  30 detecting VSX vs SVP64 etc) to rebuild the stage 3 system to not include
  31 any of those instructions, making it compliant with
  32 [sffs (slides 5 & 6)](https://ics2020.bsc.es/sites/default/files/uploaded/brian_thompto_ICS2020_SLIDES.pdf).
  33
  34 To do this, you need to rebuild Gentoo from a known working stage3
  35 archive, which we will guide you through.
  36
  37 Do note that given the rolling release nature of Gentoo, it is almost
  38 impossible to get the same system with the specific package versions we
  39 have without using the same stage3 version we used.
  40
  41 ## System requirements
  42
  43 - A POWER system that can run Debian Bookworm (meaning POWER8 and newer)
  44 - At least 80GB of hard disk space
  45
  46 Cross compiling will make this process much more complicated and is
  47 generally not recommended.
  48
  49 It is expected for you to use Debian for the host OS (anything else is
  50 unsupported: many contributors have repeatedly gotten into trouble by
  51 not following this advice)
  52
  53 ## Setting up Gentoo from stage3 and rebuilding (recommended method)
  54
  55 You will need to run these commands:
  56
  57     $ curl -O https://ftp.libre-soc.org/sffs/stage3-ppc64le-openrc-20230720T024654Z.tar.xz
  58     $ sha256sum stage3-ppc64le-openrc-20230720T024654Z.tar.xz
  59
  60 The SHA256 checksum should be
  61
  62 `2c97446d09f2790ee6cc3e4cbff0d204f6f0914083fc36998d0b08fcb28f8e90`
  63
  64     # sudo bash
  65     # mkdir gentoo-sffs && cd gentoo-sffs
  66     # tar xpvf ../stage3-ppc64le-openrc-20230720T024654Z.tar.xz \
  67                --xattrs-include='*.*' --numeric-owner
  68     # cd ..
  69
  70 Follow these steps from the Gentoo handbook:
  71
  72 * [Copy DNS info](https://wiki.gentoo.org/wiki/Handbook:PPC64/Installation/Base#Copy_DNS_info)
  73
  74 * [Mount necessary filesystems](https://wiki.gentoo.org/wiki/Handbook:PPC64/Installation/Base#Mounting_the_necessary_filesystems)
  75
  76 * [Enter the chroot environment](https://wiki.gentoo.org/wiki/Handbook:PPC64/Installation/Base#Entering_the_new_environment)
  77
  78 After which you need to add our build flags replacing the default ones in
  79 `/etc/portage/make.conf`
  80
  81     `COMMON_FLAGS="-O2 -pipe -mcpu=power9 -mno-altivec -mno-vsx -mno-crypto -mno-htm -mlong-double-64"`
  82
  83     `CPU_FLAGS_PPC=""`
  84
  85 Finally, rebuild everything on the system:
  86
  87     # emerge -e --ask --keep-going @world
  88
  89 # (Optional) Building an updated Gentoo SFFS
  90
  91 This is highly not recommended unless you specifically want to explore
  92 how newer versions of software will behave when built with the SFFS flags.
  93
  94 One way of doing this is to just set up Gentoo SFFS from the provided
  95 stage3, add out `make.conf` options and then upgrade everything, that
  96 is much simpler and easier to roll-back if need be.
  97
  98 After having a working chroot using the stage3 as outlined earlier,
  99 run these commands in the chroot:
 100
 101     # emerge-webrsync
 102     # emerge --ask --verbose --update --deep --newuse @world
 103
 104 The other way is to create a new Gentoo chroot or virtual machine under
 105 POWER using the [handbook](https://wiki.gentoo.org/wiki/Handbook:PPC64),
 106 and once you reach the [stage3
 107 tarball](https://wiki.gentoo.org/wiki/Handbook:PPC64/Installation/Stage#Unpacking_the_stage_tarball)
 108 step, download our [make.conf](link) and copy it to
 109  `/etc/portage/make.conf`, replacing the provided one, and run:
 110
 111     # emerge -e --ask --keep-going @world
 112     # emerge-webrsync
 113     # emerge --ask --verbose --update --deep --newuse @world
 114
 115 To first rebuild Gentoo for SFFS, and then update the repositories,
 116 and finally to upgrade the software you have.
 117
 118 # Testing for VSX in object files
 119
 120 To verify that our buildflags were applied by the build tools and
 121 respected by the build scripts of packages, we need to check for the
 122 inclusion of VSX as well as any other instructions not available in SFFS.
 123
 124 This is not a perfect solution and the proper way would be either to:
 125
 126 - Simulate an SFFS compliant chip running this code such as
 127   [Microwatt](https://libre-soc.org/HDL_workflow/microwatt/)
 128   (very time consuming)
 129 - Use a softcore FPGA core or ASIC of Microwatt / LibreSOC
 130   to run this code (doable, FPGA softcore in progress
 131   [in progress](https://bugs.libre-soc.org/show_bug.cgi?id=1037))
 132 - Use the finalized Libre-SOC chip (doable once it is ready)
 133
 134 Also note that this only tests for a subset of VSX instructions at the
 135 moment, more will be added in the future, this is simply a quick test to
 136 run before attempting to run on simulation and/or a softcore as both can
 137 be somewhat time consuming. In other words, if this test gives any VSX
 138 instructions, don't attempt to run the code in simulation or a softcore
 139 to avoid wasting precious time.
 140
 141 To attempt this test, you will need the scripts from our `dev-env-setup`
 142 repository
 143
 144     $ git clone https://git.libre-soc.org/git/dev-env-setup.git
 145
 146 Do *look through* the
 147 [code](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) before running
 148 any of those scripts.
 149 This is your own legal responsibility (to not run arbitrary code off
 150 of the internet) and we take no responsibility or accept any liability
 151 whatsoever for your use or misuse of them.
 152
 153 It is expected for you to use Debian for the host OS (anything else is
 154 unsupported: many contributors have repeatedly gotten into trouble by
 155 not following this advice), while all the chroots - which are developed
 156 very specifically for "reproducible builds" - run Debian 10 (Buster).
 157
 158 Edit the `test-vsx` script to point the `CHROOT` variable to your
 159 respective chroot of Gentoo, read through the script thoroughly, and run
 160
 161     $ ./test-vsx
 162
 163 If you see any number of instructions in certain binary files, those
 164 files do contain VSX and work needs to be done (patching the build
 165 scripts and/or source code) for these packages. If not, great! Try to
 166 run this in a proper SFFS environment to fully test the code you have.
 167
 168 You can also test specific binaries at any time by running this part of
 169 the script
 170
 171     $ objdump -d $OBJ_FILE | fgrep -of VSX_INSNS.txt | sort | uniq -c \
 172                 | awk '{print $2 " " $1}'
 173
 174 Where `$OBJ_FILE` is the binary you need to check.
 175
 176 ## Notes and issues
 177
 178 * Gentoo's base system packages will not complain when building for SFFS
 179   (as of July 2023, can change depending on updates)
 180 * The Linux kernel cannot boot without AltiVec and VSX, even though they
 181   are build-time options. The solution for now is to use [microwatt's 5.7
 182 kernel](https://lore.kernel.org/all/CAPweEDw710zFK8KLZY5gsQxEkQKrDiFkNRgABY9HJZ1rxpeVCg@mail.gmail.com/T/)
 183   and to incorporate the needed patches upstream in the future.
 184 * Previous note also means this is purely a chroot environment, this page
 185   will be updated once it becomes possible to boot a full VM of this build.
 186 * glibc *may* have some issues on sffs, the only way to test this is to
 187   run on microwatt which will be done soon.
 188