SFFS/gentoo_bootstrap.mdwn

   1 # Tutorial for bootstrapping Gentoo with sffs compliance from source.
   2
   3 Useful Links:
   4
   5 * bugreport: <https://bugs.libre-soc.org/show_bug.cgi?id=1131>
   6 * (External) [Read the Gentoo handbook thoroughly, it's all you need](https://wiki.gentoo.org/wiki/Handbook:PPC64)
   7 * please note this tarball contains executable binaries and it
   8   is 100% your responsibility to check them or take appropriate
   9   measures. we take no responsibility and accept no liability
  10   <https://ftp.libre-soc.org/sffs/stage3-ppc64le-openrc-20230720T024654Z.tar.xz>
  11 * TODO investigate: a mirror of stage3 snapshots is available at archive.org
  12   <https://web.archive.org/web/20221218010734/https://distfiles.gentoo.org/releases/ppc/autobuilds/20221217T033210Z/>
  13   more recent snapshot: all stage3-ppc64le-openrc-20230823T030134Z.tar.xz
  14   files:
  15   <https://web.archive.org/web/20230823160828/https://distfiles.gentoo.org/releases/ppc/autobuilds/20230823T030134Z/>
  16
  17 ## Video Tutorial
  18
  19 [todo](todo)
  20
  21 ## Reasoning:
  22
  23 Libre-SOC builds upon the OpenPOWER ISA v3.0 but does not include many
  24 parts of the architecture typically found on production silicon, like
  25 AltiVec, VSX, and 128-bit hardware floating point. Because of this,
  26 we need (at this time, might change in the future when glibc supports
  27 detecting VSX vs SVP64 etc) to rebuild the stage 3 system to not include
  28 any of those instructions, making it compliant with
  29 [sffs (slides 5 & 6)](https://ics2020.bsc.es/sites/default/files/uploaded/brian_thompto_ICS2020_SLIDES.pdf).
  30
  31 To do this, you need to rebuild Gentoo from a known working stage3
  32 archive, which we will guide you through.
  33
  34 Do note that given the rolling release nature of Gentoo, it is almost
  35 impossible to get the same system with the specific package versions we
  36 have without using the same stage3 version we used.
  37
  38 ## System requirements
  39
  40 - A POWER system that can run Debian Bookworm (meaning POWER8 and newer)
  41 - At least 80GB of hard disk space
  42
  43 Cross compiling will make this process much more complicated and is
  44 generally not recommended.
  45
  46 It is expected for you to use Debian for the host OS (anything else is
  47 unsupported: many contributors have repeatedly gotten into trouble by
  48 not following this advice)
  49
  50 ## Setting up Gentoo from stage3 and rebuilding (recommended method)
  51
  52 You will need to run these commands:
  53
  54     $ curl -O https://ftp.libre-soc.org/sffs/stage3-ppc64le-openrc-20230720T024654Z.tar.xz
  55     $ sha256sum stage3-ppc64le-openrc-20230720T024654Z.tar.xz
  56
  57 The SHA256 checksum should be
  58
  59 `2c97446d09f2790ee6cc3e4cbff0d204f6f0914083fc36998d0b08fcb28f8e90`
  60
  61     # sudo bash
  62     # mkdir gentoo-sffs && cd gentoo-sffs
  63     # tar xpvf ../stage3-ppc64le-openrc-20230720T024654Z.tar.xz \
  64                --xattrs-include='*.*' --numeric-owner
  65     # cd ..
  66
  67 Follow these steps from the Gentoo handbook:
  68
  69 * [Copy DNS info](https://wiki.gentoo.org/wiki/Handbook:PPC64/Installation/Base#Copy_DNS_info)
  70
  71 * [Mount necessary filesystems](https://wiki.gentoo.org/wiki/Handbook:PPC64/Installation/Base#Mounting_the_necessary_filesystems)
  72
  73 * [Enter the chroot environment](https://wiki.gentoo.org/wiki/Handbook:PPC64/Installation/Base#Entering_the_new_environment)
  74
  75 After which you need to add our build flags replacing the default ones in
  76 `/etc/portage/make.conf`
  77
  78     `COMMON_FLAGS="-O2 -pipe -mcpu=power9 -mno-altivec -mno-vsx -mno-crypto -mno-htm -mlong-double-64"`
  79
  80     `CPU_FLAGS_PPC=""`
  81
  82 Finally, rebuild everything on the system:
  83
  84     # emerge -e --ask --keep-going @world
  85
  86 # (Optional) Building an updated Gentoo SFFS
  87
  88 This is highly not recommended unless you specifically want to explore
  89 how newer versions of software will behave when built with the SFFS flags.
  90
  91 One way of doing this is to just set up Gentoo SFFS from the provided
  92 stage3, add out `make.conf` options and then upgrade everything, that
  93 is much simpler and easier to roll-back if need be.
  94
  95 After having a working chroot using the stage3 as outlined earlier,
  96 run these commands in the chroot:
  97
  98     # emerge-webrsync
  99     # emerge --ask --verbose --update --deep --newuse @world
 100
 101 The other way is to create a new Gentoo chroot or virtual machine under
 102 POWER using the [handbook](https://wiki.gentoo.org/wiki/Handbook:PPC64),
 103 and once you reach the [stage3
 104 tarball](https://wiki.gentoo.org/wiki/Handbook:PPC64/Installation/Stage#Unpacking_the_stage_tarball)
 105 step, download our [make.conf](link) and copy it to
 106  `/etc/portage/make.conf`, replacing the provided one, and run:
 107
 108     # emerge -e --ask --keep-going @world
 109     # emerge-webrsync
 110     # emerge --ask --verbose --update --deep --newuse @world
 111
 112 To first rebuild Gentoo for SFFS, and then update the repositories,
 113 and finally to upgrade the software you have.
 114
 115 # Testing for VSX in object files
 116
 117 To verify that our buildflags were applied by the build tools and
 118 respected by the build scripts of packages, we need to check for the
 119 inclusion of VSX as well as any other instructions not available in SFFS.
 120
 121 This is not a perfect solution and the proper way would be either to:
 122
 123 - Simulate an SFFS compliant chip running this code such as
 124   [Microwatt](https://libre-soc.org/HDL_workflow/microwatt/)
 125   (very time consuming)
 126 - Use a softcore FPGA core or ASIC of Microwatt / LibreSOC
 127   to run this code (doable, FPGA softcore in progress
 128   [in progress](https://bugs.libre-soc.org/show_bug.cgi?id=1037))
 129 - Use the finalized Libre-SOC chip (doable once it is ready)
 130
 131 Also note that this only tests for a subset of VSX instructions at the
 132 moment, more will be added in the future, this is simply a quick test to
 133 run before attempting to run on simulation and/or a softcore as both can
 134 be somewhat time consuming. In other words, if this test gives any VSX
 135 instructions, don't attempt to run the code in simulation or a softcore
 136 to avoid wasting precious time.
 137
 138 To attempt this test, you will need the scripts from our `dev-env-setup`
 139 repository
 140
 141     $ git clone https://git.libre-soc.org/git/dev-env-setup.git
 142
 143 Do *look through* the
 144 [code](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) before running
 145 any of those scripts.
 146 This is your own legal responsibility (to not run arbitrary code off
 147 of the internet) and we take no responsibility or accept any liability
 148 whatsoever for your use or misuse of them.
 149
 150 It is expected for you to use Debian for the host OS (anything else is
 151 unsupported: many contributors have repeatedly gotten into trouble by
 152 not following this advice), while all the chroots - which are developed
 153 very specifically for "reproducible builds" - run Debian 10 (Buster).
 154
 155 Edit the `test-vsx` script to point the `CHROOT` variable to your
 156 respective chroot of Gentoo, read through the script thoroughly, and run
 157
 158     $ ./test-vsx
 159
 160 If you see any number of instructions in certain binary files, those
 161 files do contain VSX and work needs to be done (patching the build
 162 scripts and/or source code) for these packages. If not, great! Try to
 163 run this in a proper SFFS environment to fully test the code you have.
 164
 165 You can also test specific binaries at any time by running this part of
 166 the script
 167
 168     $ objdump -d $OBJ_FILE | fgrep -of VSX_INSNS.txt | sort | uniq -c \
 169                 | awk '{print $2 " " $1}'
 170
 171 Where `$OBJ_FILE` is the binary you need to check.
 172
 173 ## Notes and issues
 174
 175 * Gentoo's base system packages will not complain when building for SFFS
 176   (as of July 2023, can change depending on updates)
 177 * The Linux kernel cannot boot without AltiVec and VSX, even though they
 178   are build-time options. The solution for now is to use [microwatt's 5.7
 179 kernel](https://lore.kernel.org/all/CAPweEDw710zFK8KLZY5gsQxEkQKrDiFkNRgABY9HJZ1rxpeVCg@mail.gmail.com/T/)
 180   and to incorporate the needed patches upstream in the future.
 181 * Previous note also means this is purely a chroot environment, this page
 182   will be updated once it becomes possible to boot a full VM of this build.
 183 * glibc *may* have some issues on sffs, the only way to test this is to
 184   run on microwatt which will be done soon.
 185