# NL.net proposal
+* [[questions]]
+* NLNet Project Page <https://nlnet.nl/project/LibreSoC-Video/>
+* 2019-10-031
+* Top Level bugreport <http://bugs.libre-riscv.org/show_bug.cgi?id=137>
+
## Project name
-The Libre-RISCV SoC, Video Decoding
+The Libre-RISCV SoC, Video Acceleration
## Website / wiki
One of the main "hardware accelerated blocks" of any processor intended for user applications is Video Encode and Decode. This usually means an opaque, proprietary piece of hardware, and it usually comes with proprietary firmware as well.
-In a privacy-respecting world neither of these are acceptable, therefore the goal is to develop - in an iterative fashion - not just the software but the actual hardware instructions (similar to ARM NEON) which, if fully integrated into libswscale, ffmpeg, gstreamer and other software, would make RISC-V a truly commercially competitive peer of ARM and x86 systems when it comes to video decode.
+In a privacy-respecting world neither of these are acceptable, therefore the goal is to develop - in an iterative fashion - not just the software but the actual hardware instructions (similar to ARM NEON) which, if fully integrated into libswscale, ffmpeg, gstreamer and other software, would make RISC-V a truly commercially competitive peer of ARM and x86 systems when it comes to video acceleration.
-There would thus be no opportunity and no excuse for the inclusion of spying hardware blocks or coprocessors.
+With such capability freely available for any implementor, there would be no excuse for the inclusion of spying hardware blocks or coprocessors in modern RISC-V processors, and certainly not in the Libre RISC-V SoC.
# Have you been involved with projects or organisations relevant to this project before? And if so, can you tell us a bit about your contributions?
* to implement those instructions in actual hardware, for running in an FPGA
* to follow through with the upstream submission and acceptance of customisation of relevant software libre video decode projects and toolchains.
-This needs to be done iteratively because it is only when a certain high level of functionality is reached (FPGA, full simulation) will it be possible to properly determine if the proposed instructions actually meet the requirements.
+This needs to be done iteratively because it is only when a certain high level of functionality is reached (FPGA, full simulation) will it be possible to properly determine if the proposed instructions actually meet the full requirements. In addition it may turn out that further optimisation is needed.
# Does the project have other funding sources, both past and present?
from NLNet. However that is for specifically covering the development
of the RTL (the hardware source code).
-There is no source of funds for the work on the
+There is no source of funds for the work on Video ISA development (only for its hardware implementation) or the follow through work which involves getting support for that ISA extension upstream in relevant software (ffmpeg, vlc, gstreamer) and toolchains (gcc, llvm, binutils)
# Compare your own project with existing or historical efforts.
-There are several Open VLSI Tool suites:
-
-* GNU Electric: https://www.gnu.org/software/electric/
-* MAGIC: http://opencircuitdesign.com/magic/
-* The OpenROAD Project: https://theopenroadproject.org/ (using MAGIC)
-* QFlow: http://opencircuitdesign.com/qflow/
-* Toped: http://www.toped.org.uk/
-
-and a few more. We choose Coriolis2 because of its python interface.
-The VLSI Layout is actually done as a *python* program. With nmigen
-(the HDL) being in python, we anticipate the same OO benefits to be
-achievable in coriolis2 as well.
-
-The case for the Libre RISC-V SoC itself was made already in the initial
-2018.02 proposal. That has not changed: there are no Libre / Open Projects
-approaching anything like the complexity and product market opportunities
-of the Libre RISC-V SoC, which is being designed to be a quad-core 800mhz
-multi-issue out-of-order design. All other Libre / Open processors such
-as Raven, and many more, have a goal set in advance not to exceed around
-the 350mhz mark, and are single-core.
-
-Other projects which are "open", such as the Ariane Processor, are
-developed by universities, and in the case of Ariane were *SPECIFICALLY*
-designed by and for the use of proprietary toolchains, such as those from
-Cadence, Synopsys and Mentor Graphics. Despite the source code being
-"open", there was absolutely no expectation that the processor of the
-same capability as the Libre RISC-V SoC would use Libre / Open tools.
-
-Although our first ASIC (thanks to Chips4Makers) will be only 180nm,
-single-core and a maximum of around 350mhz, this is just the first
-stepping stone to a much larger processor.
+There do exist on opencores a number of video encode and decode blocks: these are typically MPEG and h.264. However, the problem is that these are dedicated blocks, specific to those algorithms. They do not help with h.265, Theora, Dirac, vp8, vp9 and anything else that may come out.
+
+Any instructions which exist for OpenRISC1200 or MIPS will not help either, because the instructions need to be evaluated specifically for RISC-V.
+
+So the answer is: this initiative is unique, and there are no peer projects: the RISC-V initiative itself is too recent.
## What are significant technical challenges you expect to solve during the project, if any?
-Some of these have been mentioned above:
+The actual process is technically quite straightforward, and given that
+ffmpeg and so on are quite well established and platform independent,
+the hotspot areas are typically already identified (CABAC, DCT,
+Motion-estimation, YUV2RGB) and have NEON or SSE/AVX etc assembly
+routines.
-* NDAs by Foundries may interfere with the ability for Chips4Makers to
- communicate with LIP6 regarding the necessary changes to NSXLIB which
- meet the TSMC Foundry "Design Rule Checks" (DRCs).
-* Bugs or missing features in nmigen, yosys, coriolis2, NSXLIB, OpenRAM,
- and the knock-on implications throughout the chain, right the way up
- to the *actual* Libre RISC-V SoC's HDL source code itself, all need to
- be dealt with.
-* Circuit simulation and unit testing is going to be a major factor, and
- a huge utilisation of Computing power. Machines with "only" 16 GB of RAM
- and high-end quad-core processors are going to be hopelessly inadequate.
+The main challenges will be communications, particularly as this is a huge cross project initiative, covering patches and additions to at least seven separate independent software projects, as well as requiring hardware development and simulations.
## Describe the ecosystem of the project, and how you will engage with relevant actors and promote the outcomes?
-LIP6 have their own mailing list for the (transparent) discussion of
-issues related to coriolis2: <alliance-users@asim.lip6.fr>. The Libre RISC-V
+The Libre RISC-V
SoC has a full set of resources for Libre Project Management and development:
mailing list, bugtracker, git repository and wiki - all listed here:
<https://libre-riscv.org/>
all picked up the story. The list is updated and maintained here:
<https://libre-riscv.org/3d_gpu/>
+It will also be necessary to coordinate transparently across the RISC-V community resources, particularly when it comes to developing the ISA Extensions.
+
# Extra info to be submitted
* <http://libre-riscv.org/3d_gpu/>
-* <https://www-soc.lip6.fr/equipe-cian/logiciels/coriolis/>
* <https://nlnet.nl/project/Libre-RISCV/>
* <https://chips4makers.io/blog/>
projects / libreriscv.git / blobdiff