Re: [Discuss OpenCPI] Long Core and Assets Build Times

From: David Banks <dbanks_at_geontech.com>
Date: Thu, 14 Mar 19 15:10:03 -0400

Andrew,


According to Xilinx Vivado's UG901 and UG904, the Tcl parameter
"general.maxThreads" can be used to set the number of threads to be used
for synthesis and implementation. That being said, this defaults to its
maximum of 8 on Linux systems. You can check your installation's default
value for this parameter by running the following:
vivado -mode batch -source /dev/stdin <<<'puts [get_param
general.maxThreads]'

So, your installation is likely already using the global maximum of 8
maxThreads. It *could *be the case a lower number of threads could result
in faster builds on your system, but I wouldn't bet on it. If you wish to
change this parameter, you could create an "init.tcl" file for your user
that will be sourced each time you open Vivado (see UG835 for more
information). Note that I do not believe this has ever been tried in
OpenCPI. In that init.tcl script, you could add something like the
following:
set_param general.maxThreads 2

Regards,
David Banks

On Thu, Mar 14, 2019 at 1:48 PM Chris Hinkey <
chinkey_at_geontech.onmicrosoft.com> wrote:

> there are a lot of things and configurations of things that are built in
> each of the projects that are supplied.
> there are a lot of bitfiles being built in assets that you may or may not
> be using. try using the --no-assembly option and only build the assemblies
> as you need them.
>
> if a worker isn't changed there no reason that it needs to be rebuilt (the
> underlying make system deals with these dependencies for single
> workers/assemblies/). If you change a worker and don't rebuild the worker
> and build an assembly that it contains the worker will not get rebuilt as
> required to reflect the change that was made.
>
> a change to the framework (i.e opencpi version change updates to anything
> but not limited to in opencpi/runtime or opencpi/tools/ocpigen) will always
> require all projects to be rebuilt. but, I would expect your development
> process would just be updating what is in your project and not making a lot
> of changes to the framework itself not requiring a rebuild of core and
> assets.
>
> you can copy built projects as long as they are not registered. so you
> can unregister the project copy it to other computers like you mention.
>
> Is there any way to speed up this build process?
> It is possible to greatly reduce the build-time burden for the core,
> assets and your custom projects by building them in parallel, but it first
> requires an understanding of their (internal/external project)
> dependencies. As a point of reference, we've been able to reduce
> build-times of the core and assets projects targeting the Zedboad down to
> ~1 hour. (FYI - Targeting the Ultrascale++ parts seem to take longer than
> Zynq).
>
> Before we go any further, be warned that building too many "assets"
> (core/assets/custom) in parallel can render your host practically useless
> for some unknown amount of time, or possibly crash. I recommend
> incrementally adding parallel builds to gauge your systems capabilities
> (our systems have 64 GB RAM).
>
> Review the OpenCPI_HDL_Development, Section 4.2, Figure 7. This is the
> roadmap that you'll use for executing parallel builds and understanding
> cross project dependencies.
> If 'ocpidev build --hdl-platform zed' is executed from the top-level of
> the project, the HDL build order (+RCC) follows Figure 7:
> RCC workers(app & proxies)
> HDL primitive libraries
> application workers
> device workers
> adapters workers
> cards (device workers)
> platform workers (unique device workers)
> assemblies
> applications
>
> Building a single (Core) project in parallel:
> 1) For the HDL build discussion, the core project HDL primitives are a
> dependency within and across all projects and MUST be built first.
> $ ocpidev build hdl primitives --hdl-target zynq
>
> 2) Since Workers (App, Device, Adapter, & Cards) are never dependent on
> each other, then they can be build in parallel.
> $ ocpidev build -d components --hdl-target zynq &
> $ ocpidev build -d hdl/devices --hdl-target zynq &
> $ ocpidev build -d hdl/adapters --hdl-target zynq &
> $ ocpidev build -d hdl/cards --hdl-target zynq &
>
> 3) Build the desired platform(s) in parallel
> $ ocpidev build -d hdl/platforms/zed --hdl-platform zed &
> $ ocpidev build -d hdl/platforms/matchstiq_z1 --hdl-platform matchstiq_z1 &
>
> 4) Build assemblies (NO ASSEMBLIES IN CORE PROJECT, here for completeness)
> $ ocpidev build --hdl-platform matchstiq_z1 ('All' in the order in which
> they are listed in hdl/assemblies/Makefile)
> $ ocpidev build --assembly <assm_name> --hdl-platform matchstiq_z1 (A
> specific assembly)
>
> 5) Build applications (NO APPLICATION IN CORE PROJECT, here for
> completeness)
> $ ocpidev build application ('All' in the order in which they are listed
> in applications/Makefile)
>
> Building multiple (Core and Assets) projects in parallel:
> 1) Core project HDL primitives are a dependency within and across all
> projects and MUST be built first.
> $ cd ./core
> $ ocpidev build hdl primitives --hdl-target zynq
> 2) Assets project HDL primitives are a dependency within and (potentially)
> across all projects and MUST be built first.
> $ cd ./assets
> $ ocpidev build hdl primitives --hdl-target zynq
>
> 3) Core and Assets: Since Workers (App, Device, Adapter, & Cards) are
> never dependent on each other, then they can be build in parallel.
> (THIS WILL SLOW YOUR SYSTEM DOWN, RECOMMEND INCREMENTALLY ADDING ocpidev
> build... to gauge system performance)
> $ cd ./core
> $ ocpidev build -d components --hdl-target zynq &
> $ ocpidev build -d hdl/devices --hdl-target zynq &
> $ ocpidev build -d hdl/adapters --hdl-target zynq &
> $ ocpidev build -d hdl/cards --hdl-target zynq &
> $ cd ./assets
> $ ocpidev build -d components/base_comps --hdl-target zynq &
> $ ocpidev build -d components/util_comps --hdl-target zynq &
> $ ocpidev build -d components/dsp_comps --hdl-target zynq &
> $ ocpidev build -d components/misc_comps --hdl-target zynq &
> $ ocpidev build -d hdl/devices --hdl-target zynq &
> $ ocpidev build -d hdl/adapters --hdl-target zynq &
> $ ocpidev build -d hdl/cards --hdl-target zynq &
>
> 4) Build the desired platform(s) in parallel
> $ cd ./assets
> $ ocpidev build -d hdl/platforms/zed --hdl-platform zed &
> $ ocpidev build -d hdl/platforms/matchstiq_z1 --hdl-platform matchstiq_z1 &
> $ cd ./bsp_custom
> $ ocpidev build -d hdl/platforms/custom --hdl-platform custom &
>
> 6) Build assemblies (NO ASSEMBLIES IN CORE PROJECT, here for completeness)
> (Will slow system down, depending on number of parallel assemblies being
> built)
> $ cd ./core
> $ ocpidev build --hdl-platform matchstiq_z1 ('All' in the order in which
> they are listed in hdl/assemblies/Makefile)
> $ ocpidev build --assembly <assm_name> --hdl-platform matchstiq_z1 (A
> specific assembly)
> $ cd ./assets
> $ ocpidev build --hdl-platform matchstiq_z1 ('All' in the order in which
> they are listed in hdl/assemblies/Makefile)
> $ ocpidev build --assembly <assm_name> --hdl-platform matchstiq_z1 (A
> specific assembly)
>
> 7) Build applications (NO APPLICATION IN CORE PROJECT, here for
> completeness)
> $ cd ./assets
> $ ocpidev build application ('All' in the order in which they are listed
> in applications/Makefile)
>
>
> There is another level of parallel building which involves the use of
> "make -j HdlTargets=<zynq>" but this for the advanced users. We recommend
> using ocpidev as previously described.
>
>
> Can the build system utilize more threads?
> Vivado does this automatically under the hood. We don't have any tooling
> to tweak this from what i can tell from looking at the makefiles.
>
>
>
>
> ________________________________
> From: discuss <discuss-bounces_at_lists.opencpi.org> on behalf of Andrew
> Lippolis <ALippolis_at_girdsystems.com>
> Sent: Thursday, March 14, 2019 11:52 AM
> To: discuss_at_lists.opencpi.org
> Subject: [Discuss OpenCPI] Long Core and Assets Build Times
>
> I'm working with OpenCPI and have a project that will require several
> iterations of changes to OpenCPI functionality and to the core and assets
> projects. Since the core and assets projects need to get recopied and
> rebuilt for each iteration, this will turn into a time consuming process.
> For instance, just building the projects once took almost 5 hours on an 8th
> generation i7 with 32 GB of RAM! The problem is multiplied due to the fact
> that I have many colleagues that will also have to perform the same process.
>
> Is there any way to speed up this build process? Can the build system
> utilize more threads? Are there pieces that do not need to be rebuilt for
> future generations of OpenCPI? Can I copy the pre-compiled projects to
> other computers and register them? I'm welcome to any feedback or help on
> this issue.
>
> Thanks,
> Andrew Lippolis
>


Received on Thu Mar 14 2019 - 19:10:03 CDT