[Discuss OpenCPI] Re: Sharing workload between multiple workers

From: Aaron Olivarez <aaron_at_olivarez.info>
Date: Thu, 18 May 23 11:00:25 -0500

How are you executing your application? Is it using an OpencPI application
Specification (OAS) and `ocpirun` command line tool or utilizing a C++
program using the application control interface (ACI).

If you are using an OAS you can specify additional RCC containers using the
`-n` or '--processors' argument on `ocpirun` to specify the number of rcc
containers to create at execution.
It defaults to 1 but if you specify more than 1 it will round robin the
workers based on the numbers of available containers. Another option you
have is to specify a worker to run on a specific container using `-P`
argument. For syntax is -P <instance-name>=<platformname> as an example -P
parital_fft0=rcc0 -P partial_fft1=rcc1 etc. You can specify the platform in
XML as well as by adding an attribute to the instance Platform='rcc0' This
can also be done through XML.

By using the -v argument when executing `ocpirun` you can see where the
worker was ran. Below is an example of running an application with 5
workers using 5 containers and allowing OpenCPI to dictate which container
to run the worker.
```
~/opencpi-v2.4.6/projects/assets/applications$ ocpirun -v -n5 testbias5.xml
Available containers are: 0: rcc0 [model: rcc os: linux platform:
ubuntu18_04], 1: rcc1 [model: rcc os: linux platform: ubuntu18_04], 2: rcc2
[model: rcc os: linux platform: ubuntu18_04], 3: rcc3 [model: rcc os: linux
platform: ubuntu18_04], 4: rcc4 [model: rcc os: linux platform: ubuntu18_04]
Actual deployment is:
  Instance 0 file_read (spec ocpi.core.file_read) on rcc container 0:
rcc0, using file_read in /home/aaron/opencpi-v2.4.6/projects/core/artifacts/
ocpi.core.file_read.rcc.0.ubuntu18_04.so dated Fri Mar 31 11:25:20 2023
  Instance 1 bias0 (spec ocpi.core.bias) on rcc container 1: rcc1, using
bias_cc in /home/aaron/opencpi-v2.4.6/projects/core/artifacts/
ocpi.core.bias_cc.rcc.0.ubuntu18_04.so dated Fri Mar 31 11:25:24 2023
  Instance 2 bias1 (spec ocpi.core.bias) on rcc container 2: rcc2, using
bias_cc in /home/aaron/opencpi-v2.4.6/projects/core/artifacts/
ocpi.core.bias_cc.rcc.0.ubuntu18_04.so dated Fri Mar 31 11:25:24 2023
  Instance 3 bias2 (spec ocpi.core.bias) on rcc container 3: rcc3, using
bias_cc in /home/aaron/opencpi-v2.4.6/projects/core/artifacts/
ocpi.core.bias_cc.rcc.0.ubuntu18_04.so dated Fri Mar 31 11:25:24 2023
  Instance 4 bias3 (spec ocpi.core.bias) on rcc container 4: rcc4, using
bias_cc in /home/aaron/opencpi-v2.4.6/projects/core/artifacts/
ocpi.core.bias_cc.rcc.0.ubuntu18_04.so dated Fri Mar 31 11:25:24 2023
  Instance 5 bias4 (spec ocpi.core.bias) on rcc container 0: rcc0, using
bias_cc in /home/aaron/opencpi-v2.4.6/projects/core/artifacts/
ocpi.core.bias_cc.rcc.0.ubuntu18_04.so dated Fri Mar 31 11:25:24 2023
  Instance 6 file_write (spec ocpi.core.file_write) on rcc container 1:
rcc1, using file_write in
/home/aaron/opencpi-v2.4.6/projects/core/artifacts/
ocpi.core.file_write.rcc.0.ubuntu18_04.so dated Fri Mar 31 11:25:23 2023
Application XML parsed and deployments (containers and artifacts) chosen [0
s 2 ms]
Application established: containers, workers, connections all created [0 s
0 ms]
Application started/running [0 s 0 ms]
Waiting for application to finish (no time limit)
Application finished [0 s 10 ms]
```

Let me know if you are using an ACI application. It's a similar process.

Aaron

On Thu, May 18, 2023 at 10:26 AM <dwp_at_md1tech.co.uk> wrote:

> Hello,
>
> My goal is to speed up a certain process through parallelisation. I have
> used an FFT as an example but this was more about exploring the
> capabilities of openCPI.
>
> I was advised that creating new threads from within a worker is not
> recommended and that each worker exists in its own thread so it seemed
> logical to create multiple instances of a worker that can handle a portion
> of the workload and have these instances run in parallel.
> See the screenshot attached showing the 3 components I created and how
> data is sent between them.
>
> It is clear that in the current implementation this is not being achieved,
> instead the Partial FFT workers are entering their run methods one after
> the other and CPU usage is stuck at using a single core.
>
> Is this something that is possible using openCPI?
>
> What issues can arise from creating threads within workers that are
> terminated before the run method is exited?
>
> Thanks in advance,
> Dan
> _______________________________________________
> discuss mailing list -- discuss_at_lists.opencpi.org
> To unsubscribe send an email to discuss-leave_at_lists.opencpi.org
>






Received on Thu May 18 2023 - 16:00:25 CDT

This archive was generated by hypermail 2.3.0 : Fri Jun 05 2026 - 08:45:25 CDT