Note: This is still under heavy development.

To use, you’ll need to check out the streaming-plugin branches of both ROSS and ROSS-Damaris (same for CODES, if using that). Right now you can stream simulation engine data in either the GVT-based or Real time sampling modes. Virtual Time sampling for simulation engine data will be available after the development of data reduction modules for the in situ analysis system. Model data can now be streamed in any of the 3 sampling modes (as a reminder, it is only guaranteed to be causally correct in Virtual Time sampling mode). Event tracing data is not yet supported with Damaris Streaming. As a side note, I wouldn’t mix and match instrumentation modes just yet (so don’t use more than one different mode, and use the same mode if you want to stream both types of data).

Build

Follow the build instructions on the ROSS-Damaris Overview page. There’s no need to follow the instructions for the section on enabling VisIt support.

You will need a FlatBuffers install for this branch. When building ROSS with Damaris support, you’ll need to set the cmake option FLATBUF_DIR to your install directory.

Also you’ll need to add the path to libdamaris.a as well as the your ROSS/build/install/lib directory to LD_LIBRARY_PATH.

Running

There’s a new configuration option being introduced in this branch. Instead of setting all of the configuration options on the command line, you should use the configuration file. This file is only for Instrumentation settings and certain Damaris settings.

So essentially when you run your simulation you’ll do something like:

$ mpirun -np 5 ./phold --synch=3 (other ROSS/model options) --enable-damaris=1 \
--data-xml=path/to/xml/file --config-file=path/to/config-file

For the XML file, you can use the template.xml file available in the ROSS-Damaris repo for using this with simulation engine data. This is the metadata file necessary for Damaris.

For the other configuration file, you can use the example-config.cfg file in the ROSS-Damaris repo. An example of this file is:

[inst]
engine-stats = 1
model-stats = 1
num-gvt = 100
rt-interval = 1000.0
vt-interval = 500
vt-samp-end = 10000
pe-data = true
kp-data = true
lp-data = true
[damaris]
write-data = false
stream-data = true
[damaris.stream]
address = <IP address>
port = <port>

The first section [inst] is for instrumentation settings. It’s similar to how the command line options are currently for the ROSS instrumentation. For engine-stats and model-stats, 1 is for GVT-based sampling, 2 is for real time sampling, and 3 is for virtual time sampling, and 0 means no sampling at all.
So for the example here, it would run GVT-based instrumentation, sampling every 100 GVT computations and collect PE, KP, and LP level data. The write-data option is a testing feature. This lets the Damaris server write out the flatbuffers that it’s going to stream. stream-data of course turns on streaming capabilities, which will need you to set the address and the port.