Multiple camera synchronisation (wait group)

3DExpress includes functions that help synchronising the input from multiple cameras. These cover the situation when the user has a common trigger for multiple cameras that are looking at a particular scene, and the 3DExpress client program should distinguish if the incoming acquisitions are for the same scene or not.

Lost of RangeMaps, trigger line noise, buffer overflow situations, etc. may cause that some of the cameras have some undesired RangeMaps or the transmission of one has been lost.

Related functions:

The functions waitGroup allow the user to wait for multiple outputs coming from multiple cameras, and provide an easy to parse result to evaluate if the results are of the same scene or not. This result of the waitGroup function is a vector with one integer per output, describing the synchronisation state of that incoming acquisition. These integer values have special meanings.

For example, a result "1,1,2,-1,2" means that the outputs 0 and 1 are of the same scene, and the first arrived. THe output 2 and 4 are of the same scene, but later than that of the outputs 0 and 1. And the output 3 has not received any data.

Condition language

To decide whether cameras are synchronised, and thus reported belonging to the same group, a simple condition language is used to describe the decision.

There are two kinds of timestamps to be used: system timestamps, and hardware timestamps (from the camera). The system timestamp is an integer in milliseconds, set once the RangeMap received is considered finished. The camera timestamp considered comes from the first frame acquired and is in camera specific time units; usually, they count ticks of a special frequency clock in each camera, be it at 2MHz, 33MHz, etc. Please check the overloaded waitGroup method that accepts offset, to see examples of camera timestamp synchronisation.

Some examples of conditions:

Hardware timestamps (set by each camera)

In case of hardware timestamps, 3DExpress has a waitGroup function that accepts a list of offsets between the timestamps used in the conditions. This allow 3DExpress to check the conditions using the timestamps set by the cameras, each camera having possibly a different clock frequency.

Some examples of conditions using hardware timestamps:

As the timestamp counters of different cameras will be started at different times, we have to correct the time difference with some common reference. If you have a common trigger, a usual way would be to trust that the first data you get will not be lost and will be in sync. Then you can fill the offsets vector, where the index is the output id, with the timestamp value of your interest.

Example of steps to synchronise different clocks for three cameras:

// Start grabbing
const char *cond = "h1/2083=h0/33540:500,h2/10000=h0/33540:500";
std::vector<long long> offsets(3, 0);
// First acquisition
std::vector<int> res = express::waitGroup(10000, cond, offsets);
// Ommitted: check that all outputs are ready, and process them
// as required.
// The groups of 'res' will be meaningless, as we haven't
// synchronised the hardware timestamps, but we assume they
// come synchronised, and take that as a reference.
offsets[0] = o0.getRangeMapCounters().timestamp_hw;
offsets[1] = o1.getRangeMapCounters().timestamp_hw;
offsets[2] = o2.getRangeMapCounters().timestamp_hw;
// Second acquisition
res = express::waitGroup(10000, cond, offsets);
// Now the cameras will be grouped properly in 'res'.
// We may reset the offsets from time to time to received values
// to be able to handle any slight clock frequency difference.
// Omitted: output processing and next acquisitions.

Grouping algorithm in detail

Starting at the output with earliest timestamp (group 1), if there is any condition that can make another output belong to the same group, that output is taken into the group. Once all outputs are checked for group 1, and there are cameras still not grouped, then we start the group 2 with the left output of earliest timestamp, repeating the procedure until all cameras are labelled with the proper group.

Note that this function does not throw sal3d::Timeout. Any output that was not ready within the given timeout will be set to group -1. That may happen with all outputs requested.

There is the possibility to use the timestamp from internal clocks of the cameras, but as that requires multiple clock synchronisation, please check the overloaded waitGroup that includes a vector of offsets.

Any output that has not been flushed will be considered as received for waitGroup. For example, a common acquisition algorithm would be to process all outputs only if they are in group 1. In case some are at group 2, flush only those incomplete in group 1, and call waitGroup again and see if all end into the group 1.

An example of acquisition algorithm using system timestamps:

int ncams = 3;
std::vector<express::Output> o;
// Start grabbing
const char *cond = "t1=t0:1000,t2=t0:1000,t2=t1:1000";
// Acquisition loop.
// Request a result for the first 3 outputs
std::vector<int> res = waitGroup(5000, cond, 3);
if (std::find(res.begin(), res.end(), -1) != res.end())
// Some cameras not ready
if (std::count(res.begin(), res.end(), 1) == ncams)
// All cameras ready and synchronised
// Flush the cameras after the processing.
for(int i = 0; i < ncams; ++i)
// Some cameras not ready.
// Process all images in group1 (partial acquisition)
// Flush out the cameras of group 1
for(int i = 0; i < ncams; ++i)
if (res[i] == 1)