Automation Workflows processing delay

You are here:
Estimated reading time: 1 min

In order to keep our system healthy and manage high data throughput, transactional and workflow campaigns have a small delay in processing events.

How system works:

  1. data points are collected
  2. each data message is processes and goes through an ETL process
  3. produced output is added to different groups of consumers
  4. each consumer worker takes the data and acts on it
  5. processing delays in the meantime.

Point 3.

There is a maximum 3 minutes cache between new/updated campaigns are being read.

Point 4.

There is a maximum 3 minutes cache until workers pick up changes in campaigns

Point 5.

At every point (especially 2 and 4) there might be a delay when queues are pilling up with too many events, or bad network conditions, or broken functionality that requires waiting time.

Adding all these things up, you might get a 6-8 minutes delay before an event driven campaign will start processing.

Was this article helpful?
Dislike 0
Views: 16