In order to keep our system healthy and manage high data throughput, transactional and workflow campaigns have a small delay in processing events.
How system works:
- data points are collected
- each data message is processes and goes through an ETL process
- produced output is added to different groups of consumers
- each consumer worker takes the data and acts on it
- processing delays in the meantime.
Point 3.
There is a maximum 3 minutes cache between new/updated campaigns are being read.
Point 4.
There is a maximum 3 minutes cache until workers pick up changes in campaigns
Point 5.
At every point (especially 2 and 4) there might be a delay when queues are pilling up with too many events, or bad network conditions, or broken functionality that requires waiting time.
Adding all these things up, you might get a 6-8 minutes delay before an event driven campaign will start processing.