Beginning on Monday, December 21st, 2020 Oomnitza’s workflow service experienced a service disruption that affected a number of Oomnitza instances.
Starting on Monday, December 21st at 9:00am UTC/1:00am PST through, Wednesday, December 23rd at 10:00pm PST/6:00am UTC (+1 day) Oomnitza experienced a disruption to the workflow service, during which time some workflows did not complete.
Note: For some impacted customers the potential window of impact was longer than originally communicated. We are following up with those customers individually,
Some workflows that should have run during this time may have not completed and were dropped from the queue, they currently have not been rerun. We are evaluating what impacted workflows can be rerun and will reach out to impacted customers with more specifics.
The disruption was caused by a change made to the configuration of Oomnitza’s queue protocol.
Corrective and Preventative Measures
Oomnitza treats disruption in service with the utmost care and concern, and we understand the pain and frustration it may cause. Oomnitza’s Engineering team will be implementing these changes immediately to ensure there is no recurrence in future.
- Additional monitoring
- Further isolation of queued processes
Oomnitza has followed up, or will follow up with all affected clients, and will be providing impact analysis for customers that were affected. If you have identified or experienced any related issues with your organization’s workflows, please contact us immediately at firstname.lastname@example.org.