Services Outage - July 9 10:30 UTC
Incident Report for Ai-Media
Postmortem

Starting at approximately 10:28 UTC and lasting until approx 10:42 UTC on Sunday, July 9, we were experiencing processing backups/overloads on the main iCap authentication system. This affected connectivity for most users who were connected at that time and would have prevented or delayed new iCap logins during this period. This also affected Lexi jobs that were running or scheduled to run during this time, as Lexi depends on the iCap network for functionality. We believe we have identified the trigger for this overload and are working to ensure the stability of the iCap system going forward.

We apologize for any inconvenience this outage may have caused. If there are any questions or comments please email eeg.support@ai-media.tv and reference this incident (Services Outage - July 9 10:30 UTC) in your subject line, and we will get back to you as soon as possible.

Posted Jul 09, 2023 - 16:14 UTC

Resolved
After monitoring for several hours we are confident this issue is resolved. Post mortem to follow.
Posted Jul 09, 2023 - 16:00 UTC
Monitoring
We are aware of an incident that occurred today (July 9) starting around 10:30 UTC and lasting approximately 15 minutes. During this time many iCap users were logged out and some Lexi instances were not producing output. The issue looks to be resolved but we are still investigating root cause.
Posted Jul 09, 2023 - 13:32 UTC
This incident affected: iCap and Lexi.