Service Outage: East US Region

Incident Report for IndySoft

Resolved

We are pleased to confirm that the service outage affecting our East US region has been fully resolved as of 4:45 PM EST.

Final Status:
- All systems are operating normally
- Full service has been restored

Root Cause: An automated security policy update was incorrectly deployed to production systems at 2:05 PM EST, affecting performance for active user sessions in the East US region. The policy has been reverted and safeguards added to prevent recurrence

Actions Taken:
- Identified and reverted the problematic configuration
- Validated all systems are functioning correctly
- Implemented additional deployment safeguards

We sincerely apologize for any disruption this may have caused to your operations. We take service reliability seriously and are committed to preventing similar incidents in the future.

If you continue to experience any issues, please contact our support team immediately at support@indysoft.com.

Thank you for your patience and understanding during this incident.

Sincerely, IndySoft Cloud Team
Posted Jun 25, 2025 - 15:56 EDT

Update

We are continuing to investigate this issue.
Posted Jun 25, 2025 - 15:55 EDT

Update

This is an update regarding the service outage affecting our East US region cloud-based systems as of 3:20 PM EST.

Services Affected: East US region
Status: Issue appears to be mitigated - service restoration in progress

Impact: Most users should now experience normal performance

What We've Done:
- Identified and isolated the affected virtual machines
- Migrated traffic to healthy servers
- Implemented temporary mitigation measures
- Currently monitoring system stability

What We're Still Doing: Our engineering team continues to investigate the root cause to ensure this issue doesn't recur. While services appear to be stabilizing, we are:
- Monitoring all systems closely for any recurring issues
- Analyzing logs to determine the underlying cause

Current Recommendation: Users who previously experienced issues should now be able to access services normally. If you continue to experience problems, please log out and back in to ensure you're connected to an updated server pool.

Next Update: We will provide another update by 4:15 PM EST or sooner if the situation changes. A full root cause analysis will be provided once our investigation is complete.

We appreciate your patience as we work to ensure full system stability.

Thank you for your understanding.
Posted Jun 25, 2025 - 15:44 EDT

Update

We are continuing to investigate this issue.
Posted Jun 25, 2025 - 14:34 EDT

Investigating

We are currently experiencing a service outage affecting our cloud-based systems. Our team identified the issue at 2:05 PM EST and is actively investigating the root cause.

Services Affected: East US region
Impact: Some virtual machines are experiencing intermittent connectivity issues and slow response times. We have removed affected servers from the pool

What We're Doing:
Our engineering team is working with highest priority to identify and resolve the issue. We are currently investigating all possible causes and will implement a solution as quickly as possible.

Immediate Action:
If you are experiencing issues, please try logging out completely and logging back in. This will connect you to a healthy server.

Next Update:
We will provide an update within the next 60 minutes, or sooner if the situation changes.

We sincerely apologize for any inconvenience this may cause to your operations.

Thank you for your patience and understanding.
Posted Jun 25, 2025 - 14:33 EDT
This incident affected: IndySoft Cloud Software (Americas).