Author Archive for Mads Petersen

On Monday June 06 2022, from approximately 11:24 UTC a group of Kolab Now users observed an error stating: “Gateway time-out” when trying to login to the webclient, the dashboard, or connect with any desktop or mobile client. The Kolab Now main page https://kolabnow.com/ and all other services (Support, blog and knowledge base) were available with no issues for these same users.

The issue lasted for about an hour, until approximately 12:19 UTC, and was caused by a db server in one of the clusters being stuck in a large operation. The issue caused troubles with the login procedure, but had no impact on the mail flow. No mail was lost or delayed during the incident.

The operations team was warned by the monitoring system about a stuck db server in the cluster, and was having hands on keyboards right away. The issue lasted for about an hour (some users in the group saw return of service earlier than others), and was caused by this db server in the cluster being stuck in a large operation.

Further investigation and action is going into reevaluating the amount of resources assigned to the db clusters and other servers for such large operations of this type.

If you were among the group of users that was impacted by this, then please accept our apologies for any inconvenience caused.

On Friday January 21 at 15:48 UTC  users observed that the main page https://kolabnow.com/ became unaccessible and reviled an error: 503 – Server unavailable. The webclient (https://kolabnow.com/apps) was still available, but with seriously degraded performance – to the point where the service would give up and reveal the ‘Maintenance’ page.

Also IMAP, SMTP, ActiveSync, and $DAV was impacted by the failing frontends.

The service was unavailable for about 50 minutes, during which time all mails, in and out, were queued. When the service again was available, those mail queues first had to be delivered before new mail could be sent or received. This caused users to experience a delay in the traffic.

> Continue Reading

On Monday morning at 03:22 CET the kolab Now $DAV services stopped working.

While resolving a performance issue for $DAV connections, we moved around  servers from an old infrastructure to a new infrastructure, and with that the rotation of logs. Unfortunately a tiny bit were missed while configuring the new servers, which caused the log rotation attempting to reload httpd (while that wasn’t needed). In turn, reloading httpd didn’t succeed and timed out, and the new DAV servers were left in a maintenance state;  unavailable.

This happened at ~ 03:22 CET. As these are new DAV servers, they were only included in the standard monitoring, and no one got notified about the problem until it was discovered at 07:47 CET. The configuration was promptly corrected and the new DAV servers came back online right away (and was added to the monitoring notifications). At this time everything should be available and working as expected.

We apologize for any inconvenience that this may have caused.