Incident report: Spool overflow filling up disk..

Posted on: July 2nd, 2024 by

On Monday 2024-07-01 early evening (CEST) a spammer attempted to use a Kolab Now account for sending out large amounts of spam. The Kolab Now exit spam filter was sorting out the spam and redirecting it, as it was supposed to do, and none of the spam was sent out. The spammer was however stubborn and kept up the sending, which subsequently was filling up a disk and hence blocking traffic. Due to ongoing maintenance on the monitoring, the full disk was unfortunately not discovered until Tuesday morning, when the problem was immediately corrected, and queued mails were again flowing in both directions.

The problem caused a group of users (about 30%) to be unable to receive mail, and sent mail was queued until the space was again freed up and spooling was possible. No mail should have been lost during the incident.

The missing monitoring has been put back into action, and the Kolab Now Engineering team is evaluating changes that will prevent the situation from repeating.

We apologize for any inconvenience that this incident may have caused.

Urgent Service Window: More updates to frontend servers

Posted on: May 27th, 2024 by

A list of needed updates are ready to be deployed at Kolab Now – Some of these updates are of some urgency. Therefore we hereby announce an Urgent service window for:

Wednesday 2024-05-29 @ 07:00 UTC (09:00 CEST)

The service window will last for an hour.

Users can keep working, but at some point during the one hour service window, the webclient and the dashboard will be unavailable for a few minutes. Users will be logged out and will need to login again.

This blogpost will be updated with status of the work during the service window.

Please contact support with any questions or concerns in this context.

 

2024-05-29 @ 07:40 UTC – The work is done and the results are tested. All activities were successfull. If you have any issues please contact Support.

Service Window: Update of front end servers

Posted on: April 4th, 2024 by

A list of needed updates are ready to be deployed at Kolab Now – Some of these updates are of some urgency. Therefore we hereby announce a service window for:

Monday 2024 – 04 – 08 @ 07:00 UTC (09:00 CEST)

The service window will last for an hour. Users can keep working, but they will experience a short hickup outage at some point during that period.

One important fix that will be deployed during this service window is:

  • Roundcube: Update of calendar invitation cause the message “Failed to import event”, and the event is not updated. This has been resolved in the update.

Please contact support with any questions or concerns in this context.

 

2024-04-08 @ 07:32 UTC – Update: The updates has been executed and the service window is done. 

ActiveSync broken by Windows update..

Posted on: February 8th, 2024 by

Since last weekend, Support received a few reports from users who saw problems with Outlook ActiveSync synchronization towards Kolab Now. No emails were synchronized to their inbox, and the content of the Outlook Inbox would not match the inbox in the Kolab Webclient.

After some investigation it was realized, that Microsoft had provided (and automatically deployed) a new update on February 1st, and that this update caused the faulty behavior. The new version was: 2401 (various build numbers).

Rolling back Outlook resolved the issue for all users and brought back the normal behavior, and outlook would again synchronize email to the inbox.

A short version of the downgrade procedure is:

  • Find the version to downgrade to at the Microsoft site.
  • Disable automatic updates via Start -> Settings -> Windows Updates
  • Close down Outlook.
  • Open an admin shell.
    • Run: C:\Program Files\Common Files\microsoft shared\ClickToRun\OfficeC2RClient.exe /update user updatetoversion=16.0.17029.20108
    • (change 16.0.17029.20108 if you want to roll back to an even older version).
  • Start Outlook again.

Please seek further information about rollback of outlook versions at the Microsoft support site.

2024-02-23 – Update: We had a few users asking when we will have a fix for this issue. To that we must refer to the Microsoft support article about the issue, which can be found here. In short they lightly suggest, that the patch will be released on 1st of April (We do not think that this is an April Fool’s joke ).

 

Incident report: One external submission server overwhelmed by spam flood..

Posted on: January 8th, 2024 by

On the 2024-01-07 a spammer made a large flow on one of the external submission servers. The server stopped the spam mails, and saved them to a separate holding queue to make room for other users.

It took a while for the reporting to get to the operations team, but as soon as the issue was known it was swiftly resolved at ~19:00. However, meanwhile the server ran out of space, and some users (who hit that server) would have seen that the send and receive activities failed.

We apologize for the inconvenience that this issue has caused, and will focus on improving the reporting to also cover this specific issue.

Happy New Year 2024!

Posted on: January 1st, 2024 by

Cross-site Scripting (XSS) issue in the Kolab Webclient?

Posted on: December 5th, 2023 by

Lately we got a lot of questions at Support about a possible vulnerability in Roundcube, the Kolab Now webclient. The listing in question is  CVE-2023-47272 which upstream in the Roundcube development has been handled and patched.

To see the version of Roundcube running as the Kolab now webclient, login and press the ‘About’ button in the left side of the screen. This will reveal that Kolab Now is running:

Roundcube Webmail 1.5.6.3-30.1

This tells, that the fix was already installed, and that the Kolab Now webclient is not impacted by the vulnerability.

We are thankful for having an observant and active user base.

 

Announcing Service Window: 25th of October 2023

Posted on: October 20th, 2023 by

The operations team is going to make an update to the Kolab Now dashboard applications framework. Therefore we declare a service window on:

Wednesday 2023-10-25 from 07:00 UTC until 09:00 UTC.

> Continue Reading

Changes: The Kolab Now support system

Posted on: October 13th, 2023 by

As most of our users probably know, Bifrost was our support system of choice for many years. It was a great system that served us well all those years, however it also had it’s weaknesses and it is time for a change.

> Continue Reading