Blog > Post Mortem on a migration

Post Mortem on a migration

The dust is settling..

The migration happened on Tuesday, and was for a large part successful. No data was harmed during the event. All data was migrated. Kolab Now is now running on hardware that has guaranteed warranty for at least the next 5 years, and on a software platform that is making it easier for us to develop and maintain.

We did however run into a very unfortunate situation. A few very urgent matters caused our support mailbox to be quite full, and on top of that, the support ticket form at ‘https://kolabnow.com/support’ was out of order for a few hours. This resulted in response times much longer than we like them to be.

A few issues impacted users:

Invalid Certificate

SMTP endpoints was temporarily secured by an outdated certificate. This meant that users got error messages complaining about certificate errors.

This was resolved.

DAV

DAV access was interrupted due to a mismatch in the URL between old and new environment. For most clients access should resume normally now, however, at least some iPhones may have picked up a wrong CalDAV URL which has to be fixed manually.
To recover access:

Go to the Calendar account settings
Find the Kolabnow Account configuration
Remove the “/dav” from the URL so it reads:

“https://apps.kolabnow.com/principal/…”

 

Missing POP3 access

POP access was initially missing in the new environment.

This has since been rectified.

Bounced mails with ‘error 550…’

The MX records that we suggested in the past for users with private domains (see the knowledge base about DNS) pointed to the old IPs. To make sure that all IPs would work during the migration, we change the last mx record to ‘mx03.kolabnow.com’ to point at the new IPs. Then all would work, but many users only had the mx01 and mx02 in place, so they would never get to mx03.

On top of that, many private domains has a very strict SPF policy, which is good, but it did lead to the 550 rejection from recipient servers.

We changed the record mx02.kolabnow.com to also point at the new IPs, and this resolved the issue. At this time, all traffic is going through the new environment, and MX is pointing correctly. The issue is fully resolved.

Lack of catchall@ addresses

The catchall@ address functionality was initially not available.

This has since been restored.

Attachments

The allowed size of attachments were set too low (1MB), even though the client stated the size to be 50MB.

This was fixed.

Passwords with spaces

External clients didn’t validate passwords with spaces properly.

This has meanwhile been fixed.

Ongoing issues

  • A few users keep having authentication issues through external clients. We assume that this could be related to special characters such as UTF-8 sequences in passwords.
    • If you are affected we encourage you to change your password (at least temporarily), to one with ASCII characters only.
  •  The new Single Sign On for webmail was/is confusing to many users.
    • Although the SSO has been a wish from many users for a long time, we sure were announcing it too late. 
    • We will try to improve the user experience, we have many suggestions from you listed on how that can be done.
  • – A few users keep having issues with getting through the SSO process, and get stuck on a blank page.
    • For most users, switching to another browser (Firefox/Chrome) has resolved the issue.
    • This issue has eluded us so far. If you are affected by this issue, then please report to Support with ‘Which OS’ and ‘Which browser’ you are using.

Overall, the migration certainly didn’t go entirely as planned, and we want to apologize for that. We understand the frustration that a lack of functioning email, calendar and contacts can bring, and we regret the distraction that this caused.

While we were working up against a (warranty) deadline, we put a lot of work into making sure that we could migrate the quite substantial dataset with no damage to the data, minimal downtime, and with continued mail traffic. We missed a few points that would cause the inconvenient issues mentioned above. The old environment has grown large over a decade, and many details has changed over the years.

Having said that, we have successfully migrated all data onto the new infrastructure, that is ready to serve you for the coming years. This new hardware and software platform will provide us with the means to continue to ensure your data’s safety and availability.

We intend to keep things stable for the weekend, and will track down remaining issues next week. Updates and news will be made available here on this blog.

Enjoy your weekend!