Avoid big bang data migration

Data migrations are inherently high risk initiatives and, if not planned effectively, can create a significant headache for both the IT team and the organization. This is because modern technology transfers are extremely complex endeavors. However, data migration issues can often be attributed to confusion and disorganization surrounding the migration plan (if one is in place) and failure to properly prepare for the move.

Plus, every business’s data landscape is unique. It can encompass everything from legacy systems to unique databases developed in-house, each with its own level of support. Documentation may be non-existent, institutional knowledge may be limited, and key personnel may be gone. This makes the task much more complex to undertake.

Other issues that IT teams fear, or challenges they face, are data loss, compatibility issues, and hardware issues. In fact, according to analyst firm Gartner, 83% of data migrations either fail completely or exceed allocated budgets and implementation schedules.

Take an alternative approach

But should data migrations adopt such a “Big-Bang” approach? Should all data be transferred at the same time, especially if there are so many risks inherent in migration? Could a lower risk, iterative and agile alternative be adopted?

There are a few key points to consider when moving data. For example, does it make sense to migrate all existing data and has the organization considered what data will be reused? Likewise, what data modifications are required to accomplish the migration?

In my company, we partner with Atlassian, implementing and supporting applications such as Jira, Jira Service Management, and Confluence. Therefore, when I talk about data migrations, I’m going to do it with an Atlassian focus.

Understand what data you need to transfer

The quality of data an organization has within Atlassian applications greatly influences the risks and efforts associated with migrating to another platform. Complex applications with custom history and hidden scripts can be a minefield. To mitigate this risk, it is critical that the IT team consider how much history they will be transferring to the new system. Without a thorough understanding of the data they are going to transfer from source to target system, the negative impact of inaccurate, inconsistent, and irrelevant data is magnified.

Therefore, it is important to ensure that the data that populates the new system is fit for purpose and offers quantifiable improvements over the previous system.

Starting from scratch with the minimum amount of viable data provides the opportunity to focus on the data and workflows that deliver the most business value and improve efficiency and productivity.

Only move valuable data

Looking at the big bang data migration approach, it’s about moving the entire data set from the legacy system to the target system. This is usually done over the course of a weekend, and in order to mitigate as much risk as possible, multiple test migrations are often performed, which increases costs and requires significant effort. For example, a Jira migration can take up to 30 days of effort, and given the risk, cost, and effort involved, a more agile, iterative, and pragmatic approach makes perfect sense.

Iterative data migration means you’re only moving valuable data, and it’s all handled in smaller increments. However, this presents two key challenges: how to keep the data from the target and source systems operational until the migration is complete, and how to coordinate the migration of separate elements of users and business functionality without disrupting the overall continuity. of the activity.

Take a step-by-step approach

For iterative data migration to be successful, the two systems must operate in tandem during the transition period without influencing each other. Therefore, IT teams have to move business units or departments one by one, starting with new teams and projects on the new system and decommissioning old data on the existing system.

This iterative strategy and in-flight or minimal data migration allow effort that would otherwise be spent migrating from a “Big Bang” approach to delivering tangible business value. Often times, we find that up to 70-80% of the data that has not been migrated can be archived or retired.

I know firsthand that there can be extended downtime and significant complexity and effort associated with migrating large volumes of data, not to mention the time and cost of cleaning up unwanted legacy data. The iterative migration process not only offers significant benefits at a lower cost, but it also significantly reduces the impact and risk to the business.

Image credit: Sagittarius production/ Shutterstock

Gary Blower is a solutions architect, Clear vision. If you want to learn more about Atlassian iterative migrations, download the company guide: Avoid the Big Bang.

Sean N. Ayres