As part of their current modernization and digitization initiatives, many companies are deciding to move their data warehouse (DWH) or data platform to the cloud. This article discusses from a technical/organizational perspective which aspects areof particularly important for this and which strategies help to minimize anyrisks. Migration should not be seen as a purely technical exercise. "Soft" factors and business use-cases have a much higher impact.
The motivation for migrating analytical applications, and a DWH in particular, to a cloud is complex, and all reasons should be borne in mind equally.
In order to reduce complexity and achieve a high degree of automation, it is advisable not to plan too many parallel streams, but to only move individual components or layers into the cloud via "lift & shift" in a first step, wherever necessary.
In the case of a data-platform migration, different migration strategies are hence discussed for each functional component:
Migration should be carried out in three phases to ensure a structured approach, orderly monitoring of progress and proper risk minimization. The most important phase here is the assessment phase, which is firstly intended to ascertain complexity, costs and timing, and secondly also determines the specifics of the implementation.
Intertwining migration and modernization is advisable whenever new data use-cases have to be implemented rapidly.However, this applies only if changes implemented as part of modernization do not lead to significant manual testing efforts. In addition, time pressure (renewals of licenses, etc.) sometimes entails a consecutive planning of migration and modernization.
On many occasions, a data platform's value is not discussed properly because the data team and associated infrastructure are treated as pure cost centers. This works out as long as the volume of analytical requirements does not exceed implementation capacities, and infrastructure costs as well as the number of managed reports and analyses remain within limits as data volumes increase.
Very often, migration is considered for cost reasons, which is only partially expedient because development expenditure frequently exceeds potential cost savings from more modern technologies and usage-based billing models for infrastructure. A business case for migration and modernization therefore comprises a large number of components in detail, necessitating an answer to whether a) the project is commercially positive over a time horizon of 3-5 years, or b) pure optimization of the existent solution's costs while retaining technologies might be more appropriate at first?
The business case becomes positive when the value of the migrated solution exceeds the value of the old solution.This means that the business case can sometimes be negative when considering only costs. For smaller platforms, migration costs usually exceed potential savings in infrastructure expenditure. In this case, new data use-cases must be discussed in detail with regard to their added value, and medium- and long-term strategic aspects must be taken into account.es Mehrwerts im Detail diskutiert und mittel- und langfristige strategische Aspekte berücksichtigt werden.
At the beginning of the assessment and in later phases, a logical architectural representation is helpful during discussions. In particular, requirements for individual components, interfaces and modernizations can be discussed on the basis of such a visualized target architecture, and communication with the entire organization is facilitated.
The migration strategy is defined here for each individual component. This is illustrated by means of a specific example for a customer (migration to AWS):
To optimize the migration strategy, it is always necessary to mutually weigh costs, added value, time and risk, the number of components to be migrated in parallel, and what the entire organization can achieve in a certain period of time.
In addition to inventory, architectural decisions and cost estimates, an important point during the assessment is the creation of a roadmap to represent the planning of migration and modernization during and after the actual migration:
Migration as such requires little agility. Nevertheless, we recommend implementing it with an agile approach in order to ensure progress monitoring at an early stage and to be able to identify as well as resolve conflicting goals between day-to-day business and migration in a timely manner.
Risks of a migration must be identified and addressed at an early stage:
Migration poses technical and organizational challenges. It requires comprehensive transformation which affects not only IT infrastructure but also the entire organization's operations and processes. In the medium term, the project should also lead to modernization and implementation of new use-cases, as this is the only way to achieve clear monetary effects in addition to soft advantages.
If you are considering migration, we will be happy to take stock of your situation. Apart from a joint understanding of motivation, the goal of this inventory would be an initial business case as well as vendor-independent discussions of different migration strategies. In addition to cloud-readiness checks, we also offer quick checks of DWH/data-platform migration, for example.
Do you want to replace the IoT core with a multi-cloud solution and utilise the benefits of other IoT services from Azure or Amazon Web Services? Then get in touch with us and we will support you in the implementation with our expertise and the b.telligent partner network.
Domain Lead Data Platform & Data Management
Domain Lead Data Platform & Data Management
Exasol is a leading manufacturer of analytical database systems. Its core product is a high-performance, in-memory, parallel processing software specifically designed for the rapid analysis of data. It normally processes SQL statements sequentially in an SQL script. But how can you execute several statements simultaneously? Using the simple script contained in this blog post, we show you how.
Many companies with SAP source systems are familiar with this challenge: They want to integrate their data into an Azure data lake in order to process them there with data from other source systems and applications for reporting and advanced analytics. The new SAP notice on use of the SAP ODP framework has also raised questions among b.telligent's customers. This blog post presents three good approaches to data integration (into Microsoft's Azure cloud) which we recommend at b.telligent and which are supported by SAP.
First of all, let us summarize the customers' requirements. In most cases, enterprises want to integrate their SAP data into a data lake in order to process them further in big-data scenarios and for advanced analytics (usually also in combination with data from other source systems).
In the first part of this blog, the aim was to serialize a CSV file as simply as possible to Avro, and store the result in Kafka, the schema being registered in the related registry.
No previous post
No next post
