Configuration Migration

Introduction

Within enterprise applications, it’s a good practice to separate your development, test and production instances to be able to establish a clear change management process, properly test any changes before you activate them and to not impact the production environment when making changes to configurations. We believe in this best practice as well and can provide additional instances to customers upon request. This will allow you to then configure your various processes and test them out before you set them active in your production instance.

In case you need additional instances to be available, please contact your Oomnitza account executive or customer success manager who can help you to get additional instances added and deployed.

In most cases, separating your configuration from the production environment is enough to be able to test your new configurations before you set them active. In other cases, like when you have heavy integrations with other systems, we’ve even seen customers request up to 4 instances to run their change management processes. Whichever process you have, we are here to help you migrate your configuration from one instance to another so that you don’t have to manually redo the configurations in every landscape and are as such prone to data entry mistakes as well as different IDs across your landscape.

config_migration_simple.png

The purpose of the Configuration Migration Tool is to:

  • Allow you to move configuration from a source into a target instance, like config to prod
  • Restrict changes in the target (production) instance to ensure both instances stay in sync
  • Re-baseline the source instances from your production instances (coming later)

The purpose of the configuration migration tool is to establish a proper change management process for your Oomnitza instance. Please also keep in mind that using the configuration migration tool allows to push a significant amount of changes at once to the target instance and as such should be run at low usage times to avoid interruptions to your end-users since some of these changes like field changes to a large asset or user database, can take significant time to get processed.

 

Pre-requisites

In order to properly use the configuration migration process within Oomnitza, we need to consider a few prerequisites to get started:

  1. You need to define how many instances you need for this and get them deployed
    1. Typical processes include a staging and production instance or even a development, staging and production instance. A full 3 system landscape may only be required in case you have heavy configuration requirements. Typically a 2 system landscape should be sufficient.
    2. In case you require an additional instance, please reach out to your Oomnitza account executive or customer success manager.
  2. You need to request the configuration migration to be enabled for the source instance (that is the instance where your configuration happens) through a support request. You only need to request this to be enabled for the source system, not the target system. So in case you want to migrate configuration from staging to production, you only need to get this enabled within the staging instance.
  3. You need to start with a clean copy for all instances that should be included in the configuration migration process. The configuration migration will only work if both systems are in sync, meaning that all objects are absolutely identical, like each field in the business objects has not only the same technical fieldname, but also the same unique database identifier. For new customers, we typically deploy identical instances when we start. For existing customers that add additional instances later on you might want to start with a copy from production down to staging to get things started. In case you need help with this, feel free to reach out to our support team who can help you to get this started properly.
  4. Create a user in your target instance that will be used for the migration of configurations from your source to your target instance . This user should have full access to the configurations and will be the one noted as created by- and last changed by-user in the target system. For this user, you will also have to create an API Token that you will use when setting up the credential to use for the configuration migration.
  5. In the source instance, create a Credential for the above mentioned user pointing from your source instance (like staging) to your target instance (like production). A credential can be created within “Settings” > “Credentials”. You might want to create a credential without spaces in the name (please only use letters, numbers and underscore). Also it is advisable to use a system account in the target instance for the creation of these configurations.
  6. In the source instance , maintain the Global Settings for the Migration
    1. migrations.subdomain - defines the name of the target instance to where you want the configuration to migrate to. Please use the instance name without the .oomnitza.com extension.
    2. migrations.credential - defines the name of the credential to use to communicate to the target instance. 
  7. Make sure that both instances are on the same release version. In some cases during the release process at Oomnitza it may be that the 2 instances are on different releases for a short period of time (usually a few days to up to 2 weeks max) and in this case the configuration migration can not happen to avoid data inconsistency. The migration process will alert you about this when being started.
  8. You should disable configuration changes in the target system to avoid making changes in the target instance that will prevent you from using the configuration migration. You can do so by maintaining the migrations.disableConfig within Global Settings. By setting this to 1, the users in the target system will not be able to make configuration changes, which includes adding or updating customizations for any object, changing roles and others. This setting will not prevent the users from adding saved searches or Dashboards since we implemented number ranges (see below) to separate items in development from production.

The above prerequisites should be in place before you start additional configurations in your development or testing instance to make sure that you can move configurations to the other instance without running into issues. 

 

Migrating configuration between instances

In order to start the configuration migration, you first have to define the target instance within the Global Settings, see details above already. Also please review the above prerequisites to make sure you can use the migration tool. It is important to not create any new business objects, fields or roles in the configuration of your target system once this process is started to avoid the instances getting out of sync and as such the migration stops working. To prevent this from happening we highly recommend you to restrict users from making configuration changes directly in production by setting the migrations.disableConfig switch in your target instance. 

Once this is done, you can start to move configuration from the current instance to the target instance. For this, please select the menu “Migrations” within your source instance.

Within Migrations, you can move one or multiple items at a time from the current instance to the target instance. Right now we only support a limited amount of configurations to be moved but will continue to extend this over time.

The options that are available to migrate are:

To avoid any dependencies between these objects, you should always start with migrating the business objects since the other items may have dependencies on the business objects and will error out in case these dependencies are not in place (see also below section on Understanding Dependencies). In a second step you can then migrate the roles, followed by screen designs and saved searches. It is important to keep this order for migrations to avoid any inconsistencies in the target instance.

In order to support your migration activities we added a Test mode to the migration which will allow you to validate if the migration can be completed or in case there are any dependencies. Using Test instead of Migrate will verify all dependencies but not commit the final update to the target instance. The results of the test run can be found in the Sync Session log.

Also keep in mind that sometimes the items that you migrate may reference master data, like users and locations. In that case, please make sure that these items exist in the target instance already prior to migration of items that depend on them. This applies among others to integrations that reference a system user to use for creating or updating items in Oomnitza or also to credentials and API keys that you may have to cerate in the target instance directly. 

 

Migrating Business objects

When migrating a business object, like Assets or People you will always migrate the entire object. Moving only a single field is not supported right now. Also the migration can only create fields and will not delete any fields in the target system. To delete a field, please go to the target instance, run the dependency checker in the field detail screen and resolve all dependencies before you can manually delete the field from the target instance.

To migrate a business object, select Business Object as the scope for the migration and then the business objects you wish to migrate. Once you click Next and confirm your selection with the Migrate link, the system will trigger the migration of the selected business object(s). Depending on the size of these objects and the number of objects, this may take some time. 

Using the Migrations - Sync Session menu, you can always check on the status of these migrations as well as review the log for details.

Migrating Roles

Migrating Roles will move the details of a role from your source to your target instance. This includes all authorizations for the given role but not the screen designs which would need to get moved separately. Keep in mind that when your authorizations depend on certain user attributes that you should first migrate the business objects before migrating the roles.Same as with business objects, this will only update the existing roles or add new ones. Deletions would need to be carried out in the target instance locally.

Migrating Web Screen Design

The migration of web screen design will allow you to migrate the screen builder settings for a given role and object to the target instance. This will migrate the web user interface settings only and not the mobile screen design. You will find 2 drop downs that allow you to select a combination of Roles and Objects for the configuration migration. Both drop downs allow you to multi-select from the list to move multiple items at once to the target instance. Given the Primary UI role feature, the screen design can only get migrated for primary UI roles or stand-alone roles.

It is advisable to migrate the objects and roles first before you migrate the screen design for a given object and role to avoid data inconsistencies. Also this does not support delete operations, only create and update operations. To delete a given web screen design for a role, please do so in the target instance directly.

Migrating Saved Search

This menu item allows you to publish selected Saved Searches from your source into your target instance. You can only migrate public saved searches. Private saved searches can not get migrated. Also you can only add new ones or update existing ones. Deletion via migrations are not supported right now and deletions would have to be done manually in the target instance. 

Saved Searches can get created in the source instance and moved to the target instance as well as can get locally created in the target system. For this purpose we implemented separate number ranges between your source and target system. In the source system, you will be able to create and migrate saved searches with ids from 1 to 1,000,000 while in the target system, the locally generated saved searches start with id 1,000,000. If we find a saved search with an id that is larger than 1,000,000 the system will warn you about this and offer you to delete all such saved searches. This is necessary due to database constraints and without such deletion you will not be able to create additional saved searches in the allowed number range in the source system.

For saved searches there is no need to stay in complete sync between both instances. It is advisable to at least create common public saved searches in the source instance and migrate them rather than creating them directly in the target instance. Also any saved searches that you may reference in subscriptions, data exports or otherwise should be created in the source instance.

Migrating Credentials

Selecting Credential allows you to migrate one or multiple credentials into your target instance. For security purpose, we only migrate the credential definition, aka the shell of it which includes the name and description, but not the credentials themselves. This means that you will have to go to the target instance after the migration to enter the proper credentials like username and password or API Token there. This also will allow you to design the same workflows and integrations in both instances that reference these credentials but access different landscapes via the usage of credentials and global settings.

Migrating Global Settings

Via the menu Global Setting you can migrate any Global Setting to the target instance. To allow a clear separation between your development and test landscape, the tool migrates these Global Settings only upon creation but does not update the values in subsequent runs. This will allow you to have different values for these settings between your development and production landscape to e.g. connect to different subdomains for certain integrations. 

Migrating Dashboards

Using the Dashboard menu item allows you to push Dashboards and the widgets that are included in these dashboards to the target instance. Within this step, you can only add new Dashboards and Widgets to the target instance. Deletions are not supported via the migration tool and have to be done in the target instance directly. This will make sure that any widget that was locally added in the target instance will not get overwritten or deleted by accident.

Dashboards and Widgets behave in the same way as Saved Searches wherein you can create them in your dev instance and migrate them or also create them locally in your target (production) instance. 

Migrating Integrations

The Integration menu item allows you to migrate Extended Integrations including field mappings and all other settings to a target instance. This only applies to Extended Integrations. Basic Integrations are not supported here and will be replaced with Extended Integrations shortly. It is advisable that you switch to using the Extended Integration if they are already available.

When migrating Integrations, please make sure that if you reference a user within the integration that this user, identified by its username, also exists in the target instance and has proper authorizations there to perform the data integrations. 

Migrating Custom Objects

Migrating Custom Objects works much like migrating standard Business Objects. You will always migrate the entire object, and individual records cannot be migrated. Additionally, the migration can only create fields and will not delete any fields in the target system. To delete a field, please go to the target instance, run the dependency checker in the field detail screen and resolve all dependencies before you can manually delete the field from the target instance.

To migrate a custom object, select Custom Object as the scope for the migration and then select the custom objects you wish to migrate. Once you click Next and confirm your selection with the Migrate link, the system will trigger the migration of the selected business object(s). Depending on the size of these objects and the number of objects, this may take some time. 

Using the Migrations - Sync Session menu, you can always check on the status of these migrations as well as review the log for details.

Migrating Workflows

Migrating Workflows allows you to migrate your workflows to your target instance. This allows you to thoroughly test your workflows in staging before migrating them to production. If using API Blocks or Global Settings references in your migrated workflows, please make sure these setting also exist with the same names in your target instance, otherwise, you'll need to update them after performing the migration. 

 

Understanding Dependencies

When using the Configuration Migration it is important to understand that there are dependencies between the various items that can get migrated. For example when you migrate a Saved Search, you should make sure that all referenced fields from the business object are already available in the target instance. The image below show the dependencies between the various items:

ConfigurationDependencyMatrixColor.svg

The validation of these dependencies is build into the configuration migration tool and you can run the configuration migration in test mode which will validate these dependencies before you actually run the migration. This can be achieved via the Test function.

Another thing worth noting is that once you start using the Configuration Migration, you should restrict configuration changes in your target instances since this can bring the instances out of sync which will stop the migration from working. To prevent this from happening we highly recommend you to restrict users from making configuration changes directly in production by setting the migrations.disableConfig switch in your target instance. 

Next steps

With the Winter’21 release we are making the first iteration of the Migration Tool available and will continue to enhance it going forward. The plan is to step by step cover all configuration data and establish an easy and repeatable way to refresh the development system from production. We are looking forward to get your feedback on the migration tool and how we can make it even better.

 

 

 

Have more questions? Submit a request

0 Comments

Article is closed for comments.
Powered by Zendesk