top of page

Resetting your data factory dev environment from production

Working with Azure data factory, best practice suggests that you have a dev environment and a production environment. Your dev environment is connected to source control, be it git or Azure DevOps, and the proper way to work is to develop in dev, merge the changes into your git repository, and then deploy the changes to prod. And your environments are always in sync.

But… sometimes they don't.

It's not you, of course. It's some other developer who was under pressure and had little time, and he made some changes directly to the production environment.

Now you can't move new development to production, because all the data factory is moved as one, and you'll run over the changes you made to prod. You can move objects one by one by copying their JSON, but that is bad practice, and if you're working with other developers this could lead to chaos.

You need to copy the production environment and run over the dev environment, including recreating the master branch in GIT, so branches created from now on will include all the changes. Note that this procedure will run over your dev environment, and if you have changes there that were not moved to production yet, whey will be erased. You can back them up by saving objects JSON (one by one, unfortunately).

Here are the steps to reset your dev environment:

1. Export ARM template from production

On your production environment

Go to manage -> ARM template and click on "export ARM template"

From the zip folder downloaded, extract the arm_template.json file.

2. Import ARM template to dev

On your dev environment, go to manage -> ARM template

And click on import ARM template.

Click on the link "Build your own template in the editor"

Click on load file, and choose the arm_template.json you extracted in step 1.

Fill in the fields, and make sure to change attributes to match your dev environment.

Especially pay attention to the factory name and region fields. They must be exactly the same as your dev environment, so the process will run over the existing data factory.

Click on "review and create"

This step updated the live data factory, but you still need to update the GIT version.

3. Disconnect git

Go to "Git configuration" and click disconnect. You'll need to write the name of the data factory to approve the change.

4. Reconnected git while checking changes into master

Configure git again, using exactly the same configuration (same project and repo) as before.

Make sure you check "import existing resources to repository" and choose the collaboration branch (usually master or main). This will copy all the objects from the live data factory (which we updated in the previous step) into the collaboration branch.

Now your environments are in sync. Create a new branch in dev and start developing!



Guy Glantser
Guy Glantser
Aug 08, 2022

Thanks, @Chen Hirsh!

You wrote that in order to backup changes in the dev environment, you need to save objects JSON one by one.

Why is that? Why can't you export ARM template from the dev environment as a backup?

Chen Hirsh
Chen Hirsh
Aug 08, 2022
Replying to

@Guy Glantser You can backup with an ARM template, but if you need to restore, you can't use this template, since it will run over the changes you brought from prod. The only solution is to bring in the changed objects one by one (with their JSON).



Get New posts delivered straight to your inbox

Thank you for subscribing!

bottom of page