Resetting your data factory dev environment from production

Working with Azure data factory, best practice suggests that you have a dev environment and a production environment. Your dev environment is connected to source control, be it git or Azure DevOps, and the proper way to work is to develop in dev, merge the changes into your git repository, and then deploy the changes to prod. And your environments are always in sync.

But… sometimes they don't.

It's not you, of course. It's some other developer who was under pressure and had little time, and he made some changes directly to the production environment.

Now you can't move new development to production, because all the data factory is moved as one, and you'll run over the changes you made to prod. You can move objects one by one by copying their JSON, but that is bad practice, and if you're working with other developers this could lead to chaos.

You need to copy the production environment and run over the dev environment, including recreating the master branch in GIT, so branches created from now on will include all the changes. Note that this procedure will run over your dev environment, and if you have changes there that were not moved to production yet, whey will be erased. You can back them up by saving objects JSON (one by one, unfortunately).

Here are the steps to reset your dev environment:

1. Export ARM template from production

On your production environment

Go to manage -> ARM template and click on "export ARM template"

From the zip folder downloaded, extract the arm_template.json file.

2. Import ARM template to dev

On your dev environment, go to manage -> ARM template

And click on import ARM template.

Click on the link "Build your own template in the editor"

Click on load file, and choose the arm_template.json you extracted in step 1.

Fill in the fields, and make sure to change attributes to match your dev environment.

Especially pay attention to the factory name and region fields. They must be exactly the same as your dev environment, so the process will run over the existing data factory.

Click on "review and create"

This step updated the live data factory, but you still need to update the GIT version.

3. Disconnect git

Go to "Git configuration" and click disconnect. You'll need to write the name of the data factory to approve the change.

4. Reconnected git while checking changes into master

Configure git again, using exactly the same configuration (same project and repo) as before.

Make sure you check "import existing resources to repository" and choose the collaboration branch (usually master or main). This will copy all the objects from the live data factory (which we updated in the previous step) into the collaboration branch.

Now your environments are in sync. Create a new branch in dev and start developing!