Today I stumbled upon a weird behavior in Azure Data Factory (ADF) error handling.
ADF lets us add error handling in the flow control, In this example, I'm trying to copy some data, and if that fails go to on failure branch (red line). If the activity succeeded, go to on success branch (green line)
These work great (If you can call a failure great...).
Let's take another step. What if I want to run another activity at the end, no matter if the copy succeeded or failed?
This is called Finally in programming languages. After try and catch, the code under finally will run anyway.
Look at the example below. I added another activity at the end and connected it to both previous activities.
But nothing happens. The last activity does not run.
Why doesn't the last activity run? Well, it's waiting for the completion of the 2 activities it's connected to, and since the on-success one didn't run, the last activity will not start either.
What can we do?
Option 1 - duplicate the activity
Create 2 versions of the same activity, and connect one to the on-success path and one to the on-failure path.
Option 2 - use "on skip"
Connect the last activity to both previous activities with the "on skip" path. Then, even when one path completes, the other is skipped and the activity is triggered.
تعليقات