Occasionally, tasks fail or may fail to execute in the batch-ingest Airflow application. Tasks can fail for a variety of reasons that are unrelated to the nature of the data being processed. In such cases –such as when a networking error causes a task to fail– you can often troubleshoot the issue by "clearing" a task, causing it to restart.

How to clear a task

The first step to clearing tasks in Airflow is to identify them. The second step is then to clear the task (which may, in turn, automatically clear its downstream tasks).

Clear tasks with the DAG "Details" tab

One method to clear failed or zombie tasks is to use the "details" tab in the DAG overview page.

From the Airflow Admin UI, click on the "ingest" DAG link to go to the DAG overview page.

On the DAG overview page, click on the Details tab.

Now, on the Details page, click on the "queued" tab underneath "DAG Details."

Airflow's Admin UI will now present you with the list of all queued tasks. Click on the "select all" checkbox at the top of the list to select all of the queued tasks.

Now, in the "Actions" menu, select "Clear" and apply it to all of the queued tasks.

Confirm your choice to Clear the queued tasks.

Airflow should immediately prepare to run the queued tasks. You can check on the status of Airflow's progress from the Airflow Admin UI main page.

Tasks are running again.

Clear tasks with the DAG "Tree view"

Another method to clear tasks is to use the tree view DAG overview page.

From the Airflow Admin UI, click on the "ingest" DAG link to go to the DAG overview page.

On the DAG overview page, click on the "Tree View" tab.

Identify your failed task.

Click on the task's colored square to bring the controls display. Select the "Downstream" and "Recursive" options for the before pressing the "Clear" button.

Make sure to select the "Downstream" and "Recursive" options before clearing the task.

You will be presented with a list of all of the downstream tasks that will be cleared, too.  Click "OK" to confirm.