Ansible Async Poll Examples - Ansible nohup, task in background

Not all the tasks we run in our playbook are short and sweet. Some of them would take a long time but we cannot afford to sit tight and watch the screen. We got better things to do.

Some of the long-running tasks could be

Downloading a Big File from URL
Running a Script known to run for a long time
Rebooting the remote server and waiting for it to comeback

These are only few I got in my head as I write this but there could be more scenarios.

In those cases where you want to run a long-running task with Ansible. You can consider running them with async keyword in your playbook.

Especially when you are running a time taking playbook with parallism and a lot of forks you can simply start the long-running task and poll its status later, instead of blocking the entire playbook and other succeeding tasks.

So how to use this Ansible Async and Poll in our playbooks and our ad hoc commands.

That's the objective of this post.

Table of Contents

Some Long Running Script

For better understanding, I have created some funny Script to run long time. It would simply sleep for the instructed time ( obviously sleeping is also a Job )

here is the script

#!/bin/bash
echo "Long Running Job is beginning"
echo "My Job is to Sleep"
echo "Now am going to Sleep for $1 Seconds"
sleep $1
echo "Huuuhhh!!.. What time is It?"
echo "Can I get some Bed Coffee please"
exit

so it would run {sleep} as long as we set it to.

This is the script we are going to use in our examples

What does this async do in Ansible

The ansible async keyword triggers Ansible to run the task in the background which can be checked (or) followed up later, and its value will be the maximum time that Ansible will wait for that particular Job (or) task to complete before it eventually times out or complete.

So async keyword can tell Ansible how long the task should be allowed to run before Ansible gives it up and time out.

But How would ansible track the status of this job invoked in the background? How frequent it would check if the task is completed or not.

That's where the poll keyword is coming into the picture

Ansible Poll Keyword with Ansible async

The Poll keyword is auto-enabled whenever you use async and it has a default value as 10 seconds

Which means If you are not defining the poll keyword with customized time period. By default Ansible would track the status of the async task every 10 seconds.

If you think 10 seconds polling is too frequent and you want Ansible to test the status of the job every 60 seconds It is possible.

The poll keyword accepts a numeric value to know how many seconds it should wait before polling or following up with the long-running asynchronous task.

In other words, The value of the poll indicates how often to poll and check if the tasks are completed. The default poll value is 10 seconds.

A quick example of Ansible Async

Good learning is doing it practically so let us get our hands dirty and execute the following playbook to see how async and poll works

The following playbook has three tasks

Copy the script to the remote server
Run the script
Display the Output of Script with Debug

---
- name: A Playbook to test Async and Poll
  hosts: appservers
  remote_user: centos
  tasks:
    - name: Copy the script 
      copy: 
        src: "longrunningscript.sh" # the file is present on the same directory
        dest: "/tmp"
    
    - name: Execute the script
      shell:
        "chmod a+x /tmp/longrunningscript.sh &&  /tmp/longrunningscript.sh 60" # Run for 60 seconds 
      async: 120 # Maximum allowed time in Seconds
      poll: 05 # Polling Interval in Seconds
      register: scrout

    - name: Some task just to debug
      debug: var=scrout.stdout_lines

Though our script is going to run only for 60 seconds. we have given a room for it to run longer. In realtime scenarios, you would not know how long the script would run, so use your best judgement when you are setting the value of async

Let us run this playbook quickly and observe the results.

Did you notice that the second task of executing the script was waiting as long as the script is completing and blocking the succedding tasks.

we will discuss how to solve this in upcoming sections.

But there is a one more question You might ask what if I give less async time than the required time for the script. Cause we cannot judge how long the script may run in real time scnarios.

The answer is the playbook would fail if the given async time is not suffcient and print the following timeout message

Now you might ask me, Is this really asynchronous? I had to wait anyway for the task to complete.

So how to run the job actually in the background with Concurrency

If you do not want Ansible to track the status of the long running asynchronous task or job. You can tell ansible to just fire it and forget it. by setting the poll to 0 seconds

As a user, If you want to track the status of that job later. you can use the JOB ID given by ansible during the invocation to track it.

This is useful in many cases and especially when you want to start some services like weblogic, tomcat, haproxy etc using a script at the remote server and let it run

To me, if you ask. This is the actual Asynchronous approach.

If we want to track the status of this job, we can go ahead and use async_status keyword along with the job ID being returned when we run the job.

Note*

Not all Jobs and modules can be run in background with poll 0

You shouldn’t attempt to run a task asynchronously by specifying a poll value of 0 with operations that require exclusive locks (such as yum transactions) or if you expect to run other commands later in the playbook against those same resources.

An example Playbook with Poll 0 - A real concurrency

Now let us use the same playbook we used earlier with the modified Poll. You just have to update the Poll to 0 seconds and re-execute it.

---
- name: Test Playbook to test Async and Poll
  hosts: appservers
  remote_user: centos
  tasks:
    - name: Copy the script 
      copy: 
        src: "longrunningscript.sh" # the file is present on the same directory
        dest: "/tmp"
    
    - name: Execute the script
      shell:
        "chmod a+x /tmp/longrunningscript.sh &&  /tmp/longrunningscript.sh 120" # Run for 60 seconds 
      async: 180 # Maximum allowed time in Seconds
      poll: 0 # Polling Interval in Seconds
      register: scrout

    - name: Some Other Non Dependent task
      shell :
        " ls -lrt /tmp"

now execute this playbook with at least one -v so that you would be able to see the JOB ID which you can use to test the status with async_status

in the preceding screen recording, you can see how the playbook is invoked with poll 0 and tracking the job status using async_status module in AD-HOC

aync_status can be used within the playbook as well..

Ansible Async_status within the playbook

You can also use the async_status module with in the playbook as shown below.

---
- name: Test Playbook to test Async and Poll
  hosts: appservers
  remote_user: centos
  tasks:
    - name: Copy the script 
      copy: 
        src: "longrunningscript.sh" # the file is present on the same directory
        dest: "/tmp"
    
    - name: Execute the script
      shell:
        "chmod a+x /tmp/longrunningscript.sh &&  /tmp/longrunningscript.sh 120" # Run for 60 seconds 
      async: 180 # Maximum allowed time in Seconds
      poll: 0 # Polling Interval in Seconds
      register: scrout

    - name: Some Other Non Dependent task
      shell :
        " ls -lrt /tmp"
      
    - name: Checking the Job Status
      async_status:
        jid: "{{ scrout.ansible_job_id }}"
      register: job_result
      until: job_result.finished
      retries: 30

Here is the screen record of running this playbook.

Not Just async_status you can use wait_for module with different strategies to check whether the job is completed or not.

Another way to determine job completion with ansible wait_for

Consider the following playbook written to perform Server Patching and reboot.

---
  - name: Patch the server and Reboot
    hosts: app
    tasks:
      - name: Patch the server
        become: yes
        become_user: root
        tags: Patch
        shell: "yum -y update"
        register: patchresult

      - name: Reboot the server
        tags: reboot
        become: yes
        become_user: root
        shell: "sleep 5 && reboot"
        async: 1
        poll: 0
    
      - name: Wait for the reboot and reconnect 
        wait_for:
          port: 22
          host: '{{ (ansible_ssh_host|default(ansible_host))|default(inventory_hostname) }}'
          search_regex: OpenSSH
          delay: 10
          timeout: 60
        connection: local

      - name: Check the Uptime of the servers
        shell: "uptime"
        register: Uptime

      - debug: var=Uptime

At task no 2 we are rebooting the server with the same poll value 0 ( fire and forget) but instead of tracking the job status at next step we use wait_for module to make sure that the server comes back by constantly checking the ssh port of the remote server.

So you can use wait_for module as another way to track the status of the job.

to learn more about the ansible wait_for module

Ansible wait_for module examples – How to | Devops Junction

to learn more about how to perform a reboot of the server with Async and pool

Ansible Reboot system and wait_for reboot to complete

Hope this article helps.

Cheers
Sarav AK

Follow me on Linkedin My Profile
Follow DevopsJunction onFacebook orTwitter
For more practical videos and tutorials. Subscribe to our channel

Signup for Exclusive "Subscriber-only" Content