In this post we are going we are going to see how to find a list of files containing a text or a pattern and replace them at once.
As you know I have written a dedicated and brief articles on both Ansible find and replace modules with many examples.
But this article is going to be quick one unlike those brief ones with narrow focus.
As mentioned on the first line we are going to find the list of files in a directory and sub directories and look for a pattern in all of them
If the file contains the specific pattern we are looking for, we are going to replace them after taking a backup of that file.
To keep this article short and precise am not explaining the basics of find and replace modules and going straight to our objective.
If you are completely new to Ansible. Please refer to our Introduction to Ansible Playbook and Ad Hoc commands article
What we are going to do
To make this playbook relevent, am taking up some real time requirement.
Our requirement is to find the jobs which are using the outdated https://github.com/org/repo based urls and replacing them with SSH based URLs [email protected]:org/repo.git
If you are not aware, Jenkins creates a configuration file for each job we are creating on the UI and they would be present on the Jenkins Server under the JENKINS_HOME/jobs/<job name>/config.xml location
So we are going find these config.xml files and try to pattern match to find if the job is using the https based GIT urls and replace them to SSH url.
Let's go
The Ansible Playbook that find the files and replace a pattern
So we are clear on the requirement I believe which has two sub tasks
- Find the files using Ansible find module
- Look for a pattern and replace using Ansible replace module.
both these sub tasks would be represented as a task in ansible playbook.
Here is the playbook
---
- name: Replace line in file examples
hosts: jenkinsserver
tasks:
- name: Find the files
become: true
become_user: jenkins
find:
paths: /var/lib/jenkins/jobs/
file_type: file
recurse: yes
patterns:
- config.xml
use_regex: true
register: configfiles
- debug:
msg: "{{configfiles.files | map(attribute='path')| list }}"
- name: "Changing jenkins Git URL"
become: true
become_user: jenkins
replace:
path: "{{item.path}}"
regexp: '(<url>)(https:\/\/)(github\.com)(\/)(gritfy)(\/\w+)(|\/)(<\/url)'
replace: '\1git@\3:\5\6.git\8'
after: '\<hudson.plugins.git.UserRemoteConfig\>'
before: '\<\/hudson.plugins.git.UserRemoteConfig\>'
backup: yes
with_items:
- "{{configfiles.files}}"
The First task is to find the files on the path /var/lib/jenkins/jobs folder using Ansible find module.
we are using the file name as the patterns to find
The resulting file names would be saved into a register variable named configfiles
The Second task is just a debug print statement and displaying only the file names from the variable we have created on the first task.
We are using Map filter with attribute selection to do the same.
The Third Task performs the actual find and replace with in the file. It look the pattern that we have defined on the regexp
regexp: '(<url>)(https:\/\/)(github\.com)(\/)(gritfy)(\/\w+)(|\/)(<\/url)'
The regular expression pattern we have given here find various parts of the github URL and group them.
These group would be back referred based on their position.
For example :
\1 represents the <url>
\2 represents the https://github.com
The following screen shot can explain the groups better
You can see we have used these group names on the replace pattern.
replace: '\1git@\3:\5\6.git\8'
we are also using the before and after options of replace module which helps us to find the right pattern at right place.
This is briefly covered on the Ansible Replace introduction and examples article
Cheers
Sarav AK
