Creating Flow Charts is an essential element of Infrastructure Management and System Architecture. Creating Diagrams from Code is a remarkable breakthrough In recent times.
Python Diagrams is indeed a leader in this space and you can create your cloud diagrams in few minutes.
All you need is some knowledge of python and you can create beautiful Diagrams using the code.
In this article, I am going to create diagrams for all the ELBs in my aws account using Python Boto3 and Diagram as code. Completely Dynamic as the data is fetched Live using Boto
This would be a great help for SREs and DevOps/Cloud Engineers to keep a record/flowcharts of their Infrastructure setup and Maintain it Automated
With no further ado, Let us move on to the objective.
Here is a quick glimpse of what we are going to do. Looks interesting is not it ?
Prerequisites
- Python3 Must be installed
- AWS CLI must be installed and configured
- python pip must be installed to install other packages.
- Necessary IAM access to connect and pull information on ELBs from AWS account.
Setting up workspace
You can start by downloading the source code from my GITHUB repository and installing the necessary packages and running it.
We presume that your AWS CLI is already installed and configured. so we are not covering it in detail.
If you are new to AWS CLI refer to our previous article before proceeding
Download the Source code
You can download the source code from the Github repository. or just copy and paste the scripts given below.
There are two scripts. one for Application and Network load balancer another for classic Load Balancer.
- alb-diagram.py - For Application and Network Load Balancer
- clb-diagram.py - For Classic Load Balancer
Here is the script for creating Diagrams / flow charts for the Application and network Load Balancers.
alb-diagram.py
import boto3
import pprint
import sys
import json
def gettargetgroups(arn):
tgs=elb.describe_target_groups(LoadBalancerArn=arn)
tgstring=[]
for tg in tgs["TargetGroups"]:
tgstring.append(tg["TargetGroupName"])
return tgstring
def gettargetgrouparns(arn):
tgs=elb.describe_target_groups(LoadBalancerArn=arn)
tgarns=[]
for tg in tgs["TargetGroups"]:
tgarns.append(tg["TargetGroupArn"])
return tgarns
def getinstancename(instanceid):
instances=ec2.describe_instances(Filters=[
{
'Name': 'instance-id',
'Values': [
instanceid
]
},
],)
for instance in instances["Reservations"]:
for inst in instance["Instances"]:
for tag in inst["Tags"]:
if tag['Key'] == 'Name':
return (tag['Value'])
def gettargethealth(arn):
# print("arn",arn)
inss=elb.describe_target_health(TargetGroupArn=arn)
instanceids=[]
result=[]
for ins in inss["TargetHealthDescriptions"]:
ins["Name"]=getinstancename(ins['Target']['Id'])
instanceids.append(ins['Target']['Id'])
result.append(ins)
return result
def describelbs():
lbs = elb.describe_load_balancers(PageSize=400)
for lb in lbs["LoadBalancers"]:
lbjson={}
lbjson['Name']=lb["LoadBalancerName"]
lbjson['Type']=lb["Type"]
lbjson['TG']=gettargetgrouparns(lb["LoadBalancerArn"])
lbjson['TGData']=[]
TGLIST=[]
if len(lbjson["TG"]) > 0:
for tgs in lbjson['TG']:
TGD={}
TGD['Name']=tgs.split("/")[1]
tgh=gettargethealth(tgs)
if len(tgh) > 0:
TGD['Instances']=tgh
else:
TGD['Instances']=""
TGLIST.append(TGD)
lbjson['TGData'] = TGLIST
print("\n",json.dumps(lbjson, indent=4, sort_keys=True))
if __name__ == "__main__":
if len(sys.argv) < 3:
print(" – Region Name and the Profile name is mandatory – ")
print(" Syntax: python3 clb-list-json.py us-east-1 default")
exit()
region_name = sys.argv[1]
profile = sys.argv[2]
session = boto3.session.Session(profile_name=profile)
elb = session.client('elbv2')
ec2 = session.client('ec2')
describelbs()
Here is the script for creating Diagrams and flow charts for the Classic Load Balancers
clb-diagram.py
import boto3
import pprint
import json
import sys
from diagrams import Cluster, Diagram
from diagrams.aws.compute import EC2
from diagrams.aws.network import ELB
def getinstancename(instanceid):
instances=ec2.describe_instances(Filters=[
{
'Name': 'instance-id',
'Values': [
instanceid
]
},
],)
resultset = {}
for instance in instances["Reservations"]:
for inst in instance["Instances"]:
resultset["State"]=inst["State"]["Name"]
for tag in inst["Tags"]:
if tag['Key'] == 'Name':
resultset["Name"]=tag['Value']
# print (resultset)
return resultset
def getinstancehealth(lbname,instanceid):
instancestate=elb.describe_instance_health(
LoadBalancerName=lbname,
Instances = [{
'InstanceId' : instanceid
}]
)
return instancestate['InstanceStates'][0]['State']
def describelbs():
lbs = elb.describe_load_balancers(PageSize=400)
for lb in lbs["LoadBalancerDescriptions"]:
lbjson={}
lbjson['Name']=lb["LoadBalancerName"]
lbjson['HealthCheck']=lb["HealthCheck"]
lbjson['Instances']=[]
if len(lb["Instances"]) > 0:
InstanceList=[]
for instance in lb["Instances"]:
instance.update(getinstancename(instance["InstanceId"]))
instance['Health']=getinstancehealth(lb["LoadBalancerName"], instance["InstanceId"])
InstanceList.append(instance)
lbjson['Instances']=InstanceList
print("\n",json.dumps(lbjson, indent=4, sort_keys=True))
with Diagram(profile+"_CLB_"+lbjson['Name'],show=False):
lb = ELB(lbjson['Name'])
instance_group=[]
for instance in lbjson['Instances']:
instance_group.append(EC2(instance['Name']))
lb >> instance_group
if __name__ == "__main__":
if len(sys.argv) < 3:
print(" – Region Name and the Profile name is mandatory – ")
print(" Syntax: python3 ",sys.argv[0]," us-east-1 default")
exit()
region_name = sys.argv[1]
profile = sys.argv[2]
print("profilename selected:",profile)
print("regionname selected: ",region_name)
session = boto3.session.Session(profile_name=profile)
elb = session.client('elb')
ec2 = session.client('ec2')
describelbs()
Install the necessary packages
You need to install the boto python package as the script demands it
Hope you have already installed pip. now execute the following command to install boto3 with pip
pip install -r requirements.txt
Once the installation is complete we can start running our script.
Run the Scripts and Create Flowcharts/Diagrams
Run the program with the following syntax.
You need to use the right program for the right use case as there are two.
- alb-diagram.py - for Application Load Balancers and Network Load Balancers
- clb-diagram.py - for Classic Load Balancers
Here is the syntax for running alb-diagram.py for creating diagrams for Application and Network Load Balancers
python3 alb-diagram.py <region_name> <profile_name>
Here is the syntax for running clb-diagram.py for creating diagrams for Classic Load balancers
python3 clb-diagram.py <region_name> <profile_name>
If you do not have named profiles, you can simply type default as the profile name
Github link for Source code
Source code is available at the following link. Feel free to clone or create a PR for contributions
https://github.com/AKSarav/AWS_ELB_Diagram
List ELBs with Targetgroup and Instance ( Additional)
This is how it started. Earlier versions of this script which prints the JSON and TEXT output of All the ELBs in your AWS account is discussed in our other article.
Check it out.
AWS - List All ELB, NLB and CLB with Target and Target Groups | Python Boto
Conclusion
Hope this helps. There are more things I wanted to try and continue to work on the same. Any contribution or Pull requests are welcome to make this more better.
Since this is open source. feel free to update it to suit your needs and post how it helps in comments for the world to know.
Cheers
Sarav AK
Follow me on Linkedin My Profile Follow DevopsJunction onFacebook orTwitter For more practical videos and tutorials. Subscribe to our channel
Signup for Exclusive "Subscriber-only" Content
