Overview
An Automation script Linux for Log rotation, LogPurging and Compressing For weblogic and tomcat Installations. In Big infrastructure when you are running multiple Tomcat, weblogic, JBoss (or) Websphere instances or domains. The Biggest problem that you would face often as a system administrator is disk space issue.
Perhaps, the products like weblogic and WebSphere and even tomcat come with log rotation for their own instance or JVM logs but you cannot always bet on it. Since there could be other application-level logs which would be written and some STDOUT logs does not fall under the log rotation etc.
When you want to log rotate, you go for logrotate.d or some custom script to do the same. when you want to purge the logs and compress them , you will write another script to achieve that as well.
Diskspaceman does all of them under a single window. Log Purging, Log Rotation, Log Compressing (or) Archiving. You can schedule this single script in a cron to all of these tasks every single day for better server disk space management.
You can go ahead and look at the code base and examples in the Github repository using the below link (or) continue reading to know more.
GitHub Repository: https://github.com/AKSarav/diskspaceman
Short Summary
DiskSpaceMan is a Shell script, designed in bash with various modules. It was initially designed for Log Rotation and further expanded to achieve the following tasks under a single window.
- Log Rotation
- Log Purging
- Removal of Old Logs based on the Retention Period
- Compressing the Uncompressed Rotated log files
How To Execute
Step1: Download the All in one ZIP file diskspaceman.zip from the repository
Step2: Uncompress/Unzip the downloaded zip file
Step3: Change the username (or) workspace in the script. ( only if the script does not match with your username and workspace assignments)
Step4: Execute the script as a valid user, Either tomcat or weblogic along with the desired retention period
./diskspaceman.sh --retentionperiod=30days
The WorkSpace ( Directories to Search)
Based on the user of execution. The workspace (or) directories to consider for searching the logs will change. In order to update it, you need to change the value of DIRTOSEARCH variable in the diskspaceman.sh
The Code snippet with current workspace assignment is given below.
[ This is just a part of the code, Full script is on the repository]
#Determine the username if [ `whoami` == "weblogic" ] then DIRTOSEARCH="/apps/weblogic/domains/*/*/*/logs /opt/weblogic/domains/*/*/*/logs /opt/weblogic/logs/* /apps/weblogic/logs/*" LOGSDIRS=`ls -d $DIRTOSEARCH 2>/dev/null` elif [ `whoami` == "tomcat" ] then DIRTOSEARCH="/opt/tomcat/instances/*/logs /apps/tomcat/instances/*/logs /opt/tomcat/logs/* /apps/tomcat/logs/*"
For weblogic
If the script has been started as a weblogic user, the script will consider the following directories as a workspace
/apps/weblogic/domains/*/*/*/logs /opt/weblogic/domains/*/*/*/logs /opt/weblogic/logs/* /apps/weblogic/logs/*
For tomcat
If the script has been started as a tomcat user, the script will consider the following directories as a workspace
/apps/tomcat/instances/*/logs /opt/tomcat/instances/*/logs /opt/tomcat/logs/* /apps/tomcat/logs/*
You can extend the script to many other users and other workspaces based on your requirement.
BuiltIn Features
👉 Efficient Logging
The script is designed with an efficient and Debug level logging functionality. All messages to STDOUT is properly formatted with the time stamp. Take a look at the sample given below
weblogic@testserver> ./diskspaceman.sh --retentionperiod=100days 22-08-18 14:57:27 **** DISKSPACEMAN - PROCESS STARTED **** 22-08-18 14:57:27 LIST OF DIRECTORIES FOUND: [ /opt/weblogic/domains/test_domain/servers/AdminServer/logs,/opt/weblogic/domains/test_domain/servers/wls_PegaServer1/logs,/opt/weblogic/logs/pega ] 22-08-18 14:57:27 22-08-18 14:57:27 22-08-18 14:57:27 =========================================================== 22-08-18 14:57:27 PROCESSING DIRECTORY: /opt/weblogic/domains/test_domain/servers/AdminServer/logs 22-08-18 14:57:27 22-08-18 14:57:27 LIST OF FILES FOUND FOR LOGROTATION: [ access.log,AdminServer.log,test_domain.log ] 22-08-18 14:57:27 -- LOGROTATION COMPLETED SUCCESSFULLY FOR /opt/weblogic/domains/test_domain/servers/AdminServer/logs/access.log 22-08-18 14:57:27 -- LOGROTATION COMPLETED SUCCESSFULLY FOR /opt/weblogic/domains/test_domain/servers/AdminServer/logs/AdminServer.log 22-08-18 14:57:27 -- LOGROTATION COMPLETED SUCCESSFULLY FOR /opt/weblogic/domains/test_domain/servers/AdminServer/logs/test_domain.log 22-08-18 14:57:27 22-08-18 14:57:27 PURGING PROCESS STARTED 22-08-18 14:57:27 REMOVING THE 100 DAYS OLD FILES 22-08-18 14:57:27 LIST OF FILES GOING TO BE REMOVED: [ ] 22-08-18 14:57:27 22-08-18 14:57:27 G-ZIPPING THE OTHER AVAILABLE LOGS ./test_domain.log06787: 97.1% -- replaced with ./test_domain.log06787.gz ./logrotate-out.conf: 0.0% -- replaced with ./logrotate-out.conf.gz ./test_domain.log06835: 96.9% -- replaced with ./test_domain.log06835.gz ./test_domain.log06808: 97.1% -- replaced with ./test_domain.log06808.gz ./test_domain.log06826: 97.0% -- replaced with ./test_domain.log06826.gz ./test_domain.log06878: 97.1% -- replaced with ./test_domain.log06878.gz 22-08-18 14:57:27 PURGING PROCESS COMPLETED 22-08-18 14:57:27 ===========================================================
Here you could notice that all userful information is getting printed including the file name and the directory the script is processing
It can help to understand the working prinicple of this script and it comes handy for troubleshooting in case of any issues in future.
👉 Runtime Validations
The script performs multiple levels of validation to make sure nothing goes wrong
Validation1: Username validation
The script will try to validate the username as it is being invoked, to determine the correct workspace The Log directories Script is designed to dynamically switch the log directories based on the user of execution. This is to avoid an accidental execution of script as an invalid user like root , which would end up in messing up the logs and eventually result in application downtime
If the script is started as any other user, the script would print an error message and exit. The sample execution is shown below
root@testserver# whoami root root@testserver# ./diskspaceman.sh --retentionperiod=100days Correct the Errors before proceeding ERROR: Invalid User to run the script Valid Users are 1) tomcat 2) weblogic root@testserver#
Validation2: Startup Argument Validation
The script gets the retentionperiod as a startup argument, Attempt of execution without the startup argument would yield the following result
#Tried with no startup argument weblogic@testserver> ./diskspaceman.sh Please execute the script correctly ./diskspaceman.sh --retentionperiod=400days #Tried with wrong or invalid startup argument weblogic@testserver> ./diskspaceman.sh --asdfasdf Please execute the script correctly ./diskspaceman.sh --retentionperiod=400days #Spelling mistake in Startup argument weblogic@testserver> ./diskspaceman.sh --retentionpariod=300 Please execute the script correctly ./diskspaceman.sh --retentionperiod=400days weblogic@testserver>
Risk Assessment
❄️ The script does not use rm or mv command to prevent accidental log overwriting or deletion
❄️ It uses the Native logrotate command instead of manually copying and performing the log rotation
❄️ It leaves no log file (or) footprint in the file system as the output is directly printed to STDOUT, If you would like to save the output to a file you can use runtime redirection using > logfilename 2>&1
❄️ CPU load and memory load of this script have been tested and proven to be Efficient and not having any issues.
Additional Notes
- Complete Test Execution Session Output is uploaded to the GITHUB repository for your reference
- This Script can be scheduled to run every day using crontab for efficient server management and housekeeping of your Application Server.
Hope it helps.
Let me know if you need any help in modifying the script to suit your needs. You can use the comment section below
Thanks for reading.
Sarav AK
Follow me on Linkedin My Profile Follow DevopsJunction onFacebook orTwitter For more practical videos and tutorials. Subscribe to our channel
Signup for Exclusive "Subscriber-only" Content
