Shell script recipes

This is a small collection of useful bash scripts for linux administrators. Tested on CentOS 8 though should work on most of linux distributions. Environment setup assumes nginx, php-fpm, mariadb (or mysql) and optionally Elasticsearch.

Database backup via CRON

1) Create a bash script somewhere under project directory, e.g. /_myproject/_scripts/db-backup.sh with content:

#!/bin/bash
# mysql/mariadb database backup - CRON executed script
TABLES=""        # list of backuped tables or leave empty for all tables
DUMPFILE_STRUCT=myproject-structure.sql
DUMPFILE_DATA=myproject-data-$(date +%Y%m%d-%H%M%S).sql.gz

echo " "
echo "[$(date +%Y-%m-%d\ %H-%M-%S)] Starting DB backup .."
echo "Exporting structure into $DUMPFILE_STRUCT"
mysqldump --defaults-extra-file=/root/myproject/db-credentials.cnf --no-data myproject > /root/myproject/db_backup/$DUMPFILE_STRUCT

echo "Exporting data into $DUMPFILE_DATA for tables .. $TABLES"
mysqldump --defaults-extra-file=/root/myproject/db-credentials.cnf  --lock-all-tables=1 myproject $TABLES | gzip > /root/myproject/db_backup/$DUMPFILE_DATA

# on Saturdays optimize tables, lock whole DB
if [[ $(date +%u) = 6 ]] ; then
  TABLES="select concat(TABLE_SCHEMA,'.', TABLE_NAME) from information_schema.tables where data_free>0;"
  for tbl in $(mysql --defaults-extra-file=/root/myproject/db-credentials.cnf -N <<< $TABLES)
  do
	echo "[$(date +%Y-%m-%d-%H-%M-%S)] OPTIMIZING TABLE: $tbl"
	mysql --defaults-extra-file=/root/myproject/db-credentials.cnf -N <<< "optimize table $tbl"
  done
fi

echo "Database backup completed at [$(date +%Y-%m-%d\ %H-%M-%S)]"

# delete DB backups older than 7 days
find /root/myproject/db_backup/ -type f -mtime +7 -name '*.sql.gz' -execdir rm -- '{}' +

2) Create file with credentials in /root/myproject/db-credentials.cnf. Make sure that dbadmin has privileges to access dumped tables. You may want to create a new user with extended (or full) privileges or the root user.

; mysql/mariadb CRON-backup credentials
[client]
user=dbadmin
password=My.Password@123

3) Set up CRON job in e.g. /var/spool/cron/root:

# daily database 3:05 backups for last 7 days
5 3 * * * /_myproject/_scripts/db-backup.sh >> /_myproject/writable/logs/cron-root.log

Database backup files will be stored in /root/myproject/db_backup/*.*.

Restore database from backup-ed files

This is done manually and means destroying existing database. Therefore first define explicitly which backuped file will be imported, then execute script via command line:

#!/bin/bash

# edit here - set filename that should be used for restoring database
RESTORE_FILE=myproject-20221231-235959.sql.gz

echo "Started restoring database at [$(date +%Y-%m-%d\ %H-%M-%S)] .."

mysql --defaults-extra-file=/root/myproject/db-credentials.cnf myproject < /root/myproject/db_backup/structure.sql
gunzip -c /root/myproject/db_backup/$RESTORE_FILE | mysql --defaults-extra-file=/root/myproject/db-credentials.cnf myproject

echo "Restoring database completed at [$(date +%Y-%m-%d\ %H-%M-%S)]!"

Periodically check that service is running via CRON

1) Create a bash script somewhere under project directory, e.g. /_myproject/_scripts/service-running.sh with content:

#!/bin/bash

# --- start nginx if not running ---
/usr/bin/pgrep nginx > /dev/null
if [ $? -ne 0 ]; then
   echo "[$(date +%Y-%m-%d\ %H-%M-%S)] NGINX NOT RUNNING - started!"
   sudo systemctl start nginx
fi

# --- start php-fpm if not running ---
/usr/bin/pgrep php-fpm > /dev/null
if [ $? -ne 0 ]; then
   echo "[$(date +%Y-%m-%d\ %H-%M-%S)] PHP-FPM NOT RUNNING - started!"
   sudo systemctl start php-fpm
fi

# --- start mariadb/mysql if not running ---
# /usr/bin/pgrep mysql > /dev/null
/usr/bin/pgrep mariadb > /dev/null
if [ $? -ne 0 ]; then
   echo "[$(date +%Y-%m-%d\ %H-%M-%S)] MARIADB NOT RUNNING - started!"
   # sudo systemctl start mysql
   sudo systemctl start mariadb
fi

# --- start elasticsearch if not running ---
ps -ef | grep elasticsearch | grep -v grep > /dev/null
if [ $? -ne 0 ]; then
   echo "[$(date +%Y-%m-%d\ %H-%M-%S)] ELASTICSEARCH NOT RUNNING - started!"
   sudo systemctl start elasticsearch
fi

2) Set up CRON job in e.g. /var/spool/cron/root that will check services every 10 minutes:

# daily database 3:05 backups for the last 7 days
*/10 * * * * /_myproject/_scripts/service-running.sh >> /_myproject/writable/logs/cron-root.log

Periodically restart services via CRON

1) Create a bash script somewhere under project directory, e.g. /_myproject/_scripts/service-restart.sh with content:

#!/bin/bash
echo "[$(date +%Y-%m-%d\ %H-%M-%S)] Restarting sevices ..."
sudo systemctl restart php-fpm
sudo systemctl restart nginx
sudo systemctl restart mysql
sudo systemctl restart elasticsearch
echo "[$(date +%Y-%m-%d\ %H-%M-%S)] -- All services restarted --"

2) Set up CRON job in e.g. /var/spool/cron/root:

# restart services every day at 3:30
30 3 * * * /_myproject/_scripts/service-restart.sh >> /_myproject/writable/logs/cron-root.log

Renew Let's Encrypt certificate if due via CRON

1) Install certbot (nginx or apache, CentOS 8):

sudo dnf -y install epel-release certbot python3-certbot-nginx
sudo certbot --nginx

2) Set up CRON job in e.g. /var/spool/cron/root:

# Check SSL expiration each Saturday 4:15 and renew if expires in less than 30 days
15 4 * * 6 PATH=$PATH:/usr/sbin /usr/bin/certbot renew --quiet >> /myproject/writable/logs/cron-root.log

Set permissions for subdirectories unanimously

Following script will set same permissions for all subdirectories under /_data requesting confirmation for each subdir:

#!/bin/bash

# set the top directory:
ROOT="/_data"

setPermissions() {
	PATH="$1"  # take first argument
	if [ -d "$PATH" ]; then
		# ask admin to confirm whether permissions will be set for each subdirectory
		read -p "Set permissions for files and directories in $PATH (y/N)?" CONT
		if [ "$CONT" = "y" ]; then
			echo "[$(date +%Y-%m-%d\ %H:%M:%S)] Setting [nginx:nginx] for [$PATH]"
			chown -R nginx:nginx $PATH
			find $PATH -type d -print0 | xargs -0 chmod 755
			find $PATH -type f -print0 | xargs -0 chmod 644
			echo "[$(date +%Y-%m-%d\ %H:%M:%S)] Finished setting permissions in [$PATH]"
			echo " "
		fi
	fi
}

# note: remove the "-prune" argument, if you want to loop unlimited subdirectory depth
# with "-prune" only first child-level is traversed
for f in $(find $ROOT/* -type d -prune); do setPermissions $f; done

echo "DONE!"

Set permissions for subdirectories individually

This is a typical script that needs to be executed after each web site update. It ensures proper permissions for updated files and new directories, removes outdated cache files, creates missing dirs .. etc. Adjust to your project specifics.

1. Create script under project directory, e.g. /_myproject/_scripts/permissions.sh with the content bellow.
2. Run script manually via SSH command line as root user.

#!/bin/bash
# --- set system-wide permissions (opcache, session) ---
TMP="/var/lib/php/session"
echo "Setting [nginx:nginx] for [$TMP] and with writing permissions"
chown -R nginx:nginx $TMP
chmod -R uog+w $TMP

TMP="/var/lib/php/wsdlcache"
echo "Setting [nginx:nginx] for [$TMP] and with writing permissions"
chown -R nginx:nginx $TMP
chmod -R uog+w $TMP

TMP="/var/lib/php/opcache"
echo "Setting [nginx:nginx] for [$TMP] and with writing permissions"
chown -R nginx:nginx $TMP
chmod -R uog+w $TMP

# --- set website permissions ---
ROOT="/myproject"

# top directory owner nginx, set default directory / file permissions
TMP="$ROOT"
echo "Setting [nginx:nginx] for [$TMP] as READ-only"
chown -R nginx:nginx $TMP
find $TMP -type d -print0 | xargs -0 chmod 755
find $TMP -type f -print0 | xargs -0 chmod 644

# set permissions for directories NOT available to webserver - git repo, SSH scripts ..
TMP="$ROOT/.git"
echo "Setting [root:root] for [$TMP] with writing permissions"
chown -R root:root $TMP
chmod -R u+w $TMP

TMP="$ROOT/_scripts"
echo "Setting [root:root] for [$TMP] and NO writing permissions"
chown -R root:root $TMP
chmod -R uog-w-x $TMP
chmod u+x "$TMP/permissions.sh"
chmod u+x "$TMP/db-backup.sh"
chmod u+x "$TMP/service-restart.sh"
chmod u+x "$TMP/service-running.sh"

# executable CLI for CRON entry script, here we use yii script for Yii PHP framework
chmod u+x "$ROOT/app/yii"

# writable dirs
TMP="$ROOT/writable"
echo "Setting [nginx:nginx] for [$TMP] with writing permissions"
chmod -R u+w $TMP

# remove, create and set permission for assets directory
TMP="$ROOT/web/assets"
if [ ! -d "$TMP" ]; then
	mkdir $TMP
	echo "Created ASSETS directory with writing permissions [$TMP]"
fi
echo "Deleting ASSETS subdirectories [$TMP]"
rm -Rf $TMP/*
chown -R nginx:nginx $TMP
chmod -R u+w $TMP

# remove all cache files
TMP="$ROOT/writable/cache/*"
echo "Deleted files under cache directory [$TMP]"
rm -f $TMP

# make rss.xml and sitemap.xml writable to refresh these special files any time
TMP="$ROOT/web/rss.xml"
if [ -f "$TMP" ]; then
	echo "Setting writing permissions for [$TMP]"
	chmod u+w $TMP
fi

TMP="$ROOT/web/sitemap.xml"
if [ -f "$TMP" ]; then
	echo "Setting writing permissions for [$TMP]"
	chmod u+w $TMP
fi

# restart services if needed
read -p "Restart PHP-FPM (y/N)?" CONT
if [ "$CONT" = "y" ]; then
	sudo systemctl restart php-fpm
	echo "PHP-FPM restarted !"
fi

read -p "Restart NGINX (y/N)?" CONT
if [ "$CONT" = "y" ]; then
	sudo systemctl restart nginx
	echo "NGINX restarted !"
fi

read -p "Restart MariaDB/mySQL (y/N)?" CONT
if [ "$CONT" = "y" ]; then
	# sudo systemctl restart mysql
	sudo systemctl restart mariadb
	echo "MariaDB/mySQL restarted !"
fi

read -p "Restart elasticsearch (y/N)?" CONT
if [ "$CONT" = "y" ]; then
	sudo systemctl restart elasticsearch
	echo "Elasticsearch restarted !"
fi

echo "DONE!"

Related links

• Bash scripting cheatsheet
• Awesome Bash - Github
• 5 Modern Bash Scripting Techniques
• 10 Frequently Used Bash Scripts and Commands
• 13 resources for learning to write better Bash code