Published on Sep 19, 2022

Shell script recipes

This is a small collection of handy bash scripts for linux administrators. Tested on CentOS 8 though should work on most of linux distributions. Environment setup assumes nginx, php-fpm, mariadb (or mysql) and optionally Elasticsearch.

Database backup via CRON

1) Create a bash script somewhere under project directory, e.g. /_myproject/_scripts/db-backup.sh with content:

#!/bin/bash
# mysql/mariadb database backup - CRON executed script
TABLES=""        # list of backuped tables or leave empty for all tables
DUMPFILE_STRUCT=myproject-structure.sql
DUMPFILE_DATA=myproject-data-$(date +%Y%m%d-%H%M%S).sql.gz

echo " "
echo "[$(date +%Y-%m-%d\ %H-%M-%S)] Starting DB backup .."
echo "Exporting structure into $DUMPFILE_STRUCT"
mysqldump --defaults-extra-file=/root/myproject/db-credentials.cnf --no-data myproject > /root/myproject/db_backup/$DUMPFILE_STRUCT

echo "Exporting data into $DUMPFILE_DATA for tables .. $TABLES"
mysqldump --defaults-extra-file=/root/myproject/db-credentials.cnf  --lock-all-tables=1 myproject $TABLES | gzip > /root/myproject/db_backup/$DUMPFILE_DATA

# on Saturdays optimize tables, lock whole DB
if [[ $(date +%u) = 6 ]] ; then
  TABLES="select concat(TABLE_SCHEMA,'.', TABLE_NAME) from information_schema.tables where data_free>0;"
  for tbl in $(mysql --defaults-extra-file=/root/myproject/db-credentials.cnf -N <<< $TABLES)
  do
	echo "[$(date +%Y-%m-%d-%H-%M-%S)] OPTIMIZING TABLE: $tbl"
	mysql --defaults-extra-file=/root/myproject/db-credentials.cnf -N <<< "optimize table $tbl"
  done
fi

echo "Database backup completed at [$(date +%Y-%m-%d\ %H-%M-%S)]"

# delete DB backups older than 7 days
find /root/myproject/db_backup/ -type f -mtime +7 -name '*.sql.gz' -execdir rm -- '{}' +


2) Create file with credentials in /root/myproject/db-credentials.cnf. Make sure that dbadmin has privileges to access dumped tables. You may want to create a new user with extended or full privileges or the root user.

; mysql/mariadb CRON-backup credentials
[client]
user=dbadmin
password=My.Password@123


3) Set up CRON job in e.g. /var/spool/cron/root:

# daily database 3:05 backups for last 7 days
5 3 * * * /_myproject/_scripts/db-backup.sh >> /_myproject/writable/logs/cron-root.log


Database backup files will be stored in /root/myproject/db_backup/*.*.

Restore database from backup-ed files

This is done manually and means destroying existing database. Therefore first define explicitly which backuped file will be imported, then execute script via command line:

#!/bin/bash

# edit here - set filename that should be used for restoring database
RESTORE_FILE=myproject-20221231-235959.sql.gz

echo "Started restoring database at [$(date +%Y-%m-%d\ %H-%M-%S)] .."

mysql --defaults-extra-file=/root/myproject/db-credentials.cnf myproject < /root/myproject/db_backup/structure.sql
gunzip -c /root/myproject/db_backup/$RESTORE_FILE | mysql --defaults-extra-file=/root/myproject/db-credentials.cnf myproject

echo "Restoring database completed at [$(date +%Y-%m-%d\ %H-%M-%S)]!"


Periodically check that service is running via CRON

1) Create a bash script somewhere under project directory, e.g. /_myproject/_scripts/service-running.sh with content:

#!/bin/bash

# --- start nginx if not running ---
/usr/bin/pgrep nginx > /dev/null
if [ $? -ne 0 ]; then
   echo "[$(date +%Y-%m-%d\ %H-%M-%S)] NGINX NOT RUNNING - started!"
   sudo systemctl start nginx
fi

# --- start php-fpm if not running ---
/usr/bin/pgrep php-fpm > /dev/null
if [ $? -ne 0 ]; then
   echo "[$(date +%Y-%m-%d\ %H-%M-%S)] PHP-FPM NOT RUNNING - started!"
   sudo systemctl start php-fpm
fi

# --- start mysql/mariadb if not running ---
/usr/bin/pgrep mysql > /dev/null
if [ $? -ne 0 ]; then
   echo "[$(date +%Y-%m-%d\ %H-%M-%S)] MARIADB NOT RUNNING - started!"
   sudo systemctl start mysql
fi

# --- start elasticsearch if not running ---
/usr/bin/pgrep elasticsearch > /dev/null
if [ $? -ne 0 ]; then
   echo "[$(date +%Y-%m-%d\ %H-%M-%S)] ELASTICSEARCH NOT RUNNING - started!"
   sudo systemctl start elasticsearch
fi


2) Set up CRON job in e.g. /var/spool/cron/root that will check services every 10 minutes:

# daily database 3:05 backups for the last 7 days
*/10 * * * * /_myproject/_scripts/service-running.sh >> /_myproject/writable/logs/cron-root.log


Periodically restart services via CRON

1) Create a bash script somewhere under project directory, e.g. /_myproject/_scripts/service-restart.sh with content:

#!/bin/bash
echo "[$(date +%Y-%m-%d\ %H-%M-%S)] Restarting sevices ..."
sudo systemctl restart php-fpm
sudo systemctl restart nginx
sudo systemctl restart mysql
sudo systemctl restart elasticsearch
echo "[$(date +%Y-%m-%d\ %H-%M-%S)] -- All services restarted --"


2) Set up CRON job in e.g. /var/spool/cron/root:

# restart services every day at 3:30
30 3 * * * /_myproject/_scripts/service-restart.sh >> /_myproject/writable/logs/cron-root.log


Renew Let's Encrypt certificate if due via CRON

1) Install certbot (nginx or apache, CentOS 8):

sudo dnf -y install epel-release certbot python3-certbot-nginx
sudo certbot --nginx


2) Set up CRON job in e.g. /var/spool/cron/root:

# Check SSL expiration each Saturday 4:15 and renew if expires in less than 30 days
15 4 * * 6 PATH=$PATH:/usr/sbin /usr/bin/certbot renew --quiet >> /myproject/writable/logs/cron-root.log


Set permissions for subdirectories unanimously

Following script will set same permissions for all subdirectories under /_data:

#!/bin/bash

# set the top directory:
ROOT="/_data"

setPermissions() {
	PATH="$1"  # take first argument
	if [ -d "$PATH" ]; then
		read -p "Set permissions for files and directories in $PATH (y/n)?" CONT
		if [ "$CONT" = "y" ]; then
			echo "[$(date +%Y-%m-%d\ %H:%M:%S)] Setting [nginx:nginx] for [$PATH]"
			chown -R nginx:nginx $PATH
			find $PATH -type d -print0 | xargs -0 chmod 755
			find $PATH -type f -print0 | xargs -0 chmod 644
			echo "[$(date +%Y-%m-%d\ %H:%M:%S)] Finished setting permissions in [$PATH]"
			echo " "
		fi
	fi
}

# note: remove -prune argument, if you want to loop through each subdirectory
for f in $(find $ROOT/* -type d -prune); do setPermissions $f; done

echo "DONE!"


Set permissions for subdirectories individually

This is a typical script that needs to be executed after each web site update. It ensures proper permissions for updated files and new directories. It also removes outdated cache files. Adjust to your project's needs.

  1. Create script under project directory, e.g. /_myproject/_scripts/permissions.sh with the content bellow.
  2. Run script manually via SSH command line.
#!/bin/bash
# --- set system-wide permissions (opcache, session) ---
TMP="/var/lib/php/session"
echo "Setting [nginx:nginx] for [$TMP] and with writing permissions"
chown -R nginx:nginx $TMP
chmod -R uog+w $TMP

TMP="/var/lib/php/wsdlcache"
echo "Setting [nginx:nginx] for [$TMP] and with writing permissions"
chown -R nginx:nginx $TMP
chmod -R uog+w $TMP

TMP="/var/lib/php/opcache"
echo "Setting [nginx:nginx] for [$TMP] and with writing permissions"
chown -R nginx:nginx $TMP
chmod -R uog+w $TMP

# --- set website permissions ---
ROOT="/myproject"

# top directory owner nginx, set default directory / file permissions
TMP="$ROOT"
echo "Setting [nginx:nginx] for [$TMP] as READ-only"
chown -R nginx:nginx $TMP
find $TMP -type d -print0 | xargs -0 chmod 755
find $TMP -type f -print0 | xargs -0 chmod 644

# set permissions for directories NOT available to webserver - git repo, SSH scripts ..
TMP="$ROOT/.git"
echo "Setting [root:root] for [$TMP] with writing permissions"
chown -R root:root $TMP
chmod -R u+w $TMP

TMP="$ROOT/_scripts"
echo "Setting [root:root] for [$TMP] and NO writing permissions"
chown -R root:root $TMP
chmod -R uog-w-x $TMP
chmod u+x "$TMP/permissions.sh"
chmod u+x "$TMP/db-backup.sh"
chmod u+x "$TMP/service-restart.sh"
chmod u+x "$TMP/service-running.sh"

# executable CLI for CRON entry script, here we use yii script for Yii PHP framework
chmod u+x "$ROOT/app/yii"

# writable dirs
TMP="$ROOT/writable"
echo "Setting [nginx:nginx] for [$TMP] with writing permissions"
chmod -R u+w $TMP

# remove, create and set permission for assets directory
TMP="$ROOT/web/assets"
if [ ! -d "$TMP" ]; then
	mkdir $TMP
	echo "Created ASSETS directory with writing permissions [$TMP]"
fi
echo "Deleting ASSETS subdirectories [$TMP]"
rm -Rf $TMP/*
chown -R nginx:nginx $TMP
chmod -R u+w $TMP

# remove all cache files
TMP="$ROOT/writable/cache/*"
echo "Deleted files under cache directory [$TMP]"
rm -f $TMP

# make rss.xml and sitemap.xml writable to refresh these special files any time
TMP="$ROOT/web/rss.xml"
if [ -f "$TMP" ]; then
	echo "Setting writing permissions for [$TMP]"
	chmod u+w $TMP
fi

TMP="$ROOT/web/sitemap.xml"
if [ -f "$TMP" ]; then
	echo "Setting writing permissions for [$TMP]"
	chmod u+w $TMP
fi

# restart services if needed
read -p "Restart PHP-FPM (y/N)?" CONT
if [ "$CONT" = "y" ]; then
	sudo systemctl restart php-fpm
	echo "PHP-FPM restarted !"
fi

read -p "Restart NGINX (y/N)?" CONT
if [ "$CONT" = "y" ]; then
	sudo systemctl restart nginx
	echo "NGINX restarted !"
fi

read -p "Restart MariaDB (y/N)?" CONT
if [ "$CONT" = "y" ]; then
	sudo systemctl restart mysql
	echo "MariaDB restarted !"
fi

read -p "Restart elasticsearch (y/N)?" CONT
if [ "$CONT" = "y" ]; then
	sudo systemctl restart elasticsearch
	echo "Elasticsearch restarted !"
fi

echo "DONE!"


Related links

Got a question?

Synet.sk

Professional development of web applications and custom solutions. Consultancy services.

Demo

Contact


https://synet.sk