Variables & Data Types

Shell Variables

Variables are containers used to store data inside a shell script. They allow scripts to reuse information without typing the same values repeatedly. In Bash, variables do not require data types like int or string. Everything is treated as text internally.

One of the most common beginner mistakes is adding spaces around the equals sign. In Bash, variable assignment must not contain spaces.

#!/bin/bash
name="Ahmed"
age=25
course="Shell Scripting"

echo "Student: $name"
echo "Age: $age"
echo "Course: $course"

Variables can store names, numbers, file paths, command output, and even entire sentences.

Variable Naming Rules

Variable names should be meaningful and easy to understand. Good naming improves readability and helps when debugging large scripts.

#!/bin/bash
student_name="Ali"
project_name="CodeTutorium"

echo "$student_name is learning on $project_name"

Special Variables

Bash provides special predefined variables that give information about the script, arguments, and shell session.

#!/bin/bash
echo "Script Name: $0"
echo "First Argument: $1"
echo "Total Arguments: $#"

User Input with read

Scripts become interactive when they accept input from users. The read command stores keyboard input inside a variable.

#!/bin/bash
echo "Enter your name:"
read username

echo "Welcome $username"

You can also display prompts directly inside the command using the -p option.

#!/bin/bash
read -p "Enter your city: " city

echo "City: $city"

Command Substitution

Command substitution allows the output of a command to be stored inside a variable. This is extremely useful for automation and system scripting.

#!/bin/bash
today=$(date +%Y-%m-%d)
current_user=$(whoami)

echo "Date: $today"
echo "Logged in as: $current_user"

The $(command) syntax is preferred because it is cleaner and easier to read than older backtick syntax.

Readonly & Export

Sometimes variables should not be changed after assignment. The readonly keyword protects variables from modification. The export command makes variables available to child processes.

#!/bin/bash
readonly VERSION="1.0"
export APP_ENV="production"

echo "Version: $VERSION"
echo "Environment: $APP_ENV"

Quoting Rules

Quotes affect how Bash interprets variables and text. Understanding quoting rules prevents unexpected errors, especially when working with spaces.

#!/bin/bash
file="my document.txt"

touch "$file"

echo "Created: $file"

Best Practice: Always wrap variables in double quotes unless you intentionally need word splitting.

Arithmetic Operations

Bash treats variables as strings, but arithmetic can still be performed using special syntax. Integer calculations use $(( )).

#!/bin/bash
x=15
y=4

echo "Addition: $((x + y))"
echo "Subtraction: $((x - y))"
echo "Multiplication: $((x * y))"
echo "Division: $((x / y))"

For floating-point calculations, external tools like bc are commonly used.

Arrays

Arrays allow multiple values to be stored inside a single variable. Bash supports indexed arrays and associative arrays.

Indexed Arrays

#!/bin/bash
languages=("Bash" "Python" "JavaScript")

echo "First Language: ${languages[0]}"
echo "All Languages: ${languages[@]}"

Associative Arrays

#!/bin/bash
declare -A user

user[name]="Ahmed"
user[role]="Developer"

echo "${user[name]} is a ${user[role]}"

Environment vs Local Variables

Variables can exist either locally or globally. Local variables are available only inside the current shell session, while exported environment variables can be inherited by child processes.

#!/bin/bash
local_var="Local"
export global_var="Global"

bash -c 'echo $global_var'
bash -c 'echo $local_var'

Use the env or printenv command to display environment variables currently available in the shell.

Removing Variables

The unset command removes variables from memory. Once removed, the variable no longer stores any value.

#!/bin/bash
language="Bash"

echo "$language"

unset language

echo "$language"

Removing unnecessary variables helps keep scripts organized and prevents accidental reuse of old data.

Operators & Expressions

Operators are special symbols used to perform operations on values and variables. In Bash scripting, operators are used for calculations, comparisons, logical decisions, pattern matching, and file testing.

Expressions are combinations of variables, operators, and values that produce a result. Understanding operators is extremely important because they are used in almost every script.

In this lecture, we will learn:

• Arithmetic operations
• Numeric comparisons
• String and logical operators
• File test operators
• Bitwise operations
• Pattern matching and regex
• Operator precedence
• Short-circuit evaluation

Arithmetic

Arithmetic operators are used to perform mathematical calculations. Bash supports addition, subtraction, multiplication, division, modulus, and exponentiation.

#!/bin/bash

a=10
b=3

# Addition
echo $((a + b))

# Multiplication
echo $((a * b))

# Modulus (remainder)
echo $((a % b))

# Exponentiation
echo $((a ** b))

# Increment
((a++))
echo $a

Understanding Arithmetic Operators

• + → Addition
• - → Subtraction
• * → Multiplication
• / → Division
• % → Remainder
• ** → Power operator

Integer Division

Bash arithmetic works mainly with integers. Decimal calculations require external tools like bc.

#!/bin/bash

echo $((10 / 3))

echo "10 / 3" | bc -l

Numeric Comparisons

Numeric comparison operators are used to compare integer values. These are commonly used inside conditional statements.

#!/bin/bash

x=10
y=20

[[ $x -eq $y ]]  # equal

[[ $x -ne $y ]]  # not equal

[[ $x -lt $y ]]  # less than

[[ $x -gt $y ]]  # greater than

[[ $x -le $y ]]  # less or equal

[[ $x -ge $y ]]  # greater or equal

Comparison Operator Table

String & Logical Operators

String operators compare text values, while logical operators combine conditions together.

#!/bin/bash

a="hello"
b="world"
age=20
lic=true

# String equality
[[ "$a" = "$b" ]]

# Empty string check
[[ -z "$a" ]]

# Non-empty string check
[[ -n "$a" ]]

# Logical AND
[[ $age -ge 18 && $lic == true ]]

# Logical OR
[[ $a == "admin" || $age -gt 18 ]]

Logical Operators

• && → AND
• || → OR
• ! → NOT

File Test Operators

File test operators are used to check properties of files and directories. These operators are extremely important in shell scripting.

Practical File Check Example

#!/bin/bash

file="notes.txt"

if [[ -f "$file" ]]; then
    echo "File exists"
else
    echo "File missing"
fi

Bitwise Operators

Bitwise operators work directly with binary values. These are commonly used in permissions, flags, networking, and system-level programming.

#!/bin/bash

flags=6
mask=2

echo $((flags & mask))

echo $((flags | mask))

echo $((flags ^ mask))

echo $((~flags))

echo $((flags << 1))

echo $((flags >> 1))

Bitwise Operators Table

Pattern Matching & Regex

Bash supports wildcard pattern matching and regular expressions for advanced text searching.

#!/bin/bash

filename="report_2024.pdf"

[[ $filename == *.pdf ]] && echo "PDF file"

[[ $filename == report_* ]] && echo "Report prefix"

[[ $filename =~ [0-9]{4} ]] && echo "Contains year"

Common Wildcards

• * → Matches multiple characters
• ? → Matches single character
• [] → Character ranges

Regex Capture Groups

Bash automatically stores regex capture groups inside the BASH_REMATCH array.

#!/bin/bash

email="user@example.com"

[[ $email =~ ^([a-z]+)@([a-z]+\.[a-z]+)$ ]]

echo "User: ${BASH_REMATCH[1]}"

echo "Domain: ${BASH_REMATCH[2]}"

Operator Precedence & Grouping

Complex conditions should be grouped properly using parentheses. This improves readability and prevents logical errors.

#!/bin/bash

age=25
income=50000
credit=700

[[ ($age -ge 21 && $income -gt 40000) || $credit -ge 750 ]] && echo "Approved"

Short-Circuit Evaluation

Bash uses short-circuit logic for efficient execution.

• && stops if the left side is false
• || stops if the left side is true

#!/bin/bash

[[ -d /var/log ]] && touch /var/log/app.log

[[ -n "$username" ]] || username="guest"

ping -c 1 google.com &>/dev/null && echo "Online" || echo "Offline"

Compound File Tests

Real-world scripts usually combine multiple file checks together.

#!/bin/bash

config="app.conf"

if [[ -f "$config" && -r "$config" && -s "$config" ]]; then

    echo "Config is valid"

else

    echo "Invalid config" >&2

    exit 1
fi

Best Practices

• Use [[ ]] instead of old-style [ ] when possible
• Quote variables to avoid unexpected errors
• Group complex conditions clearly
• Use comments for readability
• Validate files before processing them

Control Flow

Introduction to Control Flow

Control flow determines how a script makes decisions and repeats tasks. Without control flow, scripts would execute line-by-line with no ability to react to conditions or user input.

Bash provides conditional statements and loops that allow scripts to become dynamic and interactive.

If-Elif-Else

The if statement checks conditions and executes different blocks of code depending on whether the condition is true or false.

#!/bin/bash
read -p "Enter age: " age

if [[ $age -lt 13 ]]; then
    echo "Child"
elif [[ $age -lt 20 ]]; then
    echo "Teen"
else
    echo "Adult"
fi

The condition inside [[ ]] is evaluated by Bash. If the condition is true, the matching block executes.

String Comparisons

Conditions are not limited to numbers. Bash can also compare strings and text values.

#!/bin/bash
read -p "Enter role: " role

if [[ $role == "admin" ]]; then
    echo "Access granted"
else
    echo "Access denied"
fi

String comparisons are commonly used in login systems, menus, and configuration scripts.

Case Statement

The case statement is useful when checking multiple possible values. It is cleaner and easier to manage than writing many if statements.

#!/bin/bash
read -p "Pick (start|stop|restart): " cmd

case $cmd in
    start)   echo "Starting service...";;
    stop)    echo "Stopping service...";;
    restart) echo "Restarting service...";;
    *)       echo "Unknown command";;
esac

The * symbol acts as a default option if no pattern matches.

Menu-Driven Script

Menus make scripts easier for users to interact with. Instead of memorizing commands, users can choose numbered options.

#!/bin/bash
echo "1) Date"
echo "2) Uptime"
echo "3) Disk Usage"
echo "4) Exit"

read -p "Choice: " c

case $c in
    1) date;;
    2) uptime;;
    3) df -h;;
    4) exit;;
    *) echo "Invalid choice";;
esac

Menu-based scripts are commonly used in system administration and automation tools.

Exit Codes

Every command in Linux returns an exit code after execution. A value of 0 means success, while non-zero values indicate errors or failures.

#!/bin/bash
grep "error" app.log

if [[ $? -eq 0 ]]; then
    echo "Errors found"
else
    echo "No errors found"
fi

The special variable $? stores the exit code of the last executed command.

Logical Operators

Bash supports logical operators for combining multiple conditions inside a single statement.

#!/bin/bash
age=20
country="India"

if [[ $age -ge 18 && $country == "India" ]]; then
    echo "Eligible"
fi

For Loops

The for loop repeats a block of code multiple times. It is commonly used when processing lists, files, or ranges of numbers.

#!/bin/bash
for lang in Bash Python JavaScript Go; do
    echo "Learning $lang"
done

The loop runs once for every item in the list.

Numeric Loops

#!/bin/bash
for ((i=1; i<=5; i++)); do
    echo "Number: $i"
done

While Loops

The while loop continues running as long as its condition remains true.

#!/bin/bash
count=1

while [[ $count -le 5 ]]; do
    echo "Count: $count"
    ((count++))
done

While loops are useful when the number of iterations is not known in advance.

Until Loops

The until loop is the opposite of while. It runs until a condition becomes true.

#!/bin/bash
number=1

until [[ $number -gt 5 ]]; do
    echo "Value: $number"
    ((number++))
done

Break & Continue

The break statement immediately exits a loop, while continue skips the current iteration and moves to the next one.

#!/bin/bash
for i in {1..10}; do

    if [[ $i -eq 3 ]]; then
        continue
    fi

    if [[ $i -eq 7 ]]; then
        break
    fi

    echo "Value: $i"
done

In this example, number 3 is skipped and the loop stops completely at 7.

Select Loop

The select loop automatically creates numbered menus. It is useful for interactive terminal applications.

#!/bin/bash
PS3="Choose option: "

select option in Start Stop Exit; do
    case $option in
        Start) echo "Starting...";;
        Stop) echo "Stopping...";;
        Exit) break;;
        *) echo "Invalid";;
    esac
done

The PS3 variable changes the prompt shown to the user.

Nested Conditions

Conditions can also be placed inside other conditions. This technique is called nesting and is useful for handling more complex logic.

#!/bin/bash
read -p "Enter username: " user
read -p "Enter password: " pass

if [[ $user == "admin" ]]; then

    if [[ $pass == "1234" ]]; then
        echo "Login successful"
    else
        echo "Wrong password"
    fi

else
    echo "Unknown user"
fi

Best Practices

Writing clean control flow improves script readability and reduces bugs.

• Use indentation inside conditions and loops.
• Prefer case statements for menus and multiple choices.
• Keep conditions simple and readable.
• Always test user input before processing it.
• Use comments to explain complex logic.

Loops

Loops allow a script to repeat instructions automatically. Instead of writing the same code multiple times, loops execute a block of code repeatedly until a condition changes.

Loops are extremely important in shell scripting because they help automate repetitive tasks such as:

• Processing files
• Reading log data
• Running commands repeatedly
• Monitoring systems
• Working with arrays

In this lecture, we will learn:

• For loops
• While loops
• Until loops
• Break and continue
• Nested loops
• Looping through arrays
• Safe file processing
• Practical automation examples

For Loops

The for loop is used when the number of iterations is known beforehand. It is commonly used to loop through lists, ranges, arrays, and files.

#!/bin/bash

# Loop through words
for fruit in apple banana mango; do
    echo "$fruit"
done

# C-style numeric loop
for ((i=1; i<=5; i++)); do
    echo "Count: $i"
done

# Loop through matching files
for file in *.txt; do
    echo "Found: $file"
done

Understanding the Example

• The loop variable changes automatically during each iteration
• do starts the loop body
• done marks the end of the loop
• *.txt matches all text files

Using Ranges

#!/bin/bash

for i in {1..10}; do
    echo "$i"
done

While Loop

A while loop runs as long as a condition remains true. It is useful when the number of iterations is unknown.

#!/bin/bash

count=1

while [[ $count -le 5 ]]; do

    echo "Iter $count"

    ((count++))

done

# Read file line by line

while IFS= read -r line; do

    echo "$line"

done < /etc/hosts

Why Use IFS= read -r ?

• IFS= prevents trimming spaces
• -r prevents backslash escaping
• This is the safest way to read files line by line

Until Loop

The until loop is the opposite of a while loop. It continues running until the condition becomes true.

#!/bin/bash

n=1

until [[ $n -gt 5 ]]; do

    echo "$n"

    ((n++))

done

Difference Between While and Until

Break & Continue

Bash provides loop control statements to change loop execution.

• break stops the loop completely
• continue skips the current iteration

#!/bin/bash

for i in {1..10}; do

    [[ $i -eq 3 ]] && continue

    [[ $i -eq 8 ]] && break

    echo "$i"

done

Program Output

1
2
4
5
6
7

Looping Over Arrays

Arrays work naturally with loops. You can iterate through values or indices.

#!/bin/bash

tools=(Docker Kubernetes Terraform Ansible)

# By value
for tool in "${tools[@]}"; do
    echo "Deploying with $tool"
done

# By index
for i in "${!tools[@]}"; do
    echo "$((i+1)). ${tools[$i]}"
done

Important Note

Always quote "${array[@]}" to preserve elements containing spaces.

Nested Loops

A nested loop is a loop inside another loop. The inner loop completes fully for every iteration of the outer loop.

#!/bin/bash

for row in 1 2 3; do

    for col in A B C; do

        echo -n "[$row$col] "

    done

    echo

done

Understanding Nested Loops

The outer loop controls rows while the inner loop controls columns. Nested loops are useful for:

• Tables
• Matrix operations
• Combinations
• Grid generation

Safe File Processing

Looping through files incorrectly can break filenames containing spaces. Always quote variables properly.

#!/bin/bash

# Unsafe
for f in $(find . -name "*.log"); do
    echo "$f"
done

# Safe method
find . -name "*.log" -print0 | while IFS= read -r -d '' f; do
    echo "Processing: $f"
done

Why Use -print0 ?

The -print0 option separates filenames using null bytes instead of spaces. This safely handles filenames containing spaces and special characters.

Subshell Pitfalls

When piping data into a loop, Bash may run the loop in a subshell. Variables changed inside the loop may disappear afterwards.

#!/bin/bash

# Problematic example
cat list.txt | while read line; do
    ((count++))
done

echo "$count"

# Correct solution
while read line; do
    ((count++))
done < <(cat list.txt)

echo "$count"

Practical: Batch Rename

Loops are commonly used for batch file operations such as renaming files automatically.

#!/bin/bash

c=1

for f in *.jpg; do

    mv "$f" "img_$(printf "%03d" $c).jpg"

    ((c++))

done

echo "Renamed $((c-1)) files."

Practical: System Monitor

Loops can also automate monitoring tasks. The following example checks CPU usage repeatedly.

#!/bin/bash

interval=5
attempts=0
max_attempts=6

while [[ $attempts -lt $max_attempts ]]; do

    cpu=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}')

    echo "CPU Usage: ${cpu}%"

    ((attempts++))

    sleep $interval

done

Infinite Loops

Infinite loops continue forever until manually stopped. These are useful for servers, monitoring systems, and background services.

#!/bin/bash

while true; do

    echo "Running..."

    sleep 2

done

Stopping Infinite Loops

Press Ctrl + C to interrupt the script manually.

Best Practices

• Use meaningful loop variable names
• Quote variables when working with files
• Avoid deeply nested loops when possible
• Use comments for readability
• Always test loops with small datasets first

Functions & Scripting

Introduction to Functions

Functions help organize code into reusable blocks. Instead of writing the same commands repeatedly, a function allows you to define logic once and call it whenever needed.

Functions make scripts cleaner, easier to maintain, and more modular. Large shell scripts often rely heavily on functions to separate different tasks.

Defining Functions

A function is created using a function name followed by parentheses and a block of code inside braces.

#!/bin/bash

greet() {
    local name=$1
    echo "Hello, $name!"
}

greet "Ahmed"

add() {
    echo $(($1 + $2))
}

sum=$(add 5 7)

echo "Sum: $sum"

Function arguments are accessed using positional variables like $1, $2, and so on.

Calling Functions

Functions execute only when called. Defining a function alone does not run it.

#!/bin/bash

welcome() {
    echo "Welcome to Bash Scripting"
}

welcome
welcome

In this example, the function runs twice because it is called two times.

Returning Values

Bash functions do not return values like traditional programming languages. Instead, they usually print output or use exit codes.

#!/bin/bash

multiply() {
    echo $(($1 * $2))
}

result=$(multiply 4 5)

echo "Result: $result"

The output of the function is captured using command substitution.

Local Variables

Variables inside functions should usually be declared with local. This prevents accidental modification of variables outside the function.

#!/bin/bash

g="global"

demo() {
    local l="local"
    g="changed"

    echo "$l $g"
}

demo

echo "$g"

Without local, variables inside functions become global and may affect other parts of the script.

Using Parameters in Functions

Functions can accept multiple parameters, making them flexible and reusable.

#!/bin/bash

info() {
    echo "Name: $1"
    echo "Role: $2"
}

info "Ahmed" "Developer"

Parameters are assigned in the same order they are passed during the function call.

Function Libraries

Functions can be stored in separate files and reused across multiple scripts. This approach improves code organization and prevents duplication.

#!/bin/bash

log() {
    echo "[$(date)] $*"
}

die() {
    echo "FATAL: $*" >&2
    exit 1
}

#!/bin/bash

source ./lib.sh

log "Starting application"

die "Disk full"

The source command loads another script into the current shell session.

Error Handling

Functions should properly handle errors and invalid input. Good error handling makes scripts more reliable and easier to debug.

#!/bin/bash

delete() {
    local f=$1

    if [[ ! -f "$f" ]]; then
        echo "Not found: $f" >&2
        return 1
    fi

    rm "$f"

    echo "Deleted: $f"
}

delete "/tmp/test.txt" || exit 1

The return statement exits the function and sends an exit status back to the caller.

Recursive Functions

A recursive function is a function that calls itself. Recursion is useful for solving repetitive problems like factorial calculations.

#!/bin/bash

factorial() {
    local n=$1

    if [[ $n -le 1 ]]; then
        echo 1
    else
        local prev=$(factorial $(($n - 1)))
        echo $(($n * $prev))
    fi
}

factorial 5

Recursive functions must always include a stopping condition to avoid infinite recursion.

Debugging Functions

Debugging helps identify problems inside scripts and functions. Bash provides debugging options that show commands during execution.

#!/bin/bash

set -x

test_func() {
    echo "Debugging function"
}

test_func

The set -x option prints commands before execution, making it easier to trace script behavior.

Best Practices

Writing clean and reusable functions improves script quality and maintainability.

• Keep functions focused on a single task.
• Use meaningful function names.
• Always use local variables when possible.
• Validate input before processing it.
• Store reusable functions inside libraries.

Arrays & String Manipulation

Arrays and strings are fundamental parts of Bash scripting. Arrays allow scripts to store multiple values inside a single variable, while string manipulation helps process text efficiently.

These concepts are heavily used in:

• Automation scripts
• Log processing
• Configuration management
• CSV parsing
• Text filtering
• System administration

In this lecture, we will learn:

• Indexed arrays
• Associative arrays
• Looping through arrays
• String operations
• Substring extraction
• Path manipulation
• CSV parsing

Indexed Arrays

Indexed arrays store multiple values using numeric indexes. Bash array indexing starts from 0.

#!/bin/bash

arr=("a" "b" "c" "d")

# Access element by index
echo "${arr[0]}"

# Print all elements
echo "${arr[@]}"

# Array length
echo "${#arr[@]}"

# Append new value
arr+=("e")

# Loop through array
for v in "${arr[@]}"; do
    echo "$v"
done

Understanding Array Syntax

• ${arr[0]} → first element
• ${arr[@]} → all elements
• ${#arr[@]} → total elements
• += → append new values

Looping with Array Indexes

Sometimes we need both the index and value.

#!/bin/bash

colors=("red" "green" "blue")

for i in "${!colors[@]}"; do
    echo "Index $i = ${colors[$i]}"
done

Associative Arrays

Associative arrays use keys instead of numeric indexes. They behave similarly to dictionaries or hash maps in other languages.

#!/bin/bash

declare -A user

user[name]="Ahmed"
user[age]=25

echo "${user[name]}"

for k in "${!user[@]}"; do
    echo "$k=${user[$k]}"
done

Important Notes

• declare -A creates an associative array
• Keys are strings instead of numbers
• ${!user[@]} returns all keys

Real-World Example

#!/bin/bash

declare -A servers

servers[web]="online"
servers[db]="offline"
servers[cache]="online"

for s in "${!servers[@]}"; do
    echo "$s server is ${servers[$s]}"
done

String Operations

Strings are sequences of characters. Bash provides powerful built-in tools for manipulating strings.

#!/bin/bash

s="Hello World"

# String length
echo "${#s}"

# Uppercase
echo "${s^^}"

# Lowercase
echo "${s,,}"

# Substring extraction
echo "${s:6:5}"

# Replace text
echo "${s/World/Shell}"

path="/a/b/c.txt"

# Basename
echo "${path##*/}"

# Directory name
echo "${path%/*}"

String Manipulation Operators

Removing Text from Strings

Bash can remove matching text from the beginning or end of strings.

#!/bin/bash

file="backup.tar.gz"

# Remove shortest match from front
echo "${file#*.}"

# Remove longest match from front
echo "${file##*.}"

# Remove shortest match from end
echo "${file%.*}"

# Remove longest match from end
echo "${file%%.*}"

Understanding # and %

• # removes from beginning
• % removes from end
• Double symbols remove the longest match

Splitting Strings

Strings can be split into arrays using separators.

#!/bin/bash

data="red,green,blue"

IFS=',' read -ra colors <<< "$data"

for c in "${colors[@]}"; do
    echo "$c"
done

What is IFS?

IFS stands for Internal Field Separator. It controls how Bash splits text into fields.

Practical: CSV Parser

CSV files store values separated by commas. Bash can process them line by line using loops and IFS.

#!/bin/bash

while IFS=',' read -r name age city; do

    echo "$name ($age) from $city"

done < data.csv

Understanding the CSV Parser

• IFS=',' splits columns using commas
• read -r safely reads each line
• Variables automatically store each column value

Searching Inside Strings

Bash supports pattern matching for checking string contents.

#!/bin/bash

text="server-error.log"

[[ $text == *.log ]] && echo "Log file"

[[ $text == server* ]] && echo "Starts with server"

Best Practices

• Always quote array expansions
• Use associative arrays for key-value data
• Use meaningful variable names
• Use IFS carefully when splitting strings
• Validate input before processing files

File Operations & I/O

Introduction to File Operations

File handling is one of the most important parts of shell scripting. Scripts often need to read configuration files, process logs, generate reports, or store application data.

Bash provides powerful tools for reading files, writing content, redirecting output, and managing input/output streams.

Reading Files

Files can be read line-by-line using loops. This method is memory-efficient and works well even for large files.

#!/bin/bash

while IFS= read -r line; do
    echo "$line"
done < input.txt

content=$(< input.txt)

echo "$content"

The IFS= prevents trimming whitespace, while -r stops backslashes from being interpreted as escape characters.

Reading Specific Columns

Shell scripts can split lines into multiple fields using delimiters. This is useful when processing CSV files or structured text.

#!/bin/bash

while IFS=":" read -r user shell; do
    echo "User: $user | Shell: $shell"
done < users.txt

The IFS variable defines the separator used while reading the file.

Writing Files

Bash can create new files, overwrite existing files, or append data to the end of files.

#!/bin/bash

echo "Line 1" > out.txt
echo "Line 2" >> out.txt

cat << EOF > config.cfg
host=localhost
port=8080
EOF

The > operator overwrites a file, while >> appends content without removing existing data.

Here Documents

A heredoc allows multiple lines of text to be written directly inside a script. This is commonly used for configuration files and automated setup scripts.

#!/bin/bash

cat << EOF
Welcome to Bash Scripting
This text spans multiple lines
EOF

The text continues until the ending keyword is reached.

Redirection

Redirection controls where command output goes. By default, output appears on the terminal, but Bash allows output to be redirected into files.

#!/bin/bash

./app > out.log 2> err.log
./app > all.log 2>&1
./app > /dev/null 2>&1

Redirecting output is extremely useful for logging and debugging scripts.

Pipes

Pipes connect the output of one command directly into another command. This allows powerful command chaining.

#!/bin/bash

cat users.txt | grep "Ahmed"
ps aux | grep nginx
df -h | sort

The pipe operator | passes stdout from one command as stdin to the next command.

Appending Logs

Logs are commonly appended instead of overwritten so that older data is preserved.

#!/bin/bash

echo "[$(date)] Script started" >> app.log
echo "[$(date)] Processing files" >> app.log

Appending logs helps track events and monitor script execution over time.

Checking File Existence

Before reading or writing files, scripts should verify whether the file exists to avoid unexpected errors.

#!/bin/bash

file="data.txt"

if [[ -f "$file" ]]; then
    echo "File exists"
else
    echo "File not found"
fi

The -f operator checks whether a regular file exists.

Temporary Files & Cleanup

Temporary files are useful for storing intermediate data during script execution. They should always be cleaned up afterward.

#!/bin/bash

tmp=$(mktemp /tmp/script.XXXXXX)

cleanup() {
    rm -f "$tmp"
    echo "Done."
}

trap cleanup EXIT

echo "working..." > "$tmp"

The trap command ensures cleanup happens automatically when the script exits.

File Permissions

Shell scripts often need permission management. Linux uses read, write, and execute permissions for files.

#!/bin/bash

chmod +x script.sh
ls -l script.sh

The chmod command changes file permissions, while ls -l displays them.

Input from Users

Scripts can combine file operations with user input to make interactive tools.

#!/bin/bash

read -p "Enter your note: " note

echo "$note" >> notes.txt

echo "Saved."

This technique is commonly used in note-taking utilities, logging systems, and automation scripts.

Best Practices

Proper file handling prevents data loss and improves script reliability.

• Always quote filenames to handle spaces safely.
• Check whether files exist before using them.
• Use append mode for logs whenever possible.
• Clean temporary files after execution.
• Separate stdout and stderr for easier debugging.

Process Management & Signals

Every program running on Linux is called a process. Bash scripts often create, monitor, stop, or control processes while automating tasks.

Understanding process management is important for:

• Automation scripts
• Background jobs
• System monitoring
• Server management
• Task scheduling
• Daemon services

In this lecture, we will learn:

• Running background processes
• Understanding process IDs (PID)
• Using signals
• Trapping signals safely
• Parallel processing
• Building daemon-style scripts

Background Processes

Normally, Bash waits for a command to finish before continuing. Adding & runs the process in the background so the script can continue immediately.

#!/bin/bash

# Run command in background
sleep 30 &

# Store process ID
pid=$!

echo "Job PID: $pid"

# Check if process exists
kill -0 $pid 2>/dev/null && echo "Running"

# Wait for process to finish
wait $pid

echo "Done (exit: $?)"

Important Concepts

• & → run command in background
• $! → PID of last background process
• wait → pause until process finishes
• $? → exit status of last command

Viewing Running Processes

Linux provides several commands for viewing processes.

#!/bin/bash

# Show current shell processes
ps

# Detailed process list
ps aux

# Search for process
ps aux | grep nginx

# Interactive process viewer
top

Trapping Signals

Signals are messages sent to processes. They are used to interrupt, stop, restart, or terminate programs.

The trap command lets Bash execute cleanup code when a signal is received.

#!/bin/bash

cleanup() {

    echo "Interrupted!"

    rm -f /tmp/lock

    exit 1
}

# Trap Ctrl+C and termination signals
trap cleanup SIGINT SIGTERM

touch /tmp/lock

while true; do

    echo "Working... $(date)"

    sleep 2

done

Why Trap Signals?

• Remove temporary files
• Release locks
• Close database connections
• Save progress before exiting
• Prevent corrupted state

Common Signals

Signals are identified by names and numbers.

Sending Signals Manually

#!/bin/bash

# Graceful termination
kill -TERM 1234

# Force termination
kill -KILL 1234

# Same as SIGTERM
kill 1234

Always try SIGTERM before SIGKILL. Force killing prevents cleanup operations from running.

Jobs & Job Control

Bash provides built-in commands for managing background jobs directly from the shell.

#!/bin/bash

sleep 100 &

jobs

fg %1

bg %1

Job Commands

• jobs → list background jobs
• fg → bring job to foreground
• bg → continue stopped job in background

Parallel Processing

Running multiple tasks simultaneously can improve performance dramatically.

#!/bin/bash

proc() {

    echo "Processing $1..."

    sleep 2

    echo "$1 done."
}

for f in a.txt b.txt c.txt; do

    proc "$f" &

done

wait

echo "All done!"

Understanding Parallel Execution

Without &, files process one after another. With background execution, all tasks run simultaneously.

Practical Example

#!/bin/bash

backup() {

    tar -czf "$1.tar.gz" "$1"

    echo "$1 backed up"
}

for dir in docs images videos; do

    backup "$dir" &

done

wait

echo "All backups completed."

Monitoring Processes

Scripts can monitor system activity and process health.

#!/bin/bash

while true; do

    echo "System Load:"

    uptime

    sleep 5

done

Press Ctrl+C to stop the monitoring loop.

Daemon Script

A daemon is a long-running background service. Daemon scripts are commonly used for monitoring, scheduling, or server automation.

#!/bin/bash

PIDFILE="/tmp/myd.pid"

case $1 in

  start)

         echo $$ > $PIDFILE

         trap "rm $PIDFILE" EXIT

         while true; do

             echo "run..."

             sleep 10

         done &

         ;;

  stop)

         [[ -f $PIDFILE ]] && kill $(cat $PIDFILE) && rm $PIDFILE

         ;;

  *)

         echo "Usage: $0 start|stop"

         ;;

esac

Understanding PID Files

• PID files store the running process ID
• Useful for stopping services later
• Prevents multiple copies of the same daemon

Best Practices

• Always clean temporary files with traps
• Use wait after background jobs
• Prefer graceful termination over force kill
• Track important processes using PID files
• Limit excessive parallel jobs to avoid CPU overload

Regular Expressions & grep

Text processing is one of the most powerful features of Linux and Bash scripting. System administrators and developers constantly search logs, filter data, extract patterns, and modify text files automatically.

This lecture introduces:

• grep for searching text
• Regular Expressions (Regex) for pattern matching
• sed for editing streams of text
• awk for structured text processing

These tools are essential for:

• Log analysis
• Data extraction
• Automation scripts
• Monitoring systems
• Configuration management
• CSV and report processing

grep Essentials

The grep command searches files for matching text patterns. It prints every line that matches the search condition.

#!/bin/bash

grep "error" app.log           # basic search

grep -i "error" app.log        # case-insensitive

grep -n "error" app.log        # show line numbers

grep -c "error" app.log        # count matches

grep -v "debug" app.log        # invert match

grep -r "TODO" ./src/          # recursive search

grep -C 3 "error" app.log      # show 3 context lines

Understanding Common grep Options

Practical Example

#!/bin/bash

# Find failed login attempts
grep "Failed password" /var/log/auth.log

# Count warnings
grep -c "WARNING" server.log

Regex Patterns

Regular Expressions (Regex) are patterns used to match text. Regex allows advanced searching beyond simple words.

#!/bin/bash

grep "^ERROR" app.log                 # starts with ERROR

grep "done$" app.log                  # ends with done

grep "[Ee]rror" app.log               # character class

grep "[0-9]" app.log                  # any digit

grep -E "(err|warn)" app.log          # alternation

grep -E "[0-9]{1,3}\.[0-9]{1,3}" file # partial IP pattern

Important Regex Symbols

Email Matching Example

#!/bin/bash

grep -E "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$" users.txt

This regex searches for lines that look like valid email addresses.

sed — Stream Editor

The sed command edits text streams automatically. It is extremely useful for replacing, deleting, or transforming text.

#!/bin/bash

sed 's/old/new/' file.txt      # replace first match

sed 's/old/new/g' file.txt     # replace all matches

sed '/pattern/d' file.txt      # delete matching lines

sed -n '5,10p' file.txt        # print lines 5-10

sed -i 's/foo/bar/g' file.txt  # modify file directly

Understanding sed Syntax

• s → substitute
• g → global replacement
• -i → edit file in place
• -n → suppress automatic output

Practical Example

#!/bin/bash

# Remove blank lines
sed '/^$/d' notes.txt

# Replace tabs with spaces
sed 's/\t/    /g' code.py

awk — Pattern Scanner

The awk tool processes structured text line by line. It is especially powerful for working with columns and reports.

#!/bin/bash

awk '{print $1, $3}' data.txt              # print columns

awk '$3 > 100' data.txt                    # filter rows

awk '{sum+=$2} END {print sum}' data.txt   # calculate sum

awk -F',' '{print $2}' data.csv            # custom delimiter

awk Column Variables

• $1 → first column
• $2 → second column
• NF → total fields
• NR → current line number

CSV Example

#!/bin/bash

awk -F',' '{total += $3} END {print "Total Sales:", total}' sales.csv

Combining grep, sed & awk

These tools are often combined together using pipes.

#!/bin/bash

grep "ERROR" app.log | awk '{print $5}' | sed 's/:$//' | sort | uniq -c

This pipeline:

• Finds ERROR lines
• Extracts a column
• Removes trailing colons
• Sorts results
• Counts duplicates

Best Practices

• Use quotes around regex patterns
• Test regex carefully before production use
• Use -i in sed carefully because it changes files permanently
• Prefer pipelines for readable data processing
• Use awk for structured column-based data

DevOps & Automation

Introduction to DevOps Automation

DevOps focuses on automating development, deployment, monitoring, and infrastructure tasks. Shell scripting plays a major role in DevOps because it allows repetitive operations to be automated quickly and efficiently.

Automation reduces manual work, minimizes human error, and helps maintain consistent deployments across servers and environments.

Cron Jobs

Cron is a Linux scheduling system used to run scripts automatically at specific times or intervals. Scheduled tasks are managed using the crontab command.

# crontab -e

0 2 * * * /opt/backup.sh
*/5 * * * * /opt/healthcheck.sh
@reboot /opt/startup.sh

The five fields represent minute, hour, day of month, month, and day of week.

Automated Backups

One of the most common DevOps tasks is creating automated backups for applications and databases.

#!/bin/bash

backup_dir="/backup"
date=$(date +%Y-%m-%d)

tar -czf "$backup_dir/site-$date.tar.gz" /var/www/html

echo "Backup completed"

This script compresses website files into a timestamped archive.

Systemd Services

Systemd manages services and background processes on modern Linux systems. Service files define how applications start, stop, and restart.

[Unit]
Description=My Web App
After=network.target

[Service]
ExecStart=/usr/bin/node /opt/app/server.js
Restart=always

[Install]
WantedBy=multi-user.target

After creating a service file, use systemctl commands to manage it.

#!/bin/bash

systemctl start myapp
systemctl stop myapp
systemctl restart myapp
systemctl status myapp

Deployment Scripts

Deployment scripts automate the process of updating applications on servers. Instead of manually running commands, everything can be handled in one script.

#!/bin/bash

set -e

echo "Deploying application..."

git pull origin main

npm install

npm run build

systemctl restart myapp

echo "Deployment completed"

The set -e option immediately stops the script if any command fails.

Environment Variables

Environment variables are commonly used in DevOps to store configuration values such as database credentials, ports, and API keys.

#!/bin/bash

export APP_ENV="production"
export PORT=8080

echo "Running in $APP_ENV mode on port $PORT"

Environment variables allow the same application to run in different environments without changing source code.

Log Rotation & Monitoring

Monitoring scripts help detect issues before they become critical. Logs should also be rotated regularly to prevent disks from filling up.

#!/bin/bash

usage=$(df / | tail -1 | awk '{print $5}' | tr -d '%')

if [[ $usage -gt 90 ]]; then
    echo "Disk critical: ${usage}%"
fi

curl -f http://localhost:8080/health || systemctl restart myapp

This script checks disk usage and restarts the application if the health endpoint fails.

Docker Cleanup

Docker stores images, containers, volumes, and cache data. Over time, unused resources can consume large amounts of disk space.

#!/bin/bash

docker system prune -af --volumes

echo "Docker cleanup complete"

The cleanup command removes unused containers, images, networks, and volumes.

CI/CD Basics

CI/CD stands for Continuous Integration and Continuous Deployment. It automates testing, building, and deployment whenever new code is pushed.

#!/bin/bash

echo "Running tests..."
npm test

echo "Building project..."
npm run build

echo "Deploying..."
systemctl restart myapp

CI/CD pipelines reduce deployment time and improve software reliability.

SSH Automation

Shell scripts are frequently used to automate commands on remote servers through SSH.

#!/bin/bash

ssh user@server "uptime"

ssh user@server "systemctl restart nginx"

This allows administrators to manage servers remotely without manually logging in each time.

Health Check Scripts

Health check scripts monitor services, APIs, and websites to ensure they remain operational.

#!/bin/bash

response=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8080)

if [[ $response -eq 200 ]]; then
    echo "Application healthy"
else
    echo "Application down"
fi

Health checks are often combined with cron jobs and monitoring systems.

Best Practices

Automation scripts should be safe, reliable, and easy to maintain.

• Always test scripts in a staging environment first.
• Use logging for important operations.
• Store secrets in environment variables instead of hardcoding them.
• Use set -e to stop scripts on failure.
• Monitor resource usage regularly.

Capstone Project: Automation Toolkit

Project Overview

In this capstone project, you will build a complete System Automation Toolkit. The goal is to combine everything learned throughout the course into one practical real-world application.

This toolkit acts as a mini DevOps and system administration utility capable of monitoring systems, managing backups, analyzing logs, and automating maintenance tasks.

Project Goals

The project focuses on building modular shell scripts that work together through a menu-driven interface.

• Practice writing reusable Bash functions
• Use loops, conditions, and case statements
• Work with files, logs, and system commands
• Automate common Linux administration tasks
• Improve scripting structure and organization

Features to Implement

• System health check (CPU, memory, disk)
• Backup specified directories with timestamps
• Log analyzer (find errors, count by severity)
• User management (add/remove/list users)
• Service status checker

Project Structure

Organizing scripts into folders makes large projects easier to maintain.

AutoTool/
├── autotool.sh
├── log_analyzer.sh
├── user_mgmt.sh
├── reports/
├── backups/
└── lib/
    └── utils.sh

Separating utilities into modules improves readability and reusability.

Starter Template

The main script displays the menu and calls the appropriate module based on user input.

#!/bin/bash

set -euo pipefail

source ./lib/utils.sh

show_menu() {
    echo "=== AutoTool ==="
    echo "1) Health Check  3) Log Analyzer"
    echo "2) Backup        4) User Mgmt"
    echo "5) Service Check 6) Exit"
}

health_check() {
    echo "CPU: $(top -bn1 | grep 'Cpu' | awk '{print $2}')%"
    echo "Mem: $(free -m | awk '/Mem/{printf "%.1f%%", $3/$2*100}')"
    echo "Disk: $(df -h / | tail -1 | awk '{print $5}')"
}

backup() {
    local src=$1
    local dst="/backup/$(date +%Y%m%d_%H%M%S)"

    mkdir -p "$dst"

    cp -r "$src" "$dst"

    echo "Backed up to $dst"
}

while true; do
    show_menu

    read -p "Choice: " c

    case $c in
        1) health_check;;
        2) read -p "Dir: " d; backup "$d";;
        3) ./log_analyzer.sh;;
        4) ./user_mgmt.sh;;
        5) systemctl status nginx;;
        6) exit;;
    esac
done

The set -euo pipefail option enables safer scripting by stopping execution on errors and preventing undefined variables.

Health Check Module

The health check module gathers important system metrics such as CPU usage, memory usage, and disk space.

#!/bin/bash

echo "===== SYSTEM HEALTH ====="

echo "CPU Usage:"
top -bn1 | grep "Cpu"

echo "Memory Usage:"
free -h

echo "Disk Usage:"
df -h

System monitoring scripts help administrators detect resource problems early.

Backup Module

Backups are essential for protecting data against accidental deletion, corruption, or hardware failure.

#!/bin/bash

src="/var/www/html"

dest="./backups/site-$(date +%Y%m%d_%H%M%S).tar.gz"

tar -czf "$dest" "$src"

echo "Backup saved to $dest"

The tar command compresses files into a single archive for easier storage and transfer.

Log Analyzer Module

Logs contain valuable information about system activity and application errors. Analyzing logs helps identify issues quickly.

#!/bin/bash

logfile="/var/log/syslog"

echo "ERROR COUNT:"
grep -i "error" "$logfile" | wc -l

echo "WARNING COUNT:"
grep -i "warning" "$logfile" | wc -l

The grep command filters matching lines, while wc -l counts them.

User Management Module

System administrators often automate user management tasks such as creating or deleting accounts.

#!/bin/bash

read -p "Enter username: " user

sudo useradd "$user"

echo "User created: $user"

User management operations usually require administrative privileges.

Service Status Checker

Monitoring services ensures important applications remain operational.

#!/bin/bash

service="nginx"

systemctl is-active --quiet "$service"

if [[ $? -eq 0 ]]; then
    echo "$service is running"
else
    echo "$service is stopped"
fi

This script checks whether a Linux service is currently active.

Utility Functions

Shared helper functions should be stored inside reusable library files.

#!/bin/bash

log() {
    echo "[$(date)] $*"
}

error() {
    echo "ERROR: $*" >&2
}

Using utility libraries prevents duplicate code across modules.

Generating Reports

The toolkit can generate reports summarizing system status and automation results.

#!/bin/bash

report="./reports/report.txt"

echo "System Report - $(date)" > "$report"

df -h >> "$report"

free -m >> "$report"

echo "Report generated"

Reports are useful for tracking server health over time.

Scheduling with Cron

The toolkit becomes more powerful when automated using cron jobs.

# Run health checks every hour
0 * * * * /opt/autotool/health.sh

# Run backups daily at midnight
0 0 * * * /opt/autotool/backup.sh

Automation ensures maintenance tasks run even when administrators are offline.

Bonus Challenges

• Add email alerts for critical conditions
• Schedule the health check via cron
• Generate HTML reports instead of plain text
• Add a web dashboard using a simple HTTP server

Best Practices

Large automation projects should follow professional scripting practices.

• Use modular scripts and reusable functions.
• Validate user input before processing.
• Store logs for debugging and monitoring.
• Always test scripts in a virtual machine first.
• Document your project using comments and README files.

Final Thoughts

This capstone project combines shell scripting, automation, monitoring, file handling, functions, and Linux administration into one practical toolkit.

Completing this project gives hands-on experience similar to real-world DevOps and system administration workflows.

Advanced Shell Patterns

As Bash scripts grow larger, repeating the same code becomes difficult to maintain. Advanced shell patterns help organize scripts into reusable, cleaner, and more efficient structures.

These patterns are heavily used in:

• Automation frameworks
• Deployment scripts
• DevOps tooling
• Monitoring systems
• Server provisioning
• Production-grade shell scripts

In this lecture, we will learn:

• Reusable functions
• Command substitution
• Case statements
• Input validation patterns
• Retry mechanisms
• Lock files
• Script templates

Reusable Functions

Functions allow code reuse and better organization. Instead of writing the same commands multiple times, create a function once and call it whenever needed.

#!/bin/bash

log_info() {

    echo "[INFO] $1"
}

log_error() {

    echo "[ERROR] $1" >&2
}

backup_file() {

    cp "$1" "$1.bak"

    log_info "Backup created for $1"
}

backup_file "config.txt"

Why Use Functions?

• Reduce duplicated code
• Improve readability
• Make debugging easier
• Keep scripts modular

Command Substitution

Command substitution stores command output inside variables.

#!/bin/bash

current_date=$(date)

hostname=$(hostname)

files=$(ls | wc -l)

echo "Date: $current_date"

echo "Host: $hostname"

echo "Files: $files"

The modern syntax $(command) is preferred over older backticks because it is easier to read and nest.

Case Statements

The case statement simplifies complex conditional logic. It is cleaner than writing many if-elif blocks.

#!/bin/bash

read -p "Enter option: " choice

case $choice in

    start)

        echo "Starting service..."

        ;;

    stop)

        echo "Stopping service..."

        ;;

    restart)

        echo "Restarting service..."

        ;;

    *)

        echo "Invalid option"

        ;;
esac

Wildcard Matching in case

#!/bin/bash

file="report.pdf"

case $file in

    *.txt)

        echo "Text file"

        ;;

    *.pdf)

        echo "PDF file"

        ;;

    *)

        echo "Unknown type"

        ;;
esac

Input Validation Patterns

Production scripts should always validate user input before processing it.

#!/bin/bash

read -p "Enter age: " age

if [[ $age =~ ^[0-9]+$ ]]; then

    echo "Valid number"

else

    echo "Invalid input"

fi

Regex validation helps prevent crashes caused by unexpected input values.

Retry Pattern

Network operations and API requests may fail temporarily. Retry loops automatically attempt commands again after failure.

#!/bin/bash

max_retries=5

attempt=1

while [[ $attempt -le $max_retries ]]; do

    ping -c 1 google.com &>/dev/null && break

    echo "Attempt $attempt failed"

    sleep 2

    ((attempt++))
done

echo "Finished"

Why Retries Matter

• Handle temporary failures
• Improve reliability
• Prevent immediate script termination

Lock Files

Lock files prevent multiple copies of the same script from running simultaneously.

#!/bin/bash

LOCKFILE="/tmp/app.lock"

if [[ -f $LOCKFILE ]]; then

    echo "Script already running"

    exit 1
fi

touch $LOCKFILE

trap "rm -f $LOCKFILE" EXIT

echo "Running task..."

sleep 10

The trap command ensures the lock file is removed even if the script exits unexpectedly.

Script Template Pattern

Professional shell scripts usually begin with a common structure.

#!/bin/bash

set -e

set -u

LOGFILE="app.log"

log() {

    echo "[$(date '+%H:%M:%S')] $1" | tee -a $LOGFILE
}

main() {

    log "Script started"

    echo "Processing..."

    log "Script finished"
}

main "$@"

Important Safety Options

Best Practices

• Break large scripts into functions
• Validate all external input
• Use logging for debugging
• Always clean temporary resources with traps
• Use lock files for scheduled tasks
• Write reusable script templates

Final Project — Production Automation

Project Overview

In this final project, you will combine everything learned throughout the course to build a real-world production automation system using Bash scripting.

The goal is to automate common DevOps and server administration tasks such as:

• Monitoring system health
• Backing up important files
• Rotating and compressing logs
• Cleaning temporary files
• Generating reports
• Sending alerts for failures

This type of automation is heavily used in Linux servers, cloud systems, CI/CD pipelines, monitoring platforms, and deployment infrastructure.

Project Structure

production-automation/
├── backup.sh
├── monitor.sh
├── cleanup.sh
├── rotate_logs.sh
├── report.sh
├── config.conf
├── logs/
├── backups/
└── reports/

Splitting automation tasks into separate scripts makes the system modular and easier to maintain. Each script should handle one responsibility only.

Configuration File

Hardcoding paths inside scripts is bad practice. Store configurable values in a dedicated config file.

BACKUP_DIR="./backups"
LOG_DIR="./logs"
REPORT_DIR="./reports"

SOURCE_DIR="/var/www/html"

MAX_LOG_DAYS=7
MAX_BACKUPS=5

ADMIN_EMAIL="admin@example.com"

Scripts can load configuration values using the source command.

#!/bin/bash

source ./config.conf

echo "Backup directory: $BACKUP_DIR"
echo "Admin email: $ADMIN_EMAIL"

Automated Backup System

Backups are one of the most important automation tasks in production systems. This script creates timestamped compressed backups.

#!/bin/bash

source ./config.conf

timestamp=$(date +"%Y-%m-%d_%H-%M-%S")
backup_file="$BACKUP_DIR/site_$timestamp.tar.gz"

mkdir -p "$BACKUP_DIR"

tar -czf "$backup_file" "$SOURCE_DIR"

if [[ $? -eq 0 ]]; then
    echo "Backup created: $backup_file"
else
    echo "Backup failed!" >&2
fi

The tar -czf command creates compressed archives:

• -c → Create archive
• -z → Compress using gzip
• -f → Output filename

System Monitoring

Production systems must constantly monitor CPU usage, memory usage, and disk space.

#!/bin/bash

cpu=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}')
memory=$(free -m | awk '/Mem:/ {print $3 "MB"}')
disk=$(df -h / | awk 'NR==2 {print $5}')

echo "===== SYSTEM REPORT ====="
echo "CPU Usage: $cpu%"
echo "Memory Usage: $memory"
echo "Disk Usage: $disk"

These commands are extremely common in Linux automation:

• top → CPU statistics
• free → RAM usage
• df → Disk usage
• awk → Extract columns

Log Rotation

Log files grow endlessly in production environments. Rotate and compress old logs automatically.

#!/bin/bash

source ./config.conf

find "$LOG_DIR" -name "*.log" -mtime +$MAX_LOG_DAYS | while read file; do
    gzip "$file"
    echo "Compressed: $file"
done

The -mtime option finds files older than a specific number of days.

Automatic Cleanup

Temporary files and old backups consume storage over time. Cleanup scripts help maintain server performance.

#!/bin/bash

source ./config.conf

find "$BACKUP_DIR" -type f | sort | head -n -$MAX_BACKUPS | while read file; do
    rm -f "$file"
    echo "Deleted old backup: $file"
done

Generating Reports

Automation systems should generate logs and reports for administrators.

#!/bin/bash

source ./config.conf

report="$REPORT_DIR/report_$(date +%F).txt"

mkdir -p "$REPORT_DIR"

{
    echo "===== DAILY REPORT ====="
    date
    echo

    echo "Disk Usage:"
    df -h

    echo
    echo "Memory Usage:"
    free -m

    echo
    echo "Running Processes:"
    ps aux | head
} > "$report"

echo "Report saved to $report"

Error Handling

Production scripts must fail safely and clearly report problems.

#!/bin/bash

set -euo pipefail

backup_file="/tmp/test.tar.gz"

if tar -czf "$backup_file" /important/data; then
    echo "Backup successful"
else
    echo "Backup failed" >&2
    exit 1
fi

Important production flags:

• set -e → Exit on command failure
• set -u → Error on undefined variables
• pipefail → Detect pipeline failures

Scheduling with Cron

Automation becomes truly powerful when scripts run automatically on schedules.

# Run backup every day at 2 AM
0 2 * * * /home/user/backup.sh

# Run monitor every 10 minutes
*/10 * * * * /home/user/monitor.sh

# Cleanup every Sunday
0 0 * * 0 /home/user/cleanup.sh

Open the cron editor using:

crontab -e

Production Best Practices

• Always quote variables: "$file"
• Use absolute paths in cron jobs
• Log important actions
• Validate user input
• Test scripts in safe environments first
• Use functions to organize large scripts
• Store secrets outside scripts