grep Notes

Pattern matching, recursive search, regex modes, and practical log filtering

1. What is grep? (TL;DR)

grep = Global Regular Expression Print

A command-line tool that searches text or files for lines matching a pattern and prints them.

Why grep Matters:

The Core Text Processing Trio

grep

FILTER

Find matching lines

sed

TRANSFORM

Find & replace

awk

STRUCTURE

Column-based processing

Together, these tools let you find lines, extract fields, and transform text in one pipeline.

2. Basic Syntax

grep [OPTIONS] "pattern" [file...]

Simple Examples

# Search for "error" in a file
grep "error" /var/log/syslog

# Search in multiple files
grep "failed" *.log

# Search from command output (pipe)
cat /etc/passwd | grep "root"

# Better way (no useless cat)
grep "root" /etc/passwd

3. Essential Flags - The Core Flags

Match Control

Flag Purpose Example
-i Case-insensitive grep -i "error" log.txt
-v Invert match (exclude) grep -v "debug" log.txt
-w Whole word only grep -w "fail" log.txt
-x Exact line match grep -x "hello world" file.txt 
# Find "ERROR" regardless of case
grep -i "error" /var/log/syslog

# Show lines that DON'T contain "debug"
grep -v "debug" app.log

# Match "fail" but not "failed" or "failure"
grep -w "fail" /var/log/auth.log

Output Control

Flag Purpose Example
-n Show line numbers grep -n "error" log.txt
-c Count matches grep -c "404" access.log
-l List filenames only grep -l "password" *.conf
-L List files WITHOUT match grep -L "TODO" *.py
-o Print only matched part grep -oE "[0-9.]+" log.txt
-h Hide filenames grep -h "error" *.log
-H Always show filenames grep -H "error" *.log 
# Count how many 404 errors in access log
grep -c "404" /var/log/apache2/access.log

# Find which config files contain passwords
grep -l "password" /etc/*. conf

# Extract only the IP addresses from log
grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" access.log

Context Control (Useful for Log Analysis)

Flag Purpose Example
-A n Show n lines After match grep -A 3 "error" log.txt
-B n Show n lines Before match grep -B 2 "error" log.txt
-C n Show n lines Context (both) grep -C 5 "segfault" dmesg. log 
# Show error + next 5 lines (see the stack trace)
grep -A 5 "Exception" app.log

# Show 3 lines before error (see what caused it)
grep -B 3 "failed" /var/log/auth.log

# Full context:  2 lines before and after
grep -C 2 "Connection refused" /var/log/syslog

File & Directory Control

Flag Purpose Example
-r Recursive search grep -r "api_key" /home/
--include Only search specific files grep -r --include="*. py" "import"
--exclude Skip specific files grep -r --exclude="*.log" "secret"
--exclude-dir Skip directories grep -r --exclude-dir=".git" "TODO" 
# Search all Python files for hardcoded credentials
grep -rn --include="*.py" "password\|api_key\|secret" /opt/webapp/

# Search everything except node_modules and .git
grep -rn --exclude-dir={node_modules,.git} "TODO" ./

# Find PHP files that might have SQL injection vulnerabilities
grep -rn --include="*. php" "\$_GET\|\$_POST" /var/www/html/

Regex Mode Flags

Flag Purpose
-E Extended regex (ERE) - use +, ?, |, () without escaping
-P Perl-compatible regex (PCRE) - most powerful, supports \d, \w, lookahead
-F Fixed strings (no regex, literal match) - fastest 
# ERE:  Match "error" OR "fail" OR "critical"
grep -E "error|fail|critical" /var/log/syslog

# PCRE: Extract emails using \w and \d shortcuts
grep -oP "[\w.-]+@[\w.-]+\.\w+" emails.txt

# Fixed string: Search for literal "192.168.1.1" (no regex interpretation)
grep -F "192.168.1.1" access.log

Silent and Script-Friendly

Flag Purpose
-q Quiet mode (no output, just exit code)
-s Suppress error messages 
# Use in scripts  -  check if pattern exists
if grep -q "root" /etc/passwd; then
    echo "Root user exists"
fi

# Exit codes: 
# 0 = match found
# 1 = no match
# 2 = error

4. Regular Expressions in grep

BRE vs ERE vs PCRE

Feature BRE (default) ERE (-E) PCRE (-P)
+ (one or more) \+ + +
?(zero or one) \? ? ?
| (OR) \| | |
() (grouping) \(\) () ()
\d (digit) No No Yes
\w (word char) No No Yes
\s (whitespace) No No Yes
Lookahead/behind No No Yes

Tip

Use -E for most cases, -P when you need \d, \w, or lookarounds.

Common Patterns Cheatsheet

Pattern Meaning Example
. Any single character a.c matches "abc", "adc"
* Zero or more of previous ab*c matches "ac", "abc", "abbc"
+ One or more of previous (ERE) ab+c matches "abc", "abbc" (not "ac")
? Zero or one of previous (ERE) colou?r matches "color", "colour"
^ Start of line ^root lines starting with "root"
$ End of line bash$ lines ending with "bash"
[] Character class [aeiou] matches any vowel
[^] Negated class [^0-9] matches non-digit
| OR (alternation) cat|dog matches "cat" or "dog"
() Grouping (ab)+ matches "ab", "abab"
{n} Exactly n times [0-9]{4} matches 4 digits
{n,m} Between n and m times [0-9]{2,4} matches 2-4 digits
\b Word boundary \bword\b whole word only

Practical Regex Examples

# Match IPv4 addresses
grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" access.log

# Match email addresses
grep -oP "[\w. +-]+@[\w.-]+\.[a-zA-Z]{2,}" data. txt

# Match URLs
grep -oE "https?://[a-zA-Z0-9./?=_-]+" webpage.html

# Match MAC addresses
grep -oE "([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}" arp.txt

# Match lines starting with IP and ending with "denied"
grep -E "^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+.*denied$" firewall.log

5. Cybersecurity Use Cases

Log Analysis on Kali Linux

Key Log Locations

Log File Contains
/var/log/auth.log Authentication attempts, sudo, SSH
/var/log/syslog General system messages
/var/log/apache2/access.log Apache web server access
/var/log/apache2/error.log Apache errors
/var/log/nginx/access.log Nginx access logs
/var/log/kern.log Kernel messages
/var/log/ufw.log Firewall logs (if UFW enabled)
/var/log/fail2ban.log Fail2ban actions

Hunt Failed SSH Attempts

# Find all failed SSH logins
grep "Failed password" /var/log/auth.log

# Extract just the IPs that failed
grep "Failed password" /var/log/auth. log | grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" | sort | uniq -c | sort -rn

# Find failed attempts for specific user
grep "Failed password for root" /var/log/auth.log

Hunt Successful Breaches

# Find successful logins
grep "Accepted password\|Accepted publickey" /var/log/auth.log

# Find sudo commands executed
grep "sudo:" /var/log/auth.log | grep "COMMAND"

# New user/group creation (privilege escalation indicator)
grep -E "useradd|groupadd|usermod" /var/log/auth.log

Web Server Log Analysis

# Find all 404 errors (reconnaissance indicator)
grep " 404 " /var/log/apache2/access.log

# Find potential SQL injection attempts
grep -iE "union.*select|or.*1.*=.*1|drop.*table" /var/log/apache2/access.log

# Find potential XSS attempts
grep -iE "<script|javascript:|onerror=" /var/log/apache2/access. log

# Find directory traversal attempts
grep -E "\.\./|\.\. %2f|%2e%2e" /var/log/apache2/access.log

# Top 10 IP addresses hitting your server
grep -oE "^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" /var/log/apache2/access.log | sort | uniq -c | sort -rn | head -10

Parsing Security Tool Outputs

Nmap Output Parsing

# Extract open ports from nmap output
grep -E "^[0-9]+/tcp.*open" nmap_scan.txt

# Get just port numbers
grep -oE "^[0-9]+" nmap_scan.txt

# Find hosts with specific port open
grep -B 10 "22/tcp.*open" nmap_scan.txt | grep "Nmap scan report"

Gobuster/Dirb Output

# Filter successful finds only
grep "Status: 200\|Status: 301\|Status: 302" gobuster. txt

# Remove noise, show only interesting paths
grep -v "Status: 404" gobuster.txt | grep -E "Status: [0-9]+"

Nikto Output

# Extract vulnerabilities found
grep "+ " nikto.txt | grep -v "Target\|Host"

# Find specific vulnerability types
grep -i "injection\|xss\|traversal" nikto.txt

Secret Hunting in Code

# Find hardcoded credentials
grep -rnE --include="*.py" --include="*.js" --include="*.php" \
    "password\s*=\s*['\"][^'\"]+['\"]|api_key\s*=\s*['\"][^'\"]+['\"]" /var/www/

# Find AWS keys
grep -rE "AKIA[0-9A-Z]{16}" .

# Find private keys
grep -r "BEGIN RSA PRIVATE KEY\|BEGIN OPENSSH PRIVATE KEY" .

# Find JWT tokens
grep -oE "eyJ[A-Za-z0-9_-]*\. eyJ[A-Za-z0-9_-]*\.[A-Za-z0-9_-]*" .

6. DevOps Use Cases

Docker & Kubernetes

# Filter Docker container logs for errors
docker logs my_container 2>&1 | grep -i "error\|exception\|fatal"

# Find containers by image name
docker ps -a | grep "nginx"

# Kubernetes:  Filter pod logs
kubectl logs my-pod | grep -E "ERROR|WARN"

# Find pods in specific state
kubectl get pods | grep -v "Running"

# Search across all pods in namespace
kubectl logs -l app=myapp | grep "connection refused"

System Monitoring

# Find processes using high CPU
ps aux | grep -v grep | awk '$3 > 50 {print}'

# Find zombie processes
ps aux | grep -E "Z\s"

# Check if service is running
systemctl status nginx | grep -q "active (running)" && echo "Nginx is UP"

# Monitor real-time logs for errors
tail -f /var/log/syslog | grep --line-buffered "error"

# Find listening ports
ss -tulpn | grep LISTEN
netstat -tulpn | grep LISTEN

Config File Management

# Find all config files with specific setting
grep -rn "max_connections" /etc/

# Validate nginx config (find server blocks)
grep -E "server\s*\{|listen|server_name" /etc/nginx/nginx.conf

# Find commented vs uncommented lines
grep -E "^[^#]*=" /etc/ssh/sshd_config  # Active settings only
grep -E "^#.*=" /etc/ssh/sshd_config    # Commented settings

# Check for insecure settings
grep -E "PermitRootLogin\s+yes|PasswordAuthentication\s+yes" /etc/ssh/sshd_config

CI/CD Pipeline Analysis

# Filter CI log for failed steps
cat ci_output.log | grep -E "FAILED|ERROR|error:"

# Extract test failures
grep -A 5 "FAILED" test_results.log

# Find deployment issues
grep -E "timeout|connection refused|permission denied" deploy.log

7. The Tool Trio: grep + awk + sed Pipelines

This is where pipelines become useful in real work.

Pipeline Flow Pattern

[INPUT] -> grep (FILTER) -> awk (EXTRACT) -> sed (TRANSFORM) -> [OUTPUT]

Example 1: Failed SSH Attack Analysis

Goal: Find top 10 IPs attacking via SSH, with count, sorted. 

grep "Failed password" /var/log/auth. log | \
    grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" | \
    sort | \
    uniq -c | \
    sort -rn | \
    head -10

# BREAKDOWN: 
# grep "Failed password"  -> Filter only failed login lines
# grep -oE "..."          -> Extract only IP addresses
# sort                    -> Group identical IPs together
# uniq -c                 -> Count occurrences
# sort -rn                -> Sort by count, descending
# head -10                -> Top 10 offenders

Example 2: Apache 404 Analysis with Full Pipeline

Goal: Get top 10 pages returning 404, with request count, formatted nicely.

grep " 404 " /var/log/apache2/access.log | \
    awk '{print $7}' | \
    sort | \
    uniq -c | \
    sort -rn | \
    head -10 | \
    awk '{printf "%-6s %s\n", $1, $2}'

# BREAKDOWN:
# grep " 404 "      -> Filter 404 responses
# awk '{print $7}'  -> Extract URL path (7th field in Apache log)
# sort | uniq -c    -> Count unique URLs
# sort -rn          -> Sort by frequency
# head -10          -> Top 10
# awk '{printf... }' -> Pretty format output

Example 3: Real-Time Log Monitoring with Alert

Goal: Monitor auth.log, alert on brute force attempts.

tail -f /var/log/auth.log | \
    grep --line-buffered "Failed password" | \
    grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" | \
    while read ip; do
        echo "[ALERT] Failed login attempt from:  $ip"
    done

# --line-buffered is CRUCIAL for real-time piping!

Warning Important: --line-buffered

Without --line-buffered, grep buffers output and your real-time monitoring won't work!

Example 4: Extract and Transform Nmap Data

Goal: Create clean report of open ports per host.

grep -E "Nmap scan report|open" nmap_scan.txt | \
    sed 's/Nmap scan report for /\n=== HOST:  /' | \
    sed 's/\/tcp.*open/\tPORT: /' | \
    awk '{if(/HOST/) print; else print "\t" $0}'

# BREAKDOWN:
# grep -E "..."    -> Get only host headers and open port lines
# sed (first)      -> Format host lines with "=== HOST:"
# sed (second)     -> Clean up port info
# awk              -> Indent port lines for readability

Example 5: Parse CSV with grep + awk

Goal: Find all users from "admin" department, show only name and email.

# Sample:  users.csv
# name,email,department,role
# john,[email protected],admin,superuser
# jane,jane@company. com,dev,user

grep ",admin," users.csv | \
    awk -F',' '{print $1, $2}'

# -F',' -> Set comma as field separator

Example 6: Clean and Process Web Server Stats

Goal: Get hourly request count from Apache log. 

awk '{print $4}' /var/log/apache2/access.log | \
    sed 's/\[//; s/: / /' | \
    awk '{print $2}' | \
    cut -d': ' -f1 | \
    sort | \
    uniq -c | \
    awk '{print "Hour " $2 ": 00 - " $1 " requests"}'

# BREAKDOWN:
# awk '{print $4}'    -> Extract timestamp field [10/Jan/2024:14:32:01
# sed 's/\[//...'     -> Remove [ and split date from time
# awk '{print $2}'    -> Get just time part
# cut -d':' -f1       -> Get just hour
# sort | uniq -c      -> Count per hour
# awk '{print... }'    -> Format output nicely

Example 7: Security Audit - Find Exposed Secrets

Goal: Scan codebase, find secrets, format as audit report.

grep -rnE "(password|api_key|secret|token)\s*=\s*['\"][^'\"]{8,}['\"]" /var/www/html/ 2>/dev/null | \
    awk -F:  '{print "FILE: " $1 "\nLINE: " $2 "\nCODE: " $3 "\n---"}' | \
    sed 's/password/[REDACTED]/gi'

# BREAKDOWN:
# grep -rnE "..."      -> Find secrets recursively with line numbers
# awk -F: '{... }'      -> Format as structured report (split on : )
# sed 's/password/...' -> Redact sensitive words in output

Example 8: Process ps output for Monitoring

Goal: Find high-memory processes, extract key info, format as table.

ps aux | \
    awk 'NR>1 && $4>1. 0 {print $4, $11}' | \
    sort -rn | \
    head -10 | \
    awk 'BEGIN {print "MEM%\tPROCESS\n----\t-------"} {print $1 "%\t" $2}'

# BREAKDOWN:
# ps aux                -> Get all processes
# awk 'NR>1 && $4>1.0'  -> Skip header, filter >1% memory
# sort -rn              -> Sort by memory usage
# head -10              -> Top 10
# awk 'BEGIN...'        -> Add header and format

8. Sample Logs for Practice

Create these sample files to practice your skills:

Sample auth.log

cat << 'EOF' > /tmp/sample_auth.log
Jan  4 10:15:01 kali sshd[1234]: Failed password for root from 192.168.1.100 port 22 ssh2
Jan  4 10:15:03 kali sshd[1234]: Failed password for root from 192.168.1.100 port 22 ssh2
Jan  4 10:15:05 kali sshd[1234]: Failed password for admin from 192.168.1.100 port 22 ssh2
Jan  4 10:16:01 kali sshd[1235]:  Accepted password for loki from 10.0.0.50 port 22 ssh2
Jan  4 10:17:22 kali sshd[1236]: Failed password for root from 45.33.32.156 port 22 ssh2
Jan  4 10:17:25 kali sshd[1236]: Failed password for root from 45.33.32.156 port 22 ssh2
Jan  4 10:17:28 kali sshd[1236]: Failed password for root from 45.33.32.156 port 22 ssh2
Jan  4 10:17:30 kali sshd[1236]: Failed password for root from 45.33.32.156 port 22 ssh2
Jan  4 10:17:33 kali sshd[1236]:  Failed password for root from 45.33.32.156 port 22 ssh2
Jan  4 10:18:00 kali sudo:  loki :  TTY=pts/0 ; PWD=/home/loki ; USER=root ; COMMAND=/bin/cat /etc/shadow
Jan  4 10:20:15 kali sshd[1237]:  Failed password for invalid user hacker from 203.0.113.42 port 22 ssh2
Jan  4 10:25:00 kali sshd[1238]:  Accepted publickey for loki from 10.0.0.50 port 22 ssh2
EOF

Sample access.log (Apache format)

cat << 'EOF' > /tmp/sample_access.log
192.168.1.50 - - [04/Jan/2024:10:00:01 +0000] "GET /index.html HTTP/1.1" 200 1234
192.168.1.50 - - [04/Jan/2024:10:00:02 +0000] "GET /admin HTTP/1.1" 301 512
45.33.32.156 - - [04/Jan/2024:10:01:15 +0000] "GET /../../../../etc/passwd HTTP/1.1" 400 300
45.33.32.156 - - [04/Jan/2024:10:01:16 +0000] "GET /admin' OR '1'='1 HTTP/1.1" 400 300
203.0.113.42 - - [04/Jan/2024:10:02:00 +0000] "GET /wp-admin HTTP/1.1" 404 250
203.0.113.42 - - [04/Jan/2024:10:02:01 +0000] "GET /phpmyadmin HTTP/1.1" 404 250
203.0.113.42 - - [04/Jan/2024:10:02:02 +0000] "GET /.git/config HTTP/1.1" 404 250
10.0.0.100 - - [04/Jan/2024:10:05:00 +0000] "POST /login HTTP/1.1" 200 100
10.0.0.100 - - [04/Jan/2024:10:05:30 +0000] "GET /dashboard HTTP/1.1" 200 5000
192.168.1.100 - - [04/Jan/2024:10:10:00 +0000] "GET /<script>alert(1)</script> HTTP/1.1" 400 300
EOF

Sample tcpdump-style output

cat << 'EOF' > /tmp/sample_tcpdump.log
10: 30:01.123456 IP 192.168.1.50.43210 > 10.0.0.1.80: Flags [S], seq 1234567890
10:30:01.124000 IP 10.0.0.1.80 > 192.168.1.50.43210: Flags [S. ], seq 987654321, ack 1234567891
10:30:01.124500 IP 192.168.1.50.43210 > 10.0.0.1.80: Flags [. ], ack 1
10:30:01.130000 IP 192.168.1.50.43210 > 10.0.0.1.80: Flags [P.], seq 1:100
10:30:05.000000 IP 45.33.32.156.12345 > 10.0.0.1.22: Flags [S], seq 111222333
10:30:05.001000 IP 45.33.32.156.12346 > 10.0.0.1.23: Flags [S], seq 111222334
10:30:05.002000 IP 45.33.32.156.12347 > 10.0.0.1.25: Flags [S], seq 111222335
10:30:05.003000 IP 45.33.32.156.12348 > 10.0.0.1.80: Flags [S], seq 111222336
10:30:05.004000 IP 45.33.32.156.12349 > 10.0.0.1.443: Flags [S], seq 111222337
10:30:10.000000 IP 192.168.1.50.43211 > 8.8.8.8.53: UDP, length 50
10:30:10.050000 IP 8.8.8.8.53 > 192.168.1.50.43211: UDP, length 100
EOF

9. Quick Reference Cheatsheet

Search & Display

grep "pattern" file          # Basic
grep -i "pattern" file       # Case insensitive
grep -r "pattern" /path/     # Recursive
grep -n "pattern" file       # Line numbers
grep -c "pattern" file       # Count

Advanced Filtering

grep -v "pattern" file       # Exclude
grep -E "pat1|pat2" file     # OR
grep -w "word" file          # Whole word
grep "^start" file           # Starts with
grep "end$" file             # Ends with

Extract Data

grep -o "pattern" file       # Only match
grep -oE "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+"  # IPs
grep -oP "[\w.-]+@[\w.-]+\.\w+"  # Emails

Context

grep -A 5 "pattern" file     # 5 after
grep -B 5 "pattern" file     # 5 before
grep -C 5 "pattern" file     # 5 both

File Management

grep -l "pattern" *.log      # List files
grep -L "pattern" *. log      # Non-matching
grep -r --include="*.py" "pattern" .

Regex Quick

.     # Any char     *   # 0+ times
+    # 1+ times     ?  # 0-1 times
^    # Start        $   # End
[]   # Class        |   # OR
\d   # Digit (PCRE) \w  # Word (PCRE)

Challenge 1: Basic Filtering

Using /tmp/sample_auth. log:

  1. Find all lines containing "Failed"
  2. Find all lines that do NOT contain "root"
  3. Count how many failed password attempts occurred
Show solutions 
# 1. Find all lines containing "Failed"
grep "Failed" /tmp/sample_auth.log

# 2. Find all lines that do NOT contain "root"
grep -v "root" /tmp/sample_auth.log

# 3. Count how many failed password attempts occurred
grep -c "Failed password" /tmp/sample_auth.log

Challenge 2: IP Extraction

Using /tmp/sample_auth.log:

  1. Extract all unique IP addresses
  2. Find the IP with most failed login attempts
  3. Show all activity from IP 45.33.32.156
Show solutions 
# 1. Extract all unique IP addresses
grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" /tmp/sample_auth.log | sort -u

# 2. Find the IP with most failed login attempts
grep "Failed" /tmp/sample_auth.log | \
    grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" | \
    sort | uniq -c | sort -rn | head -1

# 3. Show all activity from IP 45.33.32.156
grep "45.33.32.156" /tmp/sample_auth.log

Challenge 3: Attack Detection

Using /tmp/sample_access.log:

  1. Find all 404 responses
  2. Find potential SQL injection attempts
  3. Find potential directory traversal attempts
  4. Identify potential XSS attempts
Show solutions 
# 1. Find all 404 responses
grep " 404 " /tmp/sample_access.log

# 2. Find potential SQL injection attempts
grep -iE "or.*=.*|union.*select|drop.*table" /tmp/sample_access.log

# 3. Find potential directory traversal attempts
grep -E "\.\.\/" /tmp/sample_access.log

# 4. Identify potential XSS attempts
grep -iE "<script|alert\(" /tmp/sample_access.log

Challenge 4: Pipeline Practice

Using /tmp/sample_tcpdump.log:

  1. Extract all destination ports being scanned
  2. Find unique source IPs
  3. Count packets per source IP and format nicely
Show solutions 
# 1. Extract all destination ports being scanned by 45.33.32.156
grep "45.33.32.156" /tmp/sample_tcpdump.log | \
    grep -oE "> [0-9.]+\.[0-9]+:" | \
    awk -F'.' '{print $NF}' | tr -d ':'

# 2. Find unique source IPs
awk '{print $3}' /tmp/sample_tcpdump.log | \
    awk -F'.' '{print $1"."$2"."$3"."$4}' | sort -u

# 3. Count packets per source IP and format nicely
awk '{print $3}' /tmp/sample_tcpdump.log | \
    awk -F'.' '{print $1"."$2"."$3"."$4}' | \
    sort | uniq -c | \
    awk '{printf "%s packets from %s\n", $1, $2}'

Challenge 5: Full Security Analysis

BOSS LEVEL! Combine everything! Using all sample logs, create a security report showing:

  1. Number of failed SSH attempts per IP
  2. Suspicious web requests detected
  3. Possible port scan detected from which IP
Show solution 
#!/bin/bash
# Full Security Analysis Report

echo "+============================================================+"
echo "|            SECURITY ANALYSIS REPORT                      |"
echo "+============================================================+"
echo ""

echo "+-------------------------------------------------------------+"
echo "|  SSH BRUTE FORCE ATTEMPTS                                 |"
echo "+-------------------------------------------------------------+"
grep "Failed password" /tmp/sample_auth.log | \
    grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" | \
    sort | uniq -c | sort -rn | \
    awk '{printf "   Warning  %s attempts from %s\n", $1, $2}'

echo ""
echo "+-------------------------------------------------------------+"
echo "|  SUSPICIOUS WEB REQUESTS                                  |"
echo "+-------------------------------------------------------------+"
grep -iE "\.\./|<script|union.*select|'.*=.*'" /tmp/sample_access.log | \
    awk '{printf "    %s -> %s\n", $1, $7}'

echo ""
echo "+-------------------------------------------------------------+"
echo "|  PORT SCAN DETECTION                                      |"
echo "+-------------------------------------------------------------+"
awk '{print $3}' /tmp/sample_tcpdump.log | \
    awk -F'.' '{print $1"."$2"."$3"."$4}' | \
    sort | uniq -c | \
    awk '$1 > 3 {printf "    Possible scan from:  %s (%s packets)\n", $2, $1}'

echo ""
echo "==============================================================="
echo "                    Report complete"
echo "==============================================================="

11. Performance Tips & Best Practices

Speed Optimization

Tip Why Example
Use -F for literal strings No regex overhead - fastest mode grep -F "192.168.1.1" huge. log
Use --include/--exclude Skip unnecessary files grep -r --include="*.log" "error" /var/
Use -m to limit matches Stop after N matches grep -m 5 "error" huge.log
Use -l for filenames only Don't print matching lines grep -rl "secret" /code/
Pipe with head/tail Limit output early grep "pattern" huge.log | head -100
Use LC_all=C Byte-level matching - faster on ASCII LC_all=C grep "pattern" file

Real-World Speed Comparison

# SLOW:  Regex processing on every character
grep "192.168.1.1" massive_log.txt          # ~2.5s

# FAST: Fixed string, no regex
grep -F "192.168.1.1" massive_log.txt       # ~0.8s

# FASTEST: Fixed string + locale bypass
LC_all=C grep -F "192.168.1.1" massive_log. txt  # ~0.3s

12. Common Mistakes to Avoid

Common Errors to Avoid

Mistake Problem Fix
grep pattern * in deep dirs Misses nested files grep -r pattern .
Forgetting quotes Shell expands special chars Always quote: grep "pattern"
cat file | grep Useless use of cat (UUOC) grep pattern file
grep "a|b" for OR Literal | in BRE mode grep -E "a|b"
Regex in -F mode Pattern treated literally Remove -F or escape properly
Missing --line-buffered Real-time pipes don't stream tail -f log | grep --line-buffered
grep -r in / Searches EVERYTHING (slow) Be specific: grep -r pattern /var/log/

The "grep grep" Problem

# PROBLEM: grep finds itself in ps output! 
ps aux | grep "nginx"
# Shows:   nginx process AND the grep command itself

# SOLUTION 1: Exclude grep from results
ps aux | grep "nginx" | grep -v grep

# SOLUTION 2: Use character class trick (cleaner)
ps aux | grep "[n]ginx"
# The regex [n]ginx matches "nginx" but not "[n]ginx"

# SOLUTION 3: Use pgrep instead (best for processes)
pgrep -a nginx

13. grep Variants - Know Your Tools

Command Equivalent To Best For
grep Basic Regular Expressions (BRE) Simple searches
egrep grep -E Extended regex (deprecated, use -E)
fgrep grep -F Fixed strings, fastest (deprecated, use -F)
rgrep grep -r Recursive search (some systems)
zgrep grep for . gz files Compressed logs!
bzgrep grep for . bz2 files Bzip2 compressed files
xzgrep grep for . xz files XZ compressed files
pgrep Process grep Finding processes by name

Tip: Searching Compressed Logs

# Search through rotated/compressed logs without extracting! 
zgrep "Failed password" /var/log/auth.log. *. gz

# Search all auth logs (current + compressed)
zgrep "Failed password" /var/log/auth.log*

14. Modern Alternatives to grep

While grep is the classic, these modern tools offer extra features:

Tool Highlights Install
ripgrep (rg) Blazing fast, respects.gitignore, better defaults apt install ripgrep
ag (The Silver Searcher) Fast, code-search focused apt install silversearcher-ag
ack Designed for programmers, Perl-based apt install ack

ripgrep Quick Comparison

# grep:  Search recursively, show line numbers
grep -rn "password" /var/www/

# ripgrep: Same thing, but faster + smarter defaults
rg "password" /var/www/

# ripgrep auto-skips: .git, node_modules, binary files
# ripgrep auto-adds: colors, line numbers, file headers

When to Use What

Summary

grep is the filter tool. Use it to find the lines that matter before passing them to awk, sed, sort, or another command.

Key Takeaways

The Tool Trio Recap

grep

FILTER

"Find the lines"

awk

EXTRACT

"Get the columns"

sed

TRANSFORM

"Change the text"

Learn these three together and Linux text processing becomes much easier.

grep notes complete

Continue with sed and awk to build practical command-line pipelines.

Use grep to narrow noisy text into the few lines worth reading.

Practice with logs | Combine with awk | Clean output with sed