# File Handling in Bash Scripting

When writing bash scripts, it's common to need to work with files and directories. This includes tasks like reading and writing file contents, checking if a file exists, renaming or deleting files, and getting file metadata like permissions and size.

This article will demonstrate bash scripting techniques for effectively handling files and directories with well-commented examples.

## **Reading Files**

A common task is reading the contents of a file into a variable in your script. This allows you to store the file contents and process it later.

The `cat` command prints a file's contents to standard output. We can use command substitution to assign that output to a variable:

```bash
#!/bin/bash

# Store contents of the file in a variable
file_contents=$(cat file.txt)

echo "File contents: $file_contents"
```

This reads `file.txt` and assigns it to `$file_contents`. We echo the variable to print the contents.

For larger files, `cat` may be inefficient. An alternate is reading a file line-by-line with a while loop:

```bash
#!/bin/bash

# Read file line-by-line
while read line; do
  echo "$line" 
done < "input.txt"
```

This loops through each line of `input.txt`, assigning it to `$line` and echoing it.

## **Writing to Files**

To write to a file, use the `>` redirect operator:

```bash
#!/bin/bash 

# Write output to file
echo "This is some text" > outfile.txt
```

This writes "This is some text" into `outfile.txt`, overwriting any existing content.

To append instead of overwrite, use `>>`:

```bash
#!/bin/bash

# Append output to the end of file 
echo "Appending this line" >> appendfile.txt
```

You can also use `tee` to write to a file and also print to standard out at the same time:

```bash
echo "This gets printed and logged" | tee logfile.txt
```

## **Checking if a File Exists**

To avoid errors, check if a file exists before accessing it:

```bash
#!/bin/bash

# Check if file exists
if [ -f "file.txt" ]; then
  echo "File found" 
else
  echo "File not found"
fi
```

The `-f` flag for the `if` condition checks if the file exists and is a regular file. Other test flags include:

* `-d` - Directory exists
    
* `-e` - File exists (regular file or directory)
    
* `-s` - File exists and size is greater than 0
    

## **Getting File Metadata**

You can get useful metadata about files like size, permissions, etc.

For example, get a file's size in bytes:

```bash
#!/bin/bash

# Get file size
filesize=$(stat -c%s "file.txt")
echo "Size: $filesize bytes"
```

The `stat` command returns file info, and `%s` prints just the size.

To get a file's permissions:

```bash
#!/bin/bash

# Get file permissions
perms=$(stat -c%A "file.txt")
echo "Permissions: $perms"
```

`%A` prints the permissions in symbolic format. Other useful formats are `%U` for the user and `%G` for the group.

## **Renaming and Deleting Files**

You can rename or delete files and directories from within scripts.

**To rename a file:**

```bash
#!/bin/bash

# Rename file 
mv oldname.txt newname.txt
```

**To delete:**

```bash
#!/bin/bash

# Delete file
rm filename.txt
```

And to remove a directory and all its contents recursively:

```bash
#!/bin/bash

# Delete directory recursively
rm -r directoryname
```

### Other File Handling Commands

There are a few more useful bash commands that can be included when covering file handling in bash scripting:

* `cp` - Copy files and directories. Useful for backing up files or setting up test data. Example:
    
    ```bash
    cp file.txt file_copy.txt
    ```
    
* `mkdir` - Make new directories. Often needed to create output directories for scripts:
    
    ```bash
    mkdir /path/to/newdir
    ```
    
* `find` - Search for files recursively matching given criteria. Helps locate files based on name, size, permissions, etc:
    
    ```bash
    find /home -name "*.txt"
    ```
    
* `head`/`tail` - Read first or last N lines of a file. Useful for previewing file contents:
    
    ```bash
    head -n 10 file.txt 
    tail -n 5 file.txt
    ```
    
* `touch` - Update file timestamps or create empty files. Can force a file update:
    
    ```bash
    touch file.txt
    ```
    
* `chmod` - Change file permissions like make executable:
    
    ```bash
    chmod +x script.sh
    ```
    
* `gzip`/`gunzip` - Compress and extract gzip files:
    
    ```bash
    gzip file.txt
    gunzip file.txt.gz
    ```
    
* `tar` - Archive multiple files and dirs into a tar file:
    
    ```bash
    tar -cvf archive.tar /path/to/files
    ```
    

* Reading CSV files:
    
    ```bash
    # Read CSV file line-by-line
    while IFS="," read -r col1 col2 col3; do
      echo "$col1 | $col2 | $col3"
    done < file.csv
    ```
    
    Use IFS (input field separator) to split CSV rows into columns.
    
* Parse XML/JSON with jq:
    
    ```bash
    # Extract JSON value
    name=$(cat file.json | jq -r '.name')
    ```
    
    jq allows easy JSON parsing to extract values in scripts.
    
* File globbing - Match patterns with \* and ?:
    
    ```bash
    # Copy files matching pattern
    cp /path/*.txt /backup/
    ```
    
* Check file contents with grep:
    
    ```bash
    # Grep for match in file
    if grep -q "some_text" file.txt; then
      echo "Found"
    fi
    ```
    
    Use grep to search file contents.
    
* Create temp files and dirs:
    
    ```bash
    # Create temp file
    tmpfile=$(mktemp)
    ```
    
    mktemp makes disposable temp files.
    
* Stdout/stderr redirection:
    
    ```bash
    # Redirect stdout and stderr 
    program >out.txt 2>err.txt
    ```
    
    Send stdout and stderr output to different files.
    
* Named pipes for inter-process communication:
    
    ```bash
    # Read from named pipe
    mkfifo mypipe
    cat mypipe
    ```
    
    Multiple processes can communicate via named pipes.
    

These provide additional file and directory manipulation capabilities that can be useful in bash scripts. The key concepts remain using redirects, loops, command substitution, and conditional checks for robust file handling.

So in summary, bash provides many useful commands for reading, writing, checking files, getting metadata, renaming/deleting and more. Mastering file handling will enable you to write more powerful scripts.

The key is knowing which commands like `cat`, redirects, `stat`, `mv`, `rm` to use for different file tasks. Refer to this article for syntax examples.