Walking directories in Rust - Watching file changes

Walking directories is a common requirement for many applications: sometimes we need to search for a file, inspect metadata, or —as in this case— watch for file changes.

I will build a simple file watcher for preconfigured directories, the goal is track the changes and detect when a file is changed. This will support exclusions and many directories.

Algorithm

There are two common algorithms for walking directories, Depth-First Search (DFS), and Breadth-First Search (BFS).

• Depth-First Search (DFS): Explore as far as possible along each branch before backtracking. It fully explores a subdirectory before moving to the next sibling directory. This is often implemented recursively.
• Breadth-First Search (BFS): Explore all the nodes at the current depth level before moving on to the nodes at the next depth level. It visits all immediate subdirectories before delving into their contents. This is typically implemented using a queue.

For a file watcher, DFS is usually more efficient: most file changes happen deeper in subdirectories, so going deep first reduces unnecessary checks. I will implement the DFS. This is the diagram of the algorithm:

	%%{init: {'theme':'dark'}}%%
	flowchart TB
	A[Start] --> B[read elements in dir]
	B --> C{more entries?}
	C -- No --> Z[return changes]

	C -- Yes --> D[entry ← next]
	D --> E{entry ends with exclusion?}
	E -- Yes --> C

	E -- No --> F{is directory?}
	F -- Yes --> G[recurse]
	G --> H[append child changes]
	H --> C

	F -- No --> I[meta ← metadata entry]
	I --> J{modified > last_check?}
	J -- Yes --> K[push Change]
	J -- No --> C
	K --> C

Implementation

First, I define a custom error type JCodeError and a Change struct. I also introduce a type alias JCodeResult<T> for convenience, so functions can return Result<T, JCodeError> more concisely.

The Change struct contains the path of each element (directory or file) and the timestamp of the last modification. A Change element will be created when a change is detected.

use std::error::Error;
use std::fmt::Display;
use std::path::{Path, PathBuf};
use std::time::SystemTime;

// Optional: Custom Error type
type JCodeResult<T> = Result<T, JCodeError>;

#[derive(Debug)]
struct JCodeError(String);

impl From<std::io::Error> for JCodeError {
	fn from(value: std::io::Error) -> Self {
		Self(format!("{value}"))
	}
}

impl Display for JCodeError {
	fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
		write!(f, "{}", self.0)
	}
}

impl Error for JCodeError {}

/// A change made in a file, including the path of the
/// modified file and the timestamp of when it was modified.
#[derive(Debug)]
struct Change {
	path: PathBuf,
	timestamp: SystemTime,
}

impl Change {
	fn new(path: PathBuf, timestamp: SystemTime) -> Self {
		Self { path, timestamp }
	}
}

Then the core of the recursive algorithm:

use std::fs::{self, OpenOptions};
use std::path::{Path, PathBuf};
use std::time::SystemTime;

const EXCLUSIONS: &[&str] = &["target", ".venv", "node_modules", ".git", ".DS_Store"];

/// Get all changes from a directory, exploring the subdirectories.
///
/// # Return
/// A [`Vec`] of [`Change`] containing all the modified files
fn inspect_dir_for_changes(dir: &Path, last_check: SystemTime) -> JCodeResult<Vec<Change>> {
	let mut changed = Vec::new();

	// Verify the root is a dir
	if !dir.is_dir() {
		eprintln!("{} must be a dir", dir.to_string_lossy());
		std::process::exit(1);
	}

	// Walk the directory
	'parent: for element in fs::read_dir(dir)? {
		let element = element?;
		let element_path = element.path();

		// If it is an exclusion, skip and check the next element
		for exc in EXCLUSIONS {
			if element_path.ends_with(exc) {
				continue 'parent;
			}
		}

		// If the element is a dir, recurse
		if element_path.is_dir() {
			let mut changes = inspect_dir_for_changes(&element_path, last_check)?;
			if changes.is_empty() {
				continue;
			}

			changed.append(&mut changes);
		} else {
			// If it is a file, verify if it has been modified
			let meta = element_path.metadata()?;
			let modified = meta.modified()?;

			if modified > last_check {
				changed.push(Change::new(element_path, modified));
			}
		}
	}

	Ok(changed)
}

Now is ok! An example of the usage is this:

// Dirs to watch
const DIRS: &[&str] = &["/Users/richard/proj", "/Users/richard/dev"];

fn main() {
	loop {
		let last_check = SystemTime::now();
		// Check interval: 10 seconds
		std::thread::sleep(std::time::Duration::from_secs(10));

		for dir in DIRS {
			println!("Inspecting {dir}...");
			let changes = inspect_dir_for_changes(&PathBuf::from(dir), last_check).unwrap();
			if changes.is_empty() {
				println!("No changes detected");
			} else {
				println!("Changes detected: {:?}", changes);
				// Here we can save the data to a database, to a file
				// or trigger a hook
			}
		}
	}
}

With this simple approach, we can track file changes and trigger hooks to perform actions. This is useful for many tasks such as live-reloading servers, tracking modifications, or synchronizing files.

Some improvements and next steps to make:

• EXCLUSIONS, DIRS and interval are hardcoded, those data can be retrieved from a config file, like a toml and parsed with serde.
• This implementation can be wrapped in a struct for better abstraction.