Compare commits

..

No commits in common. "3a0684c33e20a6e4263c8a73ee43fa5133b3fd87" and "84be854186e9d4a82547393098762b4b6fca835d" have entirely different histories.

8 changed files with 600 additions and 603 deletions

View File

@ -1,17 +1,33 @@
# FileMonitor # # FileMonitor
This was a Rust Challenge that I completed in 17 hours. This was a Rust Challenge that I completed in 17 hours.
The [document](./docs/rust_coding_challenge_filemonitor.pdf) ## Review ##
describing the challenge can be found in the docs directory. I thought I solved the challenge pretty well. I took my time and went back
and refactored and commented well.
Notify was a crate I hadn't used before so that took some time to understand
and then determine how I wanted to wrap it up. I like the solution that I
came up with because it keeps most of the notify mess contained in one file
with just a little seepage out when it has to be used for monitoring.
## Platform ## One thing I was not happy about was that I had very poor git commiting during
This was created on Ubuntu and was only tested there. It should the challenge. I think it was because I was trying to keep my time lower, but
work on most other platforms, but without testing it, I do not would it really have hurt to spend a little bit of time to commit and give
know for certain. myself better backups incase of a big problem? Probably not.
This let me work on and understand rust async. I had read the rust book and
the tokio tutorial completely before doing this challenge. It wasn't that
bad. It reminds me of coding for threads a lot, which is good, but sometimes
the await gets me because I don't understand where it is blocked when it is
running sometimes, tokio-console helps here, and why calls block or what they
are blocking for. For example, why does tokio::io::stdout().write_all() seem
to block forever? It is given a few bytes to write to out, it should be able
to dump those to the console quickly enough.
## Build ## Really, i think my problem with the async IO calls was that they needed to
To build it just use a standard Cargo install with the current happen in a seperate task and be listening for messages on what to print.
Rust compiler. Make sure your Cargo is set for Crates.IO as That way they are never able to block a different task.
your registry because it uses several crates from there.
Next I think I should checkout the rayon crate, but we'll see what the next
challenge I get is.

View File

@ -1,33 +0,0 @@
# Review #
I thought I solved the challenge pretty well. I took my time and went back
and refactored and commented well. The overall amount of time I spent solving
it, 17 hours, may be rather problematic. I did have to learn some things
while I was working on it, but it took what it took.
Notify was a crate I hadn't used before so that took some time to understand
and then determine how I wanted to wrap it up. I like the solution that I
came up with because it keeps most of the notify mess contained in one file
with just a little seepage out when it has to be used for monitoring. Perhaps
I should have fully wrapped the library? It wouldn't have taken much more to
do so, it just didn't seem necessary to solving the problem.
One thing I was not happy about was that I had very poor git commiting during
the challenge. I think it was because I was trying to keep my time lower, but
would it really have hurt to spend a little bit of time to commit and give
myself better backups incase of a big problem? Probably not.
Ultimately, his let me work on and understand rust async. I had read the rust
book and the tokio tutorial completely before doing this challenge. It wasn't
that bad. It reminds me of coding for threads a lot, which is good, but
sometimesthe await gets me because I don't understand where it is blocked when
it is running sometimes, tokio-console helps here, and why calls block or what
they are blocking for. For example, why does tokio::io::stdout().write_all()
seem to block forever? It is given a few bytes to write to out, it should be
able to dump those to the console quickly enough.
Really, i think my problem with the async IO calls was that they needed to
happen in a seperate task and be listening for messages on what to print.
That way they are never able to block a different task.
Next I think I should checkout the rayon crate, but we'll see what the next
challenge I get is.

View File

@ -19,7 +19,7 @@ pub struct DirMonitor
dir: PathBuf, dir: PathBuf,
/// The files that are being monitored within the directory. /// The files that are being monitored within the directory.
monitored_files: MonitoredFiles monitored_files: MonitoredFiles,
} }
@ -31,8 +31,11 @@ impl DirMonitor
/// monitor_path: A string representation of directory to monitor. /// monitor_path: A string representation of directory to monitor.
pub fn new(monitor_path: &str) -> Self pub fn new(monitor_path: &str) -> Self
{ {
DirMonitor { dir: PathBuf::from(monitor_path), DirMonitor
monitored_files: MonitoredFiles::new() } {
dir: PathBuf::from(monitor_path),
monitored_files: MonitoredFiles::new()
}
} }
/// Scan the monitored directory and all of its sub directories and monitors /// Scan the monitored directory and all of its sub directories and monitors
@ -45,7 +48,8 @@ impl DirMonitor
match scan_dir(&self.dir, &mut self.monitored_files, &self.dir) match scan_dir(&self.dir, &mut self.monitored_files, &self.dir)
{ {
Ok(_) => Ok(_) =>
{} {
}
Err(e) => Err(e) =>
{ {
@ -65,34 +69,30 @@ impl DirMonitor
println!("{}", self.monitored_files); println!("{}", self.monitored_files);
} }
/// Start monitoring the directory asyncronously and process changes within /// Start monitoring the directory asyncronously and process changes within the directory
/// the directory until a termination message is received. /// until a termination message is received.
pub async fn monitor(&mut self, pub async fn monitor(&mut self, mut term_receiver: tokio::sync::watch::Receiver<bool>)
mut term_receiver: tokio::sync::watch::Receiver<bool>)
{ {
let mut running: bool = true; let mut running: bool = true;
// Setup the notify crate to watch the INBOX directory. // Setup the notify crate to watch the INBOX directory.
let tokio_runtime = tokio::runtime::Handle::current(); let tokio_runtime = tokio::runtime::Handle::current();
let (notify_sender, mut notify_receiver) = let (notify_sender, mut notify_receiver) = tokio::sync::mpsc::channel::<Event>(10);
tokio::sync::mpsc::channel::<Event>(10);
let wrapped_sender = NotifySender::new(tokio_runtime, notify_sender); let wrapped_sender = NotifySender::new(tokio_runtime, notify_sender);
let mut fs_watcher: RecommendedWatcher = let mut fs_watcher: RecommendedWatcher =
match notify::recommended_watcher(wrapped_sender) match notify::recommended_watcher(wrapped_sender)
{ {
Ok(watcher) => watcher, Ok(watcher) => { watcher }
Err(e) => Err(e) =>
{ {
// Just panic because we cannot watch the directories so the // Just panic because we cannot watch the directories so the program
// program would be useless and this saves from // would be useless and this saves from having to press CTRL-C.
// having to press CTRL-C.
eprintln!("Unable to create watcher: {}", e); eprintln!("Unable to create watcher: {}", e);
panic!(); panic!();
} }
}; };
if let Err(e) = if let Err(e) = fs_watcher.watch(&self.dir, notify::RecursiveMode::Recursive)
fs_watcher.watch(&self.dir, notify::RecursiveMode::Recursive)
{ {
// Just panic because we cannot watch the directories so the program // Just panic because we cannot watch the directories so the program
// would be useless and this saves from having to press CTRL-C. // would be useless and this saves from having to press CTRL-C.
@ -107,7 +107,8 @@ impl DirMonitor
// We are listening for messages from either the notify receiver or // We are listening for messages from either the notify receiver or
// the termination receiver. When ever one of them comes across we // the termination receiver. When ever one of them comes across we
// will process it. // will process it.
tokio::select! { tokio::select!
{
// Handle listening for the notify watcher events. // Handle listening for the notify watcher events.
// These are the changes that we care about from the file system. // These are the changes that we care about from the file system.
Some(mut event) = notify_receiver.recv() => Some(mut event) = notify_receiver.recv() =>
@ -131,10 +132,9 @@ impl DirMonitor
/// Scans a directory, and all of its sub directories, and creates a list of the /// Scans a directory, and all of its sub directories, and creates a list of the files
/// files inside and their last modification time. /// inside and their last modification time.
fn scan_dir(base_dir: &Path, monitored_files: &mut MonitoredFiles, dir: &Path) fn scan_dir(base_dir: &Path, monitored_files: &mut MonitoredFiles, dir: &Path) -> std::io::Result<()>
-> std::io::Result<()>
{ {
let mut dir_list: Vec<PathBuf> = Vec::new(); let mut dir_list: Vec<PathBuf> = Vec::new();
@ -150,8 +150,7 @@ fn scan_dir(base_dir: &Path, monitored_files: &mut MonitoredFiles, dir: &Path)
// Files will be added to the list of monitored files and their // Files will be added to the list of monitored files and their
// last modification time will be stored. // last modification time will be stored.
// //
// TODO: Handle symlinks. Symlinks can be either a file or another // TODO: Handle symlinks. Symlinks can be either a file or another directory.
// directory.
if meta.is_dir() if meta.is_dir()
{ {
dir_list.push(file.path().clone()); dir_list.push(file.path().clone());
@ -160,8 +159,7 @@ fn scan_dir(base_dir: &Path, monitored_files: &mut MonitoredFiles, dir: &Path)
{ {
let last_mod_time: std::time::SystemTime = meta.modified()?; let last_mod_time: std::time::SystemTime = meta.modified()?;
monitored_files.add(&strip_path_prefix(&file.path(), base_dir), monitored_files.add(&strip_path_prefix(&file.path(), base_dir), last_mod_time);
last_mod_time);
} }
} }

View File

@ -68,9 +68,7 @@ impl Event
{ {
*path = strip_path_prefix(&path, base_dir); *path = strip_path_prefix(&path, base_dir);
} }
Event::Rename { from, Event::Rename { from, to, mod_time: _ } =>
to,
mod_time: _ } =>
{ {
*from = strip_path_prefix(&from, base_dir); *from = strip_path_prefix(&from, base_dir);
*to = strip_path_prefix(&to, base_dir); *to = strip_path_prefix(&to, base_dir);
@ -98,9 +96,7 @@ impl std::fmt::Display for Event
{ {
write!(f, "[MOD] {}", path.display()) write!(f, "[MOD] {}", path.display())
} }
Event::Rename { from, Event::Rename { from, to, mod_time: _ } =>
to,
mod_time: _ } =>
{ {
writeln!(f, "[DEL] {}", from.display())?; writeln!(f, "[DEL] {}", from.display())?;
write!(f, "[NEW] {}", to.display()) write!(f, "[NEW] {}", to.display())

View File

@ -11,6 +11,7 @@ mod util;
use clap::Parser; use clap::Parser;
use tokio::sync::watch; use tokio::sync::watch;
use crate::dir_monitor::DirMonitor; use crate::dir_monitor::DirMonitor;
@ -47,7 +48,10 @@ fn create_directory(dir: &str) -> Option<String>
match std::fs::create_dir_all(inbox_dir.clone()) match std::fs::create_dir_all(inbox_dir.clone())
{ {
Ok(_) => Some(inbox_dir), Ok(_) =>
{
Some(inbox_dir)
}
Err(e) => Err(e) =>
{ {
@ -95,7 +99,8 @@ async fn main()
directory_monitor.print_monitored_files(); directory_monitor.print_monitored_files();
// Run our directory monitor. // Run our directory monitor.
let mon_task = tokio::spawn(async move { let mon_task = tokio::spawn(async move
{
// Program workflow step 2 (terminated by step 3). // Program workflow step 2 (terminated by step 3).
// This can be done async because we are waiting on // This can be done async because we are waiting on
// events to be loaded from the notify library and // events to be loaded from the notify library and
@ -117,7 +122,8 @@ async fn main()
// Run until Ctrl-C is pressed. // Run until Ctrl-C is pressed.
// Once it is pressed it will send a message to stop. // Once it is pressed it will send a message to stop.
let _term_task = tokio::spawn(async move { let _term_task = tokio::spawn(async move
{
// Program workflow step 3. // Program workflow step 3.
// Async with message passing so it can run and listen // Async with message passing so it can run and listen
// for the termination signal. It uses a watch channel // for the termination signal. It uses a watch channel
@ -130,12 +136,8 @@ async fn main()
// finish once the termination task has finished. // finish once the termination task has finished.
match mon_task.await match mon_task.await
{ {
Ok(_) => Ok(_) => {}
{} Err(e) => { eprintln!("Error during monitoring task: {}", e); }
Err(e) =>
{
eprintln!("Error during monitoring task: {}", e);
}
} }
} }
} }

View File

@ -1,5 +1,5 @@
use std::collections::HashMap; use std::collections::HashMap;
use std::path::{Path, PathBuf}; use std::path::{PathBuf, Path};
use std::time::SystemTime; use std::time::SystemTime;
use chrono::{DateTime, Local}; use chrono::{DateTime, Local};
@ -22,7 +22,10 @@ impl MonitoredFiles
/// Create a new empty set of monitored files. /// Create a new empty set of monitored files.
pub fn new() -> Self pub fn new() -> Self
{ {
MonitoredFiles { files: HashMap::new() } MonitoredFiles
{
files: HashMap::new()
}
} }
/// Process an event and change the monitored files. /// Process an event and change the monitored files.
@ -35,7 +38,7 @@ impl MonitoredFiles
self.add(&path, mod_time); self.add(&path, mod_time);
} }
Event::Modify { path, mod_time } => Event::Modify { path, mod_time} =>
{ {
self.modify(&path, mod_time); self.modify(&path, mod_time);
} }
@ -87,17 +90,11 @@ impl std::fmt::Display for MonitoredFiles
if !it.peek().is_none() if !it.peek().is_none()
{ {
writeln!(f, writeln!(f, "[{}] {}", date_time.format("%m/%d/%Y %H:%M"), path.display())?;
"[{}] {}",
date_time.format("%m/%d/%Y %H:%M"),
path.display())?;
} }
else else
{ {
write!(f, write!(f, "[{}] {}", date_time.format("%m/%d/%Y %H:%M"), path.display())?;
"[{}] {}",
date_time.format("%m/%d/%Y %H:%M"),
path.display())?;
} }
} }

View File

@ -26,8 +26,11 @@ impl NotifySender
sender: tokio::sync::mpsc::Sender<crate::event::Event>) sender: tokio::sync::mpsc::Sender<crate::event::Event>)
-> Self -> Self
{ {
NotifySender { tokio_runtime, NotifySender
sender } {
tokio_runtime,
sender
}
} }
} }
@ -47,9 +50,9 @@ impl notify::EventHandler for NotifySender
{ {
let notify_sender = self.sender.clone(); let notify_sender = self.sender.clone();
self.tokio_runtime.spawn(async move { self.tokio_runtime.spawn(async move
if let Err(e) = {
notify_sender.send(monitor_event).await if let Err(e) = notify_sender.send(monitor_event).await
{ {
eprintln!("Notify EventHandler: {}", e); eprintln!("Notify EventHandler: {}", e);
} }
@ -85,19 +88,21 @@ fn process_notify_event(event: notify::Event) -> Option<crate::event::Event>
process_remove_event(kind, event.paths) process_remove_event(kind, event.paths)
} }
_ => None _ => { None }
} }
} }
/// Process the Create event into a New event. /// Process the Create event into a New event.
fn process_create_event(event: notify::event::CreateKind, paths: Vec<PathBuf>) fn process_create_event(event: notify::event::CreateKind,
paths: Vec<PathBuf>)
-> Option<crate::event::Event> -> Option<crate::event::Event>
{ {
if event == notify::event::CreateKind::File if event == notify::event::CreateKind::File
{ {
let mod_time: std::time::SystemTime = match get_file_mod_time(&paths[0]) let mod_time: std::time::SystemTime =
match get_file_mod_time(&paths[0])
{ {
Ok(time) => time, Ok(time) => { time }
Err(e) => Err(e) =>
{ {
@ -107,31 +112,33 @@ fn process_create_event(event: notify::event::CreateKind, paths: Vec<PathBuf>)
} }
}; };
return Some(crate::event::Event::New { path: paths[0].clone(), return Some(crate::event::Event::New
mod_time: mod_time }); {
path: paths[0].clone(),
mod_time: mod_time
});
} }
None None
} }
/// Process the modify event into either a Modify event or a Rename event. /// Process the modify event into either a Modify event or a Rename event.
fn process_modify_event(event: notify::event::ModifyKind, paths: Vec<PathBuf>) fn process_modify_event(event: notify::event::ModifyKind,
paths: Vec<PathBuf>)
-> Option<crate::event::Event> -> Option<crate::event::Event>
{ {
match event match event
{ {
// This is just handling the events. On my system they came across as Any // This is just handling the events. On my system they came across as Any for
// for the MetadataKind, but the documentation seemed to hint that // the MetadataKind, but the documentation seemed to hint that modification
// modification times should be on write events. So for the sake of // times should be on write events. So for the sake of being done without testing
// being done without testing this on different platforms the Any // this on different platforms the Any modification event will trigger a time lookup.
// modification event will trigger a time lookup. notify::event::ModifyKind::Any | notify::event::ModifyKind::Metadata(_) =>
notify::event::ModifyKind::Any |
notify::event::ModifyKind::Metadata(_) =>
{ {
let mod_time: std::time::SystemTime = match get_file_mod_time(&paths let mod_time: std::time::SystemTime =
[0]) match get_file_mod_time(&paths[0])
{ {
Ok(time) => time, Ok(time) => { time }
Err(e) => Err(e) =>
{ {
@ -141,8 +148,11 @@ fn process_modify_event(event: notify::event::ModifyKind, paths: Vec<PathBuf>)
} }
}; };
Some(crate::event::Event::Modify { path: paths[0].clone(), Some(crate::event::Event::Modify
mod_time: mod_time }) {
path: paths[0].clone(),
mod_time: mod_time
})
} }
// Handling a rename event so that during testing when a file is // Handling a rename event so that during testing when a file is
@ -158,27 +168,32 @@ fn process_modify_event(event: notify::event::ModifyKind, paths: Vec<PathBuf>)
let mod_time: std::time::SystemTime = let mod_time: std::time::SystemTime =
match get_file_mod_time(&paths[1]) match get_file_mod_time(&paths[1])
{ {
Ok(time) => time, Ok(time) => { time }
Err(e) => Err(e) =>
{ {
eprintln!("Unable to open file to retrieve \ eprintln!("Unable to open file to retrieve modification time: {}",
modification time: {}",
e); e);
std::time::SystemTime::now() std::time::SystemTime::now()
} }
}; };
Some(crate::event::Event::Rename { from: paths[0].clone(), Some(crate::event::Event::Rename
{
from: paths[0].clone(),
to: paths[1].clone(), to: paths[1].clone(),
mod_time: mod_time }) mod_time: mod_time
})
} }
// This is for when a file is renamed and only the from // This is for when a file is renamed and only the from
// file is in the monitored directories. // file is in the monitored directories.
notify::event::RenameMode::From => notify::event::RenameMode::From =>
{ {
Some(crate::event::Event::Delete { path: paths[0].clone() }) Some(crate::event::Event::Delete
{
path: paths[0].clone(),
})
} }
// This is for when a file is renamed and both the to file // This is for when a file is renamed and both the to file
@ -188,36 +203,42 @@ fn process_modify_event(event: notify::event::ModifyKind, paths: Vec<PathBuf>)
let mod_time: std::time::SystemTime = let mod_time: std::time::SystemTime =
match get_file_mod_time(&paths[0]) match get_file_mod_time(&paths[0])
{ {
Ok(time) => time, Ok(time) => { time }
Err(e) => Err(e) =>
{ {
eprintln!("Unable to open file to retrieve \ eprintln!("Unable to open file to retrieve modification time: {}",
modification time: {}",
e); e);
std::time::SystemTime::now() std::time::SystemTime::now()
} }
}; };
Some(crate::event::Event::New { path: paths[0].clone(), Some(crate::event::Event::New
mod_time: mod_time }) {
path: paths[0].clone(),
mod_time: mod_time
})
} }
_ => None _ => { None }
} }
} }
_ => None _ => { None }
} }
} }
/// Process the remove event into a Delete event. /// Process the remove event into a Delete event.
fn process_remove_event(event: notify::event::RemoveKind, paths: Vec<PathBuf>) fn process_remove_event(event: notify::event::RemoveKind,
paths: Vec<PathBuf>)
-> Option<crate::event::Event> -> Option<crate::event::Event>
{ {
if event == notify::event::RemoveKind::File if event == notify::event::RemoveKind::File
{ {
return Some(crate::event::Event::Delete { path: paths[0].clone() }); return Some(crate::event::Event::Delete
{
path: paths[0].clone(),
});
} }
None None

View File

@ -2,8 +2,7 @@
/// ///
/// If it is unable to get the modification time from the file system /// If it is unable to get the modification time from the file system
/// it will default to the current system time. /// it will default to the current system time.
pub fn get_file_mod_time(path: &std::path::Path) pub fn get_file_mod_time(path: &std::path::Path) -> std::io::Result<std::time::SystemTime>
-> std::io::Result<std::time::SystemTime>
{ {
let file = std::fs::File::open(path)?; let file = std::fs::File::open(path)?;
let meta = file.metadata()?; let meta = file.metadata()?;
@ -16,10 +15,8 @@ pub fn get_file_mod_time(path: &std::path::Path)
Ok(std::time::SystemTime::now()) Ok(std::time::SystemTime::now())
} }
/// Create a new PathBuf that is the same as path but has the base removed if it /// Create a new PathBuf that is the same as path but has the base removed if it can.
/// can. pub fn strip_path_prefix(path: &std::path::Path, base: &std::path::Path) -> std::path::PathBuf
pub fn strip_path_prefix(path: &std::path::Path, base: &std::path::Path)
-> std::path::PathBuf
{ {
match &path.strip_prefix(base) match &path.strip_prefix(base)
{ {
@ -31,6 +28,9 @@ pub fn strip_path_prefix(path: &std::path::Path, base: &std::path::Path)
final_path final_path
} }
Err(_) => std::path::PathBuf::from(path) Err(_) =>
{
std::path::PathBuf::from(path)
}
} }
} }