Compare commits
No commits in common. "3a0684c33e20a6e4263c8a73ee43fa5133b3fd87" and "84be854186e9d4a82547393098762b4b6fca835d" have entirely different histories.
3a0684c33e
...
84be854186
38
README.md
38
README.md
@ -1,17 +1,33 @@
|
||||
# FileMonitor #
|
||||
# FileMonitor
|
||||
|
||||
This was a Rust Challenge that I completed in 17 hours.
|
||||
|
||||
The [document](./docs/rust_coding_challenge_filemonitor.pdf)
|
||||
describing the challenge can be found in the docs directory.
|
||||
## Review ##
|
||||
I thought I solved the challenge pretty well. I took my time and went back
|
||||
and refactored and commented well.
|
||||
|
||||
Notify was a crate I hadn't used before so that took some time to understand
|
||||
and then determine how I wanted to wrap it up. I like the solution that I
|
||||
came up with because it keeps most of the notify mess contained in one file
|
||||
with just a little seepage out when it has to be used for monitoring.
|
||||
|
||||
## Platform ##
|
||||
This was created on Ubuntu and was only tested there. It should
|
||||
work on most other platforms, but without testing it, I do not
|
||||
know for certain.
|
||||
One thing I was not happy about was that I had very poor git commiting during
|
||||
the challenge. I think it was because I was trying to keep my time lower, but
|
||||
would it really have hurt to spend a little bit of time to commit and give
|
||||
myself better backups incase of a big problem? Probably not.
|
||||
|
||||
This let me work on and understand rust async. I had read the rust book and
|
||||
the tokio tutorial completely before doing this challenge. It wasn't that
|
||||
bad. It reminds me of coding for threads a lot, which is good, but sometimes
|
||||
the await gets me because I don't understand where it is blocked when it is
|
||||
running sometimes, tokio-console helps here, and why calls block or what they
|
||||
are blocking for. For example, why does tokio::io::stdout().write_all() seem
|
||||
to block forever? It is given a few bytes to write to out, it should be able
|
||||
to dump those to the console quickly enough.
|
||||
|
||||
## Build ##
|
||||
To build it just use a standard Cargo install with the current
|
||||
Rust compiler. Make sure your Cargo is set for Crates.IO as
|
||||
your registry because it uses several crates from there.
|
||||
Really, i think my problem with the async IO calls was that they needed to
|
||||
happen in a seperate task and be listening for messages on what to print.
|
||||
That way they are never able to block a different task.
|
||||
|
||||
Next I think I should checkout the rayon crate, but we'll see what the next
|
||||
challenge I get is.
|
||||
|
@ -1,33 +0,0 @@
|
||||
# Review #
|
||||
I thought I solved the challenge pretty well. I took my time and went back
|
||||
and refactored and commented well. The overall amount of time I spent solving
|
||||
it, 17 hours, may be rather problematic. I did have to learn some things
|
||||
while I was working on it, but it took what it took.
|
||||
|
||||
Notify was a crate I hadn't used before so that took some time to understand
|
||||
and then determine how I wanted to wrap it up. I like the solution that I
|
||||
came up with because it keeps most of the notify mess contained in one file
|
||||
with just a little seepage out when it has to be used for monitoring. Perhaps
|
||||
I should have fully wrapped the library? It wouldn't have taken much more to
|
||||
do so, it just didn't seem necessary to solving the problem.
|
||||
|
||||
One thing I was not happy about was that I had very poor git commiting during
|
||||
the challenge. I think it was because I was trying to keep my time lower, but
|
||||
would it really have hurt to spend a little bit of time to commit and give
|
||||
myself better backups incase of a big problem? Probably not.
|
||||
|
||||
Ultimately, his let me work on and understand rust async. I had read the rust
|
||||
book and the tokio tutorial completely before doing this challenge. It wasn't
|
||||
that bad. It reminds me of coding for threads a lot, which is good, but
|
||||
sometimesthe await gets me because I don't understand where it is blocked when
|
||||
it is running sometimes, tokio-console helps here, and why calls block or what
|
||||
they are blocking for. For example, why does tokio::io::stdout().write_all()
|
||||
seem to block forever? It is given a few bytes to write to out, it should be
|
||||
able to dump those to the console quickly enough.
|
||||
|
||||
Really, i think my problem with the async IO calls was that they needed to
|
||||
happen in a seperate task and be listening for messages on what to print.
|
||||
That way they are never able to block a different task.
|
||||
|
||||
Next I think I should checkout the rayon crate, but we'll see what the next
|
||||
challenge I get is.
|
@ -19,7 +19,7 @@ pub struct DirMonitor
|
||||
dir: PathBuf,
|
||||
|
||||
/// The files that are being monitored within the directory.
|
||||
monitored_files: MonitoredFiles
|
||||
monitored_files: MonitoredFiles,
|
||||
}
|
||||
|
||||
|
||||
@ -31,8 +31,11 @@ impl DirMonitor
|
||||
/// monitor_path: A string representation of directory to monitor.
|
||||
pub fn new(monitor_path: &str) -> Self
|
||||
{
|
||||
DirMonitor { dir: PathBuf::from(monitor_path),
|
||||
monitored_files: MonitoredFiles::new() }
|
||||
DirMonitor
|
||||
{
|
||||
dir: PathBuf::from(monitor_path),
|
||||
monitored_files: MonitoredFiles::new()
|
||||
}
|
||||
}
|
||||
|
||||
/// Scan the monitored directory and all of its sub directories and monitors
|
||||
@ -45,7 +48,8 @@ impl DirMonitor
|
||||
match scan_dir(&self.dir, &mut self.monitored_files, &self.dir)
|
||||
{
|
||||
Ok(_) =>
|
||||
{}
|
||||
{
|
||||
}
|
||||
|
||||
Err(e) =>
|
||||
{
|
||||
@ -65,34 +69,30 @@ impl DirMonitor
|
||||
println!("{}", self.monitored_files);
|
||||
}
|
||||
|
||||
/// Start monitoring the directory asyncronously and process changes within
|
||||
/// the directory until a termination message is received.
|
||||
pub async fn monitor(&mut self,
|
||||
mut term_receiver: tokio::sync::watch::Receiver<bool>)
|
||||
/// Start monitoring the directory asyncronously and process changes within the directory
|
||||
/// until a termination message is received.
|
||||
pub async fn monitor(&mut self, mut term_receiver: tokio::sync::watch::Receiver<bool>)
|
||||
{
|
||||
let mut running: bool = true;
|
||||
|
||||
// Setup the notify crate to watch the INBOX directory.
|
||||
let tokio_runtime = tokio::runtime::Handle::current();
|
||||
let (notify_sender, mut notify_receiver) =
|
||||
tokio::sync::mpsc::channel::<Event>(10);
|
||||
let (notify_sender, mut notify_receiver) = tokio::sync::mpsc::channel::<Event>(10);
|
||||
let wrapped_sender = NotifySender::new(tokio_runtime, notify_sender);
|
||||
let mut fs_watcher: RecommendedWatcher =
|
||||
match notify::recommended_watcher(wrapped_sender)
|
||||
{
|
||||
Ok(watcher) => watcher,
|
||||
Ok(watcher) => { watcher }
|
||||
Err(e) =>
|
||||
{
|
||||
// Just panic because we cannot watch the directories so the
|
||||
// program would be useless and this saves from
|
||||
// having to press CTRL-C.
|
||||
// Just panic because we cannot watch the directories so the program
|
||||
// would be useless and this saves from having to press CTRL-C.
|
||||
eprintln!("Unable to create watcher: {}", e);
|
||||
panic!();
|
||||
}
|
||||
};
|
||||
|
||||
if let Err(e) =
|
||||
fs_watcher.watch(&self.dir, notify::RecursiveMode::Recursive)
|
||||
if let Err(e) = fs_watcher.watch(&self.dir, notify::RecursiveMode::Recursive)
|
||||
{
|
||||
// Just panic because we cannot watch the directories so the program
|
||||
// would be useless and this saves from having to press CTRL-C.
|
||||
@ -107,7 +107,8 @@ impl DirMonitor
|
||||
// We are listening for messages from either the notify receiver or
|
||||
// the termination receiver. When ever one of them comes across we
|
||||
// will process it.
|
||||
tokio::select! {
|
||||
tokio::select!
|
||||
{
|
||||
// Handle listening for the notify watcher events.
|
||||
// These are the changes that we care about from the file system.
|
||||
Some(mut event) = notify_receiver.recv() =>
|
||||
@ -131,10 +132,9 @@ impl DirMonitor
|
||||
|
||||
|
||||
|
||||
/// Scans a directory, and all of its sub directories, and creates a list of the
|
||||
/// files inside and their last modification time.
|
||||
fn scan_dir(base_dir: &Path, monitored_files: &mut MonitoredFiles, dir: &Path)
|
||||
-> std::io::Result<()>
|
||||
/// Scans a directory, and all of its sub directories, and creates a list of the files
|
||||
/// inside and their last modification time.
|
||||
fn scan_dir(base_dir: &Path, monitored_files: &mut MonitoredFiles, dir: &Path) -> std::io::Result<()>
|
||||
{
|
||||
let mut dir_list: Vec<PathBuf> = Vec::new();
|
||||
|
||||
@ -150,8 +150,7 @@ fn scan_dir(base_dir: &Path, monitored_files: &mut MonitoredFiles, dir: &Path)
|
||||
// Files will be added to the list of monitored files and their
|
||||
// last modification time will be stored.
|
||||
//
|
||||
// TODO: Handle symlinks. Symlinks can be either a file or another
|
||||
// directory.
|
||||
// TODO: Handle symlinks. Symlinks can be either a file or another directory.
|
||||
if meta.is_dir()
|
||||
{
|
||||
dir_list.push(file.path().clone());
|
||||
@ -160,8 +159,7 @@ fn scan_dir(base_dir: &Path, monitored_files: &mut MonitoredFiles, dir: &Path)
|
||||
{
|
||||
let last_mod_time: std::time::SystemTime = meta.modified()?;
|
||||
|
||||
monitored_files.add(&strip_path_prefix(&file.path(), base_dir),
|
||||
last_mod_time);
|
||||
monitored_files.add(&strip_path_prefix(&file.path(), base_dir), last_mod_time);
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -68,9 +68,7 @@ impl Event
|
||||
{
|
||||
*path = strip_path_prefix(&path, base_dir);
|
||||
}
|
||||
Event::Rename { from,
|
||||
to,
|
||||
mod_time: _ } =>
|
||||
Event::Rename { from, to, mod_time: _ } =>
|
||||
{
|
||||
*from = strip_path_prefix(&from, base_dir);
|
||||
*to = strip_path_prefix(&to, base_dir);
|
||||
@ -98,9 +96,7 @@ impl std::fmt::Display for Event
|
||||
{
|
||||
write!(f, "[MOD] {}", path.display())
|
||||
}
|
||||
Event::Rename { from,
|
||||
to,
|
||||
mod_time: _ } =>
|
||||
Event::Rename { from, to, mod_time: _ } =>
|
||||
{
|
||||
writeln!(f, "[DEL] {}", from.display())?;
|
||||
write!(f, "[NEW] {}", to.display())
|
||||
|
20
src/main.rs
20
src/main.rs
@ -11,6 +11,7 @@ mod util;
|
||||
|
||||
|
||||
use clap::Parser;
|
||||
|
||||
use tokio::sync::watch;
|
||||
|
||||
use crate::dir_monitor::DirMonitor;
|
||||
@ -47,7 +48,10 @@ fn create_directory(dir: &str) -> Option<String>
|
||||
|
||||
match std::fs::create_dir_all(inbox_dir.clone())
|
||||
{
|
||||
Ok(_) => Some(inbox_dir),
|
||||
Ok(_) =>
|
||||
{
|
||||
Some(inbox_dir)
|
||||
}
|
||||
|
||||
Err(e) =>
|
||||
{
|
||||
@ -95,7 +99,8 @@ async fn main()
|
||||
directory_monitor.print_monitored_files();
|
||||
|
||||
// Run our directory monitor.
|
||||
let mon_task = tokio::spawn(async move {
|
||||
let mon_task = tokio::spawn(async move
|
||||
{
|
||||
// Program workflow step 2 (terminated by step 3).
|
||||
// This can be done async because we are waiting on
|
||||
// events to be loaded from the notify library and
|
||||
@ -117,7 +122,8 @@ async fn main()
|
||||
|
||||
// Run until Ctrl-C is pressed.
|
||||
// Once it is pressed it will send a message to stop.
|
||||
let _term_task = tokio::spawn(async move {
|
||||
let _term_task = tokio::spawn(async move
|
||||
{
|
||||
// Program workflow step 3.
|
||||
// Async with message passing so it can run and listen
|
||||
// for the termination signal. It uses a watch channel
|
||||
@ -130,12 +136,8 @@ async fn main()
|
||||
// finish once the termination task has finished.
|
||||
match mon_task.await
|
||||
{
|
||||
Ok(_) =>
|
||||
{}
|
||||
Err(e) =>
|
||||
{
|
||||
eprintln!("Error during monitoring task: {}", e);
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(e) => { eprintln!("Error during monitoring task: {}", e); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -1,5 +1,5 @@
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::path::{PathBuf, Path};
|
||||
use std::time::SystemTime;
|
||||
|
||||
use chrono::{DateTime, Local};
|
||||
@ -22,7 +22,10 @@ impl MonitoredFiles
|
||||
/// Create a new empty set of monitored files.
|
||||
pub fn new() -> Self
|
||||
{
|
||||
MonitoredFiles { files: HashMap::new() }
|
||||
MonitoredFiles
|
||||
{
|
||||
files: HashMap::new()
|
||||
}
|
||||
}
|
||||
|
||||
/// Process an event and change the monitored files.
|
||||
@ -35,7 +38,7 @@ impl MonitoredFiles
|
||||
self.add(&path, mod_time);
|
||||
}
|
||||
|
||||
Event::Modify { path, mod_time } =>
|
||||
Event::Modify { path, mod_time} =>
|
||||
{
|
||||
self.modify(&path, mod_time);
|
||||
}
|
||||
@ -87,17 +90,11 @@ impl std::fmt::Display for MonitoredFiles
|
||||
|
||||
if !it.peek().is_none()
|
||||
{
|
||||
writeln!(f,
|
||||
"[{}] {}",
|
||||
date_time.format("%m/%d/%Y %H:%M"),
|
||||
path.display())?;
|
||||
writeln!(f, "[{}] {}", date_time.format("%m/%d/%Y %H:%M"), path.display())?;
|
||||
}
|
||||
else
|
||||
{
|
||||
write!(f,
|
||||
"[{}] {}",
|
||||
date_time.format("%m/%d/%Y %H:%M"),
|
||||
path.display())?;
|
||||
write!(f, "[{}] {}", date_time.format("%m/%d/%Y %H:%M"), path.display())?;
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -26,8 +26,11 @@ impl NotifySender
|
||||
sender: tokio::sync::mpsc::Sender<crate::event::Event>)
|
||||
-> Self
|
||||
{
|
||||
NotifySender { tokio_runtime,
|
||||
sender }
|
||||
NotifySender
|
||||
{
|
||||
tokio_runtime,
|
||||
sender
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -47,9 +50,9 @@ impl notify::EventHandler for NotifySender
|
||||
{
|
||||
let notify_sender = self.sender.clone();
|
||||
|
||||
self.tokio_runtime.spawn(async move {
|
||||
if let Err(e) =
|
||||
notify_sender.send(monitor_event).await
|
||||
self.tokio_runtime.spawn(async move
|
||||
{
|
||||
if let Err(e) = notify_sender.send(monitor_event).await
|
||||
{
|
||||
eprintln!("Notify EventHandler: {}", e);
|
||||
}
|
||||
@ -85,19 +88,21 @@ fn process_notify_event(event: notify::Event) -> Option<crate::event::Event>
|
||||
process_remove_event(kind, event.paths)
|
||||
}
|
||||
|
||||
_ => None
|
||||
_ => { None }
|
||||
}
|
||||
}
|
||||
|
||||
/// Process the Create event into a New event.
|
||||
fn process_create_event(event: notify::event::CreateKind, paths: Vec<PathBuf>)
|
||||
fn process_create_event(event: notify::event::CreateKind,
|
||||
paths: Vec<PathBuf>)
|
||||
-> Option<crate::event::Event>
|
||||
{
|
||||
if event == notify::event::CreateKind::File
|
||||
{
|
||||
let mod_time: std::time::SystemTime = match get_file_mod_time(&paths[0])
|
||||
let mod_time: std::time::SystemTime =
|
||||
match get_file_mod_time(&paths[0])
|
||||
{
|
||||
Ok(time) => time,
|
||||
Ok(time) => { time }
|
||||
|
||||
Err(e) =>
|
||||
{
|
||||
@ -107,31 +112,33 @@ fn process_create_event(event: notify::event::CreateKind, paths: Vec<PathBuf>)
|
||||
}
|
||||
};
|
||||
|
||||
return Some(crate::event::Event::New { path: paths[0].clone(),
|
||||
mod_time: mod_time });
|
||||
return Some(crate::event::Event::New
|
||||
{
|
||||
path: paths[0].clone(),
|
||||
mod_time: mod_time
|
||||
});
|
||||
}
|
||||
|
||||
None
|
||||
}
|
||||
|
||||
/// Process the modify event into either a Modify event or a Rename event.
|
||||
fn process_modify_event(event: notify::event::ModifyKind, paths: Vec<PathBuf>)
|
||||
fn process_modify_event(event: notify::event::ModifyKind,
|
||||
paths: Vec<PathBuf>)
|
||||
-> Option<crate::event::Event>
|
||||
{
|
||||
match event
|
||||
{
|
||||
// This is just handling the events. On my system they came across as Any
|
||||
// for the MetadataKind, but the documentation seemed to hint that
|
||||
// modification times should be on write events. So for the sake of
|
||||
// being done without testing this on different platforms the Any
|
||||
// modification event will trigger a time lookup.
|
||||
notify::event::ModifyKind::Any |
|
||||
notify::event::ModifyKind::Metadata(_) =>
|
||||
// This is just handling the events. On my system they came across as Any for
|
||||
// the MetadataKind, but the documentation seemed to hint that modification
|
||||
// times should be on write events. So for the sake of being done without testing
|
||||
// this on different platforms the Any modification event will trigger a time lookup.
|
||||
notify::event::ModifyKind::Any | notify::event::ModifyKind::Metadata(_) =>
|
||||
{
|
||||
let mod_time: std::time::SystemTime = match get_file_mod_time(&paths
|
||||
[0])
|
||||
let mod_time: std::time::SystemTime =
|
||||
match get_file_mod_time(&paths[0])
|
||||
{
|
||||
Ok(time) => time,
|
||||
Ok(time) => { time }
|
||||
|
||||
Err(e) =>
|
||||
{
|
||||
@ -141,8 +148,11 @@ fn process_modify_event(event: notify::event::ModifyKind, paths: Vec<PathBuf>)
|
||||
}
|
||||
};
|
||||
|
||||
Some(crate::event::Event::Modify { path: paths[0].clone(),
|
||||
mod_time: mod_time })
|
||||
Some(crate::event::Event::Modify
|
||||
{
|
||||
path: paths[0].clone(),
|
||||
mod_time: mod_time
|
||||
})
|
||||
}
|
||||
|
||||
// Handling a rename event so that during testing when a file is
|
||||
@ -158,27 +168,32 @@ fn process_modify_event(event: notify::event::ModifyKind, paths: Vec<PathBuf>)
|
||||
let mod_time: std::time::SystemTime =
|
||||
match get_file_mod_time(&paths[1])
|
||||
{
|
||||
Ok(time) => time,
|
||||
Ok(time) => { time }
|
||||
|
||||
Err(e) =>
|
||||
{
|
||||
eprintln!("Unable to open file to retrieve \
|
||||
modification time: {}",
|
||||
eprintln!("Unable to open file to retrieve modification time: {}",
|
||||
e);
|
||||
std::time::SystemTime::now()
|
||||
}
|
||||
};
|
||||
|
||||
Some(crate::event::Event::Rename { from: paths[0].clone(),
|
||||
Some(crate::event::Event::Rename
|
||||
{
|
||||
from: paths[0].clone(),
|
||||
to: paths[1].clone(),
|
||||
mod_time: mod_time })
|
||||
mod_time: mod_time
|
||||
})
|
||||
}
|
||||
|
||||
// This is for when a file is renamed and only the from
|
||||
// file is in the monitored directories.
|
||||
notify::event::RenameMode::From =>
|
||||
{
|
||||
Some(crate::event::Event::Delete { path: paths[0].clone() })
|
||||
Some(crate::event::Event::Delete
|
||||
{
|
||||
path: paths[0].clone(),
|
||||
})
|
||||
}
|
||||
|
||||
// This is for when a file is renamed and both the to file
|
||||
@ -188,36 +203,42 @@ fn process_modify_event(event: notify::event::ModifyKind, paths: Vec<PathBuf>)
|
||||
let mod_time: std::time::SystemTime =
|
||||
match get_file_mod_time(&paths[0])
|
||||
{
|
||||
Ok(time) => time,
|
||||
Ok(time) => { time }
|
||||
|
||||
Err(e) =>
|
||||
{
|
||||
eprintln!("Unable to open file to retrieve \
|
||||
modification time: {}",
|
||||
eprintln!("Unable to open file to retrieve modification time: {}",
|
||||
e);
|
||||
std::time::SystemTime::now()
|
||||
}
|
||||
};
|
||||
|
||||
Some(crate::event::Event::New { path: paths[0].clone(),
|
||||
mod_time: mod_time })
|
||||
Some(crate::event::Event::New
|
||||
{
|
||||
path: paths[0].clone(),
|
||||
mod_time: mod_time
|
||||
})
|
||||
}
|
||||
|
||||
_ => None
|
||||
_ => { None }
|
||||
}
|
||||
}
|
||||
|
||||
_ => None
|
||||
_ => { None }
|
||||
}
|
||||
}
|
||||
|
||||
/// Process the remove event into a Delete event.
|
||||
fn process_remove_event(event: notify::event::RemoveKind, paths: Vec<PathBuf>)
|
||||
fn process_remove_event(event: notify::event::RemoveKind,
|
||||
paths: Vec<PathBuf>)
|
||||
-> Option<crate::event::Event>
|
||||
{
|
||||
if event == notify::event::RemoveKind::File
|
||||
{
|
||||
return Some(crate::event::Event::Delete { path: paths[0].clone() });
|
||||
return Some(crate::event::Event::Delete
|
||||
{
|
||||
path: paths[0].clone(),
|
||||
});
|
||||
}
|
||||
|
||||
None
|
||||
|
14
src/util.rs
14
src/util.rs
@ -2,8 +2,7 @@
|
||||
///
|
||||
/// If it is unable to get the modification time from the file system
|
||||
/// it will default to the current system time.
|
||||
pub fn get_file_mod_time(path: &std::path::Path)
|
||||
-> std::io::Result<std::time::SystemTime>
|
||||
pub fn get_file_mod_time(path: &std::path::Path) -> std::io::Result<std::time::SystemTime>
|
||||
{
|
||||
let file = std::fs::File::open(path)?;
|
||||
let meta = file.metadata()?;
|
||||
@ -16,10 +15,8 @@ pub fn get_file_mod_time(path: &std::path::Path)
|
||||
Ok(std::time::SystemTime::now())
|
||||
}
|
||||
|
||||
/// Create a new PathBuf that is the same as path but has the base removed if it
|
||||
/// can.
|
||||
pub fn strip_path_prefix(path: &std::path::Path, base: &std::path::Path)
|
||||
-> std::path::PathBuf
|
||||
/// Create a new PathBuf that is the same as path but has the base removed if it can.
|
||||
pub fn strip_path_prefix(path: &std::path::Path, base: &std::path::Path) -> std::path::PathBuf
|
||||
{
|
||||
match &path.strip_prefix(base)
|
||||
{
|
||||
@ -31,6 +28,9 @@ pub fn strip_path_prefix(path: &std::path::Path, base: &std::path::Path)
|
||||
final_path
|
||||
}
|
||||
|
||||
Err(_) => std::path::PathBuf::from(path)
|
||||
Err(_) =>
|
||||
{
|
||||
std::path::PathBuf::from(path)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
Loading…
x
Reference in New Issue
Block a user