The | Crew Pkg

With crew :

controller <- crew_controller_local(workers = 8) controller$start() for (file in all_files) { controller$push( name = file, command = process_file(file) ) } results <- list() while (controller$pop()$name != "done") { Crew auto-replaces crashed workers results <- c(results, controller$pop()$result) } the crew pkg

But the real magic happens when you pair crew with targets . In a _targets.R file, changing the controller is a one-line edit: With crew : controller &lt