Pipeline update to rsync at 25 epochs
This commit is contained in:
+1
-1
@@ -1,6 +1,6 @@
|
||||
# Pipeline
|
||||
|
||||
Orchestrates ephemeral Vast.ai GPU instances: searches for an offer, creates the instance, syncs the project, trains, downloads `outputs/`, and destroys the instance automatically. Generator runs also rsync `generator/outputs/` every 50 epochs while training is still running.
|
||||
Orchestrates ephemeral Vast.ai GPU instances: searches for an offer, creates the instance, syncs the project, trains, downloads `outputs/`, and destroys the instance automatically. Generator runs also rsync `generator/outputs/` every 25 epochs while training is still running.
|
||||
|
||||
## One-time setup
|
||||
|
||||
|
||||
@@ -74,7 +74,7 @@ EUROPE_REGION_CODES = {
|
||||
}
|
||||
|
||||
# How often (in epochs) to rsync generator outputs back during training
|
||||
_GENERATOR_SYNC_INTERVAL = 50
|
||||
_GENERATOR_SYNC_INTERVAL = 25
|
||||
|
||||
|
||||
# Raised when the user aborts interactive offer selection
|
||||
|
||||
Reference in New Issue
Block a user