have you tried runpod, lambda, vast? https://www.runpod.io/ https://lambdalabs.com/ https://vast.ai/
lambda is my go-to atm. never heard of runpod, thanks!
how about parameter tracking like wandb.ai, neptune.ai, etc.? what do you guys use?