back to projects
project 03 • 2023
ML Pipeline Orchestrator
“Because training models shouldn't feel like herding cats.”
the story
Created an internal tool to orchestrate ML training pipelines. The goal was to make experimentation fast and reproducible.
Features include automatic hyperparameter tracking, distributed training across GPU clusters, and a simple YAML-based configuration system.
This project taught me that good tooling is worth 10x the investment in fancy models.
wrote this after losing a week of experiments to a crashed notebook
tools I used
KubernetesRayMLflowPythonDocker
key insight:
“Good tooling is worth 10x the investment in fancy models.”