The Kubeflow maintainers developed a machine learning model to automatically categorize issues opened by community members. However, they soon realized that frequent retraining and deployment of this model was necessary to maintain model efficacy. The solution they designed uses independent controllers for each step in the ML workflow (preprocessing, training, validation, and deployment) and applies Jupyter notebooks, GitOps, and Kubernetes. Their system heavily leverages  GitOps and reconcilers, which observe the state of the world, compute the difference between this state and the desired state of the world, and take action to progress towards the desired state. With reconcilers, the DAG can instead be specified as a series of desired states, thereby improving resilience and flexibility (e.g. different teams can still use their favorite tools). The authors describe the implementation of this system and address how they might enable lineage tracking in the future.