Training deep learning models is both computationally and memory expensive (since it is a very data intensive process). While GPUs may address the computational needs of DNN, there are fewer solutions to solve for their large memory footprint. MONeT, a new OSS project (implemented in PyTorch) from UT-Austin researchers, reduces the memory overhead necessary to train a deep learning model by jointly optimizing global techniques, such as saving and retrieving model weights, and local techniques, such as matrix multiplication optimization. Compared to hand-tuned operations and automated checkpointing, MONeT reduces average memory overhead by 3x while only increasing computational load by 9-16%. The repo also includes a “zoo” of memory-efficient schedules for training.