PhD student Stratos Dimopoulos, and his research advisors Profs. Chandra Krintz, and Rich Wolski recently received the Best Paper Award for their paper "Justice: A Deadline-aware, Fair-share Resource Allocator for Implementing Multi-analytics" at the IEEE Cluster Conference to be held September 5th - 8th in Hawaii.
In the upcoming conference, the UCSB CS research team will present Justice, a fair-share deadline-aware resource allocator for big data cluster managers. In resource constrained environments, where resource contention introduces significant execution delays, Justice outperforms the popular existing fair-share allocator that is implemented as part of Mesos and YARN. Justice uses deadline information supplied with each job and historical job execution logs to implement admission control. It automatically adapts to changing workload conditions to assign enough resources for each job to meet its deadline “just in time”. In this work, Dimopoulos, Krintz, and Wolski use trace-based simulation of production YARN workloads to evaluate Justice under different deadline formulations, and compare Justice to the existing fair- share allocation policy deployed on cluster managers like YARN and Mesos. The team finds that in resource-constrained settings, Justice improves fairness, satisfies significantly more deadlines, and utilizes resources more efficiently.