Lessons learned from using machine learning to optimize database configurations

Database management systems (DBMS) expose dozens of configurable knobs that control their runtime behavior. Setting these knobs correctly for an application's workload can improve the performance and efficiency of the DBMS. But such tuning requires considerable efforts from experienced administrators, which is not scalable for large DBMS fleets. This problem has led to research on using machine learning (ML) to devise strategies to automatically optimize DBMS knobs for any application. Current research suggests that ML can generate better DBMS configurations more quickly than what is possible with human experts. And since these ML algorithms do not require humans to make decisions, they can also scale to support tuning thousands of databases at a time. Despite the advantages of ML-based approaches, there are still several problems that one must overcome to deploy an automated tuning service for DBMSs.

In this talk, Andy will discuss the challenges in using ML to optimize DBMS knobs and the solutions we developed to address them. His presentation will be in the context of the OtterTune database tuning service. Andy will also highlight the insights learned from real-world installations of OtterTune for MySQL, Postgres, and Oracle.