NEC Labs Data Management University Awards: Unrestricted Gift
Cloud computing has emerged as a pervasive platform for deploying scalable and highly available Internet applications. Since a large fraction of the applications are data-driven, database management systems form an integral part of cloud computing systems. To facilitate data-driven applications in the cloud: elasticity, scalability, fault-tolerance, and self-manageability are fundamental requirements for database management systems (DBMS) driving such applications. Though many of the traditional DBMS driving mission critical applications have been designed to be scalable and fault-tolerant; elasticity is one critical feature often neglected, primarily due to the static resource provisioning in enterprise infrastructures. But the pay-per-use model prevalent in cloud infrastructures requires systems to be elastic in order to minimize the cost of operation. We therefore propose elasticity as a first class feature for cloud database systems. At UCSB, we are building ElasTraS, an Elastic TranSactional relational database, designed to scale out using a cluster of commodity machines while being fault-tolerant and self managing [9, 10]. ElasTraS is designed to support both classes of database needs for the cloud: (i) large databases partitioned across a set of nodes, and (ii) a large number of small and independent databases common in multi-tenant databases. Conceptually, ElasTraS operates at the granularity of partitions: for (i), each large database instance consists of hundreds or thousands of partitions, and for (ii), each partition is in itself a self-contained database, and there are hundreds to thousands of such independent single-partition databases. Database partitions in ElasTraS are the granules for load-balancing, elasticity, fault-tolerance, and transactional access. For the rest of the proposal, we concentrate on the multi-tenancy aspect of ElasTraS, where the system is managing the databases of multiple independent tenants. ElasTraS borrows from the design philosophy of scalable Key-Value stores to minimize distributed synchronization and remove scalability bottlenecks, while leveraging decades of research on transaction processing, concurrency control, and recovery to support rich functionality and transactional guarantees. Since elasticity was one of the critical features in the design space of ElasTraS, the system has been designed to allow the addition and removal of nodes from an operational system – the present implementation requires human intervention and causes some disruption in service. During the course of the proposal, we aim to make this system autonomously elastic, i.e., enable the system to seamlessly grow and shrink as per the load on the system changes – all without any human intervention. To this end, we have identified two crucial problems that we aim to solve during the course of this project: (i) model system operation and behavior and predict the necessity for elastic expansion and contraction of the system; and (ii) live migration of database partitions to allow the system to use migration as a primitive for load balancing and elasticity.