Availability and Utility of Idle Memory on Workstation Clusters.

Submitted for publication.

Abstract:

In this paper, we examine the availability and utility of idle memory in workstation clusters. We attempt to answer the following questions. First, how much of the total memory in a workstation cluster can be expected to be idle? This provides an estimate of the opportunity for hosting guest data. Second, how much memory can be expected to be idle on individual workstations? This helps determine the recruitment policy -- how much memory should be recruited on individual hosts? Third, what is the distribution of memory idle-times? This indicates how long guest data can be expected to survive; applications that access their data-sets frequently within the expected life-time of guest data are more likely to benefit from exploiting idle memory. Fourth, how much performance improvement can be achieved for off-the-shelf clusters without customizing the operating system and/or the processor firmware? Finally, how long and how frequently might a user have to wait to reclaim her machine if she volunteers to host guest pages on her machine? This helps answer the question of social acceptability. To answer the questions relating to the availability of idle memory, we have analyzed two-week long traces from two workstation pools with different sizes, locations, and patterns of use. To evaluate the expected benefits and costs, we have simulated five data-intensive applications (0.5 GB-5 GB) on these workstation pools.

Postscript.