SWEB: Scalable Digital Library and Web Servers

This project investigates the application of parallel and global computing techniques in Web-based network information systems with intensive I/O and computation. We examine issues in developing scalable Web and digital library servers on workstation clusters and parallel machines (Convex Examplar/Meiko CS-2) The main objective is to strengthen the processing capabilities of such a server to match huge expected increases in simultaneous access requests from the Internet. The scheduling component of the system actively monitors the usages of CPU, I/O channels and the interconnection network, and effectively distributes HTTP requests across processing units to exploit task and I/O parallelism. The research issues addressed are:

Scheduling and Load Balancing for Clustered Web and Digital library Servers. We have developed an optimization scheme that dynamically monitors the resource availability, uses a low-cost communication strategy for updating load information among nodes, and schedules requests based on both I/O and computation load indices. Since the accurate cost estimation for processing spatial database searching requests is difficult, we have proposed a sampling and prediction scheme to identify the relative efficiency of nodes for satisfying I/O and CPU demands of these requests. The multi-processor Web server we have developed is called SWEB.
Recently we have studied a two-level scheduling framework with a master/slave architecture for clustering Web servers. Such an architecture has advantages in dynamic resource recruitment, fail-over management and it can also improve server performance compared to a flat architecture. The key methods we propose to make this architecture efficient are the separation of static and dynamic content processing, low overhead remote execution, and reservation-based scheduling which considers both I/O and CPU utilization.
Partitioning and Scheduling for Adaptive Client-Server Computing. We also address the client-server partitionable Web applications and develop adaptive scheduling and load balancing techniques that consider client capabilities and optimize the use of both client and server resources. We have demonstrated the applications of these techniques to postscript document processing and wavelet-based progressive image browsing. We have developed a software tool called SWEB++ which implements and supports the use of adaptive scheduling strategies for programming Web applications.
Swala: our new project on Web caching and clustering support

Selected papers:

Huican Zhu and Ben Smith and Tao Yang, Scheduling Optimization for Resource-Intensive Web Requests on Server Clusters. To appear in the Proceedings of the Eleventh Annual ACM Symposium on Parallel Algorithms and Architectures (SPAA'99).
Huican Zhu, Ben Smith and Tao Yang, A Scheduling Framework for Web Server Clusters with Intensive Dynamic Content Processing Technical Report, TRCS98-29, UCSB. Dec 1998. A poster version with title ``Hierarchical Resource Management for Web Server Clusters with Dynamic Content'' will appear in Proc. of ACM SIGMETRICS'99.
V. Holmedahl, B. Smith, and T. Yang. Cooperative Caching of Dynamic Content on a Distributed Web Server in Proc. of 7th IEEE International Symposium on High Performance Distributed Computing (HPDC-7) Chicago, IL USA July 28-31, 1998. pp. 243-250.
H. Zhu, T. Yang, Q. Zheng, D. Watson, O. Ibarra and T. Smith, Adaptive Load Sharing for Clustered Digital Library Servers in Proc. of 7th IEEE International Symposium on High Performance Distributed Computing (HPDC-7) Chicago, IL USA July 28-31, 1998. pp. 235-242.
D. Andresen, T. Yang. ``SWEB++: Partitioning and Scheduling for Adaptive Client-Server Computing on WWW'', Proc. of 1998 SIGMETRICS Workshop on Internet Server Performance, June 1998.
Adaptive Partitioning and Scheduling for Enhancing WWW Application Performance, , by D. Andresen, T. Yang, O. Ibarra, O. Egecioglu. To appear in Journal of Parallel and Distributed Computing, 1998.
Dynamic Processor Scheduling with Client Resources for Fast Multi-resolution WWW Image Browsing, by D. Andresen, T. Yang, D. Watson, A. Poulakidas, Proceedings of the 11th International Parallel Processing Symposium (IPPS'97), Geneva, April, 1997. Talk slides .
Multiprocessor Scheduling with Client Resources to Improve the Response Time of WWW Applications, by D. Andresen, T. Yang, Proc. of the 11th ACM SIGARCH Inter national Conference on Supercomputing (ICS'97), 1997.
Scalability Issues for High Performance Digital Libraries on the World Wide Web by D. Andresen, T. Yang, O. Egecioglu, O. Ibarra, and T. Smith. in Proceedings of IEEE ADL '96 (Advances in Digital Libraries), IEEE, Washington D.C., May 1996.

SWEB: Towards a Scalable WWW Server on MultiComputers by D. Andresen, T. Yang, V. Holmedahl and O. Ibarra. This paper gives the system organization and scheduling algorithms, and initial experimental results. in Proccedings of the 10th International Parallel Processing Symposium (IPPS'96) , Hawaii, April, 1996. Talk slides . The journal version in Journal of Parallel and Distributed Computing, 1997.

The WWW Prototype of the Alexandria Digital Library by D.Andresen, L.Carver, R.Dolin, C.Fischer, J.Frew, M.Goodchild, O.Ibarra, R.Kothuri, M.Larsgaard, B.Manjunath, D. Nebert, J.Simpson, T.Smith, T.Yang, Q.Zheng, Proceedings of ISDL'95: International Symposium on Digital Libraries, Japan August 22 - 25, 1995. This paper gives a brief overview of the project and its relationship with the Alexandria digital library project.

The project is supported by NSF IRI94-11330 (Alexandria Digital Library), NSF CAREER CCR-9702640, NSF RIA CCR-9409695, NSF CDA-9529418, a grant from Navy NRaD, and the UC MICRO grant with a matching from SUN. This research is done in conjunction with Performance and Parallel Processing Team (Dan Andresen, Omer Egecioglu, Vegard Holmedahl, Oscar Ibarra, Thanasis Poulakidas, Ben Smith, David Watson, Dianyuan Xiong, Huican Zhu, Tao Yang) of the Alexandria Digital Library project.

Contact

Back to Parallel Systems Lab Home Page