SpiderNet: A Quality-Aware Distributed Component Composition System


Background

The Internet has evolved to become a commercial infrastructure of service delivery instead of merely providing host connectivity. Different forms of large-scale distributed systems have been developed to provide attractive service provisioning solutions, which are difficult to be implemented and deployed in the IP-layer. The examples include end system multicast, resilient routing, and peer-to-peer file sharing systems. Beyond this, we envision the emergence of value-added large-scale distributed systems, where different nodes provide not only application-level data routing but also value-added services, such as web services, data stream processing, media transcoding, language translation, and context-aware encryption or decryption. Each service can have multiple instances, which provide same or different quality-of-service (QoS) levels. Different service instances can be composed on-the-fly into a composed application services. However, the current Internet infrastructure has become inadequate for delivering various services because of the problems pertaining to scalability, implementation and deployment difficulties. More importantly, as the system becomes larger, the management complexity has become the major barrier. SpiderNet aims at providing an autonomic management framework for large-scale distributed systems to tackle the challenge of the management complexity.

 We have implemented a prototype of the SpiderNet system. Each SpiderNet node software is a multi-threaded running system written in about 13K lines of java code. The flowing figure illustrates the software architecture for one SpiderNet node. There are 6 major modules: (1) service lookup agent is responsible for discovering the list of service instances, which is implemented on top of the Pastry software; (2) service graph generator module performs initial QoS-aware service composition finding and runtime service maintenance; (3) session manager maintains session information for current active sessions; (4) data transmission module is responsible for sending, receiving, and forwarding application data; (5) overlay topology manager maintains the neighbor set; (6) monitoring module is responsible for monitoring the network/service states of neighbors.

 

 

 

 

 

 

 

 

 

 

 

 

 

Publications

 

Journal

Conference:

Related Projects:

Code Release:

Demo: