SpiderNet: A Quality-Aware Distributed Component Composition
The Internet has evolved to become a
commercial infrastructure of service delivery instead of merely
providing host connectivity.
Different forms of large-scale distributed systems have been developed
provide attractive service provisioning solutions, which are difficult
implemented and deployed in the IP-layer. The examples include end
multicast, resilient routing, and peer-to-peer file sharing systems.
this, we envision the emergence of value-added large-scale distributed
where different nodes provide not only application-level data routing
value-added services, such as web services, data stream processing,
media transcoding, language translation,
encryption or decryption. Each service can have multiple instances,
provide same or different quality-of-service (QoS)
levels. Different service instances can be composed on-the-fly into a
application services. However, the current Internet infrastructure has
inadequate for delivering various services because of the problems
to scalability, implementation and deployment difficulties. More
as the system becomes larger, the management complexity has become the
barrier. SpiderNet aims at providing an autonomic management framework
large-scale distributed systems to tackle the challenge of the
We have implemented
a prototype of the
SpiderNet system. Each SpiderNet node software
multi-threaded running system written in about 13K lines of java code.
flowing figure illustrates the software architecture for one SpiderNet
There are 6 major modules: (1) service
lookup agent is responsible for discovering the list of service
which is implemented on top of the Pastry software; (2) service
graph generator module performs initial QoS-aware
service composition finding and runtime service maintenance; (3) session manager maintains session
information for current active sessions; (4) data
transmission module is responsible for sending, receiving, and
forwarding application data; (5) overlay
topology manager maintains the neighbor set; (6) monitoring
module is responsible for monitoring the network/service
states of neighbors.
- Xiaohui Gu, Klara Nahrstedt , "On
Composing Stream Applications in Peer-to-Peer Environments",
IEEE Transactions on Parallel and Distributed Systems (TPDS),
17(8): 824-837, July, 2006.
- Xiaohui Gu, Klara Nahrstedt , "Distributed
Multimedia Service Composition with Statistical QoS
Assurances", IEEE Transactions on Multimedia (TMM),
February, 8(1): 141-151, 2006.
- Thomas Repantis,
Xiaohui Gu, Vana Kalogeraki, "Synergy:
Sharing-Aware Component Composition for Distributed Stream Processing
Systems", ACM/IFIP/USENIX International Middleware
Conference (Middleware), Melbourne,
- Xiaohui Gu, Philip S. Yu, Klara
Nahrstedt , "Optimal
Component Composition for Scalable Stream Processing", IEEE
International Conference on Distributed Computing Systems (ICDCS), Columbus, OH, 2005.
- Xiaohui Gu, Klara Nahrstedt,
Bin Yu, "SpiderNet:
An Integrated Peer-to-Peer Service Composition Framework",
IEEE International Symposium on High-Performance Distributed Computing (HPDC), Honolulu, Hawaii,
- Xiaohui Gu, Klara Nahrstedt, Rong
Chang, Christopher Ward, "QoS-Assured Service Composition in Managed
Service Overlay Networks", IEEE International Conference on
Distributed Computing Systems (ICDCS),
Providence, RI, May 19-22, 2003
- Xiaohui Gu, Klara Nahrstedt, " A
Scalable QoS-Aware Service Aggregation
Model for Peer-to-Peer Computing Grids", in Proc. of 11th IEEE
International Symposium on High Performance Distributed Computing
(HPDC-11), Edinburgh, Scotland,
July 24-26, 2002.
- Xiaohui Gu, Klara Nahrstedt, "
Dynamic QoS-Aware Multimedia Service
Configuration in Ubiquitous Computing Environments", in Proc. of
22nd IEEE International Conference on Distributed Computing Systems
(ICDCS 2002), Vienna, Austria, July 2-5, 2002.
v1.0 (available upon request by email)