DDIC: Distributed
Data-Intensive
Cloud Computing Systems
Background
The Internet has evolved to become a service delivery
infrastructure instead of merely providing host connectivity. Different
forms of large-scale distributed systems have been developped to
provide attractive service provisioning solutions. Recent industry
initiatives such as Google-IBM Cloud Computing and Amazon EC2 are
encourging real-world examples. We envision the emergence of
service-oriented data-intensive computing infrastructure, where
different nodes
provide various data analytics functions such as filtering,
correlation, aggregation.
Each service can have multiple instances that provide same or different
quality-of-service (QoS) levels. Different service instances can be
composed on-the-fly into a composed application services. However, as
such an infrastructure becomes larger, the management complexitity
becomes the major
barrier. DDIC aims at providing an automatic management
framework to tackle the complexity obstacle. DDIC has the
following goals:
- Provide automatic component management utilities (e.g., component
registration/discovery/composition/placement);
- Support generic data-intensive computing applications such as
dataflows and stream processing
- Provide performance management capabilities (e.g.,
quality-of-service provisioning);
- Provide resource management capabilities (e.g., load balancing);
- Provide security assurances;
- Provide anomaly detection and problem diagnosis.
People
Faculty
Students
- Yongmin Tan (PhD
student)
- Juan Du (PhD student)
- Raghu Kishor (MS student)
Collaborators
- Chitra Venkatramani (IBM Research)
Publications
- Thomas Repantis, Xiaohui Gu, Vana
Kalogeraki,"QoS-Aware
Shared Component Composition for Distributed
Stream Processing Systems",
IEEE Transactions on Parallel and Distributed Systems (TPDS), (to
appear, extended version of Middleware 2006 paper).
- Xiaohui Gu, Spiros Papadimitriou,
Philip S. Yu, Shu-Ping Chang, "Toward
Predictive Failure Management for Distributed Stream Processing
Systems", IEEE International Conference on Distributed Computing
Systems (ICDCS), Beijing, China, June, 2008.
- Xiaohui Gu, Zhen Wen,
Philip S. Yu, Zon-Yin Shae , "peerTalk: A Peer-to-Peer Multi-Party
Voice-Over-IP System", IEEE Transactions on Parallel and
Distributed Systems (TPDS), 2008.
- Xiaohui Gu, Philip S. Yu, Haixun Wang,
"Adaptive
Load Diffusion for Multiway Windowed Stream Joins", IEEE
International Conference on Data Engineering (ICDE), Istanbul, Turkey,
April, 2007.
- Chen Chen, Xifeng Yan, Philip S. Yu,
Jiawei Han, Dongqing Zhang, Xiaohui Gu, "Toward
Graph Containment Search and Indexing",
Very Large Data Bases Conference (VLDB), 2007.
- Kun-Lung Wu, Philip S. Yu, Bugra
Gedik, Kirsten W. Hildrum, Charu C. Aggarwal, Eric Bouillet, Wei Fan,
David A. George, Xiaohui Gu, Gang Luo, Haixun Wang, "Challenges
and Experience in Prototyping a Multi-Modal Stream Analytic and
Monitoring Application on System S", Very Large Data Bases
Conference (VLDB), 2007.
- Xiaohui Gu, Zhen
Wen, Philip S. Yu, " BridgeNet: An Adaptive Multi-Source Stream
Dissemination Service Overlay", IEEE INFOCOM
Mini-Symposium, Anchorage,
Alaska, May, 2007.
- Xiaohui Gu, Zhen Wen, Ching-Yung Lin,
Philip S. Yu, "ViCo:
An Adaptive Distributed Video Correlation System", ACM Multimedia
(SIGMM), Santa Barbara, CA, October, 2006. (full paper, acceptance
rate: 17%)
- Thomas Repantis,
Xiaohui Gu, Vana Kalogeraki, "Synergy:
Sharing-Aware Component Composition for Distributed Stream Processing
Systems", ACM/IFIP/USENIX International Middleware
Conference (Middleware), Melbourne,
Australia,
November 2006.
- Xiaohui Gu, Philip S. Yu, Klara
Nahrstedt , "Optimal
Component Composition for Scalable Stream Processing", IEEE
International Conference on Distributed Computing Systems (ICDCS), Columbus, OH, 2005.
- Xiaohui Gu, Klara Nahrstedt,
Bin Yu, "SpiderNet:
An Integrated Peer-to-Peer Service Composition Framework",
IEEE International Symposium on High-Performance Distributed Computing (HPDC), Honolulu, Hawaii,
June, 2004.
Related
Projects
Code
Release
- DDIC v1.0 (available upon email
request)