Below is a condolidated summary of our Cloud Foundations lecture series during my visit at Bilkent. It was a great experience to be back at where it all started 2+ decades ago. I had a great time seeing the first generation who taught us, the second generation, us, who learned together and several of them back as faculty, and the third generation, who is, hopefully learning something useful from us. This was long sitting in my drafts backlog; happy to finally be able push it into the open
We are seeing an accelerating growth in cloud platforms, runtimes, and programming models. Cloud discussion has shifted from utility and density to cloud-native services and design patterns. Emerging cloud services allow users to define and provision complex, distributed systems with unprecedented simplicity and agility. With the push of a button entire stacks of software can be instantiated within minutes with various configurations and customizations. Automation, continuous integration and delivery further simplify the entire lifecycle management of modern born-on-the-cloud applications. These advances also bring in new research challenges. Operational visibility into the complex, distributed applications, cloud runtimes and the underlying infrastructure is becoming a persistent pain point across end-users and providers especially for security applications. As system and configuration complexity grows, data-driven operational analytics for security, compliance, configuration and resource management emerge as key areas of focus, where traditional solutions remain ineffective or insufficient.
In this seminar I present an overview of the cloud evolution, emerging runtimes and design patterns. I describe the challenges arising from this evolution and where existing techniques fall short. I will then present our work on cloud operational visibility and analytics that aims to address some of these challenges. I propose a unique approach to leveraging cloud abstractions and implementation principles to achieve unmatched deep and seamless visibility into cloud instances, and using this deep visibility in developing operational and security analytics for the cloud. I overview two outcomes of this approach, Agentless System Crawler and the Vulnerability Advisor service. I discuss our journey developing the foundations of the visibility and security services for IBM Containers. I will share our experiences working with a production cloud and the key real-world use cases. I will provide an overview of our current research directions, open problems and opportunities in this area.