Dynamic Resource Optimization for Generative AI Workloads: A Simulation-Driven Approach to Mitigating Cold-Start Latency and Cost Inefficiency in Cloud Environments

Abstract

The rapid global adoption of Generative AI (GenAI) has precipitated a paradigm shift in cloud resource management. While GenAI offers transformative potential, it imposes significant computational demands, characterized by high variance in inference times and resource intensity. Traditional auto-scaling mechanisms, primarily designed for deterministic web traffic, often fail to address the specific "cold-start" latency issues associated with loading large model weights, leading to suboptimal performance or excessive over-provisioning costs. This study proposes a novel, simulation-driven framework for dynamic resource allocation specifically tailored for GenAI workloads. By leveraging the Abstract Behavioral Specification (ABS) language to model complex, concurrent service behaviors and integrating predictive bytecode instruction counting, we develop a multi-tiered scaling strategy. We benchmark this strategy against standard AWS Auto Scaling configurations using a diverse dataset of simulated inference requests. Our results indicate that the proposed "GenAI-Aware Scaling Engine" (GASE) reduces cold-start latency by approximately 35% while lowering idle resource costs by 22% compared to reactive baseline models. Furthermore, we demonstrate the efficacy of Ansible-based orchestration in translating these simulation-derived policies into actionable runtime configurations on Azure PaaS. These findings suggest that a shift from reactive to simulation-validated predictive scaling is essential for the sustainable scaling of enterprise-grade AI applications.