
Overview
vLLM is a high-throughput library for serving large language models behind an OpenAI-compatible API. In this codelab you build the CPU Docker image for vLLM, serve Llama-3.2-1B-Instruct locally with a Hugging Face token, and call it with the standard OpenAI Python client. You then push the image to Google Artifact Registry, deploy it on Cloud Run, point the same client code at the cloud URL, and learn about GPU and alternative hosting options plus how to clean up to avoid charges.
vLLMDockerGoogle CloudHugging FaceOpenAI
Join Stepwik
Build and share what you know
Create interactive labs and courses that help developers learn faster - then publish them to your network.
- Create labs
- Share with learners
- Grow your audience
