Building custom model provider for Strands Agents with LLMs hosted on SageMaker AI endpoints
TL;DR
This post demonstrates how to build custom model parsers for Strands agents when working with LLMs hosted on SageMaker that don't natively support the Bedrock Messages API format. We'll walk through deploying Llama 3.1 with SGLang on SageMaker using awslabs/ml-container-creator, then implementing a custom parser to integrate it with Strands agents.
Nauti's Take
Strands deployments get stuck in the default SageMakerAIModel parsing, so if you roll Llama 3.1 on SGLang you must code the parser yourself before the agent ever boots. Stream handling, usage metadata, and Bedrock-ready payloads belong in your deployment checklist now or the endpoint stays a black box with a TypeError.
Sources
