NVIDIA NeMo Switchyard Enables Dynamic Model Selection with Amazon Bedrock
NVIDIA NeMo Switchyard is an open source library that helps AI agents choose the most appropriate model for each request when integrated with Amazon Bedrock.
Model Selection and Routing with NVIDIA NeMo Switchyard
NVIDIA NeMo Switchyard is an open source library that helps an AI agent choose which model handles each request . When integrated with Amazon Bedrock, it enables dynamic model selection based on request content, improving efficiency and performance. Developers can begin by cloning the sample repository and defining model groups in LiteLLM, which allows grouping multiple model deployments under a single public model name . An example configuration in profiles/stage/litellm.yaml illustrates how to set up a routed switchyard group alongside baseline groups, specifying model names and deployment details such as bedrock/converse/global.anthropic.claude-opus-5-5 and bedrock/converse/nvidia.nemotron-super-3-120b . This setup provides flexibility for routing requests to the most appropriate model without requiring changes to the application layer.
Implementation and Configuration Details
Implementation and configuration of the route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard requires specific setup steps. The coding agent sends an OpenAI-compatible request with the model field set to switchyard, the LiteLLM model group that Switchyard routes, and the shared gateway key in the Authorization header (AWS Machine Learning). The endpoint is configured as your hostname over HTTPS, with an AWS Certificate Manager (ACM) certificate on the load balancer, and only the IPv4 CIDR range that you allow can reach it (AWS Machine Learning). The Application Load Balancer forwards only /v1/chat/completions, /v1/models, and /health/liveliness to the ECS task in a private subnet, and answers every other path with 404 (AWS Machine Learning).
Execution and Task Evaluation
In the execution and task evaluation phase of the Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard, the system selects the most efficient model for new requests. According to AWS Machine Learning, when a new request has no tool results to score, the system defaults to the efficient model, bedrock/converse/nvidia.nemotron-super-3-120b. To try the agent, users can use $GATEWAY_URL/v1 as the OpenAI-compatible base URL, the gateway key as the API key, and specify 'switchyard' as the model, as noted in AWS Machine Learning. For task execution, users can list the tasks and start with slugify on the routed group using the commands provided in AWS Machine Learning. The terminal output displays the task, selected models, tool actions, test results, and the final code diff.
Sources and reporting notes
Based on published source material. No independent hands-on testing is claimed.