New agent skill: Amazon SageMaker optimized generative AI inference for your coding agent

Overview of Amazon SageMaker Optimized Generative AI Inference

Amazon SageMaker optimized generative AI inference introduces the aws-ai-ml skill, which is available through the Agent Toolkit for AWS. This skill equips coding agents such as Kiro, Claude Code, and Codex with the ability to perform inference optimization and benchmarking. Once installed, the agent can benchmark endpoints, recommend deployment configurations, compare performance runs, and generate executable SageMaker Python SDK v3 code on your behalf.

Key Capabilities and Use Cases

The aws-ai-ml skill functions as a toolkit that integrates with any coding agent that supports the Model Context Protocol (MCP), transforming it into a SageMaker AI inference optimization expert. This skill enables users to bridge the gap between intent and infrastructure by allowing them to describe their goals in natural language, after which the agent generates executable code that can be reviewed, modified, and executed in the user's environment.

Users can ask the agent to benchmark an existing endpoint, find the right instance type for their model, or compare benchmark runs. The agent asks targeted clarifying questions, generates code grounded in real benchmarks and measured performance data, and adapts to the user's business constraints, similar to how a solutions architect would approach the task.

Getting Started with the aws-ai-ml Skill

Users can install the aws-ai-ml skill through the Agent Toolkit for AWS on their local machine or use it within an Amazon SageMaker Studio JupyterLab space. The installation process involves setting up the Agent Toolkit, installing the skill, and confirming its availability within the coding agent.

For users who prefer a managed environment, the skill can be used within Amazon SageMaker Studio with a pre-configured image. This involves launching a JupyterLab space, authorizing the coding agent, and confirming the availability of the skill within the agent's chat panel.

Prerequisites and Considerations

Before using the aws-ai-ml skill, users must ensure that their AWS credentials have the necessary permissions to call SageMaker AI APIs, including creating endpoints and running benchmark and recommendation jobs. No additional AWS Identity and Access Management (IAM) configuration is required for the skill itself.

For Kiro and Claude Code agents, skills can be discovered at runtime through the AWS MCP Server, allowing users to search for and load skills on demand without local installation. This feature enhances flexibility and reduces the need for manual configuration.

What You Can Do with the aws-ai-ml Skill

The agentic experience provided by the aws-ai-ml skill covers several capabilities across the inference optimization lifecycle. Users do not need to know which capability to invoke; they simply describe their intent, and the agent determines the next step or asks clarifying questions if needed.

Some of the key capabilities include benchmarking an existing endpoint, finding the right instance type for a model, and comparing benchmark runs. The agent generates code that evaluates models against candidate instances and configurations, presenting ranked deployment options with concrete performance metrics such as throughput, latency percentiles, time-to-first-token, and concurrency.

When benchmarking an existing endpoint, the agent generates a Python notebook that runs a load test using the Workload.synthetic() and start_benchmark() APIs from the SageMaker Python SDK. Before running any benchmark, the agent confirms that the endpoint is safe to load-test, as benchmarking drives real traffic to a live endpoint.

When comparing benchmark runs, the agent computes deltas across key metrics such as throughput, latency percentiles, and time-to-first-token, providing a single, interpretable summary of whether a change improved performance, degraded it, or had no meaningful effect.

Source: AWS Machine Learning. Analysis based on publicly available information; no hands-on testing is claimed.