Calling a model in Amazon Bedrock

Amazon Bedrock is AWS's managed service for foundation models. You call models from Amazon, Anthropic, Meta, Mistral AI and others through one API, pay per token (or for reserved capacity), and never manage a server. Most exam questions that say "use a foundation model with the least operational overhead" are pointing at Bedrock.

There are two ways to call a model at runtime. InvokeModel takes a request body in the format that model's provider defines, so the JSON for one model family isn't the JSON for another. The Converse API (Converse, and ConverseStream for streaming) takes the same shape for every model that supports it: a list of messages with user and assistant roles, an optional system prompt, and an inferenceConfig. If a question says the team wants to switch models without rewriting request code, Converse is the answer.

bedrock_runtime.converse(
    modelId="<model id or inference profile>",
    system=[{"text": "You are a support assistant. Answer in two sentences."}],
    messages=[{"role": "user", "content": [{"text": "Where is my order?"}]}],
    inferenceConfig={"maxTokens": 300, "temperature": 0.2},
)

Models don't remember earlier calls. A chat app keeps the conversation itself and sends the history back with each request. A common pattern is to store each session's turns in Amazon DynamoDB, keyed by session id, and trim or summarize old turns so the history fits in the model's context window.

Inference parameters change how the reply is picked. temperature sets randomness: low (near 0) for extraction, classification and anything that must be consistent; higher for brainstorming. topP limits choices to the most likely tokens whose probabilities add up to that share, and some models also take topK, a fixed number of candidate tokens. maxTokens caps the length of the reply, which also caps its cost. Stop sequences end the reply when the model writes a chosen string.

Streaming (ConverseStream or InvokeModelWithResponseStream) sends the reply as it's generated, so a user sees the first words in a fraction of the time the full answer takes. It doesn't make the whole answer faster, but it makes the app feel fast. The reply also says why it stopped: end_turn when the model finished, max_tokens when it hit your cap (the answer is cut off), tool_use when it wants a tool run, and a guardrail or content filter reason when one stepped in.