Troubleshooting GenAI apps

Start with the error. ValidationException means the request is wrong: a bad body or parameter, a model id that doesn't exist in that Region, or input longer than the model's context window. AccessDeniedException means IAM doesn't allow the call (often the role lacks bedrock:InvokeModel on that model or inference profile ARN, or an SCP blocks it); for some third-party models a first-use agreement or Marketplace subscription is also needed. ThrottlingException means you're over a quota: back off and retry, spread load, or raise the quota. ModelTimeoutException and server errors are worth retrying with backoff.

Context window overflow. Everything counts against the window: system prompt, conversation history, retrieved chunks, tool definitions and the reply. Fixes: send fewer or smaller chunks, summarize or drop old turns, shorten instructions, split a long document and process it in parts (summarize each, then combine), or use a model with a larger window. Count tokens before sending so you fail early with a clear message.

Truncated or incomplete answers. A stop reason of max_tokens means the reply hit your limit. If a long document is summarized badly, check whether the input was silently truncated somewhere in your own pipeline. If JSON is cut off, raise the limit or ask for less.

Retrieval problems look like model problems. Check what was actually retrieved for a bad answer before touching the prompt. Wrong chunks: try hybrid search, a reranker, better chunking or metadata filters. Nothing relevant: the document may not be ingested (check the sync job), or it may be an image or scan that wasn't converted to text. Good results that got worse over time: embeddings drift when the documents or the embedding model change, so re-embed everything with one model.

Prompt problems. If answers changed and you don't know why, compare prompt versions (Prompt Management keeps them) and run your test set on each. If output formats vary, validate against a JSON schema in code, retry with the error message, and tighten the format instructions. If the model mixes up instructions and data, separate them more clearly. Use CloudWatch Logs and X-Ray to see the exact prompt that was sent, after templating, since the bug is often in the filled-in variables.

Work methodically: reproduce with one request, look at the full logged prompt and response, change one thing, and re-run the test set. Amazon Q Developer can help read stack traces and suggest fixes in code.