A data-handling review of AI telemetry using Bedrock, OpenTelemetry, and CloudWatch documentation. It maps collection paths, minimizes content, separates destinations, and makes access and retention responsibilities visible.
At a glance
Key findings
- An observability pipeline can become a separate repository of prompts, documents, tool arguments, and generated content.
- Decide which diagnostic questions require content before enabling capture. Metadata often supports a different set of questions with less exposure.
- Review collection, export, access, and retention independently. Turning off a capture switch is not a deletion plan for existing copies.
Find the second copy of the conversation
An engineer investigating a slow AI request may need to know which model was called, how long retrieval took, whether a tool failed, and how many attempts occurred. None of those questions automatically requires a complete copy of the user's document. Yet a debugging configuration can collect that document along with the prompt, tool arguments, and response.
Map where those copies can go. Include application logs, tracing instrumentation, provider invocation logs, collector queues, dashboards, exports, and archived investigation bundles. The map is specific to the deployed application. A privacy statement about one model endpoint does not describe a separate logging library or a destination operated by the application team.
Amazon Bedrock provides one concrete example: invocation logging can collect request data, response data, and metadata, but it is disabled by default. The current documentation limits this feature to supported calls through the bedrock-runtime endpoint. That documented default does not establish what an application's own logger captures. [1]
Start with the diagnostic question
Build the telemetry specification from the investigation questions the team expects to answer. Duration can help locate a slow stage. An error class can distinguish a denied tool call from a failed connection. A model identifier and deployment revision can connect a change to an observed regression. These fields have different purposes from the text supplied to the model.
OpenTelemetry's May 2026 GenAI observability example separates ordinary metadata from opt-in content capture. The example describes model names, token counts, and durations without prompt content by default. Treat that as the behavior of the documented instrumentation, not a universal guarantee for every SDK or integration using the OpenTelemetry name. [2]
For each proposed field, write the question it answers and the audience allowed to inspect it. If nobody can name a routine diagnostic use, leave the field out until a specific need is established. If content is necessary for a bounded investigation, collect the minimum material for that purpose through an explicit exception path rather than silently broadening the default trace.
Metadata and content need different handling
Routine diagnostics and optional content capture should have explicit collection, access, and retention decisions.
Source. Bedrock logging, OpenTelemetry, and CloudWatch documentation [1]-[4].
Method. Original proposed data-handling design. Rows describe collection decisions, not measured redaction effectiveness or a verified vendor configuration.
Accessible table and figure data
| Field | Diagnostic purpose | Proposed handling |
|---|---|---|
| Duration | Locate slow stages | Routine metadata with an identified owner |
| Error class | Distinguish failure modes | Routine metadata without embedded request content |
| Model and deployment ID | Connect behavior to a version | Record approved identifiers |
| Token count | Understand usage and request shape | Record counts rather than text where sufficient |
| Prompt or retrieved document | Investigate a content-specific issue | Explicit capture decision and restricted destination |
| Tool arguments and results | Investigate a tool-specific issue | Review nested content and external copies |
| Generated response | Investigate output behavior | Apply a separate content retention decision |
| Field | Diagnostic purpose | Proposed handling |
|---|---|---|
| Duration | Locate slow stages | Routine metadata with an identified owner |
| Error class | Distinguish failure modes | Routine metadata without embedded request content |
| Model and deployment ID | Connect behavior to a version | Record approved identifiers |
| Token count | Understand usage and request shape | Record counts rather than text where sufficient |
| Prompt or retrieved document | Investigate a content-specific issue | Explicit capture decision and restricted destination |
| Tool arguments and results | Investigate a tool-specific issue | Review nested content and external copies |
| Generated response | Investigate output behavior | Apply a separate content retention decision |
Inspect the capture path rather than one log line
A short event displayed in a dashboard may reference a larger object elsewhere. Bedrock's invocation logging documentation describes separate S3 storage for binary content and JSON bodies larger than its inline limit. A review that only samples the visible log entry can therefore miss the content store that matters most. [1]
Inventory the configuration at each capture point and the credentials that can change it. Record which calls are included, whether request and response content are enabled, where references point, and which export processes create additional copies. Include failures as well as successful requests. Debugging paths are especially easy to overlook because they may only become active during an incident.
The review should produce a concrete destination list. For each destination, identify the owner, access boundary, retention setting, encryption configuration, and onward exports. Do not infer those properties from a parent service name. Two log groups or buckets in the same account can serve different audiences and have different lifecycle decisions.
Minimize before data leaves the application boundary
OpenTelemetry's sensitive-data guidance recommends collecting only what serves an observability purpose and provides processor options for removing or transforming attributes. It also warns that hashing predictable identifiers may not provide the intended anonymity. Those controls require knowledge of the application; the telemetry system cannot determine every organization's sensitive fields by itself. [4]
Prefer an explicit list of permitted fields over an assumption that anything not obviously secret is safe to export. Review nested structures, exception messages, tool results, and dynamically added attributes. A top-level field rule is incomplete if the same value appears inside a serialized request body. Keep the rules understandable enough that the owning team can maintain them after an SDK upgrade.
When content capture is approved, separate the content path from routine operational telemetry where the architecture permits it. Give it its own purpose, access decision, and expiry. This is a proposed design, not a guarantee that redaction is complete. A transformation can reduce exposure while still missing an unexpected representation or a value whose sensitivity depends on context.
Set retention and access on every destination
CloudWatch Logs retains log data indefinitely by default unless a retention period is configured. Its documentation also distinguishes expiration from physical deletion, which can occur later. That behavior matters when writing an operational retention procedure, but it does not describe an S3 export or a third-party trace backend. Each destination needs its own review. [3]
Decide who can enable content capture, who can read the resulting data, and who can export it. These need not be the same role. An investigator may require temporary access to a particular case without needing permission to change logging for every application. A platform operator may manage retention without needing routine access to private prompts.
Write the end of an investigation into the procedure. Disable the temporary capture configuration, close temporary access, identify the copies made for the case, and apply the approved retention decision to each. Do not claim that deleting a logging configuration erases all historical content. Equally, do not delete evidence subject to an authorized preservation requirement merely to satisfy a generic cleanup checklist.
Verify the policy with synthetic markers
A useful acceptance test places distinct harmless marker strings in a prompt, a retrieved document, a tool argument, and a generated response. Exercise normal and error paths, then inspect the configured sinks for those markers. Use synthetic material so the verification itself does not spread real private content into an evidence package.
The expected result should be specific. Metadata required for diagnosis should remain present. Content prohibited from routine telemetry should be absent from the destinations covered by the test. If an approved content store is in scope, verify its access and retention configuration separately. Record which sinks were inspected and which could not be inspected; absence from one dashboard does not prove absence from the pipeline.
Repeat this check when instrumentation, exporters, exception handling, or provider logging settings change. Keep the field specification and destination inventory with the application so an incident responder can tell what evidence is available without enabling every debug option. The goal is useful observability with a deliberate content boundary, not the largest possible archive of everything an assistant has seen.
Method and provenance
Primary documentation review completed August 28, 2026, with an original collection policy, destination inventory, and synthetic-marker test proposal.
No application logs, provider account, collector, or retention workflow was inspected or tested. Defaults and capture behavior are scoped to the cited documentation and integrations.
AI assistance. AI assistance was used to research sources, draft and structure the article, and prepare the visual specification. No human technical review is claimed.
Published under the Cloud Security Desk organizational byline. Read the series policy.
References
- Monitor model invocation using CloudWatch Logs and Amazon S3 AWS. Accessed .
- Inside the LLM Call OpenTelemetry. Published . Accessed .
- Working with log groups and log streams AWS. Accessed .
- Handling sensitive data, modified January 14, 2026 OpenTelemetry. Accessed .