A small request-building sequence distinguishes model context, output allowance, endpoint support and actual input-token counts without pretending the count reserves quota. Use Amazon Bedrock's supported token-counting operation on the same input that the application intends to send for inference.
At a glance
Key findings
- Use Amazon Bedrock's supported token-counting operation on the same input that the application intends to send for inference. Token counting is model-specific, so a count for one model is not a reliable count for another. It helps check whether a prompt fits the selected model and the application's input allowance. It does not generate the answer, reserve inference capacity, or tell you how many output tokens the model will eventually produce. [1][2]
- A small request-building sequence distinguishes model context, output allowance, endpoint support and actual input-token counts without pretending the count reserves quota.
- The diagrams are conceptual; documented numeric values describe AWS specifications, not a measured customer environment.
Count the exact request for the exact model
Use Amazon Bedrock's supported token-counting operation on the same input that the application intends to send for inference. Token counting is model-specific, so a count for one model is not a reliable count for another. It helps check whether a prompt fits the selected model and the application's input allowance. It does not generate the answer, reserve inference capacity, or tell you how many output tokens the model will eventually produce. [1][2]
The first prerequisite is support for the model and endpoint you use. Current Bedrock documentation distinguishes the CountTokens operation on bedrock-runtime from an Anthropic token-counting path on bedrock-mantle for applicable Claude models. Do not assume that a model supports runtime token counting because it supports Converse or has a large context window. Check its current model card and the token-counting guide before choosing the request path. [1]
This guide uses a small text-only request to explain the workflow. It is about checking prompt size before inference, not designing a full spending-control or concurrency system. A team that needs strict per-user budgets, admission control, or rate enforcement has additional application work to do. Counting the request can supply one useful input to those controls without replacing them.
The useful result is a repeatable path from one constructed request to its count, the application's decision, and the actual inference usage when a request is sent. Keep the model identifier, endpoint, relevant configuration, and time of the observation. Do not record the entire prompt by default merely to prove that counting happened. A small amount of nonsecret metadata can make the behavior reviewable without creating an unnecessary copy of user content.
Separate input size from the other limits
A context window describes the model's capacity to work with a request's context under its documented rules. A generated-output limit describes how long the response may be. Service quotas control account or model usage over time, while an application spending rule governs what the business is willing to admit. These limits are related but different. Raising one does not automatically raise the others, and a request that fits a context window can still fail for another reason. [1][3][4]
The chart compares documented context capacities for three active Nova models: Nova Micro at 128K tokens, Nova Lite at 300K, and Nova 2 Lite at 1M. It expands K and M into decimal thousands and millions for the plotted values. These are specification values from the current model cards, not measured token counts, quality scores, or recommendations to select the largest model. The cards must be checked again when choosing a model for a new deployment. [5][6][7]
Context capacity does not establish token-counting support. The Nova 2 Lite card explicitly lists runtime token counting as unsupported in the source reviewed on September 12, 2026. Its larger context window therefore cannot be used as evidence that the CountTokens example works with that model. This is the main purpose of keeping capability checks separate from numeric limits: a helpful specification chart should not silently become an unsupported API compatibility claim. [7]
Set the output allowance deliberately. InferenceConfiguration defines maxTokens as the maximum generated response length, with model-specific limits. An application asking for a short summary usually has no reason to leave its output behavior unexplained. A permitted maximum is a cap, not a promise of that much text. The application still needs to inspect the response and stop reason, because a response stopped at the cap may be incomplete for the intended task. [3][4]
Context capacity differs from token-counting support
Documented context capacity differs by model. It does not establish CountTokens support, an output allowance or an application spending limit.
Source. AWS documentation accessed September 12, 2026. [5] [6] [7]
Method. Values transcribed from cited AWS documentation. Labels are editorial; this is not measured tenant data. Scope: Context capacity from three active Nova model cards, not an inference benchmark or CountTokens support matrix. K and M expanded as decimal thousands and millions.
Accessible table and figure data
| Active model | Documented context tokens |
|---|---|
| Nova Micro | 128000 |
| Nova Lite | 300000 |
| Nova 2 Lite | 1000000 |
| Active model | Documented context tokens |
|---|---|
| Nova Micro | 128000 |
| Nova Lite | 300000 |
| Nova 2 Lite | 1000000 |
Check the model and endpoint before coding
Start from the exact model ID used in the application, including an inference profile when applicable, and identify the endpoint that serves the request. Bedrock's token-counting guide describes which operation applies and directs readers to current model details for support. A nearby code sample for another model can show the shape of the API without establishing compatibility with your selected one. Keep that distinction in the implementation note. [1]
For supported bedrock-runtime models, CountTokens accepts input in the InvokeModel or Converse shape. InvokeModel bodies are model-specific, while the Converse counting input contains the structured conversation fields supported by the counting API. Read the current schema rather than copying every field from a full Converse request indiscriminately. Parameters that govern output generation are not themselves a count of the input content. [1][2][3]
Some Claude models require the separate documented Anthropic count_tokens route on bedrock-mantle. AWS notes differences in authentication, IAM action, request shape, SDK support, and error format for that route. Treat it as a distinct integration. Do not point the normal Boto3 bedrock-runtime count_tokens method at a model that the documentation says is unsupported and interpret the resulting validation error as a prompt-size problem. [1]
Review permissions with the same specificity. Bedrock supports IAM policies, temporary credentials, resource scoping, and service-specific actions. Grant the application's intended counting and inference operations under the relevant model or supported resource boundary, rather than a blanket Bedrock administration policy. The token-counting guide names the needed actions for its example. A successful counting request does not prove permission to invoke every model or perform unrelated Bedrock operations. [1][9]
Build the request once and count that object
Construct the messages and system instructions once in application code, then pass that same content to the counting operation. Count the actual conversation, not just the last sentence the user typed. Previous messages, instructions, documents, and tool-related content can affect the real request where the chosen model and API support them. A word count in the user interface is not a substitute for model-specific tokenization. [1][2]
The example accompanying this section uses a harmless text prompt and the documented Boto3 request shape. It prints the returned inputTokens value rather than inventing an example result. The SDK must be installed and configured with an authorized identity and Region, and the selected model must support this counting path in that environment. The snippet is a bounded demonstration of the API shape, not a complete production gateway or a claim that a tenant test was performed.
After counting, apply the application's documented decision. It may reject an oversized request, ask the user to narrow the task, or construct a smaller input and count again. Do not silently change the prompt after the count and still attribute the old number to the new request. If retrieval or another component adds content later, move the counting step after that assembly point or establish a separate controlled allowance for the additional material.
Keep the content stable between the count and invocation within the application's request flow. A nonsecret request identifier or a carefully controlled digest can help correlate the two without logging plaintext. A digest is still metadata that should follow the application's data policy; do not treat it as permission to expose information about sensitive requests publicly. The important property is traceability from the count to the same constructed input, not the choice of a fashionable identifier format.
import os
import boto3
client = boto3.client("bedrock-runtime", region_name=os.environ["CSD_AWS_REGION"])
model_id = os.environ["CSD_BEDROCK_MODEL_ID"]
conversation = {
"messages": [{
"role": "user",
"content": [{"text": "Summarize why backups should be tested."}]
}],
"system": [{"text": "Use plain language and three concise sentences."}]
}
result = client.count_tokens(
modelId=model_id,
input={"converse": conversation}
)
print(result["inputTokens"])Keep counted content attached to the eventual request
Count the actual request with a supported model, apply the application allowances, and send the same content for inference.
Source. Conceptual synthesis of AWS documentation accessed September 12, 2026. [1] [2] [3] [4]
Method. Conceptual editorial synthesis of the cited service behavior; no measured outcomes. Scope: How can an application check the size of a Bedrock prompt before inference?
Accessible table and figure data
| Step | Responsibility |
|---|---|
| Build request | Include the actual messages and system instructions |
| Count request | Use the supported model and endpoint |
| Decide | Apply application input and output allowances |
| Invoke | Send the same counted content |
| Review | Compare returned usage and stop reason |
| Step | Responsibility |
|---|---|
| Build request | Include the actual messages and system instructions |
| Count request | Use the supported model and endpoint |
| Decide | Apply application input and output allowances |
| Invoke | Send the same counted content |
| Review | Compare returned usage and stop reason |
Handle a prompt that does not fit
When a request exceeds the allowed input size, first decide what information the task actually needs. A user asking about one section of a document may not need the entire document and every previous conversation turn. Retrieval, user selection, or a narrower question can reduce the input while keeping the relevant material. These are application design choices, and they should make any omitted information clear enough that the user understands the resulting answer's scope.
Avoid blindly chopping text at a byte, character, or word boundary and assuming the result fits. Tokenization is model-specific, and a truncation can remove a qualification, heading, table context, or instruction that changes the meaning. Rebuild the smaller request and count that request again through the supported path. If the application summarizes old content, recognize that the summary is a new representation with possible omissions, rather than the original source preserved at a lower cost. [1][2]
Selecting a model with a larger context window is another option, but it requires more than replacing an ID. Verify the model's capabilities, availability, data handling, output behavior, and token-counting support. The chart's larger bar says nothing about answer quality on your task or whether the model is an appropriate recipient for the data. Use the specification as a compatibility constraint and evaluate the application behavior separately. [5][6][7]
Provide a useful rejection message. Tell the user which action they can take, such as choosing fewer documents or starting a new conversation, without exposing internal service errors or private configuration. Preserve the user's input in the application's normal draft flow when appropriate so a size rejection does not force them to reconstruct work. The interface should help them complete a narrower task, not merely report a token number that has no clear next step.
Treat counting as another data submission
Counting through a hosted API sends the relevant input to that service endpoint. Treat the operation as part of the application's data flow and apply the same review of permitted content, account, Region, endpoint, and access controls that the application requires for inference. The fact that the operation returns a number does not make the submitted prompt anonymous or remove the need to evaluate where it is processed. [1][2][8]
Use authorized temporary credentials through the normal SDK configuration rather than embedding credentials in the example or application repository. Limit access to the operation and avoid putting user content into resource names, tags, or unnecessary diagnostic fields. AWS's Bedrock data-protection guidance specifically warns against placing sensitive information into free-form fields that can become part of diagnostic or billing records. The application's own logging can create additional copies even when the model operation is correctly configured. [8][9]
Log the minimum metadata needed to operate the feature. That may include model ID, endpoint family, request correlation ID, count, outcome category, and timing. Decide deliberately whether any prompt content is needed for a specific investigation, under restricted access and retention. Do not print the full request on every exception. A validation error containing request details can otherwise turn an apparently harmless preflight step into a persistent store of sensitive prompts.
Keep privacy decisions model-specific where the service's current terms and behavior require it. A general claim that no Bedrock operation ever retains any data would be broader than the narrow CountTokens API contract. Review the applicable service documentation and the organization's approved use for the selected model and endpoint. This article explains the token-counting workflow; it does not certify a particular deployment's data residency, confidentiality, or retention obligations.
Give counting failures a clear outcome
Handle unsupported input, permission denial, service throttling, and network failure as different outcomes. An unsupported model or endpoint usually requires a configuration decision. Repeatedly retrying the same incompatible request will not turn it into a supported one. A denied request should lead to a review of the approved IAM scope and identity. It should not trigger a fallback to a more powerful credential embedded in the application. [1][2][9]
For temporary failures, use the application's bounded retry and timeout policy. Counting is another request in the user flow and can add latency or fail independently of inference. If the feature is required to enforce the application's input rule, decide whether a counting failure blocks inference or uses a separately approved fallback. Do not silently skip a mandatory check just because the service returned an error at an inconvenient moment.
A fallback estimate should be labeled as an estimate. A generic characters-per-token rule is not equivalent to the model-specific count returned for the same request. If the application uses a conservative local bound for availability reasons, document its limitations and validate it against the chosen model and input types. Avoid displaying an exact-looking number that suggests service verification when the hosted counting operation never completed. [1]
Keep the original request intact while reporting the failure. A user should not lose their work because a preflight service is unavailable. Return a clear retry or input-reduction option without leaking credentials, internal endpoints, or raw provider responses. After the issue is corrected, count the current request again if its content or model changed. Reusing a count from another model or an earlier edited prompt can reintroduce the very mismatch the preflight was meant to avoid.
Compare the estimate with response usage
When inference is performed, inspect the returned usage and stop reason alongside the counting result. Converse returns structured usage information, while the exact response contract depends on the operation used. Record the input count from the preflight and the relevant actual usage fields without assuming that output tokens were known in advance. The count describes the input; the response tells the application what the invocation reported and why generation stopped. [2][3]
A discrepancy should lead to a review of the request path. Confirm that the same model, endpoint, messages, and system content were used, and inspect any retrieval or transformation that occurred after counting. Also check the documented treatment of model-specific features. Do not immediately declare the API inaccurate because the application compared different request representations or confused an input field with total input-plus-output usage.
Test the feature with deliberately small, nonsecret cases before enabling it for users. Include a supported count, a request near the application's chosen allowance, an oversized request, an unsupported model path, and a denied operation in a controlled environment. The test should verify the application's decisions and error handling rather than fabricate a benchmark. This guide's code can be syntax-checked locally, but no successful live invocation is implied without the reader's own authorized execution evidence.
The final implementation record should identify the supported counting route, selected model, request assembly point, input rule, output allowance, failure behavior, and minimal logging fields. Revisit it when the model or request format changes. Token counting becomes useful when it is attached to the actual request and a clear user-facing decision, rather than displayed as a decorative number that the application ignores when it sends a different prompt.
Method and provenance
Primary AWS documentation was retrieved and reviewed on September 12, 2026. The guide synthesizes documented service behavior into a bounded operational procedure; research records and figure data are maintained with the article.
Examples are hypothetical. Code and request shapes are checked locally where applicable, but no customer AWS account, production operation, recovery duration or benchmark was tested. Readers must verify their resource type, Region, permissions and organization controls.
AI assistance. AI assisted research organization, drafting and original visual planning. Sources, technical boundaries and final rendering are reviewed through the publication workflow.
Published under the Cloud Security Desk organizational byline. Read the practitioner guide policy.
References
- CountTokens - Amazon Bedrock AWS. Accessed .
- Converse - Amazon Bedrock AWS. Accessed .
- InferenceConfiguration - Amazon Bedrock AWS. Accessed .
- Nova Micro - Amazon Bedrock AWS. Accessed .
- Nova Lite - Amazon Bedrock AWS. Accessed .
- Nova 2 Lite - Amazon Bedrock AWS. Accessed .
- Data protection - Amazon Bedrock AWS. Accessed .
- How Amazon Bedrock works with IAM - Amazon Bedrock AWS. Accessed .