A large-object intake guide for S3 multipart upload. It covers current size and pagination limits, part identity, checksum initialization and full-object versus composite semantics, embedded completion errors, conditional writes and verified abort cleanup.
At a glance
Key findings
- The application needs its own ordered part manifest rather than treating one ListParts response as the completed upload.
- A CompleteMultipartUpload response can begin with HTTP 200 and later contain an embedded error.
- Checksum type, algorithm and initialization determine what S3 actually validates; an ETag is not a universal whole-object hash.
Define the evidence for a completed object
Treat an S3 multipart upload as complete only after the intended part manifest has been accepted, the final completion result is understood and the object has the integrity and identity evidence the application requires. Individual successful part uploads are intermediate results. Even the initial HTTP status from CompleteMultipartUpload is not sufficient on its own, because S3 can begin the response with 200 OK and later return an embedded error. [2][4]
Multipart upload separates initiation, part transfer and completion. The application obtains an upload ID, sends parts under that identity and supplies the information needed to assemble the object. Those steps create useful retry and parallelization options, but they also introduce state that must survive worker failures and ambiguous network responses. A progress bar reaching its expected byte count does not establish that the intended object was committed successfully. [2]
Define the desired object before transfer begins. The intake record should identify the destination key, expected content scope, chosen checksum behavior and owner of the completion decision. If multiple clients or workers participate, specify which component owns the authoritative upload state. This is a proposed application contract, not a new S3 API object. Its purpose is to prevent each worker's local success from being mistaken for global completion.
Keep object integrity separate from content approval. A matching checksum can establish agreement with an expected byte sequence, but it does not decide whether an uploaded document is safe to parse, belongs to the intended tenant or is permitted for downstream use. The application may need a quarantine or approval stage after storage completion. S3 completing an object is evidence about the storage operation, not every business decision involving its contents. [3][9]
The scope here is current general-purpose S3 bucket behavior. Directory buckets, Outposts and other specialized access arrangements have additional endpoint, permission and checksum constraints in the API documentation. Do not silently apply one generic endpoint or authentication recipe to those cases. Record the actual bucket and supported API path before using the acceptance model. [4][5][6][7]
Own the part manifest
Current S3 documentation lists a maximum multipart object size of 48.8 TiB and a maximum of 10000 parts, numbered from 1 through 10000. Each part can be between 5 MiB and 5 GiB, except that the final part has no minimum size. These are documented limits as reviewed on September 2, 2026, not observations about a particular upload client. Do not carry forward the older 5 TB ceiling into a current design without checking the service documentation. [1]
Choose part sizing against both the part-size range and the maximum part count. A strategy that works for ordinary files can run out of part numbers for a larger supported object. The application should reject an unsupported plan before transferring substantial data or use a supported adjustment strategy. This guide does not prescribe one universally optimal part size, because concurrency, memory and retry costs depend on the client and workload.
For each completed part, retain the part number and returned ETag, together with the checksum information required by the selected upload mode. AWS's overview says the application should maintain its own list for completion. The manifest should remain associated with the upload ID and destination object so a retry or worker restart cannot accidentally mix parts from different logical submissions. [2]
Uploading a part under an already-used part number replaces the earlier part for that multipart upload. This can be useful for retrying or correcting a part, but the owned manifest must reflect the current successful result. An earlier ETag stored by another worker may no longer identify the intended part. Coordinate updates to the upload record rather than letting independently successful workers overwrite each other's completion data without a rule. [5]
Completion requires the appropriate parts in ascending part-number order. Additional checksum rules can be stricter: AWS's detailed integrity guidance says composite-checksum multipart uploads need consecutive part numbers beginning with one. Distinguish the general ordering requirement from the selected checksum mode's requirements instead of assuming that every legal-looking sparse part list will work under every configuration. [4][9]
A manifest is also an ownership boundary. If a client can substitute the key, upload ID or part list at completion, the service should verify that those values belong to the authorized intake operation. A valid S3 request signature does not establish that the application selected the right logical upload. Keep presigned request delegation and the application's completion authority connected without treating them as the same control.
Do not confuse one page with the upload
ListParts returns at most 1000 parts in one response. The API provides truncation and part-number marker information for continuing the listing. A client that examines only the first response can miss most of a maximum-size part manifest while still receiving a perfectly successful API result. Handle pagination explicitly whenever the listing is used for verification or recovery. [1][7]
The chart compares the maximum of 10000 parts in an upload with the maximum of 1000 parts in one ListParts response. Ten thousand parts require at least ten responses; ten suffice when each returns 1000 parts. This is a calculation from documented limits, not a measured API throughput result. Smaller response pages can require more requests, so the chart is not a promise of a fixed ten-call reconciliation procedure.
Preserve the API's continuation state rather than inventing a page number from assumptions about how many parts exist. The returned marker and truncation information describe the listing contract. A recovery routine should also explain how it handles an interrupted listing and whether part transfers can continue while it is observing state. A collection of pages taken during concurrent changes needs more care than a static array returned by a local function. [7]
AWS explicitly says to use the returned list for verification and to maintain the application's own part-number and ETag list for completion. Do not replace the owned manifest with a freshly observed listing merely because listing looks easier than persisting upload state. The owned record explains what the application intended to upload and which part results it accepted. A listing alone does not capture all of that business context. [2]
In a hypothetical recovery, the application has recorded some completed parts and a worker lost its response for another part. The operator or recovery process can compare the owned record with S3's observed parts to resolve the discrepancy under a defined rule. It should not label every unrecorded part valid automatically. Confirm the upload identity, intended part content and selected checksum contract before incorporating evidence into the completion record.
Audit evidence needs similar scope. A log showing a ListParts request establishes that a listing was attempted, not that every page was processed or compared correctly. If an investigation needs evidence of object operations, configure and verify the relevant event coverage separately. The application's manifest and the platform's access records can complement one another without either replacing the other.
Upload capacity is larger than one listing response
One S3 multipart upload can contain 10000 parts, while one ListParts response returns at most 1000. Ten thousand parts require at least ten responses; ten suffice when each returns 1000 parts. Documented limits, not measured performance, checked 2026-09-02.
Source. AWS, S3 multipart limits, Current object, part count, part size and listing limits. [1]; AWS, ListParts API, Pagination and part-number markers [7].
Method. Source-derived documented values, not measured performance. Values transcribed directly. ceil(10000/1000)=10 is a lower bound on response count; ten suffice only when each response contains 1000 parts. Reviewed 2026-09-02.
Accessible table and figure data
| Documented scope | Parts |
|---|---|
| Maximum parts in one upload | 10000 |
| Maximum parts in one ListParts response | 1000 |
| Documented scope | Parts |
|---|---|
| Maximum parts in one upload | 10000 |
| Maximum parts in one ListParts response | 1000 |
Choose the checksum meaning deliberately
S3 distinguishes full-object and composite checksum types. A full-object checksum represents the object's content as a whole, while a composite checksum is built from part-level checksums. The choice affects how an expected value is calculated and verified. Do not compare two values solely because both are labeled checksum; the algorithm, checksum type and object representation must match. [9]
The detailed multipart support table is important. CRC64NVME supports full-object but not composite checksums. CRC32 and CRC32C support both. SHA-1 and SHA-256 are supported as composite checksums for multipart uploads rather than full-object checksums in that table. These are service capability distinctions, not a ranking of cryptographic strength or an assertion that every algorithm is suitable for every security purpose. [9]
ETag is not a universal substitute for the expected whole-object hash. AWS explains that multipart object ETags are not MD5 digests of the entire object. The part structure influences the resulting representation. A tool that compares a local whole-file MD5 with a multipart ETag can therefore report a mismatch even when the object transfer followed the service's documented behavior. Use the checksum contract intended for that object and upload mode. [2][9]
Specify the checksum algorithm when initiating the multipart upload. The current detailed guidance warns that for certain existing algorithms, a checksum supplied only during completion can be accepted without being validated or stored if the algorithm was not specified at initiation. For newer listed algorithms, omission at initiation can instead lead to an InvalidRequest error at completion. The safest documented recommendation is to declare the intended algorithm from CreateMultipartUpload. [9]
This initialization rule changes the acceptance evidence. Seeing a checksum-looking header in a completion request is not enough to establish that S3 validated it. Retain the initiation configuration, selected type and final stored checksum information needed to demonstrate the actual contract. The source is current product documentation rather than an executed test, so implementations should verify the behavior of the SDK version and algorithm they adopt.
S3 stores checksum information with objects and supports retrieving it for integrity checks. Define how the application obtains the expected value and how it verifies the completed object or later download. If an untrusted uploader chooses both arbitrary content and its expected checksum, a successful match establishes consistency with that declaration, not that the file is safe or belongs in an authorized workflow. [3][9]
Restoration and copying can require the same distinction. A later object representation or checksum type may differ even when the application intends to preserve the content. Use a documented integrity procedure that names what is compared, rather than treating one familiar tag value as permanent proof across every storage operation. The expected business object and the checksum representation should remain separate fields in the review record.
Read completion to its actual result
CompleteMultipartUpload can take time after beginning its response. AWS documents that an initial 200 OK can be followed by an embedded error in the response body. Supported AWS SDKs handle these errors according to their documented error and retry behavior. A custom HTTP wrapper that declares success as soon as it sees the status line can therefore report an object as complete before understanding the actual service result. [4]
Use a supported client and retain the final outcome it exposes. If a wrapper translates responses into another status format, verify that it preserves embedded failure information instead of flattening every HTTP 200 into success. This is a concrete acceptance requirement for custom integrations. The article does not claim that every SDK version or third-party wrapper has been tested, and it does not provide an unverified replacement parser.
Network ambiguity is different from a returned service failure. If the client loses the connection while completion is being processed, the application may not know whether the object was committed. A safe recovery must reconcile that state through the supported API and the owned upload record before assuming either success or absence. Retrying blindly or starting an unrelated upload can complicate the relationship between intended content and the destination key.
Conditional writes can protect the intended object-state precondition. S3 documents If-None-Match and If-Match behavior for supported write and completion operations. A failed precondition and a conflict are not identical outcomes: the conditional-write guidance distinguishes 412 and 409 cases, and some multipart conflicts require restarting the upload. Design recovery around the actual condition and returned result rather than retrying every failure with the same completion request. [8]
The storage completion decision should be connected to a final object record. Identify the expected key, checksum type and relevant metadata, then decide whether the object can leave intake and become available to downstream consumers. A storage success should not automatically override a failed business validation or content-safety check. The completion diagram separates those stages while keeping the part-transfer and abort paths visible.
Encryption remains another independent control. A correctly configured encryption key can protect stored content without proving that the completed object contains the intended bytes, and a verified checksum does not establish that key access is appropriately governed. Keep both requirements in the intake contract where they matter, with their own evidence and owners.
Transfer, completion and cleanup have separate outcomes
An owned part manifest leads to a complete result and object verification; abandoned uploads require separately verified cleanup. Conceptual flowchart, not a measured result.
Source. AWS, S3 multipart upload process, Initiation, parts, completion, checksum and abort responsibilities. [2]; AWS, CompleteMultipartUpload API, Ordered manifest and embedded errors in an initial 200 response [4]; AWS, AbortMultipartUpload API, In-flight parts and repeated abort/list verification [6]; AWS, Conditional S3 writes, If-None-Match and If-Match with completion [8].
Method. Original conceptual flowchart synthesizing the cited documentation. It represents design relationships, not observed test results. Reviewed 2026-09-02.
Accessible table and figure data
| Component | Role |
|---|---|
| Initiate upload | Key, algorithm and upload ID |
| Transfer parts | Record identifiers and checksums |
| Complete upload | Ordered manifest and conditions |
| Read final result | Success or embedded error |
| Verify object | Expected integrity and metadata |
| Abandon safely | Stop work and verify cleanup |
| Component | Role |
|---|---|
| Initiate upload | Key, algorithm and upload ID |
| Transfer parts | Record identifiers and checksums |
| Complete upload | Ordered manifest and conditions |
| Read final result | Success or embedded error |
| Verify object | Expected integrity and metadata |
| Abandon safely | Stop work and verify cleanup |
Verify abandoned-upload cleanup
Multipart parts continue consuming storage until the upload is completed or aborted under the service's rules. AWS also documents that in-flight part requests can finish around an abort operation. An application should therefore stop scheduling new work, account for outstanding transfers and verify that abandoned parts have been cleaned up. An abort request accepted once is not always the complete evidence needed to conclude that every part is gone. [2][6]
The AbortMultipartUpload reference says repeated abort attempts can be necessary when part uploads are still in progress and recommends ListParts verification to ensure part storage is freed. Use the actual upload ID and a controlled cleanup owner. Do not run an indiscriminate deletion process across unrelated uploads merely because a worker reported a timeout. Recovery should preserve the difference between abandoned work and a valid transfer that is still progressing. [6]
Acceptance cases should include an interrupted part, a replaced part number, a listing requiring more than one response, an incorrect checksum, an embedded completion error and an abort racing with outstanding work. Record the expected outcome and the evidence used to resolve it. These are proposed tests for an authorized environment; no object upload, storage operation or failure injection was performed during this article's research.
Leave the application with an explicit completion definition: the intended upload identity and part manifest were used, the final service result is known, integrity was verified under the selected checksum contract and cleanup is resolved for abandoned work. That definition is more durable than a successful HTTP status or a familiar ETag. It remains useful when SDK behavior, supported algorithms or file sizes change.
Method and provenance
Primary documentation and standards were reviewed on September 2, 2026. The guide combines source-supported behavior with explicitly framed implementation recommendations and hypothetical examples.
No customer deployment, production environment or live cloud configuration was tested. Product behavior is limited to the cited documentation and stated scope; actual versions, policies and application behavior require verification.
AI assistance. AI-assisted research synthesis, drafting and consistency checks. No human review or firsthand deployment experience is claimed.
Published under the Cloud Security Desk organizational byline. Read the practitioner guide policy.
References
- S3 multipart limits AWS. Accessed .
- S3 multipart upload process AWS. Accessed .
- S3 object integrity checks AWS. Accessed .
- CompleteMultipartUpload API AWS. Accessed .
- UploadPart API AWS. Accessed .
- AbortMultipartUpload API AWS. Accessed .
- ListParts API AWS. Accessed .
- Conditional S3 writes AWS. Accessed .
- S3 upload checksum types and initialization AWS. Accessed .