Qwen3.8-Flash-Next and GLM-5.3-Flash share a 3:1 long-context pattern

Scope and method

Qwen and Z.ai released Qwen3.8-Flash-Next and GLM-5.3-Flash on August 26, 2026. Both releases describe new model families rather than routine quantizations of older dense Transformers. Qwen calls its model an experimental preview of the architecture intended for Qwen4. Z.ai says GLM-5.3-Flash begins from a newly trained base model and is the first natively multimodal model in the GLM-5 series. [1][2][7][8]

This report compares the public architecture descriptions, model cards, released configuration files, and primary papers behind the named mechanisms. It separates three kinds of statements:

• Observed: directly stated in an official model card, configuration, release post, or primary paper.

• Derived: arithmetic applied to published configuration values.

• Inferred: an engineering or control-plane implication that follows from the published design but has not been demonstrated as a measured property of either deployment.

The report does not treat vendor benchmark tables as a common leaderboard. Qwen and Z.ai disclose different prompts, context limits, agent harnesses, generation settings, and judging arrangements. Even identical benchmark names can therefore represent different tests. [2][8]

Architecture at a glance

The two releases differ greatly in scale, yet their layer schedules are strikingly similar.

Qwen3.8-Flash-Next has 48 language-model layers arranged as 12 repetitions of three Gated DeltaNet layers followed by one Qwen Sparse Attention layer. Every layer is followed by a mixture-of-experts feed-forward block. GLM-5.3-Flash has 45 layers. Its configuration repeats three Kimi Delta Attention layers followed by one DeepSeek Sparse Attention layer 11 times, then ends with one additional Kimi Delta Attention layer. The first three GLM feed-forward blocks are dense; the remaining 42 are sparse MoE blocks. [2][9]

Figure 01 takeaway. Both models allocate roughly three quarters of their depth to recurrent or linear attention and one quarter to learned sparse retrieval.

Source. Qwen model card and GLM-5.3-Flash configuration. [2][9]

Method. The figure reduces each architecture to its published repeating layer schedule. It does not represent tensor dimensions, execution time, or communication cost.

Text description. Two vertical stacks show repeating groups of three recurrent layers and one sparse-attention layer. Qwen repeats the group 12 times. GLM repeats it 11 times and adds one final recurrent layer.

Data status. No numerical series. The diagram is a structural rendering of published layer schedules.

The table is descriptive, not a performance ranking. Parameter labels are not perfectly symmetrical. Qwen publishes a 125B main-model count and names two additional components separately. Z.ai publishes a 320B total count and an 18B active count. Active parameters also do not include every operation involved in attention, routing, normalization, embedding lookup, recurrent updates, cache handling, or multimodal encoding.

The shared design thesis: compressed memory plus selective retrieval

A conventional decoder-only Transformer stores keys and values for every preceding token in every attention layer. During generation, each new token attends back over that growing history. The mechanism is expressive, but cache memory grows with sequence length and the attention operation becomes increasingly expensive as the prompt expands.

Both new models divide that job between two mechanisms.

The first mechanism is a recurrent state update. Gated DeltaNet and Kimi Delta Attention process the sequence through a fixed-size state that is updated as tokens arrive. This state is compressed. It can preserve useful summaries and associations without maintaining a conventional key-value record at every recurrent layer. The cost of that compression is that exact historical detail can be overwritten or blended.

The second mechanism is learned sparse retrieval. Periodic sparse-attention layers keep access to token-level history, but use a lightweight indexer to select a limited subset for the expensive attention calculation. The model therefore has a compressed working state for most layers and a retrieval path for moments when precise historical evidence matters.

This division resembles a memory hierarchy more than a single replacement for attention:

1. recurrent state for continuously updated working memory;

2. sparse attention for selective historical retrieval;

3. MoE experts for conditional neural computation;

4. widened residual streams for information transport across depth;

5. multi-token prediction for drafting future tokens;

6. in Qwen's case, a large n-gram table for deterministic local-pattern lookup.

The most notable convergence is quantitative. Qwen uses recurrent layers for exactly 75 percent of its depth. GLM uses them for 75.6 percent. Independently, the two designs place sparse retrieval in approximately one layer out of four.

That similarity should not be read as proof that the mechanisms are interchangeable. Gated DeltaNet and Kimi Delta Attention use different memory controls. QSA and DSA use different indexing and retrieval structures. The common ratio instead suggests that current model builders see value in making token-level historical access periodic rather than universal.

Qwen3.8-Flash-Next: low active compute with a separate memory tier

Gated DeltaNet carries most of the sequence

Thirty-six Qwen layers use Gated DeltaNet. The underlying Gated Delta Networks work combines two ideas. Gating controls how quickly old state can be erased, while a delta update rule supports targeted changes to memory. The paper presents the combination as a response to the retrieval and long-context limitations of earlier linear Transformers. [4]

The relevant property for this comparison is not that Gated DeltaNet is lossless. It is not. Its value is that a model can carry a fixed-size state through most layers instead of expanding a conventional KV cache at every layer. Qwen then restores selective token-history access every fourth layer through QSA.

Qwen's configuration lists 48 value heads and 16 query-key heads for Gated DeltaNet, each with a dimension of 128. These dimensions are part of the recurrent mechanism and should not be compared directly with ordinary multi-head attention head counts. [2]

Qwen Sparse Attention retrieves four-token micro-blocks

QSA is explicitly block-oriented. A lightweight indexer selects up to 512 micro-blocks, with four tokens per block, for a maximum expensive-attention budget of 2,048 tokens. The sparse-attention module uses 24 query heads and two key-value heads, with a head dimension of 256. Only 64 of those 256 dimensions are assigned to rotary position encoding. [2]

At the native 262,144-token context length, 2,048 tokens represent 0.78125 percent of the full context. At a one-million-token extension, the same number represents approximately 0.205 percent. This arithmetic describes the maximum selected set, not the cost of the complete QSA operation. The indexer still has to produce a useful selection, cache structures still have to be stored or moved, and the system still has to gather the selected blocks.

Block retrieval has a practical systems rationale. Four adjacent tokens can be moved and processed more regularly than four scattered tokens. The unit also preserves short structures that are common in source code, tool output, identifiers, arguments, and structured records. The tradeoff is that selection is coarser. A useful token may bring three neighboring tokens with it, and a missed block cannot contribute to that sparse-attention layer.

QSA's two key-value heads also keep the sequence-growing cache narrower than a design that stores independent key-value heads for every query head. This is one reason that the active parameter count alone is not an adequate serving metric. Head grouping, cache representation, block layout, and kernel support affect memory and latency separately from MoE activation.

The 51B n-gram table is capacity without equivalent matrix multiplication

Qwen's most distinctive addition is a 20-million-entry n-gram embedding table with a 2,560-dimensional embedding width. Multiplying those published values gives 51.2 billion parameters, which matches the model card's rounded 51B figure. The model uses bigrams and trigrams and injects the resulting representation at layer 2. [2]

This component is not a classic n-gram language model. It does not replace the neural network's next-token distribution with a frequency table. It maps short token sequences to learned vectors that become additional inputs to the backbone.

The relevant research frames this approach as conditional memory. MoE provides conditional computation, where a router activates a small part of a large neural parameter set. N-gram lookup provides a different axis of sparsity, where deterministic addressing retrieves a small number of rows from a large static table. The conditional-memory paper argues that local and stereotyped patterns can be served by lookup instead of being reconstructed repeatedly through early neural layers. It also describes asynchronous host-memory prefetch as a way to place very large tables outside accelerator memory. [5]

Qwen's release materials make the same systems point. The table is more amenable to host-memory offload than MoE weights because the relevant rows can be identified from the token sequence before the later neural layers need them. [1][3]

This produces a useful but easily misunderstood distinction:

• Compute footprint: only a small number of n-gram rows are fetched for each token.

• Storage footprint: the complete table still has to exist, be versioned, loaded, checked, and made available with predictable latency.

The main backbone, n-gram table, and MTP module together sum to approximately 180 billion named parameters. In an unquantized two-byte representation, that would correspond to approximately 360 GB of raw parameter storage before runtime overhead. That is a storage estimate, not a deployment requirement. Implementations can quantize components, shard them, or keep the n-gram table in host memory.

Fine-grained MoE uses many narrow experts

Every Qwen layer is followed by a sparse MoE block. The model defines 512 routed experts, selects 10 per token, and includes one shared expert. Each expert has an intermediate width of 640. [2]

This is a fine-grained expert design. A token combines several narrow transformations rather than relying on one or two large experts. The shared expert provides a common path that does not depend on routing. The large n-gram table may also reduce the need for early experts to spend capacity reconstructing common local patterns, although Qwen has not published an ablation that isolates that interaction.

The active intermediate width across the 10 routed experts and one shared expert is roughly 7,040 dimensions per MoE layer. This multiplication is only a structural comparison. It is not a full FLOP estimate because projection shapes, gating, expert parallelism, token distribution, batching, and kernel fusion all matter.

Gated Residual widens the path across layers

Qwen uses four residual branches rather than one. Its Gated Residual mechanism applies an element-wise, input-dependent read gate and a scalar write gate for each branch. The control path uses a bottleneck rank of 320. [2]

Widening the residual stream gives the network more cross-layer transport capacity without making every attention head and expert proportionally wider. It may also help preserve separate representations while recurrent state is being compressed and MoE routes different tokens through different expert combinations. That interpretation is plausible, but Qwen's public material does not provide an ablation that assigns a measured benefit to that specific interaction.

Multi-token prediction addresses tokens per decoding pass

Qwen adds a separate 4B, one-layer MTP module trained across multiple future steps. Multi-token prediction trains a model to predict several future tokens from each position. The technique can improve representation learning and can support speculative generation, where a main model verifies several drafted tokens in one pass. [2][6]

MTP and n-gram memory solve different problems. The n-gram table supplies local-pattern representations to the backbone. MTP proposes future tokens. Treating both as one lookup system would obscure the model's actual division of work.

GLM-5.3-Flash: wider active computation with finer recurrent control

Kimi Delta Attention extends recurrent memory control

GLM's released configuration identifies 34 layers as KDA layers. Kimi Delta Attention extends Gated DeltaNet with finer-grained gating. The Kimi Linear paper describes independent forgetting behavior at the feature-channel level, intended to use finite recurrent state more effectively. [9][10]

In practical terms, a single recurrent head does not have to forget or retain all of its feature dimensions at the same rate. Some channels can preserve longer-lived information while others change quickly. That gives the recurrent state more expressive control than a coarser shared forget gate, at the cost of a more complex update and kernel.

The GLM configuration lists 64 KDA heads with a head dimension of 128. It also uses a short convolution kernel of four. Those values belong to the linear-attention path and should be treated as implementation dimensions rather than a direct quality measure. [9]

DeepSeek Sparse Attention provides periodic global retrieval

Eleven GLM layers use deepseek_sparse_attention. DSA separates sparse retrieval into a lightweight indexer and a more expensive attention calculation over selected history. The DeepSeek-V3.2 paper presents the mechanism as a way to reduce long-context attention complexity while preserving performance. [9][11]

GLM adds IndexPool to this path. Z.ai describes IndexPool as compressing four indexer key vectors into one through weighted pooling. The released configuration enables key-pool compression, sets index_kpool to 4, uses 32 indexer heads with 128-dimensional index heads, and sets index_topk to 2,048. [7][9]

The apparent similarity between Qwen's 2,048-token QSA budget and GLM's index_topk: 2048 should not be treated as equivalence. Qwen publishes the unit directly as 512 four-token blocks or 2,048 tokens. GLM performs four-way pooling in the indexer before selection. The selected units and subsequent cache representation are therefore different.

GLM also couples the sparse path with Multi-head Latent Attention parameters. Its configuration specifies a 1,536-dimensional query low-rank projection and a 512-dimensional key-value latent rank. MLA compresses the cache representation before it is expanded for attention, which reduces sequence-growing storage relative to conventional per-head key-value tensors. [9]

mHC constrains four-stream residual mixing

GLM uses four residual streams through Manifold-Constrained Hyper-Connections. The configuration enables mHC, sets the width multiplier to four, and specifies 20 Sinkhorn iterations. [9]

The mHC paper starts from a problem in widened residual connections. More flexible cross-layer mixing can improve capacity, but unconstrained mixing can weaken the identity-mapping property that makes residual networks stable. mHC projects the mixing structure onto a constrained manifold designed to preserve that property. [12]

The contrast with Qwen is not that one model has a residual innovation and the other does not. Both do. The difference is emphasis:

• Qwen describes dynamic read and write gating across four branches.

• GLM uses constrained four-stream mixing with an explicit normalization procedure.

These approaches create additional cross-layer bandwidth, but the public evidence does not establish which is better under equal training compute and parameter scale.

GLM uses fewer, wider experts and a dense stem

GLM has 288 routed experts, selects eight per token, and adds one shared expert. Each expert has an intermediate width of 2,048. The first three feed-forward layers are dense, each with an intermediate width of 12,288; all subsequent feed-forward layers are sparse. [9]

Compared with Qwen, GLM has fewer experts but each expert is 3.2 times wider. The approximate active expert intermediate width per sparse layer is 18,432 dimensions across eight routed experts and one shared expert. That is approximately 2.62 times Qwen's corresponding 7,040-dimensional total. Again, this is a structural indicator, not a full computation estimate.

The dense first three layers give every token a common early transformation before sparse expert routing begins. The design may stabilize early feature construction or give the router a more uniform representation, but Z.ai has not published an ablation that isolates the reason.

GLM is configured for a 1,048,576-token maximum

The released configuration sets max_position_embeddings to 1,048,576. Qwen's model card lists 262,144 tokens natively and extension to 1,000,000. [2][9]

A context setting establishes addressable sequence length, not uniform recall. Long-context quality depends on training distribution, positional treatment, recurrent retention, sparse selection, cache precision, and the task. A model can accept a one-million-token prompt and still fail to retrieve a relevant passage. The architecture changes the economics of testing that behavior; it does not remove the need to test it.

GLM's serving design is explicitly disaggregated

Z.ai's release post describes a custom SGLang serving engine optimized for its architecture. The published stack includes tensor parallelism for linear attention and the language-model head, ReplaySSM, W8A8 quantization, mixed INT8, FP8, and BF16 cache formats, and Layer Split. At cluster scale, Z.ai separates multimodal encoding, prompt prefill, and token decoding into independently scalable worker pools through an Encode-Prefill-Decode architecture. Z.ai reports a threefold end-to-end serving improvement over its initial baseline. These are vendor measurements, not independently reproduced results. [7]

This serving description matters because GLM is not only a smaller-active-parameter version of a conventional model. Its performance depends on scheduling recurrent-state work, sparse retrieval, MoE traffic, cache formats, multimodal encoding, and distinct worker pools.

Where each model places capacity

The clearest architectural difference is not total parameters. It is the location and form of capacity.

Qwen uses a smaller active neural path and adds a large, deterministically addressed memory table. GLM uses a larger active neural path, wider experts, a wider hidden state, more granular recurrent gating, and a configured one-million-token maximum.

The active-share percentages are simple divisions. They do not describe attention cost, n-gram traffic, routing, recurrent updates, dense GLM stem layers, vision encoding, or multi-token verification.

This leads to different likely bottlenecks.

For Qwen, accelerator arithmetic may be comparatively low, but host memory capacity, lookup-table placement, deterministic prefetch, and host-to-device bandwidth become first-class engineering concerns. A cache miss or delayed transfer in a 51B table can affect latency even though the lookup itself is computationally cheap.

For GLM, the active neural workload is larger. Wider experts and a 4,096-dimensional hidden state increase arithmetic and expert-state movement. The one-million-token setting and disaggregated worker pools also make cache formats, queueing, state transfer, and network topology central to service behavior.

Neither profile is universally superior. A high-concurrency deployment with ample host RAM and optimized prefetch may favor Qwen's lower active compute. A deployment that values a wider active model and values a released one-million-token maximum setting may favor GLM. Hardware, runtime maturity, batch shape, prompt reuse, quantization, and network design can reverse a theoretical advantage.

The sparse indexer becomes part of model behavior

Sparse attention introduces a less visible failure mode than ordinary full attention. Before the model can reason over an exact historical token, an indexer has to make that token or block available to the expensive attention path.

A sparse layer can therefore fail in two distinct ways:

1. the selected evidence is present, but the attention and downstream layers use it incorrectly;

2. the evidence is never selected for that layer.

The second failure is structurally specific to learned sparse retrieval. It can be masked by later layers, recurrent state, repeated evidence, or local context, but it cannot be corrected inside a sparse-attention operation that never received the relevant history.

Qwen's repeated QSA layers and GLM's repeated DSA layers give the network multiple opportunities to retrieve evidence at different depths. The query representation changes as it moves through the model, so later indexers can make different choices. Still, operators should not assume that successful short-context tests validate one-million-token retrieval.

A useful evaluation set should vary more than needle position. It should include:

• several semantically similar passages where only one contains the authoritative value;

• conflicting versions of a policy or configuration with explicit timestamps;

• references split across micro-block boundaries;

• long tool traces with repeated identifiers;

• repository-scale code questions that require relationships across files;

• multimodal documents where the relevant text is inside an image or table;

• prompts that place untrusted instructions near the target evidence;

• tests repeated under quantized and unquantized cache settings.

The output should record not only accuracy, but also prompt length, selected runtime configuration, cache type, serving engine version, batch size, prefix-cache state, and tail latency. Without those fields, a long-context result is difficult to reproduce.

What the architecture changes in the cloud control plane

The following implications are inferred from the published designs. They are not claims that either model contains a known vulnerability.

Model topology becomes service topology

A dense model can already span many accelerators. Hybrid recurrent, sparse-attention, and MoE models add more differentiated state. A deployment may contain expert shards, recurrent-state buffers, sparse index structures, multimodal encoders, prefill workers, decode workers, host-resident lookup tables, and cache-transfer services.

That topology should be documented as an application dependency graph, not hidden behind one model endpoint. Each state transition should have a named owner, transport, integrity check, retention decision, and failure mode.

Qwen's n-gram table is an integrity boundary

A host-resident n-gram table is static model state, not ordinary transient cache. The serving path should be able to prove which table version was loaded, how it maps token sequences to rows, and whether the files were altered between build and runtime.

Controls should include signed artifacts, immutable object versions, checksums at load, host hardening, least-privilege access to table files, and telemetry for prefetch failure or fallback behavior. These measures follow from the role of the table. The release material does not prescribe a specific control implementation.

GLM's Encode-Prefill-Decode split creates service identities

Separating encoding, prefill, and decode into independent pools creates queue and state-transfer boundaries. A multimodal request may be transformed by one worker class, converted into model state by another, and continued by a third.

The control questions are concrete:

• Which identity can submit work to each pool?

• Can one tenant reference another tenant's encoded or prefetched state?

• Are cache handles scoped, unguessable, and expired?

• Is cache transfer encrypted and authenticated inside the cluster?

• Can telemetry correlate one request across all three stages?

• What happens when a worker restarts after producing partial state?

The same questions apply to any disaggregated inference system, but Z.ai's published EPD design makes them part of the stated architecture rather than an optional deployment detail.

MoE communication is an east-west control path

A token can be routed to several experts, and those experts may be distributed across devices or nodes. The inference request therefore produces internal east-west traffic whose destination depends on model routing.

Network policy should permit the required expert communication without turning the entire inference cluster into an unrestricted trust zone. At the same time, packet-level logs alone may not explain a model request because expert selection changes by token. Operational telemetry needs request, batch, shard, and model-version correlation at a higher layer.

Recurrent state and sparse caches need lifecycle controls

Long-context deployments may retain recurrent state, latent KV representations, selected indices, and prefix-cache entries. These objects can outlive an individual kernel call and may be reused across generation steps or requests.

The service should define whether state is tenant-specific, request-specific, or shareable; how it is zeroed or evicted; whether it is written to host memory; and how it behaves during retry. A cache optimization that is correct for public prompts may be unacceptable for sensitive prompts if tenancy boundaries are weak.

Multimodal encoding expands the input boundary

Both models include vision encoders. GLM explicitly separates multimodal encoding in its serving topology. Image and video handling introduce decoders, resize paths, patch extraction, token expansion, and metadata processing before the language model sees the input.

The model endpoint can therefore appear healthy while the consequential parser or transformation path sits elsewhere. File-size limits, media-type validation, decompression limits, sandboxing, dependency inventory, and request tracing belong in the AI service review.

Figure 06 takeaway. The model endpoint is only the front door. Qwen and GLM place consequential state in different worker, memory, cache, and communication paths.

Source. Qwen release materials, Z.ai release post, and primary mechanism papers. [1][3][5][7]

Method. Inferred control-plane diagram. It illustrates review boundaries and is not an observed vendor deployment.

Text description. The Qwen side shows an API gateway, accelerator workers, host n-gram memory, sparse index and cache state, and expert shards. The GLM side shows API routing into encoder, prefill, and decode pools, with sparse index and latent cache state plus expert shards. Identity, integrity, and telemetry boundaries are marked around state transfers.

Data status. No numerical series. The figure is an inferred review map, not an observed deployment.

A practical evaluation plan

An architecture comparison becomes useful only when it changes the test plan. The following matrix is designed for teams evaluating either model on their own infrastructure.

Results should be reported by workload class. A model that performs well on code repositories may not behave the same on policy archives, multimodal manuals, log streams, or agent transcripts. Aggregating all prompts into one average can hide the sparse-selection and retrieval failures that the architecture is designed to manage.

What cannot be concluded from the launch material

There is no apples-to-apples throughput result

Qwen and Z.ai describe efficiency improvements against different baselines and serving stacks. Qwen recommends specialized engines including SGLang, vLLM, and TokenSpeed. Z.ai describes a custom SGLang stack and reports gains over its own initial baseline. Those statements do not establish which model is faster on the same hardware, precision, batch mix, prompt distribution, and runtime. [2][7]

Active parameters are not end-to-end compute

The 6B and 18B figures are useful, but they do not count every operation. Attention projections, recurrent updates, indexer scoring, cache movement, routing, normalization, vision encoding, n-gram lookup, dense stem layers, and MTP verification all contribute to latency and resource use.

One-million-token support is not one-million-token recall

A context setting defines what the model and runtime can address. It does not prove equal recall across positions or workload types. Sparse selection and recurrent memory should be tested under realistic conflict, repetition, and adversarial distraction.

Equal top-k numbers do not mean equal retrieval work

Qwen's budget is explicitly 512 four-token blocks. GLM's configuration combines index_topk: 2048 with four-way key pooling. The selected units, cache representation, and gather pattern are different. A direct percentage comparison would be misleading.

Benchmark tables should not be merged into one ranking

The release materials use different contexts, harnesses, generation settings, timeouts, judges, and sometimes proprietary evaluation services. GLM's model card, for example, lists 400K context for DeepSWE and one million tokens for NL2Repo. Qwen documents separate evaluation methods for its table. The scores are useful within each vendor's disclosed setup, but they do not answer the architecture question by themselves. [2][8]

Assessment

The important result is convergence, not a winner.

Both model families have moved away from the assumption that every layer needs direct, full-resolution access to every previous token. They rely on recurrent state for most of the network and insert sparse retrieval at regular intervals. Both widen the residual path, use sparse experts, support multimodal input, and add multi-token prediction.

Qwen pushes farther toward heterogeneous memory. It activates 6B neural parameters, combines many narrow experts, and adds a 51B n-gram table that can be served as deterministic conditional memory. Its architecture is designed to exchange some neural computation for sparse lookup and host-memory capacity.

GLM invests more in the active neural path. It activates 18B parameters, uses a 4,096-dimensional hidden state, wider experts, channel-wise recurrent forgetting, MLA-based sparse attention, mHC residual mixing, and a configured 1,048,576-token maximum. Its release also treats disaggregated multimodal serving as part of the system design.

The likely successor to all-attention Transformers is not a single new attention operator. It is a hierarchy of memory and computation, with different mechanisms assigned to local patterns, compressed history, selective token or block retrieval, conditional neural work, cross-layer transport, and speculative decoding.

For practitioners, that hierarchy changes the assurance boundary. The right review object is no longer only the model file or API endpoint. It is the complete path through tokenizer, lookup memory, recurrent state, sparse indexer, expert shards, cache formats, worker pools, and runtime configuration. Architecture and control plane now have to be evaluated together.

References

1. Qwen Team. “Qwen3.8-Flash-Next: A New Architecture, Towards Ultimate Cost-Efficiency.” August 26, 2026. qwen.ai source ↗

2. Qwen Team. “Qwen3.8-Flash-Next Model Card.” Hugging Face, revision f5d0827. huggingface.co source ↗

3. Qwen Team. “Qwen3.8-Flash-Next Repository.” GitHub, revision 513aa6e. github.com source ↗

4. Yang, Songlin, Jan Kautz, and Ali Hatamizadeh. “Gated Delta Networks: Improving Mamba2 with Delta Rule.” ICLR 2025. arxiv.org source ↗

5. Cheng, Xin, et al. “Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models.” 2026. arxiv.org source ↗

6. Gloeckle, Fabian, et al. “Better & Faster Large Language Models via Multi-token Prediction.” 2024. arxiv.org source ↗

7. Z.ai. “GLM-5.3-Flash: Frontier Intelligence, Flash Cost.” August 26, 2026. z.ai source ↗

8. Z.ai. “GLM-5.3-Flash Model Card.” Hugging Face, revision 3f1971b. huggingface.co source ↗

9. Z.ai. “GLM-5.3-Flash Configuration.” Hugging Face, revision 3f1971b. huggingface.co source ↗

10. Kimi Team. “Kimi Linear: An Expressive, Efficient Attention Architecture.” 2025. arxiv.org source ↗

11. DeepSeek-AI. “DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models.” 2025. arxiv.org source ↗

12. Xie, Zhenda, et al. “mHC: Manifold-Constrained Hyper-Connections.” 2025. arxiv.org source ↗

Publication note

This report is based on materials available on August 26, 2026. Architecture fields may change if model repositories or serving implementations are revised. Derived values are identified in the text. Operational implications are analytical inferences, not observed security findings.