Every one of the 62 layers runs ordinary causal softmax attention over the full context with grouped-query attention: 48 query heads of dimension 128 share 8 key/value heads (6 query heads per KV head). There is no latent compression, no token selection, no sliding window (sliding_window = null) and no output gate or sink.
Cost. Each layer caches 2 × 8 × 128 = 2,048 values per token, 4 KiB in BF16, so the stack holds about 248 KiB per token: roughly 52 GB for one 204,800-token sequence (half that with an FP8 KV cache). Prefill FLOPs grow quadratically with context.
Why MiniMax kept it. MiniMax-Text-01 used a Lightning-attention hybrid. For M2 the team continued pretraining hybrid sliding-window variants for hundreds of billions to trillions of tokens, varying SWA/full ratios, RoPE settings, intra- vs. inter-layer mixing and sink tokens. In pretraining evals the hybrid tied on MMLU (85.5 vs. 85.6) and MATH (60.3 both) but lost on RULER-128K common-word extraction (90.0 → 72.0), RULER-128K multi-query (99.0 → 93.0), HELMET in-context learning (75.8 → 72.7) and MTOB in-context translation (−15.0 and −17.6). After SFT it lost on SWE-bench Verified (54.7 → 50.2), Terminal-Bench (26.7 → 23.8) and τ²-telecom (32.5 → 21.0), but won on IFBench, XBench-ds and τ²-retail. MiniMax's blog attributes the long-context loss to retrieval and induction heads that form early in pretraining and that continued pretraining struggles to rewire, and says efficient attention still lacked mature low-precision, prefix-caching and speculative-decoding support.