## deepspeedai/DeepSpeed — v0.19.3…v0.19.4

_40 commits._

### Features
- **Unmanaged gradient accumulation: ZeRO stage 3 support (#8217)** (e75d115)
- **Unmanaged gradient accumulation: ZeRO stage 2 support (#8203)** (3c737a1)
- **Add Triton grouped-GEMM for MoE experts on Ampere/Ada (#8180)** (df84f6d)
- **Add keyword argument support to activation checkpointing (#8182)** (ebf1531)
- **Add managed_gradient_accumulation for ZeRO stage 0/1 (#8184)** (7a31fe6)
- **[AutoTP] Enable HF `colwise_gather_output` to support `lm_head` replace (#8146)** (48d54e0)
- **Enable support for Tutel when k != 1 for shared moe (#8174)** (dc81c9e)

### Fixes
- **Fix ZeRO-3 all_reduce param fetch stride for padded parameters (#8158)** (4e3acb1)
- **Fix MLU_Accelerator conformance to the DeepSpeedAccelerator ABC (#8208)** (437226b)
- **Fix DeepCompile ZeRO-3 gathered parameter ownership (#8157)** (d8dd9e4)
- **MoE dispatch: fix silent Tutel + TP corruption, speed up native and Tutel paths (#8195)** (619e319)
- **[AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP (#8168)** (eec237e)
- **Fix WarmupLR collapsing multi-group base LRs to group 0's (#8171)** (6c58fb7)
- **Fix autotuning get_val_by_key to search all nested subdicts (#8177)** (4b69365)

### Backend
- **Update version.txt pre-0.19.4 release (#8218)** (c455031)
- **Stop trim_mean sorting the caller's list in place (#8199)** (b8b4480)
- **Share DeepNVMe pinned-tensor manager and route swap buffers through I/O handles (#8212)** (e1d6b4f)
- **Make PipelineModule.set_checkpoint_interval actually change the interval (#8178)** (87e4c8c)
- **Reduce redundant work in AutoEP token routing (#8209)** (58aab42)
- **Consolidate LR scheduler tensor-LR tests into a contract suite (#8206)** (6498adc)
- **Wiring up offload_opt_states (#8186)** (76928ff)
- **Preserve tensor learning rates in OneCycle (#8205)** (f6a386a)
- **Remove redundant DCO workflow (#8188)** (d2bd087)
- **Preserve tensor learning rates across scheduler updates (#8202)** (3b64148)
- **Clamp capacity to num_tokens in MoE gating drop branches (complete #5353) (#8155)** (c43cf21)
- **Skip CUDA transformer forward tests when TransformerBuilder is incompatible (#8200)** (7e07be2)
- **Enable ZeRO-3 linear wrapper for existing models (#8189)** (617061d)
- **Release GIL in blocking AIO bindings (#8196)** (e2aae1b)
- **Warn when zero.Init silently falls back to a single rank (#8084) (#8089)** (a375b7f)
- **Derive AutoEP rank splits from the per-expert count exchange (#8190)** (35d1c7d)
- **[AutoTP] Preserve universal checkpoint metadata for HuggingFace tp_plan (#8187)** (d6fbd47)
- **Clarify merge commit exemption for signoff check in AGENTS.md and CLAUDE.md (#8194)** (5cc0617)
- **Scope DeepCompile compiler state to graph and engine lifecycles (#8159)** (615e6d5)
- **Make the OneCycle stair counts actually change the schedule (#8179)** (3169c2d)
- **Guard LRRangeTest and OneCycle schedulers against zero step sizes (#8166)** (90e30f4)
- **Stabilize fork-sensitive nightly tests and AutoSP coverage (#8172)** (86fd223)
- **NVMe write warning (#8164)** (010c6e9)
- **[AutoTP] Allow ZeRO stage 3 inference with tensor parallelism (#8167)** (d326520)
- **Run pull request code only inside a Modal Sandbox (#8170)** (39adc2b)
- **Validate warmup_type in WarmupCosineLR like WarmupLR (#8151)** (886790b)

_Recap by [Repo Wrapped](https://repowrapped.com/gh/deepspeedai/DeepSpeed?utm_source=github-action)._