## NVIDIA/Megatron-LM — v2.5…v2.6

_62 commits._

### Features
- **onlly support pp=1** (7b58544)
- **Add seq-length argument to distribued BERT example** (3f57f46)

### Fixes
- **fix bugs in fused softmax** (0d350c8)
- **Improve and fix bugs about fused softmax layer** (40becfc)
- **pushed a fix for torch ddp** (6a0ef5b)
- **fixed flag name bug** (db08a63)
- **Merge branch 'patch-1' of https://github.com/eric-haibin-lin/Megatron-LM into github_small_prs** (2c26bf7)
- **Merge branch 'patch-1' of https://github.com/eqy/Megatron-LM into github_small_prs** (d7d788a)
- **Merge branch 'lmcafee/copygrad-fix-v2' into 'main'** (7574026)
- **simd fix for upper traingular softmax kernel** (62424a7)
- **fused kernels simd fix for short sequences** (5e22689)
- **Merge branch 'main' of ssh://gitlab-master.nvidia.com:12051/ADLR/megatron-lm into lmcafee/copygrad-fix-v2** (21a11ca)
- **fix typo in mappings.py** (92636ee)
- **fix a typo** (61fcc02)
- **fix typo** (e515f02)

### Backend
- **Merge branch 'slym/pyt21.08_nvfuser' into 'main'** (3860e99)
- **Use nvfuser at pytorch >= 1.10** (dba2506)
- **Merge branch 'server' into 'main'** (89e8d27)
- **Adding API server** (3fe6821)
- **Merge branch 'dist_chkpt_act' into 'main'** (136d63c)
- **Merge branch 'github_fused_softmax' into 'main'** (0be4052)
- **removed unused variable** (bc7b353)
- **Modify comments** (b0b017f)
- **chagne PR by reviews** (c6b7763)
- **Remove useless include** (116820a)
- **tested** (cb5e611)
- **some cleanup** (c61dc22)
- **added for pp** (b8940b9)
- **resolved conflicts** (7f2cc3a)
- **resolved conflicts** (30b92cf)
- **resolved conflicts** (8cb389b)
- **Merge branch 'slym/partial_chpt' into 'main'** (23266c5)
- **Merge branch 'mem_opt_for_contig_buffer' into 'main'** (3715b0b)
- **update readme and arguement definition** (99f4767)
- **more updates on examples** (6985e58)
- **Checkpoint a set number of invidividual Transformer layers** (c1e0689)
- **removed contiguous buffer for checkpointed activation** (e923ec5)
- **made contiguous buffer in local ddp default** (e8fb052)
- **set grads to none for the contig. buffers** (df6e3cd)
- **Merge branch 'lmcafee/empty-cache' into 'main'** (68797d9)
- **Merge branch 'github_small_prs' into 'main'** (8e8d760)
- **minor changes from github issues** (60078cf)
- **Merge branch 'lmcafee/empty-cache' into 'main'** (bd315c3)
- **Merge branch 'iter_read_update' into 'main'** (5ca20cd)
- **updated argument name** (52b2296)
- **simplified the iteration read check across ranks** (ede0a58)
- **Merge branch 'github_small_prs' into 'main'** (a8f4edc)
- **Merge branch 'leftover_fixes' into 'main'** (6ee3d21)
- **Merge branch 'typo' of https://github.com/hwijeen/Megatron-LM into github_small_prs** (f69f699)
- **Merge branch 'update-sample-bert-pretrain-args' of https://github.com/roclark/Megatron-LM into github_small_prs** (2ef470e)
- **Merge branch 'main' of https://github.com/ktaebum/Megatron-LM into github_small_prs** (562d1b2)
- **added across rank sync for checkpoint iteration laod, fixed type for timing, and validation iterations** (0722f87)
- **Merge branch 'torch_dist_env_init' into 'main'** (91fcbd6)
- **changed torch distributed init method from tcp to env** (ee4fd50)
- **Destroy more groups in `destroy_model_parallel`** (eddf759)
- **added evaluation logic; finalized flag levels** (9dec537)
- **added flag/logic for emptying unused memory** (3bd2e97)
- **Merge branch 'lmcafee/pipe-forward-only' into 'main'** (78a69ea)
- **Merge branch 'fused_kernels_simd_fix' into 'main'** (2387ce0)
- **added asserts/checks for local ddp and params_have_main_grad** (f597f02)
- **removed saving of input/output tensors for forward-only passes of pipeline schedules** (0865c4d)
- **(conditionally) release grad/main_grad memory in copy_model_grads_to_main_grads, after copy** (6f2bff5)

_Recap by [Repo Wrapped](https://repowrapped.com/gh/NVIDIA/Megatron-LM?utm_source=github-action)._