## NVIDIA/Megatron-LM — v2.0…v2.5 _306+ commits._ ### Features - **Add temporary assert to finetuning until it can be fixed.** (113c636) - **Add arg needed for tokenizer.** (2c247af) ### Fixes - **Use timers kwargs correctly to prevent bug with new p2p_communication API** (e727de9) - **Merge branch 'lmcafee/tbmem-fix' into 'main'** (f3be8ea) - **Merge branch 'lmcafee/zerograd-fix' into 'main'** (3202f23) - **Merge branch 'patch-1' of https://github.com/stas00/Megatron-LM into github-pr** (4a35d50) - **Merge branch 'typo-fix' of https://github.com/devrimcavusoglu/Megatron-LM into github-pr** (e1318f0) - **Typo fix in readme** (07ca952) - **fix copy-n-paste error** (3f75537) - **One more quick fix.** (657d7cd) - **Quick fix of copy/paste error.** (edb76ab) - **fix one more issue** (0c01c2f) - **Cherry-pick fix from development repo.** (8aa4619) ### Backend - **Merge branch 'sc21' into 'main'** (e269e20) - **scripts for sc21** (de7dc40) - **Merge branch 'bugfix' into 'main'** (6a68098) - **Merge branch 'main_p2p' into 'main'** (a676bc2) - **Make it possible to pass in tensor shapes to communication methods in p2p_communication.py** (1dccefd) - **Merge branch 'small_refactor' into 'main'** (3db6517) - **Use helper method in megatron/schedules.py as intended** (77bff38) - **fixed help message; removed redundant destination variable** (bc5a8e2) - **added comment explaining why fp32_from_float16_groups should be zeroed here** (4e64903) - **switched tensorboard memory logging from opt-out to opt-in** (236a5ec) - **Merge branch 'readme_update' into 'main'** (c107527) - **Merge branch 'main-readme' into 'main'** (a279d35) - **updated data processing readme** (5f2cb26) - **Address Mohammad's comment** (a037a9c) - **Update README to have a small note about interleaved schedule** (ab9a797) - **added memory stats (allocated/reserved) to tensorboard logging** (24b7c3c) - **fixed zero_grad for fp32_from_float16_groups** (4fd6432) - **Merge branch 'github-pr' into 'main'** (90e0a0d) - **Merge branch 't5' of https://github.com/stas00/Megatron-LM into github-pr** (7898c9a) - **Merge branch 'main_retriver_merge_dpr' into 'main'** (82b69e8) - **updated readme** (4c92ca8) - **updated readme** (32da2e7) - **updated readme** (baf2e2a) - **updated readme** (9d350c9) - **Merge branch 't5_scripts' into 'main'** (2be1e51) - **Merge branch 'main_retriver_merge_dpr' into 'main'** (598d7ee) - **Merge branch 'main_retriver_merge_dpr' of ssh://gitlab-master.nvidia.com:12051/ADLR/megatron-lm into main_retriver_merge_dpr** (98113c6) - **addressed comments** (2845047) - **Clean up README.md a bit** (473127f) - **Adding readme** (c45109e) - **Adding readme** (e287bf0) - **Adding readme** (293554a) - **Adding readme** (8661ca2) - **Adding readme** (bab5cc4) - **Adding readme** (1095d7e) - **Adding readme** (d562d7b) - **Adding readme** (a983cab) - **fixed the evaluation hangs** (e46f326) - **fixed the tensor size miss-mass issue** (ebfbfce) - **resolved hang issue** (04c79f3) - **Update T5 scripts** (3dadd16) - **Merge branch 'main' into main_retriver_merge_dpr** (84eb016) - **updating script** (c7c65bb) - **Merge branch 'main_retriver_merge_dpr' into 'main'** (83c4d95) - **Merge branch 'vit_pipeline_fixes' into 'main'** (01fc083) - **updating no load rng** (fda81a2) - **updating the scripts** (63121a9) - **added exit interval for finetuning** (d078e54) - **Merge branch 'finetune_assert' into 'main'** (217f54b) - **updated the evaluation script for retriver** (825375c) - **updated the evaluation script for retriver** (a41e478) - **updated the evaluation script for retriver** (f21a662) - **updated the evaluation script for retriver** (dfb6a9b) - **Fixed issues with ICT pretraining** (7577931) - **renaming the folders** (8e44d61) - **additional cleaning** (2529380) - **cleaning the code** (2eaf6c7) - **vit pipeline fixes** (ccae9db) - **before cleaning the comments** (7a0710e) - **Merge branch 'main' into main_retriver_merge_dpr** (4a09bb3) - **t5 fixes** (2dae74b) - **Merge branch 'github-main' into 'main'** (42c1cf4) - **Merge internal main and github main back into one branch.** (5d65de5) - **Merge branch 'readme_fix' into 'main'** (27b6c87) - **Merge branch 'readme_fix' into 'main'** (3f38ecf) - **Merge branch 'preprocess_fix' into 'main'** (bba90f7) - **Merge branch 'numpy_seed' into 'main'** (3747571) - **Ensure numpy random seed is within range.** (1c4c360) - **Merge branch 'arg_checks' into 'main'** (002cde6) - **Merge branch 't5_docs' into 'main'** (78bad98) - **Adding T5 to docs and a bit of cleanup.** (306eb24) - **Update arguments checks.** (8044c7b) - **Merge branch 'v0_checkpoint_fixes' into 'main'** (ee76a50) - **fixed compatiblity with v0 checkpoints** (26b49aa) - **debugging DPR** (dca47cf) - **evaluation works!** (f64977f) - **added pre ad post process** (7e335e1) - **added pre ad post process** (5409341) - **fixing model evaluation of retriver** (f926720) - **DPR finetune and evaluation** (6d03d7a) - **DPR ongoing** (d2d5086) - **DPR evaluation debugging** (220637f) - **removed commnets** (a8d172b) - **removed commnets** (f415dc8) - **removed commnets** (8004731) - **adding dpr code** (b9fcb7b) - **Merge branch 'main' into main_retriver_merge_dpr** (957d1c9) - **implementation dpr** (06076c7) - **Merge branch 't5_merge' into 'main'** (2ff004a) - **Merge branch 'main_generate' into 'main'** (716a324) - **Merge branch 'main_dedup' into 'main'** (7a5768a) - **addressed comments** (e5ec27d) - **addressed comments** (5a6431f) - **addressed comments** (5c2ce59) - **addressed reviews** (0fa728a) - **added more comments** (f938e19) - **added more comments** (c49b464) - **Merge branch 'main' into main_dedup** (7c3d8b7) - **modified the params** (44bfcb3) - **added this function for evaluation** (045959c) - **Integrate code from t5_main into existing code.** (48a5e0d) - **Merge branch 'main' into github-main** (aed2f75) - **Merge branch 'add_ref' into 'main'** (f32a638) - **added link to the pipeline papers** (9ec547c) - **Merge branch 'main' into main_retriver_merge_dpr** (cdde433) - **implementing DPR** (10ff060) - **Merge branch 'release_fixes' into 'main'** (8cfef1b) - **Release fixes** (50a4b5f) - **Merge branch 'interleaved_bugfix' into 'main'** (23632ee) - **Small bugfix to make sure refactored code works with interleaved schedule** (6fd7818) - **Merge branch 'pipeline_refactor' into 'main'** (3fc035d) - **Addressed MR comments, mostly adding comments to code.** (e270f68) - **Merge branch 'main' into main_dedup** (ee7b19e) - **More features added** (d413bd5) - **updated filter_ngrams.py** (f559787) - **Merge branch 'bfloat_jit' into 'main'** (f2d64c0) - **removed the checks for bfloat jitting** (d28716e) - **added parallelism for computing jaccard similaity** (43d307d) - **Fixing text generation and zeroshot eval and addressing comments.** (64a83fb) - **Tasks seems to be working.** (b938ec5) - **pipeline code simplification** (3b91262) - **Merge branch 'extra_assertion' into 'main'** (2f3a2d6) - **Make sure pipeline-model-parallel size is greater than 2 for interleaved schedule** (182841f) - **Merge branch 'main' into main_retriver_merge_ict_eval** (a5acbf5) - **Added more feature in train data deduplication** (882683d) - **Merge branch 'main_retriver_merge_ict_eval' into 'main'** (a6e00d9) - **ICT zeroshot evaluation** (fcfd094) - **fixed another issue** (4056539) - **Merge branch 'bfloat_fused_softmax' into 'main'** (c534679) - **Bfloat fused softmax + fused layer norm** (0fa7175) - **Fixed based on review recoemmendation** (43c9137) - **Merge branch 'ninja_compilation_fix' into 'main'** (d9b1c68) - **refactored the fused kernels build** (0d5188c) - **Merge branch 'softmax_perf' into 'main'** (876096d) - **fixes to upper triangular masked softmax fusion kernel** (3b12ab1) - **minor fixes** (531152d) - **softmax data load/store optimization** (b1a8337) _Recap by [Repo Wrapped](https://repowrapped.com/gh/NVIDIA/Megatron-LM?utm_source=github-action)._