diff options
| author | Tim Dettmers <tim.dettmers@gmail.com> | 2021-11-28 21:18:11 -0800 |
|---|---|---|
| committer | Tim Dettmers <tim.dettmers@gmail.com> | 2021-11-28 21:18:11 -0800 |
| commit | 2f8083bd8b084290f888fe59b329d98ebd6dd468 (patch) | |
| tree | da534579bd762e93cd42b69a5e14c36f4b643979 /CHANGELOG.md | |
| parent | ca2078a697ae3adfb84255ae398f79623dc4ea2a (diff) | |
Added AdamW. #10 #13
Diffstat (limited to 'CHANGELOG.md')
| -rw-r--r-- | CHANGELOG.md | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index beaa256..e943fa2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -42,3 +42,7 @@ Docs: Features: - Added Adagrad (without grad clipping) as 32-bit and 8-bit block-wise optimizer + - Added AdamW (copy of Adam with weight decay init 1e-2) + +Bug fixes: + - Fixed a bug where weight decay was incorrectly applied to 32-bit Adam |