From 2f8083bd8b084290f888fe59b329d98ebd6dd468 Mon Sep 17 00:00:00 2001
From: Tim Dettmers <tim.dettmers@gmail.com>
Date: Sun, 28 Nov 2021 21:18:11 -0800
Subject: Added AdamW. #10 #13

---
 CHANGELOG.md | 4 ++++
 1 file changed, 4 insertions(+)

(limited to 'CHANGELOG.md')

diff --git a/CHANGELOG.md b/CHANGELOG.md
index beaa256..e943fa2 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -42,3 +42,7 @@ Docs:
 
 Features:
  - Added Adagrad (without grad clipping) as 32-bit and 8-bit block-wise optimizer
+ - Added AdamW (copy of Adam with weight decay init 1e-2)
+
+Bug fixes:
+ - Fixed a bug where weight decay was incorrectly applied to 32-bit Adam
-- 
cgit v1.2.3