summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGustaf Rydholm <gustaf.rydholm@gmail.com>2022-10-02 03:25:58 +0200
committerGustaf Rydholm <gustaf.rydholm@gmail.com>2022-10-02 03:25:58 +0200
commit1e912845afc1503133c929140918fe5916ef7681 (patch)
treefac97ae2ff0327f5e8608909bd757421acbb89b7
parent4e44486aa0e87459bed4b0fe423b16e59c76c1a0 (diff)
Update README
-rw-r--r--README.md6
1 files changed, 4 insertions, 2 deletions
diff --git a/README.md b/README.md
index 8511b83..f549cda 100644
--- a/README.md
+++ b/README.md
@@ -75,5 +75,7 @@ Ideas of mine that did not work unfortunately:
- [ ] fix linting
- [ ] Modularize the decoder
- [ ] Add kv cache
-- [ ] Fix stems
-- [ ] residual attn
+- [x] Fix stems
+- [x] residual attn
+- [x] single kv head
+- [x] fix rotary embedding