From 1e912845afc1503133c929140918fe5916ef7681 Mon Sep 17 00:00:00 2001 From: Gustaf Rydholm Date: Sun, 2 Oct 2022 03:25:58 +0200 Subject: Update README --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8511b83..f549cda 100644 --- a/README.md +++ b/README.md @@ -75,5 +75,7 @@ Ideas of mine that did not work unfortunately: - [ ] fix linting - [ ] Modularize the decoder - [ ] Add kv cache -- [ ] Fix stems -- [ ] residual attn +- [x] Fix stems +- [x] residual attn +- [x] single kv head +- [x] fix rotary embedding -- cgit v1.2.3-70-g09d2