diff options
| author | Gustaf Rydholm <gustaf.rydholm@gmail.com> | 2022-10-02 03:25:58 +0200 |
|---|---|---|
| committer | Gustaf Rydholm <gustaf.rydholm@gmail.com> | 2022-10-02 03:25:58 +0200 |
| commit | 1e912845afc1503133c929140918fe5916ef7681 (patch) | |
| tree | fac97ae2ff0327f5e8608909bd757421acbb89b7 | |
| parent | 4e44486aa0e87459bed4b0fe423b16e59c76c1a0 (diff) | |
Update README
| -rw-r--r-- | README.md | 6 |
1 files changed, 4 insertions, 2 deletions
@@ -75,5 +75,7 @@ Ideas of mine that did not work unfortunately: - [ ] fix linting - [ ] Modularize the decoder - [ ] Add kv cache -- [ ] Fix stems -- [ ] residual attn +- [x] Fix stems +- [x] residual attn +- [x] single kv head +- [x] fix rotary embedding |