diff options
| author | Gustaf Rydholm <gustaf.rydholm@gmail.com> | 2023-08-25 23:19:51 +0200 |
|---|---|---|
| committer | Gustaf Rydholm <gustaf.rydholm@gmail.com> | 2023-08-25 23:19:51 +0200 |
| commit | e6cc7c9ff78b171ba2218ac421242ab399143029 (patch) | |
| tree | 440ed179d3290076f7552c10369c0c816cde93f8 /README.md | |
| parent | 6968572c1a21394b88a29f675b17b9698784a898 (diff) | |
Update reqs
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 3 |
1 files changed, 3 insertions, 0 deletions
@@ -80,3 +80,6 @@ Ideas of mine that did not work unfortunately: - [x] residual attn - [x] single kv head - [x] fix rotary embedding +- [ ] simplify attention with norm +- [ ] tie embeddings +- [ ] cnn -> tf encoder -> tf decoder |