diff options
| author | Gustaf Rydholm <gustaf.rydholm@gmail.com> | 2022-09-13 21:54:38 +0200 |
|---|---|---|
| committer | Gustaf Rydholm <gustaf.rydholm@gmail.com> | 2022-09-13 21:54:38 +0200 |
| commit | 7db6e95f5ae404e276f22ef2c07e72285a33c490 (patch) | |
| tree | c3cd16ceee423bc0c4ae5125b05878c5b5d546ef | |
| parent | 78601f86c4eb3a3b040cb1bde22a550b48eabe91 (diff) | |
Update Readme
| -rw-r--r-- | README.md | 7 |
1 files changed, 4 insertions, 3 deletions
@@ -30,7 +30,6 @@ make generate ## Train - Use, modify, or create a new experiment found at `training/conf/experiment/`. To run an experiment we first need to enter the virtual env by running: @@ -52,15 +51,17 @@ Create a picture of the network and place it here Ideas of mine that did not work unfortunately: +* Efficientnet was apparently a terrible choice of an encoder + - A ConvNext module heavily copied from lucidrains [x-unet](https://github.com/lucidrains/x-unet) + * Use VQVAE to create pre-train a good latent representation - Tests with various compressions did not show any performance increase compared to training directly e2e, more like decrease to be honest - This is very unfortunate as I really hoped that this idea would work :( - I still really like this idea, and I might not have given up just yet... - * Axial Transformer Encoder - Added a lot of extra parameters with no gain in performance - - Cool idea, but on a single GPU, nah... not worth it! + - Cool idea, but on a single GPU * Word Pieces - Might have worked better, but liked the idea of single character recognition more. |