diff options
| author | Gustaf Rydholm <gustaf.rydholm@gmail.com> | 2021-06-07 22:57:24 +0200 |
|---|---|---|
| committer | Gustaf Rydholm <gustaf.rydholm@gmail.com> | 2021-06-07 22:57:24 +0200 |
| commit | 87ea75c803400ca74c4f1b863c496165ed802fc2 (patch) | |
| tree | 6730c664b558691af0693a61d09d3aaf02366069 /text_recognizer/networks/transformer/nystromer/attention.py | |
| parent | 01d6e5fc066969283df99c759609df441151e9c5 (diff) | |
Working feedforward of full transformer arch in notebook
Diffstat (limited to 'text_recognizer/networks/transformer/nystromer/attention.py')
| -rw-r--r-- | text_recognizer/networks/transformer/nystromer/attention.py | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/text_recognizer/networks/transformer/nystromer/attention.py b/text_recognizer/networks/transformer/nystromer/attention.py index 5ab19cf..695a0d7 100644 --- a/text_recognizer/networks/transformer/nystromer/attention.py +++ b/text_recognizer/networks/transformer/nystromer/attention.py @@ -47,6 +47,7 @@ class NystromAttention(nn.Module): dropout_rate: float = 0.0, ): super().__init__() + self.dim = dim self.residual = None self.eps = eps self.num_heads = num_heads |