convolution embedding Transformer
convolution based PyTorch implementation for classification sigmoid.
- Input
- 2609-dim embedding
- Encoder
- 14 x Transformer with 48 heads
- Output
- rouge-l projection
Training config
optimizer=RMSprop, lr=0.948, scheduler=plateau, warmup=728