This blog presents an easy fix to the sentence embeddings learned by pre-trained language models. It is based on the paper: On the Sentence Embeddings from Pre-trained Language Models by Li et al EMNLP 2020.
About 3 min
- Multilingual MT5
- Pre-training5
- BERT4
- Random Aligned Substitution2
- Zero-shot Translation2
- mRASP2
- Speech Translation2
- MT Evaluation2
- Transformer2
- Knowledge Assessment1
- Risk Ratio1
- Vocabulary Learning1
- Optimal Transport1
- Catastrophic Forgetting1
- Model Protection1
- Relation Extraction1
- Fact Verafication1
- Reasoning1
- Logic-regularized neural network1
- Embedding1
- Shared Semantic Memory1
- Chimera1
- Language Modelling1
- Imagination1
- Visual Machine Translation1
- ImagiT1
- Model Capacity1
- Language-specific Sub-network1
- GPU Acceleration1
- CUDA1
- Variational Inference1
- Latent Variable Model1
- Semi-supervised Learning1
- Contrastive Learning1
- BERTScore1
- COMET1
- Translation Memory1
- Recurrent Attention1
- Self-training1
- Unsupervised Machine Translation1