A high performance open-source library for NLP Transformer model training and inferencing.
About 4 min
A high performance open-source library for NLP Transformer model training and inferencing.
Upon its emergence, the Transformer Neural Networks [1] dominates the sequence-to-sequence tasks. It even outperforms the Google Neural Machine Translation model in specific tasks. Specifically, the multi-head attention mechanism that depends on element-wise dot-product is deemed as one of the critical building blocks to get things to work. But is it really that important?