A Non-sequential Model for Machine Text Translation

A typical sequential DNN has 1 input head and 1 output head. However, the common text translation model with encoder-decoder, or encoder-decoder with attention has another input at middle of the DNN, so it's still DNN but not sequential. The model is fed as following:

Encoder_Out = encoder(Xleftlang)
Decoder_Out = decoder(Encoder_Out,Xrightlang)