Error Training Vision Encoder Decoder for Image Captioning

lewtun
Sat, 20 Nov 2021 10:28:33 +0000

Great detective work @seanbenhur ! I think that adding the BOS token is the cause of the problem and you can resize the embedding layer as follows:

model.resize_token_embeddings(len(tokenizer))

Does resizing the embedding layer solve the problem?