[t5gemma] resize_token_embeddings does not effect to decoder.embed_tokens

[KoichiYasuoka]

System Info

• transformers version: 5.5.1
• Platform: Linux-6.6.113+-x86_64-with-glibc2.35
• Python version: 3.12.13
• Huggingface_hub version: 1.9.2
• Safetensors version: 0.7.0
• Accelerate version: 1.13.0
• Accelerate config: not found
• DeepSpeed version: not installed
• PyTorch version (accelerator?): 2.10.0+cpu (NA)
• Using distributed or parallel set-up in script?:

(Google Colaboratory)

Who can help?

@zucchini-nlp

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Quick reproduce:

from transformers import T5GemmaForConditionalGeneration
mdl = T5GemmaForConditionalGeneration.from_pretrained("harshaljanjani/tiny-t5gemma-test")
e = mdl.get_input_embeddings()
f = mdl.model.decoder.embed_tokens
g = mdl.get_output_embeddings()
print(e.num_embeddings, f.num_embeddings, g.out_features)
assert e.num_embeddings == f.num_embeddings == g.out_features
e = mdl.resize_token_embeddings(e.num_embeddings + 1)
f = mdl.model.decoder.embed_tokens
g = mdl.get_output_embeddings()
print(e.num_embeddings, f.num_embeddings, g.out_features)
assert e.num_embeddings == f.num_embeddings == g.out_features

Expected behavior

All e.num_embeddings f.num_embeddings and g.out_features should be increased to 256001.

[KoichiYasuoka]

Sorry @zucchini-nlp but I've just found another bug in resize_token_embeddings of T5Gemma. It seems similar to #45276, but slightly different.

[zucchini-nlp]

Oh... I'll check it today

[KoichiYasuoka]

Thank you @zucchini-nlp for including T5Gemma in #45324. Well, do you have a plan to include T5Gemma2 similar way?

from transformers import T5Gemma2ForConditionalGeneration
mdl = T5Gemma2ForConditionalGeneration.from_pretrained("kingabzpro/t5gemma2-latex-ocr-1k")
e = mdl.get_input_embeddings()
f = mdl.model.decoder.embed_tokens
g = mdl.get_output_embeddings()
print(e.num_embeddings, f.num_embeddings, g.out_features)
assert e.num_embeddings == f.num_embeddings == g.out_features
e = mdl.resize_token_embeddings(e.num_embeddings + 1)
f = mdl.model.decoder.embed_tokens
g = mdl.get_output_embeddings()
print(e.num_embeddings, f.num_embeddings, g.out_features)
assert e.num_embeddings == f.num_embeddings == g.out_features