ValueError: Trying to set a tensor of shape torch.Size([4096]) in "weight" (which has shape torch.Size([4096, 4096])), this look incorrect.
#4
by
penut85420
- opened
Traceback (most recent call last):
File "/mnt/ssd4t/Penut/Testing/240520-VLM/hello.py", line 34, in <module>
model = AutoModelForVision2Seq.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ssd4t/Penut/Miniconda3/envs/Py311/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ssd4t/Penut/Miniconda3/envs/Py311/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3754, in from_pretrained
) = cls._load_pretrained_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ssd4t/Penut/Miniconda3/envs/Py311/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4214, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ssd4t/Penut/Miniconda3/envs/Py311/lib/python3.11/site-packages/transformers/modeling_utils.py", line 887, in _load_state_dict_into_meta_model
set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
File "/mnt/ssd4t/Penut/Miniconda3/envs/Py311/lib/python3.11/site-packages/accelerate/utils/modeling.py", line 358, in set_module_tensor_to_device
raise ValueError(
ValueError: Trying to set a tensor of shape torch.Size([4096]) in "weight" (which has shape torch.Size([4096, 4096])), this look incorrect.
version: transformers==4.41.0
Sorry, I forgot to add monkey_patch code for yivl.
You can refer to latest readme file to modify example code.
For training and more usage, please refer to code in llama-factory: https://github.com/hiyouga/LLaMA-Factory/blob/main/src/llamafactory/model/utils/visual.py
BUAADreamer
changed discussion status to
closed