Dont work
I get this error:
Error: llama runner process has terminated: GGML_ASSERT(ggml_can_mul_mat(a, b)) failed
Hey @Milor123 I actually uploaded this as part of a work-in-progress PR. It should work if you're using https://github.com/ggerganov/llama.cpp/pull/9209
Hey @Milor123 I actually uploaded this as part of a work-in-progress PR. It should work if you're using https://github.com/ggerganov/llama.cpp/pull/9209
What do you mean bro, do you refer to the <|user|>\n<|image_1|>\n{prompt}<|end|>\n<|assistant|>\n ??
Should i use it as template in Modelfile for ollama? I am not sure about of how set it. I've used the phi template for create the model in ollama, could you please guide me?
Thank u very much bro !
Hey @Milor123 I actually uploaded this as part of a work-in-progress PR. It should work if you're using https://github.com/ggerganov/llama.cpp/pull/9209
Can you please add instructions on how to pip install this branch via pip, and include a code sample for chat completion?
Can't run in Ollama...
$ ollama create phi-3.5-v -f modelfile
transferring model data
using existing layer sha256:ad0a1ee23ea9d88e932b493a4c077dea95c0f52a0f57a604509504c6ebc3df12
using existing layer sha256:9a7383c19d46afba480b39fda2ed75bfb19e3ca68932105a85aa1286bd861dad
creating new layer sha256:a6e8f0354abcf24d61c9c6c0835b798d7a617a2faf0503568b05036414a6face
creating new layer sha256:04df43857439ce1507af8177a47d6709b1c94c611b05e3cd55d49dbdc73443ae
creating new layer sha256:bc39d5b11e2e1b483f5b21dc11a325671f37ce28d884daf24794ab582a34c3ec
writing manifest
success
$ ollama list
NAME ID SIZE MODIFIED
phi-3.5-v:latest 1ec79dc9fc5a 4.7 GB About a minute ago
$ ollama run phi-3.5-v
Error: llama runner process has terminated: signal: abort trap
Request for gguf -
Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int4