Llama3.1 gguf model
Hello
I want to start using llama3.1. Previously, I used llama-2-7b-chat.Q5_0.gguf, which had some accuracy issues. Now, I'd like to switch to llama3.1. Could you please share the best model from your list so I can review it? through from it.
I can't in good conscience recommend specific models out of my pool because I haven't used most of them myself. Then, I am personally not very happy with llama-3 nor llama-3.1, and, lastly, it really depends on what your goals are. If you want an assistant, go with the llama-3.1-8b-instruct. Otherwise, probably a good place would be the localllama or localllm subreddits (where you can regularly find suggestions).
I'm yet to see a llama 3.1 model better than version 3.0. Somehow Meta did the 8B version dumber than 3.0 and I don't know why they did that. The new model is good in conversation, but awful in reasoning. 😂
I'm yet to see a llama 3.1 model better than version 3.0. Somehow Meta did the 8B version dumber than 3.0 and I don't know why they did that. The new model is good in conversation, but awful in reasoning. 😂
Try the 405B ones they are really intelligent from my experience. Especially its self-merge BigLlama-3.1-681B-Instruct is as of now the most intelligent open model I've ever tried. The 8B and 70B are just downscaled 405B models as far I'm aware. While the additional training between llama 3.0 and llama 3.1 had quite an impact on 70B I agree with you that 8B showed almost no improvements. I wouldn't expect much 8B improvement in the future should they decide to downscale an even better trained 405B. Other than llama 3.1 I still really like dolphin-2.9.2-qwen2-72b while waiting for a larger Llama 3.1 dolphin finetune. Nemotron-4-340B-Instruct is looking quite promising so far as well but seams to be slightly worse than 405B in reasoning. If you are forced to use 8B and want to try the best llama 3.1 based 8B model dolphin-2.9.4-llama3.1-8b is the one I would choose.
The great unwashed do not understand :)