Model Feedback
Post any feedback you have on the models here
what is Hypernetworks ?
what is Hypernetworks ?
If you are using the automatic1111 webui to generate images, hypernetworks are an additional component that can be loaded to further finetune and refine image generation. Other versions of stable diffusion may have hypernetworks implemented as well, but I dont know if the hypernetwork will work with all implementations.
Does the new LORA version give better results than the old checkpoint?
The 1.5 checkpoint needs the hypernet to match the portrait artstyle, I assume there is not a lot of point in using the LORA with some other model if you still need the 1.5 model for the hypernet?
Every one of those rows is the LORA version used with a different loaded base checkpoint, so you don't need to use the full model checkpoint I trained with the LORA, you can use whatever model checkpoint you want to try. This makes the LORA version far more versatile and adaptable and I think it should make more sense that the concept of a "starsector portrait" is not really a particularly complex idea, so devoting an entire model checkpoint to it is pretty overkill. You can use the LORA with the model checkpoint trained here as well,
The results using the LORA heavily depend on the model checkpoint you choose to use it with, base models that have much more training on people will produce much better portrait and face features for example. Also, like a hypernet LORA can be easily adjusted in how much of an effect they apply to the image as well so you can much more easily fine tune things. You can use it with the model checkpoint trained here, if you do I would basically just use it in place of the hypernet.
Here is an example I just quickly made using the LORA with the version 2.1 trained ckpt from here and without the hypernet, sweeping the applied strength of the LORA
LORA is applied in the prompt using < lora:Starsector_Portrait_LORA_1.13.0.e8:.1 > so the labels in the image is the applied strength changing from 0.1 - 1.0
So you can see that when used with the old trained checkpoints I made it functions similarly to the hypernet, but the fact the LORA can be used with any base checkpoint is really I think what makes it shine.
The model checkpoint is pretty much a product of its early time released months ago when training capabilities and technologies were much less developed, so I see the lora as pretty much a replacement for the old full checkpoint version. The LORA being just a single building block you can plug into other building blocks to make the image you want compare to the checkpoint which is more like the building foundation much more set and less adaptable.
The LORA seems to be a good step forward and can generate a lot more variety while mostly keeping the style training intact.
From starting at :8> LORA strength you can see odd artifacts around the edges of the hair and with the eyes.
The old 1.5 checkpoint version does the same thing, I wonder what causes it, the model trying to do sharpening effect?
Have you considered retraining the model with added portraits from 0.96?
It has 36 new portraits (a 25% increase).
Have you considered retraining the model with added portraits from 0.96?
It has 36 new portraits (a 25% increase).
Yup already added to the training set. I have been testing a bunch of different settings and changes to the training process to try and continue to improve things which is taking its time. Eventually I should have a new LORA version I am happy with that is much improved using the increased dataset and improved training process.
One of the things I have been trying to incorporate is some of the things I learned making a Stellaris style portrait LORA which let you include a species as part of the prompt. So with more complex captioning and training in Starsector the goal would be that you could specify a particular faction for style queues in the output.