TheSkullery
/

LD-Zephyria-37b

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Edit model card

Model Information

Base Model: unsloth/Mistral-Small-Instruct-2409

Strategy: Late Duplication

Total Layers: 55

Duplication Start: Layer 28 (50.9% of model)

Duplicated Layers: 21 (38.2% of model)

Unique Final Layers: 7 (12.7% of model)

Model Characteristics

Models down_proj and o_proj layers have been nulled and will require healing
Emphasizes complex feature extraction before duplication
Smallest duplicated section among all strategies
Ideal for tasks requiring extensive unique feature processing
May excel in tasks that benefit from a wide range of unique features before refinement

Configuration Visualization


[        Unique        ][    Duplicated    ][ Unique ]
0 ------------------- 27 28 ------------ 48 49 --- 54
        50.9%               38.2%           10.9%

Downloads last month: 1

Safetensors

Model size

37.5B params

Tensor type

BF16

·

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for TheSkullery/LD-Zephyria-37b

Base model

unsloth/Mistral-Small-Instruct-2409

Finetuned

(6)

this model

Quantizations