llama3.1-gutenberg-8B-Heretic
A decensored version of nbeerbower/llama3.1-gutenberg-8B, made using Heretic v1.1.0
| llama3.1-gutenberg-8B-Heretic | Original model (llama3.1-gutenberg-8B) | |
|---|---|---|
| Refusals | 3/100 | 97/100 |
| KL divergence | 0.0615 | 0 (by definition) |
Heretic Abliteration Parameters
| Parameter | Value |
|---|---|
| direction_index | 13.98 |
| attn.o_proj.max_weight | 1.34 |
| attn.o_proj.max_weight_position | 19.54 |
| attn.o_proj.min_weight | 1.33 |
| attn.o_proj.min_weight_distance | 12.82 |
| mlp.down_proj.max_weight | 1.22 |
| mlp.down_proj.max_weight_position | 24.24 |
| mlp.down_proj.min_weight | 1.06 |
| mlp.down_proj.min_weight_distance | 11.39 |
llama3.1-gutenberg-8B
VAGOsolutions/Llama-3.1-SauerkrautLM-8b-Instruct finetuned on jondurbin/gutenberg-dpo-v0.1.
Method
Finetuned using 2x RTX 4060 for 3 epochs.
- Downloads last month
- 5