Model Merging

LEGIONHETO supports three model merging algorithms.

SLERP (Spherical Linear Interpolation)

Smooth interpolation between models:

from legionheto.export import slerp_merge

merged = slerp_merge(model1, model2, t=0.5)

TIES (Trim, Elect Sign, and Merge)

Selective merging with conflict resolution:

from legionheto.export import ties_merge

models = [base_model, adapter1, adapter2]
weights = [0.5, 0.3, 0.2]

merged = ties_merge(models, weights, density=0.6)

DARE (Drop And REscale)

Random dropping with rescaling:

from legionheto.export import dare_merge

merged = dare_merge(models, weights, drop_rate=0.5, rescale=True)

Using the Merger Class

from legionheto.export import ModelMerger

merger = ModelMerger([model1, model2, model3])

# SLERP merge
merged = merger.merge_slerp(t=0.5)

# TIES merge
merged = merger.merge_ties(density=0.6)

# DARE merge
merged = merger.merge_dare(drop_rate=0.5)