We’ve made a lot of improvements to model2vec since it came out, many of which target the baseline performance of our distillation process.

Our newest shiny release is here! 0.3.8! This is a small release in line for a big one we’ll be releasing next week. See here for the release notes.

We’re super excited to announce the release of semhash, our semantic deduplication and dataset multitool (other features coming soon).

This blogpost describes the Tokenlearn method, which is a method to pre-train Model2Vec models.

This blog was first posted on the Hugging Face blog. We’re also posting it here for archival purposes.