It would work the same way, you would just need to connect with your local model. For example, change the code to find the embeddings with your local model, and store that in Milvus. After that, do the inference calling your local model.

I’ve not used inference with local API, can’t help with that, but for embeddings, I used this model and it worked quite fast, plus was a top2 model in Hugging Face. Leaderboard. Model.

I didn’t do any training, just simple embed+interference.

Milvus documentation has a nice example: link. After this, you just need to use a persistent Milvus DB, instead of the ephimeral one in the documentation.

Let me know if you have further questions.

OP can also use an embedding model and work with vectorial databases for the RAG.

I use Milvus (vector DB engine; open source, can be self hosted) and OpenAI’s text-embedding-small-3 for the embedding (extreeeemely cheap). There’s also some very good open weights embed modelsln HuggingFace.

You missed the entire point of the comment by incorrectly calling my country “Columbia”. I don’t even know what to say.

Let’s not waste any of our time and go ask in any Venezuelan forum about the topic.

It is not my intention to be rude. I’m from Colombia, follow Venezuela’s status closely (from media on a broad range of the political spectrum) see Venezuelan emmigrants daily and have met quite a few Venezuelans, and yet Lemmy is the only place I’ve ever seen with people really convinced that Venezuelans love Maduro, and the current situation of the country is because of the sanctions.

It feels almost surreal, and reminds me when some people on Reddit were convinced they knew better than me what’s my country’s political status, all while mistakenly calling the country “Columbia”.

I’m not trying to argue that you should blindly trust my opinions here, but really, really, Venezuela is in a bad spot, nobody likes Maduro’s dictatorship, and the sanctions are not the main causes of any of that (but they do help). Either that or somehow almost everybody in whole Latin American has a very biased opinion from first-hand experiences, and only people from other continents can see that.

Yeah, the rigged ones lol. There’s even mathematical evidence of it being rigged, with votes accounting for exact percentages with just 2 decimal places, for every single candidate.

Venezuela hasn’t publish the official acts, nor let international observers be present in the elections. There was heavy repression on elections day as well, plus some offices not letting people vote.

It is the poorer population that suffers the most. That’s the reason Venezuela has such a big emigration crisis, and every latinamerican country has also seen such a massive influx of poor emigrants. I experience this firsthand, almost daily.

It is not rich people that the militia constantly murders/kidnap.

Absolutely, yes.

I kinda doubt that will happen. For instance, look at Venezuela: Venezuelans are beyond fed up with Maduro’s dictatorship, but there’s nothing they can do against the government forces.

Governments will do anything they can to prevent a paradigm change.