From 433c5b2b5287005e4c034ed0071e6488d4daf5ab Mon Sep 17 00:00:00 2001 From: Gustaf Rydholm Date: Sun, 5 May 2024 22:07:44 +0200 Subject: Update rag text --- content/projects/retrieval-augmented-generation.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/content/projects/retrieval-augmented-generation.md b/content/projects/retrieval-augmented-generation.md index 17d1358..d3316d3 100644 --- a/content/projects/retrieval-augmented-generation.md +++ b/content/projects/retrieval-augmented-generation.md @@ -12,15 +12,19 @@ tags: draft: false --- -I implemented a retrieval augmented generation (RAG) -[program](https://github.com/aktersnurra/rag) for fun with the goal of being able to +I implemented a retrieval augmented generation (RAG) +[program](https://github.com/aktersnurra/rag) for fun with the goal of being able to search my personal library. My focus was to make this run locally with only open -source models. This was achieved with `ollama` and `sentence-transformers` for -downloading and running these models locally. However, the project was expanded to -integrate with cohere and its rerank and command-r+ models, since I was curious about -the command-r+ performance. These models can be downloaded and run locally, but it took -ages for my computer to generate any output, since the command-r+ is huge. +source models. This was achieved with [`ollama`](ollama.com) and +[`sentence-transformers`](https://github.com/UKPLab/sentence-transformers) for +downloading and running these models locally. +However, the project was expanded to +integrate with cohere and their rerank and command-r+ models, since I was curious about +the command-r+ performance. These models can be downloaded and run locally, but it took +ages for my computer to generate any output, since the command-r+ is 104B parameter +model. The obvious and cool benefit of the command-r+ is that it generates citations +from the context in the answer. Here is a [presentation](/rag.html) that gives a brief overview of what a RAG system is, and how it can be improved with reranking. -- cgit v1.2.3-70-g09d2