summaryrefslogtreecommitdiff
path: root/content/projects/retrieval-augmented-generation.md
diff options
context:
space:
mode:
Diffstat (limited to 'content/projects/retrieval-augmented-generation.md')
-rw-r--r--content/projects/retrieval-augmented-generation.md18
1 files changed, 11 insertions, 7 deletions
diff --git a/content/projects/retrieval-augmented-generation.md b/content/projects/retrieval-augmented-generation.md
index 17d1358..d3316d3 100644
--- a/content/projects/retrieval-augmented-generation.md
+++ b/content/projects/retrieval-augmented-generation.md
@@ -12,15 +12,19 @@ tags:
draft: false
---
-I implemented a retrieval augmented generation (RAG)
-[program](https://github.com/aktersnurra/rag) for fun with the goal of being able to
+I implemented a retrieval augmented generation (RAG)
+[program](https://github.com/aktersnurra/rag) for fun with the goal of being able to
search my personal library. My focus was to make this run locally with only open
-source models. This was achieved with `ollama` and `sentence-transformers` for
-downloading and running these models locally. However, the project was expanded to
-integrate with cohere and its rerank and command-r+ models, since I was curious about
-the command-r+ performance. These models can be downloaded and run locally, but it took
-ages for my computer to generate any output, since the command-r+ is huge.
+source models. This was achieved with [`ollama`](ollama.com) and
+[`sentence-transformers`](https://github.com/UKPLab/sentence-transformers) for
+downloading and running these models locally.
+However, the project was expanded to
+integrate with cohere and their rerank and command-r+ models, since I was curious about
+the command-r+ performance. These models can be downloaded and run locally, but it took
+ages for my computer to generate any output, since the command-r+ is 104B parameter
+model. The obvious and cool benefit of the command-r+ is that it generates citations
+from the context in the answer.
Here is a [presentation](/rag.html) that gives a brief overview of what a RAG system
is, and how it can be improved with reranking.