Building a RAG Pipeline

Retrieval-Augmented Generation (RAG) combines a large language model with your own document base. Instead of relying on the model’s training knowledge, the pipeline retrieves relevant passages from your sources and passes them as context to the LLM. This keeps answers current, traceable, and grounded in your data.

A complete RAG pipeline consists of six steps. The noris “LLM as a Service” platform handles embedding, reranking, and final generation. Storage and similarity search happen in the vector database of your choice.

Step 1: Embed Documents

First, convert each document section into a dense vector. Use the POST /v1/embeddings endpoint for this. Send the plain text of each section in the input field and choose the platform’s embedding model as model.

curl -X POST https://ai.noris.de/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "vllm/release/harrier-oss-v1-0.6b",
    "input": "The noris Cloud Stack offers a complete Kubernetes platform for enterprises."
  }'

The response contains a vector (a list of floats) of fixed dimensionality under data[0].embedding. Also store metadata such as source, chapter, or date for each section.

Step 2: Store in a Vector Database

Store the vectors in the vector database of your choice, for example pgvector, Weaviate, Qdrant, Milvus, or Pinecone. The noris platform doesn’t lock you into a specific database. Store at minimum the embedding vector, the original text, and metadata for each record. Choose cosine similarity as the distance metric, since the platform’s embedding models are optimized for it.

Step 3: Embed the Query

When a user asks a question, its text must be transformed into the same vector space as the documents. Use exactly the same endpoint and model as in step 1. Only the content of input differs.

curl -X POST https://ai.noris.de/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "vllm/release/harrier-oss-v1-0.6b",
    "input": "Which Kubernetes platform does noris offer?"
  }'

Take the vector from data[0].embedding and pass it to your vector database.

Step 4: Retrieve Candidates

Run a similarity search in your vector database that returns the top_k closest sections to the user’s query. Typical top_k values range from 20 to 50, deliberately larger than the final count, because in the next step a reranker filters out the truly relevant hits. Depending on your database, you can apply additional filters (e.g. language, date, access rights) to narrow the candidate set.

Step 5: Reranking

Embedding-based search is fast but not always precise: it measures coarse semantic proximity but only partially understands nuances like negation or relevance hierarchies. A cross-encoder reranker scores each query-candidate pair jointly and re-sorts the list. The POST /v1/rerank endpoint takes the query, the list of candidate texts, and the desired count top_n.

curl -X POST https://ai.noris.de/v1/rerank \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "vllm/release/bge-reranker-v2-m3",
    "query": "Which Kubernetes platform does noris offer?",
    "documents": [
      "The noris Cloud Stack offers a complete Kubernetes platform for enterprises.",
      "noris operates data centers in Nuremberg and Frankfurt.",
      "Kubernetes is a container orchestration system.",
      "With noris Managed Kubernetes, teams scale workloads automatically."
    ],
    "top_n": 2
  }'

The response contains the indices of the sorted documents along with a relevance score under results. Keep only the top_n best hits, these then go into the context for the LLM.

Step 6: LLM Request with Context

Now for the actual generation. Pass the ranked sections as context within the system message to the /chat/completions endpoint. Instruct the model to answer strictly based on the context and to openly admit missing knowledge.

curl -X POST https://ai.noris.de/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "vllm/release/gpt-oss-120b",
    "messages": [
      {
        "role": "system",
        "content": "Answer questions strictly based on the following context. If the information is not contained, say: \"Not stated in the context.\"\n\nContext:\n1) The noris Cloud Stack offers a complete Kubernetes platform for enterprises.\n2) With noris Managed Kubernetes, teams scale workloads automatically."
      },
      {"role": "user", "content": "Which Kubernetes platform does noris offer?"}
    ]
  }'

Model Recommendations

Step	Recommended Model	Purpose
1 & 3: Embedding	vllm/release/harrier-oss-v1-0.6b	Compact, fast, RAG-optimized.
5: Reranking (multilingual)	vllm/release/bge-reranker-v2-m3	High precision, many languages.
5: Reranking (high throughput)	vllm/release/jina-reranker-v2-base-multilingual	0.3B parameters, efficient at scale.
6: Generation	vllm/release/gpt-oss-120b	Strong reasoning for precise answers.
6: Generation (long context)	vllm/release/glm-5-2	Up to 1M tokens of context, ideal for large candidate sets.

For further selection guidance, see Model Selection.