RAG

What is RAG?

Retrieval Augmented Generation (RAG) is an architecture pattern in AI that connects a Large Language Model (LLM) to an external knowledge source such as a vector database, company documents, or live search before generating a response.

Instead of relying solely on static parametric knowledge learned during training, RAG retrieves relevant facts dynamic to the user’s prompt and injects them into the context window.

A standard RAG pipeline operates in three key phases:

• Ingestion & Indexing (Offline):

  • Raw documents (PDFs, docs, database records) are broken down into smaller text chunks.
  • Chunks pass through an embedding model to convert text into vector embeddings.
  • Embeddings are indexed and stored in a vector database (e.g., Pinecone, Qdrant, Milvus, or pgvector).

• Retrieval (Runtime):

  • The user submits a prompt (e.g., “What is our company’s Q3 refund policy?”).
  • The query is converted into an embedding using the same model.
  • A vector similarity search (k-NN / cosine similarity) identifies the top matching chunks in the vector store.

• Generation (Runtime):

  • The retrieved chunks are appended to the user prompt as context (e.g., “Answer the query using ONLY the following source snippets: […]”).
  • The LLM processes the augmented prompt and generates a grounded, accurate response.

Relational vs Vector database

The primary difference between a SQL database (Relational Database) and a Vector database lies in how they store data and how they search through it. SQL databases look for precise, logical matches, while vector databases look for conceptual similarity.

Core Differences at a Glance

Feature	SQL Database (Relational)	Vector Database
Data Structure	Structured rows, columns, and tables with fixed schemas.	High-dimensional numeric arrays (embeddings) and metadata.
Search Logic	Deterministic: Exact matches, keyword matching, and boolean logic (=, <, >, LIKE).	Probabilistic: Semantic similarity search based on distance metrics (e.g., Cosine, Euclidean).
Query Goal	Answers explicit, factual questions (e.g., “Find user ID 402”).	Finds conceptually related data (e.g., “Find articles about reducing server lag”).
Consistency	High Integrity: Strict ACID compliance for reliable transactions.	Fuzzy/Approximate: Focuses on speed and relevance rather than rigid transactional guarantees.
Core Use Case	Banking, ERP systems, inventory, and structured records.	AI/LLM applications, RAG (Retrieval-Augmented Generation), and recommendation engines.
Examples	PostgreSQL, MySQL, SQL Server.	Pinecone, Chroma, Qdrant.

How They Differ in Action

1. SQL Databases: Exact Matches

Traditional SQL engines communicate via structured, explicit logic. If you query a table for the keyword "USA", it will not automatically pull up records containing "United States" or "North America" unless you explicitly write code to link them. It excels at filtering, sorting, and joining strict data points (like dates, strings, and integers) with absolute precision.

2. Vector Databases: Semantic Meaning

Vector databases store unstructured data (like text, images, or audio) that has been converted by AI models into mathematical coordinates called embeddings. Because these numbers represent the meaning of the data, the database uses Approximate Nearest Neighbor (ANN) algorithms to scan billions of data points instantly. It understands that “reduce latency” and “optimize performance” are conceptually close, even if they share zero identical keywords.

Why Vector Databases Matter for RAG?

• Semantic Search: They convert text and queries into numerical lists called vector embeddings, allowing the system to match concepts even if the phrasing is completely different.
• Context Grounding: They supply large language models (LLMs) with precise, domain-specific data, which reduces hallucinations and keeps answers accurate.
• Speed at Scale: They use specialized indexing techniques (like HNSW) to perform fast approximate nearest neighbor searches across millions of data chunks.
• Flexibility: Opinions are mixed on whether a standalone vector database is always mandatory. Some developers prefer traditional relational databases or hybrid keyword-vector systems for production reliability, but vector databases remain the standard default for semantic retrieval.

Whats the need for multi demension?

A multi-dimensional database (like an OLAP cube or a vector database) solves the problem of contextual correlation across multiple attributes simultaneously, which a single-dimensional database (like a flat key-value store or a simple linear list) cannot do efficiently

Here is a practical example contrasting how they handle a complex problem:

Imagine you are building a music streaming app that suggests songs based on three distinct attributes: tempo (BPM), mood (happiness scale), and acousticness.

A user comes along looking for a track that is energetic but melancholy, using organic acoustic instruments (e.g., a fast, sad acoustic guitar song).

How a Single-Dimensional Database Fails

A single-dimensional database can only index and sort data along one independent line at a time.

• The Indexing Limit: It can easily find the fastest songs by sorting the Tempo column. It can also find the saddest songs by filtering the Mood column.

• The Problem: It cannot calculate how “close” a song is across all three attributes at the exact same time.

• The Workaround: To find a match, it has to perform a rigid query like: WHERE tempo > 120 AND mood < 3 AND acousticness > 0.8.

• The Failure: If no song perfectly matches those exact boundaries, it returns zero results. It cannot recognize that a song with a tempo of 118 and acousticness of 0.79 is actually an excellent, near-perfect match, because it evaluates each dimension in isolation.

How a Multi-Dimensional Database Solves It

A multi-dimensional database plots each song as a single coordinate point in a 3D geometric space (X = Tempo, Y = Mood, Z = Acousticness).

       [Mood (Sad)]
           ^
           |    * (Target Vibe)
           |   /
           |  /  * (Song A: 118 BPM, Sad, Acoustic) -> Very Close!
           | /
           +-------------------> [Tempo (Fast)]
          /
         /
        v
 [Acousticness]

• True Spatial Search: Instead of checking boxes, the database treats the user’s request as a target coordinate point in that 3D room.

• Nearest Neighbour Calculation: It measures the actual physical distance (using math like Euclidean distance) from the target point to all surrounding songs.

• The Solution: It instantly identifies Song A as the top recommendation because it is geometrically the closest item in the room, even if its exact numbers didn’t hit a rigid, pre-defined filter.

While a single-dimensional database forced you to look through separate, isolated lists, the multi-dimensional database looks at the entire combined shape of the data at once to deliver nuanced, relative, and contextual results.

The Evolving Middle Ground: Hybrid Options

You do not always have to choose one over the other. Many modern applications use a hybrid approach:

1. Vector Extensions: Traditional relational databases have added vector capabilities (e.g., PostgreSQL with pgvector). This allows you to run relational SQL queries and vector similarity searches inside a single system.

2. Dedicated Vector Databases: For massive scale (billions of embeddings) or specialized high-performance AI indexing, standalone systems like Pinecone remain the preferred choice.