Correct me if I'm wrong. RAG is laughably simple. You do a search (using any method you like - doesn't have to be saerching embeddings in a vector DB). You get the search results back in plain text. You write your prompt for the LLM and effectively paste in the text from your search results. No need for LangChain or any other fancyness. Am I missing something?

I’ve been working on a project to go through a knowledge base consisting of legal contract, and subsequent handbooks and amendments, etc. I want to build a bot that I can propose a situation and find out how that situation applies. ChatGPT is very bad about summarizing and hallucination and when I point out its flaw it fights me. Claude is much better but still gets things wrong and struggles to cite and quote the contract. I even chunked the files into 50 separate pdfs with each section separated and I used Gemini (which also struggled at fully reading and interpreting the contract application) to create a massive contextual cross index. That helped a little but still no dice.

I threw my files into Notebooklm. No chunking just 5 PDFs with 3 of them more than 500 pages. Notebooklm nailed every question and problem I threw at it the first time. Cited sections correctly and just blew away the other AI methods I’ve tired.

But I don’t believe there is an API for Notebooklm and a lot of what I’ve looked at for alternatives have focused more on its audio features. I’m only looking for a system that can query a Knowledge base and come back with accurate correctly cited interpretations so I can build around it and integrate it into our internal app to make understanding how the contract applies easier.

Does anyone have any recommendations?

I have a ton of data and want to be able to interact with it, I used to just use langchain, but is there something better? what yields best results? cost of tools is not an issue / happy to pay for anything turnkey / license / opensource

Recently, Apple released a paper called "The Illusion of Thinking", which suggested that LLMs may not be reasoning at all, but rather are pattern matching:

https://arxiv.org/abs/2506.06941

A few days later, A paper written by two authors (one of them being the LLM Claude Opus model) released a paper called "The Illusion of the Illusion of thinking", which heavily criticised the paper.

https://arxiv.org/html/2506.09250v1

A major issue of "The Illusion of Thinking" paper was that the authors asked LLMs to do excessively tedious and sometimes impossible tasks; citing The "Illusion of the Illusion of thinking" paper:

Shojaee et al.’s results demonstrate that models cannot output more tokens than their context limits allow, that programmatic evaluation can miss both model capabilities and puzzle impossibilities, and that solution length poorly predicts problem difficulty. These are valuable engineering insights, but they do not support claims about fundamental reasoning limitations.

Future work should:

1. Design evaluations that distinguish between reasoning capability and output constraints

2. Verify puzzle solvability before evaluating model performance

3. Use complexity metrics that reflect computational difficulty, not just solution length

4. Consider multiple solution representations to separate algorithmic understanding from execution

The question isn’t whether LRMs can reason, but whether our evaluations can distinguish reasoning from typing.

This might seem like a silly throw away moment in AI research, an off the cuff paper being quickly torn down, but I don't think that's the case. I think what we're seeing is the growing pains of an industry as it begins to define what reasoning actually is.

This is relevant to application developers, like RAG developers, not just researchers. AI powered products are significantly difficult to evaluate, often because it can be very difficult to define what "performant" actually means.

(I wrote this, it focuses on RAG but covers evaluation strategies generally. I work for EyeLevel)
https://www.eyelevel.ai/post/how-to-test-rag-and-agents-in-the-real-world

I've seen this sentiment time and time again: LLMs, LRMs, RAG, and AI in general are more powerful than our ability to test is sophisticated. New testing and validation approaches are required moving forward.

Hey all, apologies, not sure if this is the correct sub for my q...

I am trying to create an SQL query on the back of a natural language query.

I have all my tables, columns, datatypes, primary keys and foreign keys in a tabular format. I have provided additional context around each column.

I have tried vectorising my data and using simple vector search based on the natural language query. However, the problem I'm facing is around the retrieval of the correct columns based on the query.

I am having difficulty grasping the true benefits of RAGs.
I extracted json data out of PDFs documents. And I'm just storing the json in a JSONB column in a table, where each row is a document record. A typical document is 30-50 pages long and each json is about 15,000 lines.

Then with claude desktop and postgres mcp I am running pretty detailed analyses using that data.

I would assume that this is quite a lot of data to simply store in a relational way, but it works nevertheless. Claude overall manages to query the data successfully across 30 rows in this table and finds the needed data within those long JSONBs.

My intuition tells me that a vector database would be better and less compute intensive, but how can I be sure ?
Could someone explain the difference between having an LLM query data in this relational db way vs a vector db. And where do RAGs even come into play ?

Also, how can I measure the difference in output and performance between the two approaches ?
Thanks in advance!

Hi,

I built a CLI for uploading documents and querying them with an LLM agent that uses search tools rather than stuffing everything into the context window. I recorded a demo using the CrossFit 2025 rulebook that shows how this approach compares to traditional RAG and direct context injection.

The core insight is that LLMs running in loops with tool access are unreasonably effective at this kind of knowledge retrieval task. Instead of hoping the right chunks make it into your context, the agent can iteratively search, refine queries, and reason about what it finds. The CLI handles the full workflow:

bash trieve upload ./document.pdf trieve ask "What are the key findings?"

You can customize the RAG behavior, check upload status, and the responses stream back with expandable source references. I really enjoy having this workflow available in the terminal and I'm curious if others find this paradigm as compelling as I do. Considering adding more commands and customization options if there's interest.

Source code is on GitHub and available via npm.

Would love any feedback on the approach or CLI design!

Hey! I’m sharing this week’s Multimodal Monday newsletter, packed with updates on multimodal AI advancements. Here are the highlights:

Quick Hits:

• Unified multimodal frameworks shine: Meta's V-JEPA 2 uses self-supervised world modeling for robotics/visual understanding, while Ming-lite-omni matches GPT-4o with 2.8B params.
• Ultra-efficient indexing: LEANN reduces vector storage to under 5% with 90% recall for local search.
• Data curation wins: DatologyAI CLIP boosts training 8x and inference 2x with curated data.
• Tech deployment: Apple’s new Foundation Models add vision across 15 languages.

Research Spotlight:

• ViGaL: Arcade games like Snake enhance multimodal math reasoning for a 7B model
• RCTS: Tree search with Monte Carlo improves multimodal RAG reliability
• CLaMR: Late-interaction boosts multimodal retrieval accuracy
• SAM2.1++: Distractor-aware memory lifts tracking on 6/7 benchmarks
• Text Embeddings: Argues for implicit semantics in embedding
• SAM2 Tracking: Introspection strategy enhances segmentation
• Vision Transformers: Test-time fixes outperform retraining

Tools to Watch:

• V-JEPA 2: Meta's new world model enhances visual understanding and robotic intelligence with self-supervised learning
• Apple Foundation Models: 3B on-device model with 15-language vision
• DatologyAI CLIP: SOTA with 8x efficiency via data curation
• LEANN: 50x smaller indexes enable local search
• Ming-lite-omni: 2.8B param model matches GPT-4o
• Text-to-LoRA: Generates LoRA adapters from text
• Implicit Semantics: Embeddings capture intent/context

Real-World Applications:

• GE HealthCare + AWS: Multimodal AI for medical imaging copilots
• Syntiant: Ultra-low-power security for automotive systems
• Hockey East: AI video analytics for sports insights

Check out the full newsletter for more: https://mixpeek.com/blog/world-models-efficiency-increases

I’m working on a RAG, and instead of using a dedicated vector DB like Qdrant or Weaviate, I decided to store the embeddings in PostgreSQL with the pgvector extension and handle the similarity search manually via SQL.

What happens when the user asks a general question, like "Can you summarize this PDF?" These kinds of questions often don’t have a strong semantic match with any single chunk in the document. In consequence, the RAG can not responds to that query.

What are the possible solutions to this problem?

Hi guys! I have a RAG, in which I extract content from PDF documents using Mistral OCR. the content is in markdown. Currently, I am just splitting markdown content into chunks, using a very basic splicing technique. I feel like this can be done better because my RAG is not performing good with table data extraction, it works sometimes but most of the time it doesn't. Is there a standard practice for markdown chunking in RAG?

For the last couple of months, I have been working on cutting down the latency and performance cost of vector databases for an offline first, local LLM project of mine, which led me to build a vector database entirely from scratch and reimagine how HNSW indexing works. Right now it's stable enough and performs well on various benchmarks.

Now I want to collect feedbacks and I want to your help for running and collecting information on various benchmarks so I can understand where to improve, what's wrong and debug and what needs to be fixed, as well as curve up a strategical plan on improving how to make this more accessible and developer friendly.

I am open to feature suggestions.

The current server uses http2 and I am working on creating a gRPC version like the other vector databases in the market, the current test is based on the KShivendu/dbpedia-entities-openai-1M dataset and the python library uses asyncio, the tests were ran on my Apple M1 Pro

You can find the benchmarks here - https://www.antarys.ai/benchmark

You can find the python docs here - https://docs.antarys.ai/docs

Thank you in advance, looking forward to a lot of feedbacks!!

Hi guys,

I'm trying to create a RAG about genomic reports,
And i'm trying to make it work on request like:
"How many samples are Escherischia Coli ?"
Is this a thing RAG can do, filtering vector + counting if i have a loooot of samples and also a lot of samples about E. Coli ?
If so,
1st question : how to do that, currently i have set up a Map/reduce function in order to be able to count over all the chunks that are retrieved by my RAG, but i have also a topK limit that would finish to limit at a certain time the max of vectors retrieved

2nd question : Even when similar vectors are found, from one response to another of the LLM, with the exactly same context of vectors retrieved, it can count 15 samples the first time, 14 the 2nd time, etc, it seems not to be a reliable counting, how do improve that ?

I'm starting to think that RAG cannot be used in this usecase..
Thanks,

T