RAG and data platforms for companies

I build the retrieval that gets the knowledge of a company to a language model, and the data platform under it: migrations, models, sync bridges and analytics. It is for teams whose assistant answers from the wrong document, or whose data is not yet in a shape that a model, or a person, can query.

RAG, retrieval-augmented generation, means that a model answers from documents or records fetched at the moment of the question, not from what it memorized in training. Retrieval decides the answer: a model that reads the wrong file answers with confidence from it. A company needs this work when its knowledge is spread over documents, databases and business tools, and when an answer from the wrong source costs more than no answer.

What I build

  • Hybrid retrieval. Vector search and full-text search over the same corpus, merged in the database. On LeanManufacturing.mx this is one SQL function on Neon Postgres with pgvector, under a widget that answers from the published documentation. The embeddings are optional: without them the search falls back to the lexical index.
  • Retrieval for agents. How an agent locates a document: a search tool, a curated index or both, chosen by measuring which one reaches the right file.
  • Migrations, models and analytics. From collections in a document store to a relational schema with keys and constraints, dimensional models that keep history, and the dashboards that read them.
  • Sync bridges that keep what they move. Integrations between business systems with persistent storage, duplicate detection and automatic recovery from errors.

Where answers go wrong: the retrieval or the data under it

  • The assistant writes a fluent answer and takes it from the wrong document.
  • Records about the same customer sit in separate systems with no shared key.
  • A sync moves data between systems and deletes it afterwards, so no history is left.
  • Duplicates, mixed formats and placeholder dates reach the model as if they were facts.

The first is a retrieval fault, and it can be measured. A common design gives the agent a hand-written index that summarizes every file and tells it what to open. In a pre-registered study on a corpus of prose documents, an agent with that index as its only locator reached the right document on about half of the questions, and an agent with plain search on four out of five. The agent with the index read more files and landed on the wrong one more often. A public replication with a released corpus pointed the same way. The result concerns that kind of index on those corpora, not an index used together with search, so on your corpus the answer is measured, not assumed.

How the work runs

  1. A frozen question set. We write real questions, each with the document or record that answers it, and freeze the set before anything is tuned.
  2. Reach before accuracy. For every question I record whether the system reached the right source, apart from whether the answer was correct. Not finding the document and misreading it are different faults.
  3. The simplest search that passes. Lexical search first, then vectors where they find more. Combining methods is tested, not assumed: in a companion study a better ranker found far more documents, and fusing methods made the results worse.
  4. The data underneath. Where the sources are not ready I model them: keys and constraints, deduplication, history, and incremental loads that process only what changed.
  5. Operation. Reindexing after a content change is one authenticated request. Sync jobs get a dry-run mode, switches that turn a destination off without a deployment and retries with backoff.

What you receive

The schema and its migrations, the loading and sync jobs, the index and the retrieval function, the question set with its results, and the dashboard or assistant built on top, all in your database and your cloud accounts.

What this does not include

I do not promise that a vector database fixes wrong answers, and I do not add an index, a layer of summaries or a second search method without a measurement that shows it helps. Retrieval does not repair data that is wrong at the source: duplicates and placeholder values are handled first.

Where the evidence is

For a business in the warranty management industry I built the pipeline that migrates and synchronizes data from MongoDB to PostgreSQL, the relational schema and the analytics dashboard on top. I turned an inherited sync bridge between PropertyWare and ServiceFusion into a persistent warehouse with a dimensional model and full history, while the system stayed in production. LeanManufacturing.mx shows hybrid retrieval on a public site. The retrieval study is published with its corpus, questions, harness and adjudicator.

Work and evidence

Enterprise analytics platform (opens in a new tab)

My contribution: Worked on MongoDB migration, data modeling and the analytics application described in the case study.

What this supports: Experience integrating corporate data and building its application layer.

What it does not establish: Reported record volumes and page timings are case-study claims, not an agent load test.

Source reviewed:

Reading More, Finding Less (opens in a new tab)

My contribution: Authored a pre-registered study and public replication of agent retrieval routing.

What this supports: Evaluation methodology with a released corpus, frozen questions, harness and adjudicator.

What it does not establish: Results concern the studied document corpus and routing policies, not all RAG or agent systems.

Source reviewed:

Common questions

What is RAG and when does a company need it?

RAG means the model answers from documents or records retrieved at the moment of the question. A company needs it when its assistant must answer from its own manuals, policies, catalogs or records, and when an answer taken from the wrong source has a cost.

Do we need a vector database for RAG?

Not always. On LeanManufacturing.mx the vectors live in Postgres with pgvector, next to a full-text index, and the search still works on the lexical index alone. In my retrieval research, plain search over files beat a hand-written index. I start with the simplest search and add vectors when a measurement shows that they find more.

Why does our AI assistant answer from the wrong document?

Most often because retrieval sent it there. In a study I ran, an agent guided by a hand-written index had read a wrong file in 43 of its 47 wrong answers, and when it did reach the right file it was correct most of the time. The repair is in how documents are found, and that is what I measure first.

Do you take RAG and data projects for companies in the United States and Mexico?

Yes. I work remotely from Monterrey, Mexico, with teams in both countries, in English and in Spanish, on contract or in a senior role. The database, the pipelines and the index run in your accounts.

Can you clean up our data before building the assistant?

Yes, and often that is the first job. I have migrated document stores to relational models, added deduplication and history to a sync between business systems, and built the analytics on top. An assistant on unreliable data repeats its errors with confidence.

Related questions

Open to senior AI engineering roles and contract engagements, remote in the US, Mexico and Latin America.