Retail IQ is Systech’s reference implementation of a real-time, governed AI chatbot for e-commerce — built on Databricks Lakebase and Unity Catalog, and delivered through LakeBuild, our accelerator for implementing solutions on Lakebase. It shows how to connect conversational AI to live transactional data without ETL lag, connection contention, or bolt-on security layers.
The lesson from building it: when most people think about AI chatbots, they think about the model. But the harder problem isn’t the intelligence that generates a response — it’s the data layer underneath it. Lakebase and Unity Catalog are what let Retail IQ answer from live operational reality instead of yesterday’s snapshot.
The Problem No One Talks About: The OLTP Gap
E-commerce support teams are buried under repetitive, transactional queries. “Where is my order?” “Has my refund been processed?” “Am I still eligible to return this?” These questions make up the majority of all support volume.
The standard answer is an AI chatbot. But here’s what most chatbot implementations quietly get wrong: they don’t have access to live data.
Traditional architectures force an uncomfortable choice:
- Expose the operational application database directly to AI workloads, and accept connection contention, weak governance, no audit trail, and AI query load degrading production performance.
- Replicate data via batch ETL into a separate AI store, and accept that the bot always answers from yesterday’s snapshot.
The result is a familiar pattern:
- Bots that confidently tell customers their order is on the way — when it was actually delayed hours ago.
- Bots that quote return policies that don’t match the customer’s actual order date.
- Bots that erode trust precisely when customers need it most.
This is the OLTP Gap: the missing bridge between live transactional reality and AI intelligence.
Lakebase: Closing the Gap for Good
Lakebase is Databricks’ managed, PostgreSQL-compatible OLTP engine — built natively inside the Databricks Data Intelligence Platform. And it closes the OLTP Gap completely.
Here’s what that means in practice:
| Architecture | Data Freshness | Governance | Complexity |
|---|---|---|---|
| Batch ETL to AI data store | Hours or days | Separate, inconsistent | High |
| Direct exposure of operational app DB | Real-time | Weak, unaudited | High |
| Lakebase inside Databricks | Real-time | Unified via Unity Catalog | Low |
With Lakebase, the batch ETL layer is gone. The BotBrain API — the component that answers transactional customer queries — constructs SQL and queries Lakebase directly. No replication, no sync jobs, just a governed path from customer question to live order data.
When a customer asks “Where is my order?”, the flow is: intent classification → BotBrain API → SQL against Lakebase → live status, ETA, and tracking returned, with a response synthesized in under 1.5 seconds. The record comes from the same data store that drives the e-commerce platform: no stale cache, no estimated status.
Unity Catalog: Governance That Comes Standard
Connecting AI to live operational data raises the governance bar: it has to be baked in, not bolted on. Because Lakebase runs inside Databricks, it inherits Unity Catalog governance automatically — no separate access control tool, no reconciliation between AI-data and operational-data policies.
What Unity Catalog provides for every Lakebase query:
Row-level security: customers only see their own orders, enforced at the data layer rather than in application code.
Column-level masking: sensitive fields (payment details, PII, internal logistics) are masked at query time, so the BotBrain API never receives unmasked data even when it queries a table that contains it.
Audit logging: every query the BotBrain API executes is logged automatically — which session triggered it, what data was returned, and when.
Full data lineage: Unity Catalog traces operational source → Lakebase → BotBrain API → customer response, every hop tracked and auditable.
This matters for any production deployment handling customer records at scale. When a regulator asks “who accessed this customer’s order data?”, the answer is in Unity Catalog — automatically, with no additional instrumentation required.
What Lakebase Stores
Lakebase is the single source of truth for all transactional AI queries: orders (live status, tracking, ETA), returns (eligibility, stage, refund status), delays (notifications and updated arrival windows), and customers (profile, preferences, recent activity). The BotBrain API queries these tables directly on every invocation, and sub-second latency means live querying replaces pre-fetching and caching entirely — always fresher than any cache could be.
Tappable Order Cards: Lakebase UX in Action
The most visible demonstration is the tappable order card experience. When a customer opens the chat widget, the system queries Lakebase using their session identity and renders recent orders as interactive cards (order number, item, date, current status). The customer taps an order and its ID flows to the BotBrain API for follow-up queries. Without sub-second OLTP, this would require pre-fetching into a cache and risking stale data; with Lakebase the cards load live every time, and manual order-ID entry — the worst friction point in mobile support — disappears entirely.
MLflow + Lakebase: Observability All the Way Down
Connecting AI to a live operational database creates a new observability requirement: visibility into the AI layer and the data layer at the same time. MLflow Tracing handles both. Every Lakebase query is logged with SQL text, latency, row count, and the session that triggered it, alongside model call logs, response quality scores, and sentiment data. When something goes wrong, engineers trace the full chain from customer message to response.
The Policy Layer: RAG Complements Lakebase
Not every customer question is transactional. “Can I return this after 30 days?” “What happens if my package is lost?” These are policy questions, and they need a different path.
For policy intelligence, Retail IQ uses a RAG (Retrieval-Augmented Generation) pipeline on Mosaic AI Vector Search. Return rules, delay SLAs, and compensation policies are embedded and indexed; on a policy question, the system retrieves the relevant chunks and synthesizes a grounded answer with citations to the source.
The division of labor is clean: Lakebase handles the “what is” (live facts about this customer and order), RAG handles the “what if” (policy rules and eligibility). Together they cover the full query spectrum without hallucination in either domain.
Escalation: Handing Off with Full Context
When the bot escalates to a human agent (triggered by sentiment signals, legal language, unresolved queries, or direct customer request), the Freshdesk ticket is pre-populated with a live Lakebase snapshot: current status, ETA, items, order value, and the full session query log. The agent arrives with real data and never asks the customer to repeat their order ID.
The broader point: when the AI layer is connected to live operational data, that data becomes an asset at every stage of the customer journey, not just during bot interactions.
The LakeBuild Reference Architecture
LakeBuild is Systech’s accelerator for implementing solutions on Lakebase. Retail IQ is built on the LakeBuild reference architecture shown below — ingestion and storage through to the serving layer, with Unity Catalog as the unified governance layer running beneath everything.
At its core: Lakebase Tables are the source of truth for live transactional data. A DLT pipeline handles incremental embeddings into Mosaic AI Vector Search for the policy RAG layer. Agent Bricks orchestrate the agent workflow, and Genie provides natural-language access to governed data for analytical queries. The serving layer delivers results across Databricks Apps, AI/BI dashboards, agentic actions, and API integrations.
Critically, Unity Catalog spans the entire stack — governing access control, lineage, model registry, and tool permissions across every component. It’s not a layer you add later. It’s the foundation everything else runs on.
The Architecture, Summarized
Four layers. One platform. One OLTP engine.
- Layer 1Freshchat — Customer-facing widget with tappable order cards and guided button menus
- Layer 2Python/Flask middleware — Intent classification, deterministic routing, session management, escalation handling
- Layer 3Databricks + Lakebase — BotBrain API querying Lakebase for live transactional data; RAG pipeline for policy retrieval; MLflow for unified observability
- Layer 4Freshdesk — Automated ticket creation with live Lakebase order snapshots
All transactional data lives in Lakebase. Unity Catalog governs access. MLflow logs every query. No external database. No ETL lag. No data movement.
What the Architecture Delivers: Traditional vs. Lakebase
| Metric | Traditional Architecture | RetailIQ on Lakebase |
|---|---|---|
| Time to answer “Where is my order?” | 5–10 minutes (agent queue) | 1.5 seconds (Lakebase → BotBrain) |
| Data freshness in AI responses | Hours/days (batch ETL) | Real-time (direct Lakebase queries) |
| Order ID entry friction | Manual typing on mobile | Zero — tappable cards from Lakebase |
| Governance model | Per-tool, inconsistent | Unified via Unity Catalog |
| Audit trail for data access | None | Automatic, per-query via Unity Catalog |
| Infrastructure components | 5+ separate tools | 1 platform — Databricks + Lakebase |
The Takeaway: Data Architecture Is AI Architecture
Retail IQ works because it knows, not because it guesses well. That distinction traces directly to Lakebase and Unity Catalog: Lakebase eliminates the latency and data movement that have historically separated AI systems from live operational data, and Unity Catalog ensures access to that data is governed, audited, and traceable by default.
The lesson is the one we opened with. The intelligence of any AI is bounded by the freshness and trustworthiness of the data it can access. Invest in the data layer first.