Beyond Basic RAG: A Technical Deep Dive using DocsQL

In our previous post, we explored why traditional RAG systems fail and introduced the concept of intent-driven query architecture. Today, we're diving into the technical implementation details of how we built this system using PromptQL for our own documentation.

The Challenge: Transform our DocsQL system from a traditional RAG approach that treated every query identically into an intelligent router that applies the right processing strategy based on user intent.

The Result: 70% reduction in average response time, 75% fewer unnecessary vector operations, and near-zero hallucination for CLI commands and configuration examples.

The Classification Framework

The heart of our system is fast intent classification that routes queries before they hit expensive operations. Here's how we implemented it:

<!-- globals/metadata/promptql-config.hml -->
<query_classification>
Before responding, classify the question:
- General Information: Conceptual questions, "what is", "how does", "why"
→ Answer directly, offer follow-up
- Guide Request: "how do I", "show me how", "walk me through"
→ Requires CLI validation
- Example Request: "show me an example", "what does X look like"
→ Requires metadata validation
- Troubleshooting: Error messages, "not working"
→ May require validation depending on solution
</query_classification>

This classification step is lightweight but critical: it determines the entire response pathway and eliminates expensive operations when they don't add value.

Implementation Pattern: General Information

For the 60% of queries that are conceptual questions, we bypass retrieval entirely:

<!-- globals/metadata/promptql-config.hml -->
<general_information_pattern>
For conceptual questions about PromptQL:
1. Answer directly using natural language knowledge
2. Keep response concise (1-3 sentences + key points if needed)
3. ALWAYS end with: "Would you like me to show you how to set this up,
or would you prefer to see a specific example?"
4. NO data queries unless user requests guide/example in follow-up
</general_information_pattern>

The Performance Win: Simple questions like "What is PromptQL?" now respond in 0.5 seconds instead of 13-45 seconds, while maintaining the same accuracy and offering natural progression to more detailed help.

Implementation Pattern: CLI Validation

The most critical implementation challenge was preventing hallucinated CLI commands. Here's our exact validation protocol:

<!-- globals/metadata/promptql-config.hml -->
<cli_validation_process description="Use only when providing CLI commands">
Before providing any CLI command information, validate the command exists:
1. Check command existence: Query app.pql_docs_doc_content for pages with URLs:
https://promptql.io/docs/reference/cli/commands/ddn_[command]_[subcommand]/
- Commands use underscores in URLs (e.g., ddn_connector_init)
- Commands use spaces in actual CLI usage (e.g., ddn connector init)
- ALWAYS include trailing slash
2. Extract exact usage and flags from documentation page:
- Exact command syntax
- Available flags and descriptions
- Required vs optional parameters
3. Include placeholders or example values:
- ddn connector init <my_connector> -i (placeholders)
- ddn connector init my_connector -i (example values)
4. Never invent CLI commands or flags - if command doesn't exist,
tell user it doesn't exist
Example validation query (always include trailing slash):
SELECT page_url, title, content
FROM app.pql_docs_doc_content
WHERE page_url = 'https://promptql.io/docs/reference/cli/commands/ddn_connector_init/'
</cli_validation_process>

The Accuracy Win: This targeted validation nearly completely eliminated CLI command hallucination. Commands offered by the assistant are verified against our actual documentation before the response is generated.

Metadata Validation for Examples

When users request configuration examples, we implemented specific validation against our documentation metadata:

<!-- globals/metadata/promptql-config.hml -->
<metadata_validation_process description="MANDATORY for configuration examples>
For configuration examples (YAML, JSON, etc.):
1. Query documentation for actual example files
2. Verify syntax against schema documentation
3. Ensure version compatibility across components
4. Validate that all referenced elements actually exist
5. Provide working examples, not plausible-looking syntax
</metadata_validation_process>

The Reliability Win: The combination of intent-driven classification and targeted validation created a dramatic shift in our system's reliability profile. Before implementing this architecture, our assistant had a significant hallucination problem, particularly with CLI commands and configuration syntax. Users would receive confident-sounding but completely invalid commands, leading to frustration and lost trust.

The PromptQL Query Pipeline

Our implementation leverages PromptQL's structured data capabilities to create an intelligent routing system capable of querying over unstructured data, when needed:

-- Pseudocode representation of our routing logic
IF intent = 'general_info' THEN
RETURN model_knowledge()           -- Skip expensive retrieval
ELSIF intent = 'cli_guide' THEN
VALIDATE exact_commands()          -- Ensure accuracy
ELSIF intent = 'example_request' THEN
VALIDATE syntax_examples()         -- Prevent hallucination
ELSE
EXECUTE full_rag_pipeline()        -- Complex troubleshooting
END IF

This approach mirrors database query optimization, where different execution paths are chosen based on query complexity and data access patterns.

Resource Efficiency at Scale

The performance improvements compound as query volume grows:

Before (Traditional RAG):

• 100% of queries → Full embedding + vector search pipeline
• 13-45 seconds per query, regardless of complexity
• High computational costs for unnecessary operations
• Flat performance profile frustrating users

After (Intent-Driven):

• 60% of queries → Direct model response (0.5 seconds)
• 40% of queries → Targeted validation (10-20 seconds)
• 75% reduction in vector operations
• Natural performance profile matching user expectations

Accuracy Improvements

The targeted validation protocols transformed reliability:

• Near-zero hallucination for CLI commands through exact documentation verification
• Validated configuration examples ensuring working syntax instead of plausible-looking failures
• Maintained speed for conceptual questions while adding precision where it matters

This implementation demonstrates one of PromptQL's core strengths: bringing structured data intelligence to unstructured problems. Just as PromptQL excels at complex database queries, the same principles apply to optimizing LLM workflows.

The intent-driven architecture shows how structured thinking about query patterns, validation protocols, and response optimization can transform AI application performance. Traditional RAG treats every query as an unstructured retrieval problem. PromptQL's approach applies the same optimization principles that make databases efficient to make LLM systems fast, reliable, and accurate.

Ready to implement intent-driven systems? This technical foundation provides the blueprint for moving beyond basic RAG in any application: whether you’re powering search, decision-making, automation, or building intelligence into your own platforms. It’s about structuring knowledge and interactions so your systems don’t just respond, they reason. Reach out to get started!