Knowledge Base Integration

Embed your knowledge base directly in the system prompt for accurate, context-aware responses

NavTalk’s knowledge base integration allows you to embed your custom knowledge base directly in the instructions (system prompt). This approach ensures that the AI has immediate access to your domain-specific information without requiring external API calls or vector databases.

How It Works

Instead of using RAG (Retrieval-Augmented Generation) with external vector databases, NavTalk integrates knowledge bases by embedding the content directly in the system prompt. This provides:

• Immediate access to all knowledge base content
• No latency from external API calls
• Context-aware responses based on your custom data
• Simplified architecture without vector databases

1. Prepare Your Knowledge Base Content

   Organize your knowledge base content into a structured format that can be embedded in the system prompt:

   JavaScript

   // Example: Organizing knowledge base content
   const knowledgeBase = `
   # Product Documentation
   
   ## Features
   - Real-time audio processing with sub-500ms latency
   - Support for 60+ languages with automatic detection
   - WebSocket and WebRTC integration
   
   ## API Endpoints
   - Unified WebSocket: wss://transfer.navtalk.ai/wss/v2/realtime-chat
   
   ## Supported Voices
   alloy, ash, ballad, cedar, coral, echo, marin, sage, shimmer, verse
   
   # FAQs
   
   **Q: What audio format is required?**
   A: PCM16, 24kHz sample rate, base64 encoded.
   
   **Q: How do I handle interruptions?**
   A: The system automatically detects speech and handles interruptions using server-side VAD.
   
   # Company Policies
   
   - Support hours: Monday-Friday, 9 AM - 6 PM EST
   - Response time: Within 24 hours for technical support
   - Data retention: 30 days for conversation logs
   `;
   
   // Combine with your system instructions
   const systemPrompt = `You are a helpful assistant for NavTalk's Real-time Digital Human API.
   
   ${knowledgeBase}
   
   When answering user questions:
   1. Use the knowledge base information above to provide accurate answers
   2. If the question is about features or APIs, reference the specific documentation
   3. For technical questions, provide detailed information from the knowledge base
   4. If information is not in the knowledge base, say so clearly
   `;
   

   # Example: Organizing knowledge base content
   knowledge_base = """
   # Product Documentation
   
   ## Features
   - Real-time audio processing with sub-500ms latency
   - Support for 60+ languages with automatic detection
   - WebSocket and WebRTC integration
   
   ## API Endpoints
   - Unified WebSocket: wss://transfer.navtalk.ai/wss/v2/realtime-chat
   
   ## Supported Voices
   alloy, ash, ballad, cedar, coral, echo, marin, sage, shimmer, verse
   
   # FAQs
   
   **Q: What audio format is required?**
   A: PCM16, 24kHz sample rate, base64 encoded.
   
   **Q: How do I handle interruptions?**
   A: The system automatically detects speech and handles interruptions using server-side VAD.
   
   # Company Policies
   
   - Support hours: Monday-Friday, 9 AM - 6 PM EST
   - Response time: Within 24 hours for technical support
   - Data retention: 30 days for conversation logs
   """
   
   # Combine with your system instructions
   system_prompt = """You are a helpful assistant for NavTalk's Real-time Digital Human API.
   
   {}
   
   When answering user questions:
   1. Use the knowledge base information above to provide accurate answers
   2. If the question is about features or APIs, reference the specific documentation
   3. For technical questions, provide detailed information from the knowledge base
   4. If information is not in the knowledge base, say so clearly
   """.format(knowledge_base)
   

   Best Practices for Knowledge Base Content:

   • Use clear headings and structure (Markdown format works well)
   • Keep sections concise and scannable
   • Include FAQs for common questions
   • Add examples for technical concepts
   • Update regularly to reflect current information
2. Embed Knowledge Base in System Prompt

   Set the instructions field in your session configuration to include your knowledge base:

   JavaScript

   // Load or define your knowledge base
   const knowledgeBase = `
   # Your Knowledge Base Content Here
   ...
   `;
   
   const systemPrompt = `You are a helpful assistant. Use the following knowledge base to answer questions accurately:
   
   ${knowledgeBase}
   
   When users ask questions, prioritize information from the knowledge base above.`;
   

   # Load or define your knowledge base
   knowledge_base = """
   # Your Knowledge Base Content Here
   ...
   """
   
   system_prompt = """You are a helpful assistant. Use the following knowledge base to answer questions accurately:
   
   {}
   
   When users ask questions, prioritize information from the knowledge base above.
   """.format(knowledge_base)
   

   You can load knowledge base content from files, databases, or APIs when initializing your session, then embed it in the system prompt.

3. Configure Session with Knowledge Base

   Include the knowledge base-enhanced system prompt in your session configuration. Send configuration in the WebSocket onopen event handler:

   JavaScript

   async function sendSessionConfig() {
     // Load your knowledge base (from file, API, database, etc.)
     const knowledgeBase = await loadKnowledgeBase();
     
     // Build system prompt with knowledge base
     const systemPrompt = `You are a helpful assistant for our company.
   
   ${knowledgeBase}
   
   When answering questions, use the knowledge base information above. If the information is not available in the knowledge base, let the user know.`;
   
     const config = {
       voice: 'cedar',
       prompt: systemPrompt,  // Knowledge base embedded here
       tools: []  // Optional: Function calling tools (OpenAI models only)
     };
   
     socket.send(JSON.stringify({
       type: 'realtime.input_config',
       data: { content: JSON.stringify(config) }
     }));
   }
   
   // Call this function in socket.onopen
   socket.onopen = async () => {
     console.log('WebSocket connection established');
     await sendSessionConfig();
   };
   
   // Example: Load knowledge base from various sources
   async function loadKnowledgeBase() {
     // Option 1: Load from a file
     // const fs = require('fs');
     // return fs.readFileSync('knowledge-base.md', 'utf-8');
     
     // Option 2: Load from API
     // const response = await fetch('https://your-api.com/knowledge-base');
     // return await response.text();
     
     // Option 3: Load from database
     // return await database.getKnowledgeBase();
     
     // Option 4: Static content
     return `
   # Product Features
   - Feature 1: Description
   - Feature 2: Description
   
   # FAQs
   Q: Common question?
   A: Answer here
     `;
   }
   

   import json
   
   async def send_session_config(websocket):
       # Load your knowledge base (from file, API, database, etc.)
       knowledge_base = await load_knowledge_base()
       
       # Build system prompt with knowledge base
       system_prompt = f"""You are a helpful assistant for our company.
   
   {knowledge_base}
   
   When answering questions, use the knowledge base information above. If the information is not available in the knowledge base, let the user know."""
   
       config = {
           'voice': 'cedar',
           'prompt': system_prompt,  # Knowledge base embedded here
           'tools': []  # Optional: Function calling tools (OpenAI models only)
       }
   
       await websocket.send(json.dumps({
           'type': 'realtime.input_config',
           'data': { 'content': json.dumps(config) }
       }))
   
   # Call this function when WebSocket connection opens
   async def on_open(websocket):
       print('WebSocket connection established')
       await send_session_config(websocket)
   
   # Example: Load knowledge base from various sources
   async def load_knowledge_base():
       # Option 1: Load from a file
       # with open('knowledge-base.md', 'r', encoding='utf-8') as f:
       #     return f.read()
       
       # Option 2: Load from API
       # async with aiohttp.ClientSession() as session:
       #     async with session.get('https://your-api.com/knowledge-base') as response:
       #         return await response.text()
       
       # Option 3: Load from database
       # return await database.get_knowledge_base()
       
       # Option 4: Static content
       return """
   # Product Features
   - Feature 1: Description
   - Feature 2: Description
   
   # FAQs
   Q: Common question?
   A: Answer here
       """
   

   The prompt field supports up to 4096 tokens (approximately 16,000 characters) by default. For larger knowledge bases, you may need to:

   • Summarize or prioritize the most important content
   • Use function calling to query external knowledge bases for specific information
   • Structure your knowledge base to fit within the token limit
4. Optional: Combine with Function Calling for Large Knowledge Bases

   For knowledge bases that exceed the system prompt token limit, you can combine prompt-based knowledge with function calling to query additional information. Note: Function calling is only supported for OpenAI models (gpt-realtime, gpt-realtime-mini).

   JavaScript

   socket.onopen = () => {
     console.log('WebSocket connection established');
     
     // Core knowledge base in system prompt (most important/frequent info)
     const systemPrompt = `You are a helpful assistant. Core knowledge base:
   
   ${coreKnowledgeBase}
   
   For detailed queries beyond the core knowledge base, use the function calling tool to query additional resources.`;
   
     const config = {
       voice: 'cedar',
       prompt: systemPrompt,
       tools: [
         {
           type: 'function',
           name: 'function_call_judge',
           description: 'Query external knowledge base or API when user questions require detailed information not in the core knowledge base.',
           parameters: {
             type: 'object',
             properties: {
               userInput: {
                 type: 'string',
                 description: 'The user\'s question that needs detailed information'
               }
             },
             required: ['userInput']
           }
         }
       ]
     };
     
     socket.send(JSON.stringify({
       type: 'realtime.input_config',
       data: { content: JSON.stringify(config) }
     }));
   };
   

   import json
   
   def on_open(ws):
       # Core knowledge base in system prompt (most important/frequent info)
       system_prompt = f"""You are a helpful assistant. Core knowledge base:
   
   {core_knowledge_base}
   
   For detailed queries beyond the core knowledge base, use the function calling tool to query additional resources."""
   
       config = {
           'voice': 'cedar',
           'prompt': system_prompt,
           'tools': [
               {
                   'type': 'function',
                   'name': 'function_call_judge',
                   'description': 'Query external knowledge base or API when user questions require detailed information not in the core knowledge base.',
                   'parameters': {
                       'type': 'object',
                       'properties': {
                           'userInput': {
                               'type': 'string',
                               'description': 'The user\'s question that needs detailed information'
                           }
                       },
                       'required': ['userInput']
                   }
               }
           ]
       }
       
       ws.send(json.dumps({
           'type': 'realtime.input_config',
           'data': { 'content': json.dumps(config) }
       }))
   

   Hybrid Approach: Put your most frequently accessed knowledge in the system prompt for instant access, and use function calling to query detailed documentation or databases when needed.

Knowledge Base Content Organization

Structured Format

Organize your knowledge base with clear structure:

# Product Documentation
## Features
- Feature 1
- Feature 2

## API Reference
### Endpoints
- Endpoint 1
- Endpoint 2

# FAQs
**Q:** Question 1?
**A:** Answer 1

**Q:** Question 2?
**A:** Answer 2

Prioritize Frequently Asked Content

Place the most commonly asked information at the top of your knowledge base. This ensures it’s easily accessible within the token limit.

Use Clear Examples

Include concrete examples in your knowledge base to help the AI provide accurate, detailed responses:

## Code Examples

### JavaScript Connection
\`\`\`javascript
const socket = new WebSocket('wss://transfer.navtalk.ai/wss/v2/realtime-chat?license=YOUR_KEY&name=CHARACTER_NAME');
\`\`\`

### Python Connection
\`\`\`python
import websockets
websocket = await websockets.connect('wss://transfer.navtalk.ai/wss/v2/realtime-chat?license=YOUR_KEY&name=CHARACTER_NAME')
\`\`\`

Keep Content Concise

Due to token limits, keep each section concise and focused. Remove redundant information and prioritize essential facts.

Token Limits: The system prompt has token limits (default ~4096 tokens). For very large knowledge bases, consider:

1. Summarizing and prioritizing content
2. Using a hybrid approach with function calling
3. Rotating knowledge base content based on conversation context