> ## Documentation Index
> Fetch the complete documentation index at: https://docs.navtalk.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Base Integration

> Embed your knowledge base directly in the system prompt for accurate, context-aware responses

NavTalk's knowledge base integration allows you to embed your custom knowledge base directly in the `instructions` (system prompt). This approach ensures that the AI has immediate access to your domain-specific information without requiring external API calls or vector databases.

## How It Works

Instead of using RAG (Retrieval-Augmented Generation) with external vector databases, NavTalk integrates knowledge bases by embedding the content directly in the system prompt. This provides:

* **Immediate access** to all knowledge base content
* **No latency** from external API calls
* **Context-aware responses** based on your custom data
* **Simplified architecture** without vector databases

<Steps>
  <Step title="Prepare Your Knowledge Base Content">
    Organize your knowledge base content into a structured format that can be embedded in the system prompt:

    <CodeGroup>
      ```javascript JavaScript theme={null}
      // Example: Organizing knowledge base content
      const knowledgeBase = `
      # Product Documentation

      ## Features
      - Real-time audio processing with sub-500ms latency
      - Support for 60+ languages with automatic detection
      - WebSocket and WebRTC integration

      ## API Endpoints
      - Unified WebSocket: wss://transfer.navtalk.ai/wss/v2/realtime-chat

      ## Supported Voices
      alloy, ash, ballad, cedar, coral, echo, marin, sage, shimmer, verse

      # FAQs

      **Q: What audio format is required?**
      A: PCM16, 24kHz sample rate, base64 encoded.

      **Q: How do I handle interruptions?**
      A: The system automatically detects speech and handles interruptions using server-side VAD.

      # Company Policies

      - Support hours: Monday-Friday, 9 AM - 6 PM EST
      - Response time: Within 24 hours for technical support
      - Data retention: 30 days for conversation logs
      `;

      // Combine with your system instructions
      const systemPrompt = `You are a helpful assistant for NavTalk's Real-time Digital Human API.

      ${knowledgeBase}

      When answering user questions:
      1. Use the knowledge base information above to provide accurate answers
      2. If the question is about features or APIs, reference the specific documentation
      3. For technical questions, provide detailed information from the knowledge base
      4. If information is not in the knowledge base, say so clearly
      `;
      ```

      ```python Python theme={null}
      # Example: Organizing knowledge base content
      knowledge_base = """
      # Product Documentation

      ## Features
      - Real-time audio processing with sub-500ms latency
      - Support for 60+ languages with automatic detection
      - WebSocket and WebRTC integration

      ## API Endpoints
      - Unified WebSocket: wss://transfer.navtalk.ai/wss/v2/realtime-chat

      ## Supported Voices
      alloy, ash, ballad, cedar, coral, echo, marin, sage, shimmer, verse

      # FAQs

      **Q: What audio format is required?**
      A: PCM16, 24kHz sample rate, base64 encoded.

      **Q: How do I handle interruptions?**
      A: The system automatically detects speech and handles interruptions using server-side VAD.

      # Company Policies

      - Support hours: Monday-Friday, 9 AM - 6 PM EST
      - Response time: Within 24 hours for technical support
      - Data retention: 30 days for conversation logs
      """

      # Combine with your system instructions
      system_prompt = """You are a helpful assistant for NavTalk's Real-time Digital Human API.

      {}

      When answering user questions:
      1. Use the knowledge base information above to provide accurate answers
      2. If the question is about features or APIs, reference the specific documentation
      3. For technical questions, provide detailed information from the knowledge base
      4. If information is not in the knowledge base, say so clearly
      """.format(knowledge_base)
      ```
    </CodeGroup>

    <Note>
      **Best Practices for Knowledge Base Content**:

      * Use clear headings and structure (Markdown format works well)
      * Keep sections concise and scannable
      * Include FAQs for common questions
      * Add examples for technical concepts
      * Update regularly to reflect current information
    </Note>
  </Step>

  <Step title="Embed Knowledge Base in System Prompt">
    Set the `instructions` field in your session configuration to include your knowledge base:

    <CodeGroup>
      ```javascript JavaScript theme={null}
      // Load or define your knowledge base
      const knowledgeBase = `
      # Your Knowledge Base Content Here
      ...
      `;

      const systemPrompt = `You are a helpful assistant. Use the following knowledge base to answer questions accurately:

      ${knowledgeBase}

      When users ask questions, prioritize information from the knowledge base above.`;
      ```

      ```python Python theme={null}
      # Load or define your knowledge base
      knowledge_base = """
      # Your Knowledge Base Content Here
      ...
      """

      system_prompt = """You are a helpful assistant. Use the following knowledge base to answer questions accurately:

      {}

      When users ask questions, prioritize information from the knowledge base above.
      """.format(knowledge_base)
      ```
    </CodeGroup>

    <Note>
      You can load knowledge base content from files, databases, or APIs when initializing your session, then embed it in the system prompt.
    </Note>
  </Step>

  <Step title="Configure Session with Knowledge Base">
    Include the knowledge base-enhanced system prompt in your session configuration. Send configuration in the WebSocket `onopen` event handler:

    <CodeGroup>
      ```javascript JavaScript theme={null}
      async function sendSessionConfig() {
        // Load your knowledge base (from file, API, database, etc.)
        const knowledgeBase = await loadKnowledgeBase();
        
        // Build system prompt with knowledge base
        const systemPrompt = `You are a helpful assistant for our company.

      ${knowledgeBase}

      When answering questions, use the knowledge base information above. If the information is not available in the knowledge base, let the user know.`;

        const config = {
          voice: 'cedar',
          prompt: systemPrompt,  // Knowledge base embedded here
          tools: []  // Optional: Function calling tools (OpenAI models only)
        };

        socket.send(JSON.stringify({
          type: 'realtime.input_config',
          data: { content: JSON.stringify(config) }
        }));
      }

      // Call this function in socket.onopen
      socket.onopen = async () => {
        console.log('WebSocket connection established');
        await sendSessionConfig();
      };

      // Example: Load knowledge base from various sources
      async function loadKnowledgeBase() {
        // Option 1: Load from a file
        // const fs = require('fs');
        // return fs.readFileSync('knowledge-base.md', 'utf-8');
        
        // Option 2: Load from API
        // const response = await fetch('https://your-api.com/knowledge-base');
        // return await response.text();
        
        // Option 3: Load from database
        // return await database.getKnowledgeBase();
        
        // Option 4: Static content
        return `
      # Product Features
      - Feature 1: Description
      - Feature 2: Description

      # FAQs
      Q: Common question?
      A: Answer here
        `;
      }
      ```

      ```python Python theme={null}
      import json

      async def send_session_config(websocket):
          # Load your knowledge base (from file, API, database, etc.)
          knowledge_base = await load_knowledge_base()
          
          # Build system prompt with knowledge base
          system_prompt = f"""You are a helpful assistant for our company.

      {knowledge_base}

      When answering questions, use the knowledge base information above. If the information is not available in the knowledge base, let the user know."""

          config = {
              'voice': 'cedar',
              'prompt': system_prompt,  # Knowledge base embedded here
              'tools': []  # Optional: Function calling tools (OpenAI models only)
          }

          await websocket.send(json.dumps({
              'type': 'realtime.input_config',
              'data': { 'content': json.dumps(config) }
          }))

      # Call this function when WebSocket connection opens
      async def on_open(websocket):
          print('WebSocket connection established')
          await send_session_config(websocket)

      # Example: Load knowledge base from various sources
      async def load_knowledge_base():
          # Option 1: Load from a file
          # with open('knowledge-base.md', 'r', encoding='utf-8') as f:
          #     return f.read()
          
          # Option 2: Load from API
          # async with aiohttp.ClientSession() as session:
          #     async with session.get('https://your-api.com/knowledge-base') as response:
          #         return await response.text()
          
          # Option 3: Load from database
          # return await database.get_knowledge_base()
          
          # Option 4: Static content
          return """
      # Product Features
      - Feature 1: Description
      - Feature 2: Description

      # FAQs
      Q: Common question?
      A: Answer here
          """
      ```
    </CodeGroup>

    <Note>
      The `prompt` field supports up to **4096 tokens** (approximately 16,000 characters) by default. For larger knowledge bases, you may need to:

      * Summarize or prioritize the most important content
      * Use function calling to query external knowledge bases for specific information
      * Structure your knowledge base to fit within the token limit
    </Note>
  </Step>

  <Step title="Optional: Combine with Function Calling for Large Knowledge Bases">
    For knowledge bases that exceed the system prompt token limit, you can combine prompt-based knowledge with function calling to query additional information. **Note: Function calling is only supported for OpenAI models (gpt-realtime, gpt-realtime-mini).**

    <CodeGroup>
      ```javascript JavaScript theme={null}
      socket.onopen = () => {
        console.log('WebSocket connection established');
        
        // Core knowledge base in system prompt (most important/frequent info)
        const systemPrompt = `You are a helpful assistant. Core knowledge base:

      ${coreKnowledgeBase}

      For detailed queries beyond the core knowledge base, use the function calling tool to query additional resources.`;

        const config = {
          voice: 'cedar',
          prompt: systemPrompt,
          tools: [
            {
              type: 'function',
              name: 'function_call_judge',
              description: 'Query external knowledge base or API when user questions require detailed information not in the core knowledge base.',
              parameters: {
                type: 'object',
                properties: {
                  userInput: {
                    type: 'string',
                    description: 'The user\'s question that needs detailed information'
                  }
                },
                required: ['userInput']
              }
            }
          ]
        };
        
        socket.send(JSON.stringify({
          type: 'realtime.input_config',
          data: { content: JSON.stringify(config) }
        }));
      };
      ```

      ```python Python theme={null}
      import json

      def on_open(ws):
          # Core knowledge base in system prompt (most important/frequent info)
          system_prompt = f"""You are a helpful assistant. Core knowledge base:

      {core_knowledge_base}

      For detailed queries beyond the core knowledge base, use the function calling tool to query additional resources."""

          config = {
              'voice': 'cedar',
              'prompt': system_prompt,
              'tools': [
                  {
                      'type': 'function',
                      'name': 'function_call_judge',
                      'description': 'Query external knowledge base or API when user questions require detailed information not in the core knowledge base.',
                      'parameters': {
                          'type': 'object',
                          'properties': {
                              'userInput': {
                                  'type': 'string',
                                  'description': 'The user\'s question that needs detailed information'
                              }
                          },
                          'required': ['userInput']
                      }
                  }
              ]
          }
          
          ws.send(json.dumps({
              'type': 'realtime.input_config',
              'data': { 'content': json.dumps(config) }
          }))
      ```
    </CodeGroup>

    <Note>
      **Hybrid Approach**: Put your most frequently accessed knowledge in the system prompt for instant access, and use function calling to query detailed documentation or databases when needed.
    </Note>
  </Step>
</Steps>

## Knowledge Base Content Organization

<AccordionGroup>
  <Accordion title="Structured Format">
    Organize your knowledge base with clear structure:

    ```markdown theme={null}
    # Product Documentation
    ## Features
    - Feature 1
    - Feature 2

    ## API Reference
    ### Endpoints
    - Endpoint 1
    - Endpoint 2

    # FAQs
    **Q:** Question 1?
    **A:** Answer 1

    **Q:** Question 2?
    **A:** Answer 2
    ```
  </Accordion>

  <Accordion title="Prioritize Frequently Asked Content">
    Place the most commonly asked information at the top of your knowledge base. This ensures it's easily accessible within the token limit.
  </Accordion>

  <Accordion title="Use Clear Examples">
    Include concrete examples in your knowledge base to help the AI provide accurate, detailed responses:

    ```markdown theme={null}
    ## Code Examples

    ### JavaScript Connection
    \`\`\`javascript
    const socket = new WebSocket('wss://transfer.navtalk.ai/wss/v2/realtime-chat?license=YOUR_KEY&name=CHARACTER_NAME');
    \`\`\`

    ### Python Connection
    \`\`\`python
    import websockets
    websocket = await websockets.connect('wss://transfer.navtalk.ai/wss/v2/realtime-chat?license=YOUR_KEY&name=CHARACTER_NAME')
    \`\`\`
    ```
  </Accordion>

  <Accordion title="Keep Content Concise">
    Due to token limits, keep each section concise and focused. Remove redundant information and prioritize essential facts.
  </Accordion>
</AccordionGroup>

<Warning>
  **Token Limits**: The system prompt has token limits (default \~4096 tokens). For very large knowledge bases, consider:

  1. Summarizing and prioritizing content
  2. Using a hybrid approach with function calling
  3. Rotating knowledge base content based on conversation context
</Warning>