Feature: Implement Web Search Tool for LLM Assistant (NF.SM3) #115

New issue

Closed

opened 2025-10-28 18:32:26 +00:00 by rwachters · 0 comments

rwachters commented

2025-10-28 18:32:26 +00:00

(Migrated from github.com)

Feature: Implement Web Search Tool for LLM Assistant (NF.SM3)

Labels: feature, backend, frontend, LLM, tooling, enhancement

Executive Summary

This issue outlines the implementation of a web search tool for LLM assistants that support tool calling. The feature will allow users to enable/disable a web search tool for specific prompts, using DuckDuckGo as the default search engine, and visualize its usage and output in the UI. This is a significant feature requiring changes across data, domain, backend, API, and frontend layers.

User Story Requirements (NF.SM3)

A web search tool can be turned on/off for a specific prompt.
The tool is only available for LLMs that support tool calling.
The tool uses DuckDuckGo to perform the search.
The tool returns the search results to the assistant.
The tool can be configured to use a different search engine.
When the tool is called, the UI displays a notification that the tool is being used.
When an assistant message is generated, it includes a list of tools that were used to generate it.
The user can select an item in the list to view the tool's output.

Architecture Overview

The implementation requires changes across multiple layers:

Data Layer: New database tables for tool definitions, tool call records, and session-specific tool configurations.
Domain Models: New common models for tools and tool calls.
Backend Services: Tool management service, tool execution service, and enhanced message service to orchestrate tool calling.
LLM Integration: Enhanced chat strategies for OpenAI and Ollama to support tool calling.
API Layer: New endpoints for tool management and session-specific tool configuration.
Frontend: UI for tool configuration, tool call visualization, and notifications.

Implementation Plan (Phases)

Phase 1: Foundation (Backend Data & Services)

Create database tables (ToolDefinitionTable, ToolCallTable, SessionToolConfigTable) and update ExposedDataManager (server main).
Create DAO interfaces and Exposed implementations for ToolDefinition (eu.torvian.chatbot.common.models.tool.ToolDefinition), ToolCall (eu.torvian.chatbot.common.models.tool.ToolCall), and SessionToolConfig (server/src/main/kotlin/eu/torvian/chatbot/server/data/tables/SessionToolConfigTable.kt).
Create domain models: ToolDefinition (eu.torvian.chatbot.common.models.tool.ToolDefinition), ToolType (eu.torvian.chatbot.common.models.tool.ToolType), ToolCall (eu.torvian.chatbot.common.models.tool.ToolCall), ToolCallStatus (eu.torvian.chatbot.common.models.tool.ToolCallStatus), and RawChatMessage (eu.torvian.chatbot.common.models.llm.RawChatMessage).
Create ToolService (eu.torvian.chatbot.server.service.core.ToolService) and its implementation.
Create ToolExecutor interface (eu.torvian.chatbot.server.service.tool.ToolExecutor), WebSearchToolExecutor implementation (server/src/main/kotlin/eu/torvian/chatbot/server/service/tool/impl/WebSearchToolExecutor.kt), and ToolExecutorFactory (eu.torvian.chatbot.server.service.tool.ToolExecutorFactory).
Add unit tests for DAOs and services.

Phase 2: LLM Integration

Enhance LLM API models (OpenAiApiModels & OllamaApiModels) to support tool calling.
Update LLMCompletionResult and LLMStreamChunk to include tool calls.
Update ChatCompletionStrategy interface and implementations (OpenAIChatStrategy, OllamaChatStrategy) to handle tools.
Modify MessageServiceImpl to implement the tool calling loop, context building with RawChatMessage, and tool execution.
Update LLMApiClient and LLMApiClientKtor to pass tools.
Add integration tests for LLM strategies and message service.

Phase 3: API Layer

Create tool management routes in configureToolRoutes.kt (server/src/main/kotlin/eu/torvian/chatbot/server/ktor/routes/configureToolRoutes.kt).
Create API request/response models (ToolResource, CreateToolRequest, UpdateToolRequest, SetToolEnabledRequest) under common module.
Add configureToolRoutes to ApiRoutesKtor.
Add route tests.

Phase 4: Frontend Foundation

Create ToolApi (app/src/commonMain/kotlin/eu/torvian/chatbot/app/service/api/ToolApi.kt) and its implementation.
Create ToolRepository (app/src/commonMain/kotlin/eu/torvian/chatbot/app/repository/ToolRepository.kt) and its implementation.
Update ChatViewModel and ChatState to manage tool data.
Add repository tests.

Phase 5: UI Components

Create ToolConfigPanel (app/src/commonMain/kotlin/eu/torvian/chatbot/app/compose/chatarea/ToolConfigPanel.kt) for session-specific tool configuration.
Create ToolCallDisplay (app/src/commonMain/kotlin/eu/torvian/chatbot/app/compose/chatarea/ToolCallDisplay.kt) to show tool calls within messages.
Create ToolNotification (app/src/commonMain/kotlin/eu/torvian/chatbot/app/compose/chatarea/ToolNotification.kt) for tool execution status.
Update MessageItem to integrate ToolCallDisplay.
Update MessageStreamEvent with new tool-related events.

Phase 6: Polish & Testing

Conduct end-to-end testing.
Improve error handling and performance.
Update documentation.

Key Considerations

Tool Calling Loop: MessageServiceImpl will implement a loop to handle multiple tool calls in a single turn, creating/updating a single assistant message.
Context Building: The buildContext method will convert ChatMessage history into RawChatMessage objects, including tool calls and their results, for LLM API communication. Tool results are not stored as separate chat messages but are reconstructed for context.
Streaming Tool Calls: Support for accumulating and processing tool call deltas from streaming LLM responses (OpenAI) and complete tool calls (Ollama).
Model Capability Check: Utilize LLMModel.hasCapability(LLMModelCapabilities.TOOL_CALLING) to ensure tool usage is restricted to capable models.
Provider Differences: Account for variations between OpenAI (tool call IDs, string arguments) and Ollama (no tool call IDs, JSON object arguments).
Error Handling: Graceful handling of tool execution failures, reporting errors back to the LLM and UI.
Initial Data: Initialize a default web search tool in DatabaseInitializer (server.main.DatabaseInitializer).

Estimated Effort

Approximately 17-23 days (3.5-4.5 weeks).

Reference

This issue is based on the detailed implementation report: docs/plans/web search tool/report.md

Notes / Suggested next steps

Add the issue to the appropriate project board and milestone.
Assign to the engineer(s) who will own backend/core LLM integration first (Phase 1 & 2).
Create smaller follow-up issues per phase (data schema, LLM integration, API routes, frontend) to track progress.

# Feature: Implement Web Search Tool for LLM Assistant (NF.SM3) **Labels**: `feature`, `backend`, `frontend`, `LLM`, `tooling`, `enhancement` ## Executive Summary This issue outlines the implementation of a web search tool for LLM assistants that support tool calling. The feature will allow users to enable/disable a web search tool for specific prompts, using DuckDuckGo as the default search engine, and visualize its usage and output in the UI. This is a significant feature requiring changes across data, domain, backend, API, and frontend layers. ## User Story Requirements (NF.SM3) - [ ] A web search tool can be turned on/off for a specific prompt. - [ ] The tool is only available for LLMs that support tool calling. - [ ] The tool uses DuckDuckGo to perform the search. - [ ] The tool returns the search results to the assistant. - [ ] The tool can be configured to use a different search engine. - [ ] When the tool is called, the UI displays a notification that the tool is being used. - [ ] When an assistant message is generated, it includes a list of tools that were used to generate it. - [ ] The user can select an item in the list to view the tool's output. ## Architecture Overview The implementation requires changes across multiple layers: 1. **Data Layer**: New database tables for tool definitions, tool call records, and session-specific tool configurations. 2. **Domain Models**: New common models for tools and tool calls. 3. **Backend Services**: Tool management service, tool execution service, and enhanced message service to orchestrate tool calling. 4. **LLM Integration**: Enhanced chat strategies for OpenAI and Ollama to support tool calling. 5. **API Layer**: New endpoints for tool management and session-specific tool configuration. 6. **Frontend**: UI for tool configuration, tool call visualization, and notifications. ## Implementation Plan (Phases) ### Phase 1: Foundation (Backend Data & Services) - Create database tables (`ToolDefinitionTable`, `ToolCallTable`, `SessionToolConfigTable`) and update `ExposedDataManager` (server main). - Create DAO interfaces and Exposed implementations for `ToolDefinition` (`eu.torvian.chatbot.common.models.tool.ToolDefinition`), `ToolCall` (`eu.torvian.chatbot.common.models.tool.ToolCall`), and `SessionToolConfig` (`server/src/main/kotlin/eu/torvian/chatbot/server/data/tables/SessionToolConfigTable.kt`). - Create domain models: `ToolDefinition` (`eu.torvian.chatbot.common.models.tool.ToolDefinition`), `ToolType` (`eu.torvian.chatbot.common.models.tool.ToolType`), `ToolCall` (`eu.torvian.chatbot.common.models.tool.ToolCall`), `ToolCallStatus` (`eu.torvian.chatbot.common.models.tool.ToolCallStatus`), and `RawChatMessage` (`eu.torvian.chatbot.common.models.llm.RawChatMessage`). - Create `ToolService` (`eu.torvian.chatbot.server.service.core.ToolService`) and its implementation. - Create `ToolExecutor` interface (`eu.torvian.chatbot.server.service.tool.ToolExecutor`), `WebSearchToolExecutor` implementation (`server/src/main/kotlin/eu/torvian/chatbot/server/service/tool/impl/WebSearchToolExecutor.kt`), and `ToolExecutorFactory` (`eu.torvian.chatbot.server.service.tool.ToolExecutorFactory`). - Add unit tests for DAOs and services. ### Phase 2: LLM Integration - Enhance LLM API models (OpenAiApiModels & OllamaApiModels) to support tool calling. - Update `LLMCompletionResult` and `LLMStreamChunk` to include tool calls. - Update `ChatCompletionStrategy` interface and implementations (`OpenAIChatStrategy`, `OllamaChatStrategy`) to handle tools. - Modify `MessageServiceImpl` to implement the tool calling loop, context building with `RawChatMessage`, and tool execution. - Update `LLMApiClient` and `LLMApiClientKtor` to pass tools. - Add integration tests for LLM strategies and message service. ### Phase 3: API Layer - Create tool management routes in `configureToolRoutes.kt` (`server/src/main/kotlin/eu/torvian/chatbot/server/ktor/routes/configureToolRoutes.kt`). - Create API request/response models (`ToolResource`, `CreateToolRequest`, `UpdateToolRequest`, `SetToolEnabledRequest`) under `common` module. - Add `configureToolRoutes` to `ApiRoutesKtor`. - Add route tests. ### Phase 4: Frontend Foundation - Create `ToolApi` (`app/src/commonMain/kotlin/eu/torvian/chatbot/app/service/api/ToolApi.kt`) and its implementation. - Create `ToolRepository` (`app/src/commonMain/kotlin/eu/torvian/chatbot/app/repository/ToolRepository.kt`) and its implementation. - Update `ChatViewModel` and `ChatState` to manage tool data. - Add repository tests. ### Phase 5: UI Components - Create `ToolConfigPanel` (`app/src/commonMain/kotlin/eu/torvian/chatbot/app/compose/chatarea/ToolConfigPanel.kt`) for session-specific tool configuration. - Create `ToolCallDisplay` (`app/src/commonMain/kotlin/eu/torvian/chatbot/app/compose/chatarea/ToolCallDisplay.kt`) to show tool calls within messages. - Create `ToolNotification` (`app/src/commonMain/kotlin/eu/torvian/chatbot/app/compose/chatarea/ToolNotification.kt`) for tool execution status. - Update `MessageItem` to integrate `ToolCallDisplay`. - Update `MessageStreamEvent` with new tool-related events. ### Phase 6: Polish & Testing - Conduct end-to-end testing. - Improve error handling and performance. - Update documentation. ## Key Considerations - **Tool Calling Loop**: `MessageServiceImpl` will implement a loop to handle multiple tool calls in a single turn, creating/updating a single assistant message. - **Context Building**: The `buildContext` method will convert `ChatMessage` history into `RawChatMessage` objects, including tool calls and their results, for LLM API communication. Tool results are not stored as separate chat messages but are reconstructed for context. - **Streaming Tool Calls**: Support for accumulating and processing tool call deltas from streaming LLM responses (OpenAI) and complete tool calls (Ollama). - **Model Capability Check**: Utilize `LLMModel.hasCapability(LLMModelCapabilities.TOOL_CALLING)` to ensure tool usage is restricted to capable models. - **Provider Differences**: Account for variations between OpenAI (tool call IDs, string arguments) and Ollama (no tool call IDs, JSON object arguments). - **Error Handling**: Graceful handling of tool execution failures, reporting errors back to the LLM and UI. - **Initial Data**: Initialize a default web search tool in `DatabaseInitializer` (`server.main.DatabaseInitializer`). ## Estimated Effort Approximately 17-23 days (3.5-4.5 weeks). ## Reference This issue is based on the detailed implementation report: `docs/plans/web search tool/report.md` --- ### Notes / Suggested next steps - Add the issue to the appropriate project board and milestone. - Assign to the engineer(s) who will own backend/core LLM integration first (Phase 1 & 2). - Create smaller follow-up issues per phase (data schema, LLM integration, API routes, frontend) to track progress.