Sub-issue: Implement Web Search Tool - Backend (NF.SM3.B) #117

New issue

Closed

opened 2025-10-28 18:36:56 +00:00 by rwachters · 0 comments

rwachters commented

2025-10-28 18:36:56 +00:00

(Migrated from github.com)

Sub-issue: Implement Web Search Tool - Backend (NF.SM3.B)

Labels: backend, feature, LLM, tooling, sub-issue

Parent Issue: https://github.com/rwachters/chatbot/issues/115

Executive Summary

This sub-issue covers all backend development required for the web search tool feature. This includes setting up the data layer, implementing core tool services, integrating tool calling with LLM strategies, and exposing necessary APIs for tool management and session-specific configurations.

User Story Requirements (Backend Focus)

The tool is only available for LLMs that support tool calling.
The tool uses DuckDuckGo to perform the search.
The tool returns the search results to the assistant.
The tool can be configured to use a different search engine (backend configuration).
When the tool is called, the UI displays a notification that the tool is being used (backend sends event).
When an assistant message is generated, it includes a list of tools that were used to generate it (backend provides data).
The user can select an item in the list to view the tool's output (backend provides data).

Implementation Plan

Phase 1: Foundation (Backend Data & Services)

Create database tables (ToolDefinitionTable, ToolCallTable, SessionToolConfigTable) and update ExposedDataManager (server main).
Create DAO interfaces and Exposed implementations for ToolDefinition, ToolCall, and SessionToolConfig.
Create domain models: ToolDefinition, ToolType, ToolCall, ToolCallStatus, and RawChatMessage.
Create ToolService and its implementation.
Create ToolExecutor interface, WebSearchToolExecutor, and ToolExecutorFactory.
Add unit tests for DAOs and services.

Phase 2: LLM Integration

Enhance LLM API models (OpenAiApiModels & OllamaApiModels) to support tool calling.
Update LLMCompletionResult and LLMStreamChunk to include tool calls.
Update ChatCompletionStrategy interface and implementations (OpenAIChatStrategy, OllamaChatStrategy) to handle tools.
Modify MessageServiceImpl to implement the tool calling loop, context building with RawChatMessage, and tool execution.
Update LLMApiClient and LLMApiClientKtor to pass tools.
Add integration tests for LLM strategies and message service.

Phase 3: API Layer

Create tool management routes in configureToolRoutes.kt.
Create API request/response models (ToolResource, CreateToolRequest, UpdateToolRequest, SetToolEnabledRequest).
Add configureToolRoutes to ApiRoutesKtor.
Add route tests.

Key Considerations (Backend)

Tool Calling Loop: The MessageServiceImpl will implement a loop to handle multiple tool calls in a single turn, creating/updating a single assistant message.
Context Building: The buildContext method will convert ChatMessage history into RawChatMessage objects, including tool calls and their results, for LLM API communication. Tool results are not stored as separate chat messages but are reconstructed for context.
Streaming Tool Calls: Support for accumulating and processing tool call deltas from streaming LLM responses (OpenAI) and complete tool calls (Ollama).
Model Capability Check: Utilize LLMModel.hasCapability(LLMModelCapabilities.TOOL_CALLING) to ensure tool usage is restricted to capable models.
Provider Differences: Account for variations between OpenAI (tool call IDs, string arguments) and Ollama (no tool call IDs, JSON object arguments).
Error Handling: Graceful handling of tool execution failures, reporting errors back to the LLM.
Initial Data: Initialize a default web search tool in DatabaseInitializer.

Estimated Effort

Approximately 10-14 days.

Reference

This issue is based on the detailed implementation report: docs/plans/web search tool/report.md

# Sub-issue: Implement Web Search Tool - Backend (NF.SM3.B) **Labels**: `backend`, `feature`, `LLM`, `tooling`, `sub-issue` **Parent Issue**: https://github.com/rwachters/chatbot/issues/115 ## Executive Summary This sub-issue covers all backend development required for the web search tool feature. This includes setting up the data layer, implementing core tool services, integrating tool calling with LLM strategies, and exposing necessary APIs for tool management and session-specific configurations. ## User Story Requirements (Backend Focus) - [ ] The tool is only available for LLMs that support tool calling. - [ ] The tool uses DuckDuckGo to perform the search. - [ ] The tool returns the search results to the assistant. - [ ] The tool can be configured to use a different search engine (backend configuration). - [ ] When the tool is called, the UI displays a notification that the tool is being used (backend sends event). - [ ] When an assistant message is generated, it includes a list of tools that were used to generate it (backend provides data). - [ ] The user can select an item in the list to view the tool's output (backend provides data). ## Implementation Plan ### Phase 1: Foundation (Backend Data & Services) - Create database tables (`ToolDefinitionTable`, `ToolCallTable`, `SessionToolConfigTable`) and update `ExposedDataManager` (server main). - Create DAO interfaces and Exposed implementations for `ToolDefinition`, `ToolCall`, and `SessionToolConfig`. - Create domain models: `ToolDefinition`, `ToolType`, `ToolCall`, `ToolCallStatus`, and `RawChatMessage`. - Create `ToolService` and its implementation. - Create `ToolExecutor` interface, `WebSearchToolExecutor`, and `ToolExecutorFactory`. - Add unit tests for DAOs and services. ### Phase 2: LLM Integration - Enhance LLM API models (OpenAiApiModels & OllamaApiModels) to support tool calling. - Update `LLMCompletionResult` and `LLMStreamChunk` to include tool calls. - Update `ChatCompletionStrategy` interface and implementations (OpenAIChatStrategy, OllamaChatStrategy) to handle tools. - Modify `MessageServiceImpl` to implement the tool calling loop, context building with `RawChatMessage`, and tool execution. - Update `LLMApiClient` and `LLMApiClientKtor` to pass tools. - Add integration tests for LLM strategies and message service. ### Phase 3: API Layer - Create tool management routes in `configureToolRoutes.kt`. - Create API request/response models (`ToolResource`, `CreateToolRequest`, `UpdateToolRequest`, `SetToolEnabledRequest`). - Add `configureToolRoutes` to `ApiRoutesKtor`. - Add route tests. ## Key Considerations (Backend) * **Tool Calling Loop**: The `MessageServiceImpl` will implement a loop to handle multiple tool calls in a single turn, creating/updating a single assistant message. * **Context Building**: The `buildContext` method will convert `ChatMessage` history into `RawChatMessage` objects, including tool calls and their results, for LLM API communication. Tool results are not stored as separate chat messages but are reconstructed for context. * **Streaming Tool Calls**: Support for accumulating and processing tool call deltas from streaming LLM responses (OpenAI) and complete tool calls (Ollama). * **Model Capability Check**: Utilize `LLMModel.hasCapability(LLMModelCapabilities.TOOL_CALLING)` to ensure tool usage is restricted to capable models. * **Provider Differences**: Account for variations between OpenAI (tool call IDs, string arguments) and Ollama (no tool call IDs, JSON object arguments). * **Error Handling**: Graceful handling of tool execution failures, reporting errors back to the LLM. * **Initial Data**: Initialize a default web search tool in `DatabaseInitializer`. ## Estimated Effort Approximately 10-14 days. ## Reference This issue is based on the detailed implementation report: docs/plans/web search tool/report.md