Torvian.eu/chatbot
Watch
1
0
Fork
You've already forked chatbot
 0
Code Issues Pull requests Projects Releases 12 Packages Wiki Activity Actions

Phase 4: Local MCP Tool Execution via WebSocket #126

New issue
Closed
opened 2025-11-22 22:53:26 +00:00 by rwachters · 0 comments
rwachters commented 2025-11-22 22:53:26 +00:00 (Migrated from github.com)

Phase 4: Local MCP Tool Execution via WebSocket

Epic: NF.EA1 - Local MCP Servers

Description

This issue details the implementation of the end-to-end tool execution flow. When an LLM decides to call a tool hosted on a local MCP server, the request must be securely routed from the main server to the correct client, executed locally on the user's machine, and the result sent back to the LLM.

This requires upgrading the primary chat message endpoint from a standard HTTP request/response model to a persistent WebSocket connection to facilitate real-time, bidirectional communication.

User Stories & Acceptance Criteria

✅ Task Checklist

  • US4.1: Implement Local MCP Tool Executor (Client-Side)

    • Create a new client-side service, LocalMCPToolExecutor.
    • This service will have a primary method, execute(toolCallRequest), which takes the details of the tool call from the server.
    • The execute method will orchestrate the entire local execution process:
      • Use LocalMCPServerManager to ensure the correct MCP server process is running.
      • Use MCPClientService to send the tool call parameters to the local process via MCP.
      • Await the result from the local process.
      • Return the tool execution result (or an error) to its caller.

  • US4.2: Implement WebSocket Message Processing Integration (Server-Side)

    • Upgrade the POST /api/v1/sessions/{sessionId}/messages endpoint to support WebSocket connections.
    • Define a clear WebSocket message protocol for tool execution, including message types for ToolCallRequest (server-to-client) and ToolCallResult (client-to-server).
    • When the LLM service generates a tool call for a local MCP tool, the server must:
      • Identify the correct client WebSocket connection.
      • Send a ToolCallRequest message to that client.
    • The server must listen on the WebSocket for an incoming ToolCallResult message from the client.
    • Upon receiving the result, the server forwards it back to the LLM service to continue the generation.

  • US4.3: Implement Client-Side WebSocket Handler for Tool Execution

    • Implement the client-side WebSocket logic to connect to the server's new endpoint.
    • Create a handler that listens for incoming WebSocket messages from the server.
    • When a ToolCallRequest message is received, the handler must invoke the LocalMCPToolExecutor with the request payload.
    • Once the LocalMCPToolExecutor completes, the handler must take the result, wrap it in a ToolCallResult message, and send it back to the server over the WebSocket.
    • The handler must gracefully manage execution errors and report them back to the server.

Architectural Context

  • Shift to WebSockets: The core architectural change in this phase is the move from a stateless HTTP API to a stateful WebSocket connection for chat sessions. This is non-negotiable for tool calling, as the server needs to be able to push a tool execution request to the client at any time and wait for a response.

  • Separation of Roles: This implementation solidifies the roles of the server and client in the tool-calling process:

    • Server: Acts as a message broker and orchestrator. It facilitates communication between the LLM and the client but never executes the local tool itself. Its job is to route requests and results.
    • Client: Acts as the execution agent. It is responsible for the entire lifecycle of the local tool call: receiving the request, managing the OS process, communicating via MCP, and reporting the result.

  • End-to-End Flow:

    1. LLM tells the Server: "Call tool search_web(query='AI')".
    2. Server sends a ToolCallRequest WebSocket message to the Client.
    3. Client's WebSocket handler receives the message and calls LocalMCPToolExecutor.
    4. The executor starts the local MCP server process and sends the command over its stdin.
    5. The local process executes the search and writes the result to its stdout.
    6. The executor reads the result and returns it to the WebSocket handler.
    7. The handler sends a ToolCallResult WebSocket message back to the Server.
    8. Server gives the result to the LLM, which then formulates a final response.
# Phase 4: Local MCP Tool Execution via WebSocket **Epic**: `NF.EA1 - Local MCP Servers` ## Description This issue details the implementation of the end-to-end tool execution flow. When an LLM decides to call a tool hosted on a local MCP server, the request must be securely routed from the main server to the correct client, executed locally on the user's machine, and the result sent back to the LLM. This requires upgrading the primary chat message endpoint from a standard HTTP request/response model to a persistent WebSocket connection to facilitate real-time, bidirectional communication. --- ## User Stories & Acceptance Criteria ### ✅ Task Checklist - [ ] **US4.1: Implement Local MCP Tool Executor (Client-Side)** - [ ] Create a new client-side service, `LocalMCPToolExecutor`. - [ ] This service will have a primary method, `execute(toolCallRequest)`, which takes the details of the tool call from the server. - [ ] The `execute` method will orchestrate the entire local execution process: - Use `LocalMCPServerManager` to ensure the correct MCP server process is running. - Use `MCPClientService` to send the tool call parameters to the local process via MCP. - Await the result from the local process. - Return the tool execution result (or an error) to its caller. - [ ] **US4.2: Implement WebSocket Message Processing Integration (Server-Side)** - [ ] Upgrade the `POST /api/v1/sessions/{sessionId}/messages` endpoint to support WebSocket connections. - [ ] Define a clear WebSocket message protocol for tool execution, including message types for `ToolCallRequest` (server-to-client) and `ToolCallResult` (client-to-server). - [ ] When the LLM service generates a tool call for a local MCP tool, the server must: - Identify the correct client WebSocket connection. - Send a `ToolCallRequest` message to that client. - [ ] The server must listen on the WebSocket for an incoming `ToolCallResult` message from the client. - [ ] Upon receiving the result, the server forwards it back to the LLM service to continue the generation. - [ ] **US4.3: Implement Client-Side WebSocket Handler for Tool Execution** - [ ] Implement the client-side WebSocket logic to connect to the server's new endpoint. - [ ] Create a handler that listens for incoming WebSocket messages from the server. - [ ] When a `ToolCallRequest` message is received, the handler must invoke the `LocalMCPToolExecutor` with the request payload. - [ ] Once the `LocalMCPToolExecutor` completes, the handler must take the result, wrap it in a `ToolCallResult` message, and send it back to the server over the WebSocket. - [ ] The handler must gracefully manage execution errors and report them back to the server. --- <details> <summary><h3>Architectural Context</h3></summary> - **Shift to WebSockets**: The core architectural change in this phase is the move from a stateless HTTP API to a stateful WebSocket connection for chat sessions. This is non-negotiable for tool calling, as the server needs to be able to push a tool execution request to the client at any time and wait for a response. - **Separation of Roles**: This implementation solidifies the roles of the server and client in the tool-calling process: - **Server**: Acts as a message broker and orchestrator. It facilitates communication between the LLM and the client but never executes the local tool itself. Its job is to route requests and results. - **Client**: Acts as the execution agent. It is responsible for the entire lifecycle of the local tool call: receiving the request, managing the OS process, communicating via MCP, and reporting the result. - **End-to-End Flow**: 1. LLM tells the Server: "Call tool `search_web(query='AI')`". 2. Server sends a `ToolCallRequest` WebSocket message to the Client. 3. Client's WebSocket handler receives the message and calls `LocalMCPToolExecutor`. 4. The executor starts the local MCP server process and sends the command over its `stdin`. 5. The local process executes the search and writes the result to its `stdout`. 6. The executor reads the result and returns it to the WebSocket handler. 7. The handler sends a `ToolCallResult` WebSocket message back to the Server. 8. Server gives the result to the LLM, which then formulates a final response. </details>
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
test/credential-validator-fixture-umask
feat/credential-profile-agent-roles
fix/credential-secret-tree-removal
feat/terminal-session-credential-attach
feat/worker-terminal-session
feat/worker-command-uid-isolation
v0.11.0
v0.10.0
v0.9.0
v0.8.0
v0.7.0
v0.6.0
v0.5.1
v0.5.0
v0.4.0
v0.3.0
v0.2.0
v0.1.0
Labels
Clear labels   
Epic 5

   
MCP

   
P1

   
PR-0

   
PR-1

   
api

   
api-layer

   
architecture

   
area:backend

   
area:frontend

   
arrow

   
authentication

   
backend

   
bug
Something isn't working

  
chat

   
client

   
clipboard

   
common-module

   
compose

   
cross-cutting

   
cross-cutting-requirement

   
data-layer

   
data-models

   
database

   
desktop-client

   
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
epic

   
epic-1

   
epic-2

   
epic-3

   
epic-4

   
epic-5

   
epic-6

   
epic-7

   
epic-llm-config

   
error-handling

   
feature

   
frontend

   
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
implementation

   
infrastructure

   
installer

   
invalid
This doesn't seem right

  
kotlin

   
ktor

   
llm

   
medium-priority

   
message

   
multiplatform

   
needs-review

   
needs-triage

   
packaging

   
performance

   
phase-1

   
phase-2

   
phase-3

   
phase-4

   
phase-5

   
phase3

   
phase:3

   
priority: high

   
question
Further information is requested

  
refactor

   
refactoring

   
resource-sharing

   
security

   
server

   
server-module

   
services

   
session

   
session-management

   
settings

   
setup

   
shutdown

   
state

   
storage

   
streaming

   
sub-issue

   
tests

   
tooling

   
type:task

   
ui

   
ui/ux

   
user-accounts

   
user-management

   
ux

   
viewmodel

   
websocket

   
windows

   
windows-integration

   
wontfix
This will not be worked on

No labels
Epic 5
MCP
P1
PR-0
PR-1
api
api-layer
architecture
area:backend
area:frontend
arrow
authentication
backend
bug
chat
client
clipboard
common-module
compose
cross-cutting
cross-cutting-requirement
data-layer
data-models
database
desktop-client
documentation
duplicate
enhancement
epic
epic-1
epic-2
epic-3
epic-4
epic-5
epic-6
epic-7
epic-llm-config
error-handling
feature
frontend
good first issue
help wanted
implementation
infrastructure
installer
invalid
kotlin
ktor
llm
medium-priority
message
multiplatform
needs-review
needs-triage
packaging
performance
phase-1
phase-2
phase-3
phase-4
phase-5
phase3
phase:3
priority: high
question
refactor
refactoring
resource-sharing
security
server
server-module
services
session
session-management
settings
setup
shutdown
state
storage
streaming
sub-issue
tests
tooling
type:task
ui
ui/ux
user-accounts
user-management
ux
viewmodel
websocket
windows
windows-integration
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Torvian.eu/chatbot#126
No description provided.