Add user approval workflow for tool calls: AWAITING_APPROVAL, USER_DENIED + denialReason field #132

New issue

Closed

opened 2025-12-15 22:49:12 +00:00 by rwachters · 0 comments

rwachters commented

2025-12-15 22:49:12 +00:00

(Migrated from github.com)

Summary

Introduce a user-approval workflow for LLM-initiated tool calls so the client can approve or deny each tool execution before it runs. When denied by the user the tool call should be marked with a dedicated status USER_DENIED and the optional denialReason must be stored.

Background

Currently tool calls go through statuses such as PENDING, SUCCESS, and ERROR. We want the LLM to be able to request a tool call and then have the client (user) explicitly approve or deny the call. Denials should be tracked separately from execution errors.

High-level changes

Add new enum values to ToolCallStatus:
- AWAITING_APPROVAL — tool call saved and waiting for user approval
- USER_DENIED — user explicitly denied the tool call
Add a nullable denialReason: String? field to the ToolCall model and corresponding DB column denial_reason.
Add models for approval protocol:
- ToolCallApprovalRequest (server→client): toolCallId, toolName, inputJson, summary of definition
- ToolCallApprovalResponse (client→server): toolCallId, approved: Boolean, denialReason: String?
Add events to API models so server can request approval and client can respond:
- ChatEvent / ChatStreamEvent: add ToolCallApprovalRequested(request: ToolCallApprovalRequest)
- ChatClientEvent: add ToolCallApproval(response: ToolCallApprovalResponse)
Server flow: in ChatServiceImpl.executeAndUpdateToolCalls (server), when encountering a tool that should be executed by server or local MCP, save it with AWAITING_APPROVAL and emit ToolCallApprovalRequested. Wait for a matching ToolCallApprovalResponse from the client's approval flow before executing. If denied, update ToolCall to USER_DENIED and set denialReason.
Client flow/UI: show a ToolCallApprovalDialog when ToolCallApprovalRequested is received. User can Approve or Deny (optionally provide denial reason). Emit ChatClientEvent.ToolCallApproval event with the response. Integrate the approval response into the SendMessageUseCase flows by merging a MutableSharedFlow<ToolCallApprovalResponse> into clientEvents so server receives client decisions.

Files & code locations (starting points)

Common models:
- common/src/commonMain/kotlin/.../models/tool/ToolCallStatus.kt
- common/src/commonMain/kotlin/.../models/tool/ToolCall.kt
- Add ToolCallApprovalRequest.kt and ToolCallApprovalResponse.kt under same folder
API events:
- common/src/commonMain/kotlin/.../models/api/core/ChatEvent.kt
- common/src/commonMain/kotlin/.../models/api/core/ChatStreamEvent.kt
- common/src/commonMain/kotlin/.../models/api/core/ChatClientEvent.kt
Server:
- server/src/main/kotlin/.../service/core/impl/ChatServiceImpl.kt — update executeAndUpdateToolCalls to use approval flow and new statuses
- server/src/main/kotlin/.../service/mcp/LocalMCPExecutor.kt — wrap request in approval flow (if MCP execution also needs approval)
- server/src/main/kotlin/.../data/tables/ToolCallTable.kt — add new column denial_reason = text("denial_reason").nullable()
- server/src/main/kotlin/.../data/dao/ToolCallDao.kt and exposed/ToolCallDaoExposed.kt — update signatures/SQL mapping to include denialReason
Client (app):
- app/src/commonMain/.../viewmodel/chat/usecase/SendMessageUseCase.kt — wire up a MutableSharedFlow<ToolCallApprovalResponse> and merge it into clientEvents for the repository
- app/src/commonMain/.../service/mcp/LocalMCPToolCallMediator.kt implementations — ensure MCP flows can accept approvals
- UI: app/src/commonMain/.../compose/chatarea/ToolCallApprovalDialog.kt (new), update MessageItem.kt, ToolCallBadge.kt, and ToolCallDetailsDialog.kt to visualize AWAITING_APPROVAL and USER_DENIED statuses

DB migration notes

SQLite (Exposed): run a migration to add the denial_reason column (nullable):

ALTER TABLE tool_calls ADD COLUMN denial_reason TEXT;
No enum DB migration needed for ToolCallStatus since the enum is stored by name; ensure the column width (currently 50) is sufficient.

Testing & QA

Unit tests (server): update ToolCallDaoExposedTest to assert denialReason is stored/updated correctly. Add tests for executeAndUpdateToolCalls path where approval is requested and denied/approved.
Integration tests (client/server): simulate a ToolCallApprovalRequested and ensure the client sends ChatClientEvent.ToolCallApproval and server acts accordingly.
UI tests: show approval dialog, deny with and without reason, ensure denialReason is sent and persisted.

Acceptance criteria

Server emits ToolCallApprovalRequested before executing any tool that requires client approval.
Client shows an approval dialog allowing Approve/Deny + optional reason.
On Approve: tool executes as before and ultimately transitions to SUCCESS/ERROR depending on execution result.
On Deny: tool call record is updated to USER_DENIED and denialReason contains the user's reason (or null if none). No execution occurs.
Existing tests continue to pass after updates, and new tests cover the approval/deny flow.

Implementation notes / suggestions

Consider an approval timeout to auto-deny after configurable timeout. If desired, server should set a reasonable default (e.g., 60s or session-specific).
For multiple simultaneous tool calls consider a batched approval UI or an "Approve All / Deny All" option.
Optionally add a per-tool "always approve" preference to reduce modal frequency.

Checklist

Add AWAITING_APPROVAL and USER_DENIED to ToolCallStatus
Add denialReason to ToolCall model and DB table
Add approval request/response models
Add ChatEvent/ChatStreamEvent/ChatClientEvent variants
Server: emit approval requests and wait for responses
Server: set USER_DENIED + denialReason for denied tool calls
Client: UI dialog and flow integration
Update DAOs, tests, and add migration SQL

Labels

["enhancement","backend","frontend"]

If you want different labels or an assignee, tell me and I can update the issue or create a follow-up.

Summary Introduce a user-approval workflow for LLM-initiated tool calls so the client can approve or deny each tool execution before it runs. When denied by the user the tool call should be marked with a dedicated status `USER_DENIED` and the optional `denialReason` must be stored. Background Currently tool calls go through statuses such as `PENDING`, `SUCCESS`, and `ERROR`. We want the LLM to be able to request a tool call and then have the client (user) explicitly approve or deny the call. Denials should be tracked separately from execution errors. High-level changes - Add new enum values to `ToolCallStatus`: - `AWAITING_APPROVAL` — tool call saved and waiting for user approval - `USER_DENIED` — user explicitly denied the tool call - Add a nullable `denialReason: String?` field to the `ToolCall` model and corresponding DB column `denial_reason`. - Add models for approval protocol: - `ToolCallApprovalRequest` (server→client): toolCallId, toolName, inputJson, summary of definition - `ToolCallApprovalResponse` (client→server): toolCallId, approved: Boolean, denialReason: String? - Add events to API models so server can request approval and client can respond: - `ChatEvent` / `ChatStreamEvent`: add `ToolCallApprovalRequested(request: ToolCallApprovalRequest)` - `ChatClientEvent`: add `ToolCallApproval(response: ToolCallApprovalResponse)` - Server flow: in `ChatServiceImpl.executeAndUpdateToolCalls` (server), when encountering a tool that should be executed by server or local MCP, save it with `AWAITING_APPROVAL` and emit `ToolCallApprovalRequested`. Wait for a matching `ToolCallApprovalResponse` from the client's approval flow before executing. If denied, update `ToolCall` to `USER_DENIED` and set `denialReason`. - Client flow/UI: show a `ToolCallApprovalDialog` when `ToolCallApprovalRequested` is received. User can Approve or Deny (optionally provide denial reason). Emit `ChatClientEvent.ToolCallApproval` event with the response. Integrate the approval response into the `SendMessageUseCase` flows by merging a `MutableSharedFlow<ToolCallApprovalResponse>` into `clientEvents` so server receives client decisions. Files & code locations (starting points) - Common models: - `common/src/commonMain/kotlin/.../models/tool/ToolCallStatus.kt` - `common/src/commonMain/kotlin/.../models/tool/ToolCall.kt` - Add `ToolCallApprovalRequest.kt` and `ToolCallApprovalResponse.kt` under same folder - API events: - `common/src/commonMain/kotlin/.../models/api/core/ChatEvent.kt` - `common/src/commonMain/kotlin/.../models/api/core/ChatStreamEvent.kt` - `common/src/commonMain/kotlin/.../models/api/core/ChatClientEvent.kt` - Server: - `server/src/main/kotlin/.../service/core/impl/ChatServiceImpl.kt` — update `executeAndUpdateToolCalls` to use approval flow and new statuses - `server/src/main/kotlin/.../service/mcp/LocalMCPExecutor.kt` — wrap request in approval flow (if MCP execution also needs approval) - `server/src/main/kotlin/.../data/tables/ToolCallTable.kt` — add new column `denial_reason = text("denial_reason").nullable()` - `server/src/main/kotlin/.../data/dao/ToolCallDao.kt` and `exposed/ToolCallDaoExposed.kt` — update signatures/SQL mapping to include `denialReason` - Client (app): - `app/src/commonMain/.../viewmodel/chat/usecase/SendMessageUseCase.kt` — wire up a `MutableSharedFlow<ToolCallApprovalResponse>` and merge it into `clientEvents` for the repository - `app/src/commonMain/.../service/mcp/LocalMCPToolCallMediator.kt` implementations — ensure MCP flows can accept approvals - UI: `app/src/commonMain/.../compose/chatarea/ToolCallApprovalDialog.kt` (new), update `MessageItem.kt`, `ToolCallBadge.kt`, and `ToolCallDetailsDialog.kt` to visualize AWAITING_APPROVAL and USER_DENIED statuses DB migration notes - SQLite (Exposed): run a migration to add the `denial_reason` column (nullable): ALTER TABLE tool_calls ADD COLUMN denial_reason TEXT; - No enum DB migration needed for `ToolCallStatus` since the enum is stored by name; ensure the column width (currently 50) is sufficient. Testing & QA - Unit tests (server): update `ToolCallDaoExposedTest` to assert `denialReason` is stored/updated correctly. Add tests for `executeAndUpdateToolCalls` path where approval is requested and denied/approved. - Integration tests (client/server): simulate a `ToolCallApprovalRequested` and ensure the client sends `ChatClientEvent.ToolCallApproval` and server acts accordingly. - UI tests: show approval dialog, deny with and without reason, ensure denialReason is sent and persisted. Acceptance criteria - Server emits `ToolCallApprovalRequested` before executing any tool that requires client approval. - Client shows an approval dialog allowing Approve/Deny + optional reason. - On Approve: tool executes as before and ultimately transitions to `SUCCESS`/`ERROR` depending on execution result. - On Deny: tool call record is updated to `USER_DENIED` and `denialReason` contains the user's reason (or null if none). No execution occurs. - Existing tests continue to pass after updates, and new tests cover the approval/deny flow. Implementation notes / suggestions - Consider an approval timeout to auto-deny after configurable timeout. If desired, server should set a reasonable default (e.g., 60s or session-specific). - For multiple simultaneous tool calls consider a batched approval UI or an "Approve All / Deny All" option. - Optionally add a per-tool "always approve" preference to reduce modal frequency. Checklist - [ ] Add `AWAITING_APPROVAL` and `USER_DENIED` to `ToolCallStatus` - [ ] Add `denialReason` to `ToolCall` model and DB table - [ ] Add approval request/response models - [ ] Add ChatEvent/ChatStreamEvent/ChatClientEvent variants - [ ] Server: emit approval requests and wait for responses - [ ] Server: set `USER_DENIED` + `denialReason` for denied tool calls - [ ] Client: UI dialog and flow integration - [ ] Update DAOs, tests, and add migration SQL Labels ["enhancement","backend","frontend"] If you want different labels or an assignee, tell me and I can update the issue or create a follow-up.