Torvian.eu/chatbot

Watch

Fork 0

Auto-Approval / Auto-Denial for Tool Calls — server + routes + DB + tests #134

New issue

Closed

opened 2025-12-17 22:03:17 +00:00 by rwachters · 0 comments

rwachters commented

2025-12-17 22:03:17 +00:00

(Migrated from github.com)

Auto-Approval/Auto-Denial Tool Call Feature - Implementation Summary

Overview

Successfully implemented auto-approval and auto-denial functionality for tool calls, allowing users to configure which tools should be automatically approved or denied without manual intervention.

Changes Made

1. Database Layer

New Table: UserToolApprovalPreferencesTable

Location: server/src/main/kotlin/eu/torvian/chatbot/server/data/tables/UserToolApprovalPreferencesTable.kt
Schema:
- userId (FK to UsersTable) - User who owns the preference
- toolDefinitionId (FK to ToolDefinitionTable) - Tool being configured
- autoApprove (BOOL, default: true) - Whether to auto-approve (true) or auto-deny (false)
- conditions (TEXT, nullable) - Reserved for future conditional logic
- denialReason (TEXT, nullable) - Reason text provided to LLM on auto-denial
- Composite Primary Key: (userId, toolDefinitionId)
- Index: On userId for efficient lookups

2. Common Models

UserToolApprovalPreference

Location: common/src/commonMain/kotlin/eu/torvian/chatbot/common/models/tool/UserToolApprovalPreference.kt
Shared data model for client-server communication
Includes all table fields plus serialization support

API Request DTO

Location: common/src/commonMain/kotlin/eu/torvian/chatbot/common/models/api/tool/SetToolApprovalPreferenceRequest.kt
Used for setting/updating preferences via REST API

3. Data Access Layer

UserToolApprovalPreferenceDao Interface

Location: server/src/main/kotlin/eu/torvian/chatbot/server/data/dao/UserToolApprovalPreferenceDao.kt
Methods:
- getPreference(userId, toolDefId) - Retrieve single preference
- getAllPreferencesForUser(userId) - List all preferences for a user
- setPreference(...) - Create or update preference (upsert)
- deletePreference(userId, toolDefId) - Delete specific preference
- deleteAllPreferencesForUser(userId) - Cleanup on user deletion

UserToolApprovalPreferenceDaoExposed

Location: server/src/main/kotlin/eu/torvian/chatbot/server/data/dao/exposed/UserToolApprovalPreferenceDaoExposed.kt
Exposed ORM implementation using SQL DSL
Validates user and tool existence before operations
Uses upsert for setPreference to handle create-or-update scenarios

Error Types

Location: server/src/main/kotlin/eu/torvian/chatbot/server/data/dao/error/UserToolApprovalPreferenceError.kt
NotFound - Preference doesn't exist
InvalidToolDefinition - Referenced tool doesn't exist
InvalidUser - Referenced user doesn't exist

4. Service Layer

ToolService Interface Extensions

Added methods:

setToolApprovalPreference(...) → Either<SetToolApprovalPreferenceError, UserToolApprovalPreference>
getToolApprovalPreference(...) → Either<GetToolApprovalPreferenceError, UserToolApprovalPreference>
getAllApprovalPreferencesForUser(userId) → List
deleteToolApprovalPreference(...) → Either<DeleteToolApprovalPreferenceError, Unit>

ToolServiceImpl Implementation

Location: server/src/main/kotlin/eu/torvian/chatbot/server/service/core/impl/ToolServiceImpl.kt
Added UserToolApprovalPreferenceDao dependency
Implemented all four new methods with proper error handling
Wraps DAO operations in transactions

Error Types

Location: server/src/main/kotlin/eu/torvian/chatbot/server/service/core/error/tool/ToolServiceErrors.kt
SetToolApprovalPreferenceError - ToolNotFound, UserNotFound
GetToolApprovalPreferenceError - PreferenceNotFound
DeleteToolApprovalPreferenceError - PreferenceNotFound

Error Mappings

Location: server/src/main/kotlin/eu/torvian/chatbot/server/service/core/error/tool/ToolServiceErrorExtensions.kt
Added extension functions to convert service errors to API errors

5. Auto-Approval Logic Integration

ChatServiceImpl Updates

Location: server/src/main/kotlin/eu/torvian/chatbot/server/service/core/impl/ChatServiceImpl.kt

Changes:

Added userId: Long parameter to processNewMessage() and processNewMessageStreaming()
Added userId: Long parameter to executeAndUpdateToolCalls() private function
Injected UserToolApprovalPreferenceDao into ChatServiceImpl constructor

Auto-Approval Flow (in executeAndUpdateToolCalls):

Check if user has a preference for the tool
If preference exists:
- If autoApprove == true: Create synthetic approval response → proceed to execution
- If autoApprove == false: Set status to USER_DENIED with denialReason → skip execution
If no preference exists:
- Request manual approval from user (existing flow)
- Wait for user response (5-minute timeout)
- Handle approval/denial based on user input

Benefits:

Reduces manual intervention for trusted/untrusted tools
Maintains backward compatibility (no preference = manual approval)
Supports conditional logic with conditions field (reserved for Phase 2)
Allows LLM to understand auto-denial reasons via denialReason field

6. API Routes

ToolResource Extensions

Location: common/src/commonMain/kotlin/eu/torvian/chatbot/common/api/resources/ToolResource.kt
Added ApprovalPreferences resource class for type-safe routing

Endpoints

Location: server/src/main/kotlin/eu/torvian/chatbot/server/ktor/routes/configureToolRoutes.kt

Added endpoints:

GET /api/v1/tools/approval-preferences - List all preferences for current user
PUT /api/v1/tools/approval-preferences - Set/update preference
GET /api/v1/tools/approval-preferences/{toolId} - Get specific preference
DELETE /api/v1/tools/approval-preferences/{toolId} - Delete preference

All endpoints require USER_JWT authentication.

7. Dependency Injection

Koin DI Updates

daoModule.kt: Registered UserToolApprovalPreferenceDaoExposed
serviceModule.kt: Updated ChatServiceImpl and ToolServiceImpl with new dependencies

8. Database Scheman

Schema Registration

Location: server/src/main/kotlin/eu/torvian/chatbot/server/main/ExposedDataManager.kt
Added UserToolApprovalPreferencesTable to schema creation/dropping

9. Route Handler Updates

WebSocket Route

Location: server/src/main/kotlin/eu/torvian/chatbot/server/ktor/routes/configureSessionRoutes.kt
Updated calls to processNewMessage() and processNewMessageStreaming() to pass userId from route context

10. Testsn

ChatServiceImplTest

Location: server/src/test/kotlin/eu/torvian/chatbot/server/service/core/impl/ChatServiceImplTest.kt
Updated all test mocks to include UserToolApprovalPreferenceDao
Updated processNewMessage() calls to include userId parameter
All tests passing ✓

Future Enhancements

Conditional Auto-Approval (Phase 2):
- Implement JSON-based condition evaluation in the conditions field
- Examples: approve only if input matches pattern, only during certain hours, etc.
Preference Management UI:
- Add frontend screens for users to manage their tool approval preferences
- Show which tools are auto-approved/auto-denied
Bulk Preference Operations:
- Batch update multiple tool preferences at once
- Import/export preference configurations
Preference Templates:
- Pre-configured preference sets for common use cases
- Share preferences between users/teams

Build Status

✅ Server Build: Successful (./gradlew server:assemble)
✅ Server Tests: All passing (./gradlew server:test)
✅ No compilation errors

Architecture Notes

Separation of Concerns: DAO → Service → API Route layers clearly separated
Error Handling: Logical errors using Arrow's Either type
Transaction Safety: All DB operations wrapped in transactions via TransactionScope
Backward Compatibility: Existing manual approval flow unchanged when no preference exists
User Context: userId now flows through ChatService for user-specific operations

# Auto-Approval/Auto-Denial Tool Call Feature - Implementation Summary ## Overview Successfully implemented auto-approval and auto-denial functionality for tool calls, allowing users to configure which tools should be automatically approved or denied without manual intervention. ## Changes Made ### 1. **Database Layer** #### New Table: UserToolApprovalPreferencesTable - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/data/tables/UserToolApprovalPreferencesTable.kt` - **Schema**: - `userId` (FK to UsersTable) - User who owns the preference - `toolDefinitionId` (FK to ToolDefinitionTable) - Tool being configured - `autoApprove` (BOOL, default: true) - Whether to auto-approve (true) or auto-deny (false) - `conditions` (TEXT, nullable) - Reserved for future conditional logic - `denialReason` (TEXT, nullable) - Reason text provided to LLM on auto-denial - **Composite Primary Key**: (userId, toolDefinitionId) - **Index**: On userId for efficient lookups ### 2. **Common Models** #### UserToolApprovalPreference - **Location**: `common/src/commonMain/kotlin/eu/torvian/chatbot/common/models/tool/UserToolApprovalPreference.kt` - Shared data model for client-server communication - Includes all table fields plus serialization support #### API Request DTO - **Location**: `common/src/commonMain/kotlin/eu/torvian/chatbot/common/models/api/tool/SetToolApprovalPreferenceRequest.kt` - Used for setting/updating preferences via REST API ### 3. **Data Access Layer** #### UserToolApprovalPreferenceDao Interface - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/data/dao/UserToolApprovalPreferenceDao.kt` - **Methods**: - `getPreference(userId, toolDefId)` - Retrieve single preference - `getAllPreferencesForUser(userId)` - List all preferences for a user - `setPreference(...)` - Create or update preference (upsert) - `deletePreference(userId, toolDefId)` - Delete specific preference - `deleteAllPreferencesForUser(userId)` - Cleanup on user deletion #### UserToolApprovalPreferenceDaoExposed - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/data/dao/exposed/UserToolApprovalPreferenceDaoExposed.kt` - Exposed ORM implementation using SQL DSL - Validates user and tool existence before operations - Uses upsert for setPreference to handle create-or-update scenarios #### Error Types - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/data/dao/error/UserToolApprovalPreferenceError.kt` - `NotFound` - Preference doesn't exist - `InvalidToolDefinition` - Referenced tool doesn't exist - `InvalidUser` - Referenced user doesn't exist ### 4. **Service Layer** #### ToolService Interface Extensions Added methods: - `setToolApprovalPreference(...)` → Either<SetToolApprovalPreferenceError, UserToolApprovalPreference> - `getToolApprovalPreference(...)` → Either<GetToolApprovalPreferenceError, UserToolApprovalPreference> - `getAllApprovalPreferencesForUser(userId)` → List<UserToolApprovalPreference> - `deleteToolApprovalPreference(...)` → Either<DeleteToolApprovalPreferenceError, Unit> #### ToolServiceImpl Implementation - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/service/core/impl/ToolServiceImpl.kt` - Added UserToolApprovalPreferenceDao dependency - Implemented all four new methods with proper error handling - Wraps DAO operations in transactions #### Error Types - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/service/core/error/tool/ToolServiceErrors.kt` - `SetToolApprovalPreferenceError` - ToolNotFound, UserNotFound - `GetToolApprovalPreferenceError` - PreferenceNotFound - `DeleteToolApprovalPreferenceError` - PreferenceNotFound #### Error Mappings - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/service/core/error/tool/ToolServiceErrorExtensions.kt` - Added extension functions to convert service errors to API errors ### 5. **Auto-Approval Logic Integration** #### ChatServiceImpl Updates - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/service/core/impl/ChatServiceImpl.kt` **Changes**: 1. Added `userId: Long` parameter to `processNewMessage()` and `processNewMessageStreaming()` 2. Added `userId: Long` parameter to `executeAndUpdateToolCalls()` private function 3. Injected `UserToolApprovalPreferenceDao` into ChatServiceImpl constructor **Auto-Approval Flow** (in executeAndUpdateToolCalls): 1. Check if user has a preference for the tool 2. If preference exists: - If `autoApprove == true`: Create synthetic approval response → proceed to execution - If `autoApprove == false`: Set status to USER_DENIED with denialReason → skip execution 3. If no preference exists: - Request manual approval from user (existing flow) - Wait for user response (5-minute timeout) - Handle approval/denial based on user input **Benefits**: - Reduces manual intervention for trusted/untrusted tools - Maintains backward compatibility (no preference = manual approval) - Supports conditional logic with `conditions` field (reserved for Phase 2) - Allows LLM to understand auto-denial reasons via `denialReason` field ### 6. **API Routes** #### ToolResource Extensions - **Location**: `common/src/commonMain/kotlin/eu/torvian/chatbot/common/api/resources/ToolResource.kt` - Added `ApprovalPreferences` resource class for type-safe routing #### Endpoints - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/ktor/routes/configureToolRoutes.kt` Added endpoints: - `GET /api/v1/tools/approval-preferences` - List all preferences for current user - `PUT /api/v1/tools/approval-preferences` - Set/update preference - `GET /api/v1/tools/approval-preferences/{toolId}` - Get specific preference - `DELETE /api/v1/tools/approval-preferences/{toolId}` - Delete preference All endpoints require `USER_JWT` authentication. ### 7. **Dependency Injection** #### Koin DI Updates - **daoModule.kt**: Registered `UserToolApprovalPreferenceDaoExposed` - **serviceModule.kt**: Updated `ChatServiceImpl` and `ToolServiceImpl` with new dependencies ### 8. **Database Schema**n #### Schema Registration - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/main/ExposedDataManager.kt` - Added `UserToolApprovalPreferencesTable` to schema creation/dropping ### 9. **Route Handler Updates** #### WebSocket Route - **Location**: `server/src/main/kotlin/eu/torvian/chatbot/server/ktor/routes/configureSessionRoutes.kt` - Updated calls to `processNewMessage()` and `processNewMessageStreaming()` to pass `userId` from route context ### 10. **Tests**n #### ChatServiceImplTest - **Location**: `server/src/test/kotlin/eu/torvian/chatbot/server/service/core/impl/ChatServiceImplTest.kt` - Updated all test mocks to include `UserToolApprovalPreferenceDao` - Updated `processNewMessage()` calls to include userId parameter - All tests passing ✓ ## Future Enhancements 1. **Conditional Auto-Approval** (Phase 2): - Implement JSON-based condition evaluation in the `conditions` field - Examples: approve only if input matches pattern, only during certain hours, etc. 2. **Preference Management UI**: - Add frontend screens for users to manage their tool approval preferences - Show which tools are auto-approved/auto-denied 3. **Bulk Preference Operations**: - Batch update multiple tool preferences at once - Import/export preference configurations 4. **Preference Templates**: - Pre-configured preference sets for common use cases - Share preferences between users/teams ## Build Status ✅ **Server Build**: Successful (./gradlew server:assemble) ✅ **Server Tests**: All passing (./gradlew server:test) ✅ **No compilation errors** ## Architecture Notes - **Separation of Concerns**: DAO → Service → API Route layers clearly separated - **Error Handling**: Logical errors using Arrow's Either type - **Transaction Safety**: All DB operations wrapped in transactions via TransactionScope - **Backward Compatibility**: Existing manual approval flow unchanged when no preference exists - **User Context**: userId now flows through ChatService for user-specific operations