Implement File References in Chat Messages #164

New issue

Closed

opened 2025-12-29 09:48:51 +00:00 by rwachters · 0 comments

rwachters commented

2025-12-29 09:48:51 +00:00

(Migrated from github.com)

Overview

Add comprehensive file reference support to allow users to attach files to messages, send them inline or as non-inline references, and display them as interactive badges throughout the chat interface. Files can be referenced without sending content (for LLM tool use with MCP) or with content included (for direct context, text files only).

Implementation Steps

Create data models - Add FileReference data class in common/src/commonMain/kotlin/eu/torvian/chatbot/common/models/core/ with stored properties: basePath, relativePath, fileSize, lastModified, mimeType, content (optional String), inlinePosition (optional Int for cursor placement); add computed properties fileName (from relativePath) and isInline (from inlinePosition != null)
Extend database schema - Add fileReferences TEXT column (JSON array) to ChatMessageTable and update ChatMessage.UserMessage and ChatMessage.AssistantMessage to include fileReferences: List<FileReference> property
Update persistence layer - Modify MessageDaoExposed to serialize/deserialize file references using kotlinx.serialization JSON, update toUserMessage and toAssistantMessage mappers to handle file references column with default empty list for existing messages
Extend API contracts - Add fileReferences: List<FileReference> field to ProcessNewMessageRequest and InsertMessageRequest, add validation in ChatService for 5MB total content size limit across all file references with content
Implement UI badge components - Create FileReferenceBadge.kt composable (similar to ToolCallBadge) showing file name and file icon, create FileReferenceDetailsDialog.kt modal (similar to ToolCallDetailsDialog) displaying full path, base path, relative path, size, modified date, MIME type, whether content is included, and scrollable content preview
Add file picker UI - Extend InputArea with file attachment button (in action row below text field), add badge display section for attached non-inline file references with remove button, implement base path selector UI with automatic detection of common parent path from selected files, add content inclusion toggle per file (default off for files > 100KB)
Add state management - Extend ChatAreaState with attachedFileReferences: List<FileReference> and basePathOverride: String? (stored in ViewModel only, per session), add actions to ChatAreaActions for file operations: onAddFiles, onRemoveFile, onSelectBasePath, onToggleFileContentInclusion
Display badges in messages - Update MessageItem to render non-inline file reference badges below tool call badges section in new "Files" section, render inline file reference badges within the message content at their character positions using custom text rendering, add click handlers to open FileReferenceDetailsDialog with file details
Implement LLM context integration - Update ChatService message preparation to build LLM context string: append non-inline references as "\n\nReferenced files: [file1.txt, file2.txt]" at end of message, insert inline references at inlinePosition by injecting file content when present or "[reference: filename.txt]" placeholder when absent, preserve original message content structure
Add platform-specific file picker - Create expect fun selectFiles(): List<SelectedFileInfo>? in commonMain with data class for file path/size/lastModified, implement desktop version in desktopMain using JFileChooser with multi-select, read file metadata and attempt UTF-8 content reading with validation (null byte check), calculate common parent path across selections for default base path, enforce 5MB total content limit with user-friendly error
Implement file re-reading on reuse - When user selects file references from previous messages for new message (future feature), re-read files from disk using stored basePath + relativePath, update content, fileSize, and lastModified fields, preserve relativePath and basePath from original, show error notification if file not found or not readable, allow user to browse for moved file

Key Constraints

File content size limit: 5MB total per message across all file references with content included
Only support text-based files - validate UTF-8 decodability and null byte checks
Base path stored in ViewModel only (per session), not persisted to server
Server only receives content field from FileReference if present - no file handling on backend
Inline content injection: inject actual file content at inlinePosition, or "[reference: filename.txt]" if content absent

Considerations for Implementation

Database migration - Add fileReferences column to existing chat_messages table with ALTER TABLE statement, default to empty JSON array '[]' for existing rows
Large file content warnings - Show warning icon/tooltip on files > 1MB when user enables content inclusion, suggest using reference-only mode for large files
MIME type detection - Use file extension mapping as primary method with fallback to "application/octet-stream", validate text MIME types (text/*, application/json, application/xml) against UTF-8 decode success
Message content length limits - Consider separate limit on final message content length after inline file content injection (e.g., 1MB text limit for LLM context window considerations)
Relative path normalization - Normalize path separators to forward slash / for cross-platform consistency in stored references, convert back to platform-specific separator when reading files

Future Implementation

Add/remove inline file references in UI - Implement custom TextField with "#" key trigger for file reference popup, show list of attached files for current message, allow inline insertion at cursor position with visual badge rendering, support removal via backspace or click-to-remove
Select file references from previous messages - Dialog to browse conversation history, filter messages with file references, select existing references, option to insert inline or non-inline in current message with automatic file re-reading

Tech Stack

Kotlin 2.2.0
Compose 1.8.2 with Material 3
Ktor 3.2.3 (Server & HTTP Client)
Exposed 0.61.0
kotlinx.datetime
kotlinx.serialization
Arrow 2.1.2

## Overview Add comprehensive file reference support to allow users to attach files to messages, send them inline or as non-inline references, and display them as interactive badges throughout the chat interface. Files can be referenced without sending content (for LLM tool use with MCP) or with content included (for direct context, text files only). ## Implementation Steps 1. **Create data models** - Add `FileReference` data class in `common/src/commonMain/kotlin/eu/torvian/chatbot/common/models/core/` with stored properties: `basePath`, `relativePath`, `fileSize`, `lastModified`, `mimeType`, `content` (optional String), `inlinePosition` (optional Int for cursor placement); add computed properties `fileName` (from `relativePath`) and `isInline` (from `inlinePosition != null`) 2. **Extend database schema** - Add `fileReferences` TEXT column (JSON array) to `ChatMessageTable` and update `ChatMessage.UserMessage` and `ChatMessage.AssistantMessage` to include `fileReferences: List<FileReference>` property 3. **Update persistence layer** - Modify `MessageDaoExposed` to serialize/deserialize file references using kotlinx.serialization JSON, update `toUserMessage` and `toAssistantMessage` mappers to handle file references column with default empty list for existing messages 4. **Extend API contracts** - Add `fileReferences: List<FileReference>` field to `ProcessNewMessageRequest` and `InsertMessageRequest`, add validation in `ChatService` for 5MB total content size limit across all file references with content 5. **Implement UI badge components** - Create `FileReferenceBadge.kt` composable (similar to `ToolCallBadge`) showing file name and file icon, create `FileReferenceDetailsDialog.kt` modal (similar to `ToolCallDetailsDialog`) displaying full path, base path, relative path, size, modified date, MIME type, whether content is included, and scrollable content preview 6. **Add file picker UI** - Extend `InputArea` with file attachment button (in action row below text field), add badge display section for attached non-inline file references with remove button, implement base path selector UI with automatic detection of common parent path from selected files, add content inclusion toggle per file (default off for files > 100KB) 7. **Add state management** - Extend `ChatAreaState` with `attachedFileReferences: List<FileReference>` and `basePathOverride: String?` (stored in ViewModel only, per session), add actions to `ChatAreaActions` for file operations: `onAddFiles`, `onRemoveFile`, `onSelectBasePath`, `onToggleFileContentInclusion` 8. **Display badges in messages** - Update `MessageItem` to render non-inline file reference badges below tool call badges section in new "Files" section, render inline file reference badges within the message content at their character positions using custom text rendering, add click handlers to open `FileReferenceDetailsDialog` with file details 9. **Implement LLM context integration** - Update `ChatService` message preparation to build LLM context string: append non-inline references as "\n\nReferenced files: [file1.txt, file2.txt]" at end of message, insert inline references at `inlinePosition` by injecting file `content` when present or `"[reference: filename.txt]"` placeholder when absent, preserve original message content structure 10. **Add platform-specific file picker** - Create `expect fun selectFiles(): List<SelectedFileInfo>?` in `commonMain` with data class for file path/size/lastModified, implement desktop version in `desktopMain` using `JFileChooser` with multi-select, read file metadata and attempt UTF-8 content reading with validation (null byte check), calculate common parent path across selections for default base path, enforce 5MB total content limit with user-friendly error 11. **Implement file re-reading on reuse** - When user selects file references from previous messages for new message (future feature), re-read files from disk using stored `basePath + relativePath`, update `content`, `fileSize`, and `lastModified` fields, preserve `relativePath` and `basePath` from original, show error notification if file not found or not readable, allow user to browse for moved file ## Key Constraints - File content size limit: **5MB total per message** across all file references with content included - Only support **text-based files** - validate UTF-8 decodability and null byte checks - Base path stored in **ViewModel only** (per session), not persisted to server - Server only receives `content` field from FileReference if present - no file handling on backend - Inline content injection: inject actual file `content` at `inlinePosition`, or `"[reference: filename.txt]"` if content absent ## Considerations for Implementation 1. **Database migration** - Add `fileReferences` column to existing `chat_messages` table with ALTER TABLE statement, default to empty JSON array `'[]'` for existing rows 2. **Large file content warnings** - Show warning icon/tooltip on files > 1MB when user enables content inclusion, suggest using reference-only mode for large files 3. **MIME type detection** - Use file extension mapping as primary method with fallback to `"application/octet-stream"`, validate text MIME types (text/*, application/json, application/xml) against UTF-8 decode success 4. **Message content length limits** - Consider separate limit on final message content length after inline file content injection (e.g., 1MB text limit for LLM context window considerations) 5. **Relative path normalization** - Normalize path separators to forward slash `/` for cross-platform consistency in stored references, convert back to platform-specific separator when reading files ## Future Implementation - **Add/remove inline file references in UI** - Implement custom TextField with "#" key trigger for file reference popup, show list of attached files for current message, allow inline insertion at cursor position with visual badge rendering, support removal via backspace or click-to-remove - **Select file references from previous messages** - Dialog to browse conversation history, filter messages with file references, select existing references, option to insert inline or non-inline in current message with automatic file re-reading ## Tech Stack - Kotlin 2.2.0 - Compose 1.8.2 with Material 3 - Ktor 3.2.3 (Server & HTTP Client) - Exposed 0.61.0 - kotlinx.datetime - kotlinx.serialization - Arrow 2.1.2