E4.S7: Allow User to Select LLM Model & Settings Per Session #85

New issue

Closed

opened 2025-09-12 23:16:44 +00:00 by rwachters · 0 comments

rwachters commented

2025-09-12 23:16:44 +00:00

(Migrated from github.com)

🚀 Feature Request: Allow User to Select LLM Model & Settings Per Session

User Story ID: E4.S7

📝 Description

As a user, within an active chat session, I want to easily choose which configured LLM Model and specific settings profile will be used for the next assistant response, allowing me to change the AI's approach mid-conversation. This provides granular control over AI behavior during a conversation.

✅ Acceptance Criteria

UI Element for Selection: A UI element (e.g., a dropdown or selector) is visibly present in the active chat session area, showing the currently selected LLM Model and its associated settings profile.
Model & Settings List Display: Clicking the UI element displays a comprehensive list of all configured LLM models (from E4.S2) and, for each model, its available settings profiles (from E4.S5).
Selection & UI Update: Users can select a different model and/or settings profile from the displayed list. Upon selection, the UI element for the current session updates to reflect the new choice.
Backend Update: The frontend makes a call to the backend updateSessionDetails API endpoint, sending the sessionId and the IDs of the newly selected currentModelId and currentSettingsId.
LLM Integration: The backend's LLM processing logic (as per E1.S4) retrieves and uses these updated currentModelId and currentSettingsId from the ChatSession record before making the actual LLM API call for the next assistant response.
Data Persistence: The updated currentModelId and currentSettingsId for the ChatSession are persisted in the database (as per E2.S2).
Asynchronous Operations: All database updates and UI state changes related to this feature are performed asynchronously using Kotlin Coroutines (as per E7.S6).

🔗 Dependencies

This feature depends on the successful implementation of the following:

E4.S2: View Configured LLM Models List: To allow users to see and select from a list of available LLM models.
E4.S5: Manage Model Settings Profiles List: To allow users to see and select from a list of available settings profiles for a given model.
E1.S4: Get LLM Response and Display Assistant Message in Thread: The backend logic for generating LLM responses must utilize the selected model and settings.
PR 9: Wire Backend API - Session & Group Management: Specifically, the updateSessionLeafMessage route needs to be extended or a new updateSessionDetails route needs to be added to handle updating currentModelId and currentSettingsId.
PR 15: Implement Chat Actions ViewModel Logic: The ChatViewModel will need methods to handle model/settings selection and update the session.

⏱️ Effort Estimate

Medium (M)

💡 Potential Implementation Notes

Consider using Compose's DropdownMenu or a custom dialog for selecting models and settings.
The API call to update the session should be a PUT request to /api/v1/sessions/{sessionId}/model and /api/v1/sessions/{sessionId}/settings or a single PUT to /api/v1/sessions/{sessionId}/details encompassing both.
The ChatSessionTable will need to store currentModelId and currentSettingsId as nullable foreign keys. Remember to set ON DELETE SET NULL for these foreign keys in the Exposed schema (E7.S4).

## 🚀 Feature Request: Allow User to Select LLM Model & Settings Per Session **User Story ID:** E4.S7 ### 📝 Description As a user, within an active chat session, I want to easily choose which configured LLM Model and specific settings profile will be used for the next assistant response, allowing me to change the AI's approach mid-conversation. This provides granular control over AI behavior during a conversation. ### ✅ Acceptance Criteria * **UI Element for Selection:** A UI element (e.g., a dropdown or selector) is visibly present in the active chat session area, showing the currently selected LLM Model and its associated settings profile. * **Model & Settings List Display:** Clicking the UI element displays a comprehensive list of all configured LLM models (from `E4.S2`) and, for each model, its available settings profiles (from `E4.S5`). * **Selection & UI Update:** Users can select a different model and/or settings profile from the displayed list. Upon selection, the UI element for the current session updates to reflect the new choice. * **Backend Update:** The frontend makes a call to the backend `updateSessionDetails` API endpoint, sending the `sessionId` and the IDs of the newly selected `currentModelId` and `currentSettingsId`. * **LLM Integration:** The backend's LLM processing logic (as per `E1.S4`) retrieves and uses these updated `currentModelId` and `currentSettingsId` from the `ChatSession` record before making the actual LLM API call for the next assistant response. * **Data Persistence:** The updated `currentModelId` and `currentSettingsId` for the `ChatSession` are persisted in the database (as per `E2.S2`). * **Asynchronous Operations:** All database updates and UI state changes related to this feature are performed asynchronously using Kotlin Coroutines (as per `E7.S6`). ### 🔗 Dependencies This feature depends on the successful implementation of the following: * **`E4.S2: View Configured LLM Models List`**: To allow users to see and select from a list of available LLM models. * **`E4.S5: Manage Model Settings Profiles List`**: To allow users to see and select from a list of available settings profiles for a given model. * **`E1.S4: Get LLM Response and Display Assistant Message in Thread`**: The backend logic for generating LLM responses must utilize the selected model and settings. * **`PR 9: Wire Backend API - Session & Group Management`**: Specifically, the `updateSessionLeafMessage` route needs to be extended or a new `updateSessionDetails` route needs to be added to handle updating `currentModelId` and `currentSettingsId`. * **`PR 15: Implement Chat Actions ViewModel Logic`**: The `ChatViewModel` will need methods to handle model/settings selection and update the session. ### ⏱️ Effort Estimate * **Medium (M)** ### 💡 Potential Implementation Notes * Consider using Compose's `DropdownMenu` or a custom dialog for selecting models and settings. * The API call to update the session should be a `PUT` request to `/api/v1/sessions/{sessionId}/model` and `/api/v1/sessions/{sessionId}/settings` or a single `PUT` to `/api/v1/sessions/{sessionId}/details` encompassing both. * The `ChatSessionTable` will need to store `currentModelId` and `currentSettingsId` as nullable foreign keys. Remember to set `ON DELETE SET NULL` for these foreign keys in the Exposed schema (`E7.S4`).