⚡ Technical Documentation

1. Current State: The Foundation 🏗️ (Where We Are Now)

Technical guide covering **1. current state: the foundation 🏗️ (where we are now)**

👤
Author
Cosmic Lounge AI Team
📅
Updated
6/1/2025
⏱️
Read Time
15 min
Topics
#llm #ai #model #fine-tuning #gpu #docker #api #server

📖 Reading Mode

📖 Table of Contents

⚡ Mount-E Chat Evolution Roadmap 🚀

(An ever-evolving journey towards secure, agentic AI within the RCMP)

This roadmap outlines the strategic direction for the Mount-E Chat AI system, detailing our current capabilities, ongoing development efforts, and future aspirations. Our focus remains steadfast on leveraging the power of AI, particularly Local LLMs and the Model Context Protocol, within a secure, on-premises environment to enhance RCMP operations, directly informed by user needs and technical readiness.

🌌 1. Current State: The Foundation 🏗️ (Where We Are Now)

We’ve established a robust and secure base for our AI capabilities, prioritizing data control and compliance.

🌟 System Architecture:

Underpinning everything is a secure, on-premises deployment model.

  • Frontend: OpenWebUI serves as the user-friendly “Mount-E Chat” interface, customized for RCMP workflows. 🖥️

  • Backend: Ollama is the engine, chosen for its ability to run LLMs locally and support structured outputs like JSON. ⚙️

  • Containerization: We’re utilizing Docker and Docker Compose for deploying and managing OpenWebUI and Ollama. 🐳

🌟 Hardware:

  • Currently operating on a Debian 12 workstation powered by an NVIDIA RTX 4090 GPU. 💻

  • A significant upgrade to a more powerful server system, likely featuring an H100 setup with redundancy, is planned within the next year. 🚀

🌟 Key Capabilities & Drivers:

  • Leveraging open-source pre-trained LLMs, including smaller, domain-specific models. 🧠

  • Initial success in custom document generation based on RCMP guidelines. 📄

  • Implementation of standardized JSON output formats using OpenWebUI filters for consistent reporting. 📊

  • Exploration into multimodal AI, specifically vision (handwriting/evidence recognition, basic image generation). 👁️

  • Established a model evaluation framework using DeepEval, initially for summarization and JSON output compliance. ✅

  • Driven by critical needs for enhanced data security, privacy, regulatory compliance, customization, and performance. 🔒

🌌 2. Ongoing Development: Building Momentum 📈 (Near-Term / Mid-Term Focus)

Our current efforts are focused on refining existing functionalities, integrating new capabilities, and preparing specific tested solutions for wider deployment.

🌟 Current Needs for Implementation (Immediate/Near-Term):

  • Setting up Granite models for Creator models. 🛠️

  • Deploying the tested Creator models (Briefing Notes, Info Notes, Broadcast) from staging to the live system for the general group. 🚀

  • Implementing the tested RAG configuration on the live system, using the mxbai-rerank-large-v1 model with a TopK of 10 and MinScore of 0.5. (> ⚠️ Note: This will require redoing the current knowledge base). 🔄

  • Preparing the working test of the Synopsis of Reports to Crown Council for Major Crime and IHIT for implementation/pilot. 🧪

🌟 Infrastructure Enhancement (Near-Term):

  • Implementing the planned H100 server infrastructure upgrade to boost computational power and reliability, projected within about a year. ⚡

🌟 Core AI Advancement (Near-Term):

  • Continuing research and experimentation with advanced LLM fine-tuning techniques tailored for specific RCMP tasks using custom datasets. 🔬

  • Optimizing the RAG pipeline for secure knowledge retrieval from RCMP sources, exploring embedding models and re-ranking techniques. 🔗

  • Expanding DeepEval with more RCMP-specific tests and developing a user-friendly evaluation interface. 📊

🌟 Capability Integration (Mid-Term):

  • Prototyping Agentic AI through task-specific web applications via the OpenWebUI API to streamline RCMP workflows. 🤖

  • Actively investigating and researching the incorporation of the Model Context Protocol (MCP) to enable standardized interaction with external tools and data sources, particularly for enabling local intranet search capabilities. 🌐

🌟 Governance & Collaboration:

  • Maintaining ongoing, direct collaboration with RCMP divisions to ensure the system is user-centric and meets operational needs.🤝

  • Prioritizing ethical, legal, and privacy considerations throughout development and deployment. ⚖️

🌌 3. User and Tester Feedback: Guiding Our Evolution 🗣️

Direct feedback from current users and testers is invaluable in shaping the direction of Mount-E Chat, highlighting both the challenges in understanding the technology and the most impactful desired capabilities.

🌟 Challenges in Understanding & Adoption:

  • General confusion among most users regarding the full potential and capabilities of the technology. 🤔

  • The broad functionality of the base chatbot is often overwhelming for users. 🤯

🌟 Current High-Impact Use Cases (Power Users):

  • Summarization of documents. 📝

  • Sorting information within documents. 🗂️

  • Information hunting within documents (especially via drag-and-drop). 🔍

🌟 Most Frequent & Desired Requests:

  • Creation of Standard Operating Procedures (SOPs). 📜

  • Creation of documents in Word and PowerPoint formats. 📊

  • Template creation and template fill-out capabilities (“doing their paperwork”). 📋

  • Automation of complex, multi-part tasks (“doing their job”). 🤖

🌟 Specific Requested Capabilities:

  • Briefing Notes generation (now moving to implementation). 📄

  • Statement analysis. 🗣️

  • Video editing (aspirational, likely multimodal). 🎬

  • Enhanced Speech-to-Text (STT). 🎤

  • Policy searching. 🔍

  • General knowledge searching (including local intranet). 🌐

🌟 Understanding Gap:

  • Many user groups currently lack a clear understanding of how the technology can specifically benefit their daily tasks and workflows. 🤷

🌌 4. Future Vision: Agentic Evolution ✨ (Long-Term Aspirations)

Looking ahead, we envision Mount-E Chat evolving into a sophisticated, autonomous AI system deeply integrated into RCMP operations, directly addressing the needs and opportunities identified.

🌟 Advanced Agentic Frameworks:

  • Building sophisticated AI agents capable of executing complex, multi-step workflows securely within the RCMP environment, coordinating tasks across various internal systems, including automating paperwork and complex operational procedures as requested by users. 🤖

🌟 Seamless Data & Tool Interaction:

  • Implementing advanced RAG systems that dynamically query diverse internal data sources (databases, file systems, legacy systems) through standardized MCP tools, enabling comprehensive policy and knowledge searching. 🔗

  • Enabling powerful natural language querying and analysis across federated internal data silos, using MCP as an abstraction layer to navigate complexity. 🌐

🌟 Transformative Operational Impact:

  • Developing secure internal chatbots and assistants integrated with enterprise systems (HR, IT) via MCP to streamline employee support. 💬

  • Creating AI agents for automated compliance checks, report generation, and document creation (including SOPs and templates) that pull information from internal sources via MCP. 🤖

  • Enabling proactive and personalized services for both RCMP personnel and potentially citizens, leveraging secure access to internal data. 🤝

  • Providing enhanced support through agents that can handle complex queries by accessing internal knowledge and interacting with backend systems via MCP. 🙋

  • Optimizing resource allocation and efficiency through AI-powered analysis and planning. 📈

  • Strengthening internal communication and collaboration within RCMP teams. 🗣️

🌟 Addressing Specific User Needs:

  • Developing specialized agentic flows and models for tasks like Statement Analysis and Synopsis of Reports to Crown Council, building on current successes. 🤖

  • Investigating and potentially implementing capabilities like enhanced Speech-to-Text and exploring the feasibility of complex tasks like video editing within the secure framework. 🎤

🌟 User Empowerment & Education:

  • Focusing on strategies to improve user understanding of the technology’s capabilities and how it can address their specific needs and automate tasks. 🎓

🌟 The Human-AI Partnership:

  • Realizing a collaborative “digital workforce” where human officers focus on strategic and interpersonal tasks, augmented by efficient AI agents that handle routine and complex paperwork/information retrieval. 🤝🤖
Back to Knowledge Base Explore More Knowledge