In an on-premises deployment, Ozgar is installed in the customer’s own data center or server environment. All application components and data reside inside the organization’s firewall, and the customer’s IT team manages the infrastructure. The Large Language Model (LLM) integration can either be run on-premises (within the firewall on local hardware) or invoked via a cloud API from the on-prem installation. On-prem deployments are typically chosen by institutions with highly sensitive data and strict compliance rules.

Pros:

  • Data residency and control: All data and processing remain on the customer’s internal servers, meeting stringent regulatory and security requirements. Ozgar’s architecture supports the customer’s regulatory and security policies; the customer remains responsible for defining and enforcing those controls (e.g. provisioning encrypted volumes).
  • Isolation: No external dependencies if an on-prem LLM is used, which can be important for privacy.
  • Custom infrastructure: Full control over infrastructure and network configuration to meet internal policies.

Cons:

  • Longer deployment timeline: Setting up on-prem infrastructure and (if applicable) an on-prem LLM can take several weeks (~3–4 weeks for full deployment).
  • Resource and maintenance burden: The customer must provision and manage the necessary hardware (compute, storage) and handle ongoing maintenance, updates, and scaling internally.
  • Scaling limitations: Expanding capacity or upgrading components may be slower due to procurement and internal change processes.

 

 

Hardware and Infrastructure Requirements

 

For a typical production deployment of Ozgar on Linux x86_64/ARM, we recommend a baseline server (or VM) provisioned with 8 vCPU cores and 24 GB of RAM. This capacity easily accommodates Ozgar’s containerized components — including its backend services, database, and indexing modules — under normal workloads.

Ozgar is packaged as Kubernetes deployments and must be run within a Kubernetes cluster on Linux x86_64/ARM nodes. The deployment environment should have network connectivity to the required data sources (e.g. source code repositories, databases, document stores) and, if using a cloud-hosted LLM API, outbound internet access for connecting to that service. If an on-premises LLM is used (instead of a cloud LLM API), additional hardware will be needed: specifically, one or more GPU-enabled servers capable of hosting the chosen LLM model. The size and number of GPU machines will depend on the model (for example, hosting a large transformer model may require multiple high-memory GPU nodes). This is only necessary if the organization opts to keep the LLM completely on-prem; in all other cases, Ozgar can integrate with an external LLM service without specialized AI hardware on the client side.

Recommended Hardware for On-Premises LLM Hosting

  • GPU Accelerator
    • NVIDIA RTX PRO 6000 with 96 GB of GPU memory (sufficient for models up to 32 billion parameters)
  • CPU & System Memory
    • 8 vCPUs dedicated to data preprocessing, batching, and GPU workflow orchestration
    • 64 GB RAM (minimum)
  • Local Storage
    • 1 TB enterprise-grade NVMe SSD
    • ≥ 1 500 MB/s sustained I/O
  • Encryption
    • AES-256 at rest (e.g. LUKS on Linux) with keys managed via your KMS
  • Platform Support
    • Linux x86_64/ARM

 

 

Implementation Phases

The following are the typical implementation phases for deploying Ozgar AI, from project initiation through go-live and support:

  1. Kickoff & Access: Conduct a project kickoff meeting with stakeholders (IT managers, architects, DevOps, and business sponsors) to align on objectives and the deployment approach. In this phase, the target environment is defined (on-premises or cloud), including network zones and security requirements. Key use cases for Ozgar are identified (e.g. codebase documentation generation, Q&A, modernization guidance) to focus the deployment scope. The team also gathers required access credentials for relevant systems – such as read access to source code repositories, database schemas, and any documentation portals – so that Ozgar can ingest these sources.

  2. Container Deployment: Set up the Ozgar application in the chosen environment by provisioning the necessary container infrastructure. This involves preparing Docker or Kubernetes clusters as needed and deploying the Ozgar backend containers onto this infrastructure. Network configurations are applied (for example, ingress rules, service mesh or proxy settings) to allow user access to the application within the organization’s network.

  3. Ingestion & Indexing: Configure Ozgar’s connectors to ingest data from the organization’s sources and build its knowledge index. This includes connecting to source code repositories (for instance, AS/400 code libraries or Git repositories) and to relevant databases (such as AS/400 or other databases via JDBC) to extract schema and metadata. Additionally, connectors for other documentation sources (wiki pages, ticketing systems, etc.) can be set up if those are in scope. Once connectors are configured, Ozgar ingests the content and creates the search indices – including vector embeddings for semantic search, full-text indexes for documentation, and any graph or metadata indices required.

  4. LLM Agent Setup: Integrate and configure the AI language model that powers Ozgar’s Q&A and guidance capabilities. Depending on the deployment choice, this means connecting Ozgar to an enterprise LLM service – either an on-premises model deployment or a cloud LLM API (such as an OpenAI or Azure OpenAI service). By the end of this phase, the AI agent is operational and able to answer questions using the indexed knowledge base (e.g. developers can ask about specific programs and get answers generated by the LLM with references to the ingested code/docs).

  5. User Management Setup: Configure user accounts and roles within Ozgar in preparation for go-live. If Ozgar has its own user management, the admin team will set up the user directory, define roles/permissions (for example, who can administer the system vs. who can query and view documentation), and map users to their roles. The rollout plan for users is also finalized – determining how the platform will be introduced to users (e.g. starting with a pilot group then expanding) and ensuring support materials or training are ready for those users.

  6. Go-Live & Monitoring: Launch the Ozgar platform to all intended users and begin production use. This involves announcing the go-live to the user community and providing them with the necessary information to access the system. Once live, the customer team closely monitors the platform’s metrics and usage: for example, tracking the number of queries and requests, success rates of answers, system performance, CPU/memory usage of the containers. For on-premises deployments, the Ozgar team provides a runbook and tooling to the customer’s IT staff to aid in ongoing monitoring and operations. This ensures the customer can handle typical maintenance tasks and understand the system’s operational profile.

  7. Support & Maintenance: After go-live, the focus shifts to ongoing support and continuous improvement of the solution. The Ozgar team will provide software updates and security patches as they become available, along with instructions or automated scripts (runbooks) for applying these updates. The customer is responsible for executing these updates and maintaining the installation according to their own procedures.

 

 

Customer Responsibilities

Successful deployment of Ozgar requires certain actions and resources to be provided by the customer team. The key customer responsibilities include:

  • Provide access to code, database catalog and existing documentation: Supply the Ozgar deployment team with read-access to relevant source code repositories, database catalogs, and any existing documentation. This access (arranged by the customer’s IT/DevOps teams) is needed so that Ozgar can ingest the source code and related materials.

  • Approve hosting and network setup: Review and approve the chosen deployment model (on-premises or public cloud) and ensure the necessary network configurations (firewall rules, VPN connectivity, etc.) are in place. This task is typically handled by an IT manager or infrastructure team lead, since it involves aligning with the customer’s IT policies.

  • Assign a technical point of contact: Designate a technical lead from the customer’s side to coordinate with the Ozgar team. This person (often a project sponsor or lead architect) will handle onboarding tasks, facilitate communications, and ensure internal prerequisites are met. They will be the go-to person for scheduling access, clarifying requirements, and receiving progress updates during the project.

  • Define and enforce regulatory and security policies: The customer remains responsible for their own compliance controls; Ozgar provides the tooling to support these policies.

  • Data retention & backup: Implementing backup policies e.g. using Kubernetes volume snapshots or regular database dumps; definition of retention periods (e.g., 30/60/90 days); and maintain a recovery runbook for accidental node or pod deletions or data loss scenarios.

 

 

Ozgar AI Deliverables

As part of the deployment, the Ozgar team will provide the following deliverables and services to ensure a successful implementation:

  • Containerization & LLM Integration: Prebuilt Ozgar Docker/Kubernetes container images are provided, along with deployment scripts or Helm charts as needed for easy setup. The team also assists in configuring the connection to the customer’s chosen LLM service – whether that is an on-premises model deployment or a cloud-based LLM API – ensuring the AI component is properly integrated into the Ozgar platform.
  • Prepared Knowledge Base: The Ozgar team will configure the knowledge base for the customer’s environment, setting up connectors for the source code repositories and databases, and prepare initial indices. This includes ensuring that the source code and database metadata are ingested and indexed properly so that Ozgar can utilize them for Q&A and documentation generation. (If some connectors or content types are not available in the current version, the team will work with the customer on possible workarounds or include them when those features become available.)
  • Admin Setup Support: Direct assistance is provided in setting up initial user accounts, teams, roles, and project boundaries within the Ozgar application. The Ozgar team will guide the customer’s administrators through the configuration of role-based access control and multi-project setups (if the organization wants to segment the knowledge base by project or department). This one-time setup support helps align the tool with the customer’s user management policies from the start.
  • End-User Playbook: A comprehensive end-user guide (playbook) is provided, containing example queries, usage tips, and FAQs tailored to the customer’s use cases. This documentation serves as a reference for developers and other end users on how to use Ozgar AI (e.g. how to phrase questions in chatbot, tailoring documentation pages), and it provides best practices to get the most out of the platform. The playbook helps accelerate user adoption and addresses common questions that may arise post-launch.