3. Technical Overview

System Architecture

RangerIO is built on a modern, layered architecture designed for performance, security, and maintainability.

High-Level Architecture

Architecture Principles

Offline-First Design
- No required network connectivity
- All processing happens locally
- Optional model downloads during setup

Modular Components
- Loosely coupled services
- Independent scaling
- Easy to maintain and extend

Hardware Agnostic
- Runs on CPU-only systems
- Automatically leverages available GPUs
- Optimizes based on detected hardware

AI & Machine Learning

Local Language Models

RangerIO supports multiple open-source language models optimized for local execution.

Model Comparison

Quantization Explained

Models are available in different quantization levels:

Q4 (4-bit)
- Smallest size (~25% of original)
- Fastest inference
- Slight quality trade-off
- Recommended for: Quick tasks, low-memory systems

Q5 (5-bit)
- Balanced size (~31% of original)
- Good performance
- Minimal quality loss
- Recommended for: General use

Q8 (8-bit)
- Larger size (~50% of original)
- Slower inference
- Highest quality
- Recommended for: Maximum accuracy, high-memory systems

Inference Engine

The local inference engine provides:

Optimizations:
- Dynamic batching
- KV cache management
- Memory-efficient attention mechanisms
- Context window management

Hardware Acceleration:
- Automatic GPU detection and utilization
- Mixed precision (FP16, FP32)
- Tensor core optimization (NVIDIA)
- Metal Performance Shaders (Apple)

Performance Features:
- Streaming responses for better UX
- Parallel inference for batch operations
- Intelligent layer distribution across CPU/GPU

RAG (Retrieval-Augmented Generation)

The RAG system enhances LLM responses with relevant context from your data.

How RAG Works

Three-Phase Process:

1. Ingestion Phase
2. Documents are split into semantic chunks
3. Each chunk is embedded with a local ONNX embedding model (fastembed) using binary quantization to fit consumer hardware

4. Vectors are indexed in the vector database

5. Retrieval Phase

6. User question is converted to a query vector
7. Similar chunks are retrieved using cosine similarity

8. Context is ranked by relevance score

9. Generation Phase

10. Top-ranked chunks are added to the LLM prompt
11. LLM generates contextually-aware answer
12. Sources are automatically cited in the response

RAG Enhancements

Contextual Embeddings
- Chunks include surrounding context
- Better retrieval accuracy
- Reduced false positives

Hybrid Search
- Combines semantic (vector) search
- With keyword (BM25) search
- Best of both approaches

Reranking
- Secondary relevance scoring
- Improves result quality
- Reduces noise in context

Query Expansion
- Generates related queries
- Improves recall
- Better coverage of information

Embedding Models

Used for vector representation of text:

fastembed (ONNX)
- Local, dependency-free embedding
- Binary quantized vectors to stay within consumer-hardware memory budgets
- BM25 keyword retrieval used alongside vector search (hybrid)

Data Processing Architecture

Multi-Database Strategy

RangerIO uses specialized databases for different workloads:

Metadata Store
- Lightweight embedded database
- Stores project configurations
- User preferences and settings
- Import history and lineage

Analytics Engine
- Columnar database for performance
- Handles large-scale aggregations
- SQL query execution
- Multi-format data loading

Vector Database (LanceDB)
- Embedded, file-based vector store
- Efficient nearest-neighbor queries with binary quantization
- Hybrid (vector + BM25) search
- Incremental updates per document

Data Flow

Complete data ingestion and processing pipeline:

Processing Stages:

1. Import - Multi-format data loading
2. Parsing - Format detection and extraction
3. Validation - Schema and quality checks
4. Profiling - Statistical analysis and metadata
5. Storage - Distributed across specialized databases
6. Embedding - Vector generation for AI queries
7. Indexing - Fast retrieval preparation

Hardware Optimization

Automatic Hardware Detection

On startup, RangerIO detects:
- CPU architecture and capabilities
- Available RAM and disk space
- GPU vendor and model
- Compute capabilities (CUDA cores, Metal support)

GPU Acceleration

NVIDIA GPUs (CUDA)
- Automatic CUDA detection
- Tensor core utilization
- Multi-GPU support
- Dynamic memory management

AMD GPUs on Windows (Vulkan)
- Cross-compiled Vulkan backend
- Works on x86_64 without AMD-specific runtimes
- Memory-efficient operations

Apple Silicon (Metal)
- Native M1/M2/M3+ support
- Unified memory architecture
- Power-efficient inference

CPU Fallback
- Always available on every supported platform
- Auto-selected when no compatible GPU is detected

Memory Management

Dynamic Allocation:
- Monitors available RAM
- Adjusts batch sizes automatically
- Prevents out-of-memory errors

Model Loading:
- Loads only required layers
- Offloads inactive models
- Efficient model switching

Context Management:
- Sliding window for long contexts
- Automatic truncation when needed
- KV cache optimization

Security Architecture

Privacy by Design

No telemetry, no analytics, no phone-home:
- No usage statistics, no license validation calls, no auto-update probes
- Crash reporting is opt-in and local-only

Governed cloud handoff:
- The Stage 2 connector is opt-in; disable it and RangerIO is fully offline
- When enabled, sensitive values are masked before any cloud round-trip and restored when the answer returns
- The mapping between masked stand-ins and real values lives only on your device

Data Isolation:
- Each project in a separate directory
- No cross-project data sharing
- All raw rows, dossiers, and PII overrides stored locally

Data Protection

At-Rest:
- Pair with OS-level full-disk encryption (FileVault, BitLocker)
- Secure file permissions on the project directory

In-Transit:
- All backend communication on localhost
- Cloud handoff (when enabled) uses TLS to the chosen provider's endpoint

Performance Characteristics

Throughput

Data Ingestion:
- CSV: ~100MB/sec
- Excel: ~50MB/sec
- JSON: ~80MB/sec
- Database: Network dependent

AI Inference:
- Varies by model, hardware, and quantization
- Smaller GGUF models (1-3B) typically run multiple tokens/sec on CPU; significantly faster on GPU or Apple Silicon
- Cloud-routed responses via the Stage 2 connector are limited by network latency and provider speed

Vector Search:
- ~1-5ms latency for queries
- Handles millions of vectors
- Sub-second response times

Scalability Limits

Data Volumes:
- Files: Up to available RAM (typically 10GB+)
- Databases: Query-based, no limit
- Vector store: Millions of documents

Concurrent Users:
- Single-user desktop application
- One active session at a time

Technology Stack

Core Technologies

Backend:
- RESTful API server
- Async request handling
- WebSocket support for streaming

Frontend:
- Modern JavaScript framework
- Component-based architecture
- Real-time updates

Data Processing:
- High-performance data libraries
- Parallel processing
- Streaming operations

AI/ML:
- Local inference engines
- Vector similarity libraries
- NLP preprocessing

Deployment

Electron Application:
- Cross-platform desktop app (macOS and Windows)
- Bundled runtime environment
- Manual updates only — no auto-update, no phone-home

Development Mode:
- Separate FastAPI backend (port 9000) and Vite frontend (port 5173)
- Hot reload for development
- Debug logging

Extensibility

Model Support

Current local: Qwen3, Qwen3.5, Phi-3.5-mini, DeepSeek-R1 distilled, and any other GGUF-compatible llama.cpp model

Cloud: Claude Desktop via the Stage 2 connector today; ChatGPT and Perplexity in named phases (Stage 2.1 and 2.2)

Data Formats

Current: 15+ formats

Extensible: Plugin architecture for custom formats

Integrations

Current: REST APIs, databases

Roadmap: GraphQL, gRPC, message queues