Project Overview

QaptaanLM-0.75B is an efficient, compact hybrid-attention foundation language model engineered for localized source code generation, technical comprehension, and long-context reasoning. By stripping the vision transformer components from base architectures down to 752M dense parameters (Qwen3_5ForCausalLM / QaptaanForCausalLM), the model underwent full-parameter Continued Pre-Training (CPT) across 1,000,013,824 tokens of KapCode-1B on Google TPU v5e-8 accelerator pods, achieving 12,485 tokens/sec throughput and rapid loss convergence from 12.8064 to 2.4413.

Architecture & Features

• Hybrid Linear Attention Backbone: 24 decoder layers arranged in 6 repeating macro-blocks combining 18 Gated DeltaNet linear attention layers with 6 Gated Grouped-Query Attention (GQA) layers (8 query heads / 2 KV heads, 256 head dim), delivering $O(N)$ linear computational/memory scaling while preserving associative multi-hop reasoning.
• Multi-Session Distributed TPU CPT: Successfully executed 61,036 optimization steps on Google TPU v5e-8 (8 pod cores, 128 GB HBM) across 3 chained sessions with asynchronous Orbax checkpointing, achieving an average throughput of 12,485 tok/s (0.762 steps/s) and minimum cross-entropy loss of 1.5911.
• Exact Weight Alignment & Clean Safetensors Export: Authored custom configuration_qaptaan.py and modeling_qaptaan.py with fast $O(1)$ single-token caching, exact recurrence alignment, and tied word embeddings (tie_word_embeddings=True), exporting a verified 752M-parameter (1.50 GB) PyTorch Safetensors release.
• Fill-in-the-Middle (FIM) Infilling: 50% of code tokens trained with Prefix-Suffix-Middle (PSM) formatting using native delimiters (<|fim_prefix|>, <|fim_middle|>, <|fim_suffix|>) to empower zero-shot IDE code completion.
• Liger Kernel Loss Fusion: Custom Triton fused Cross-Entropy loss computation directly paired with vocabulary projection, reducing backward pass VRAM by 40%–60% and accelerating training throughput by 15%–20% on NVIDIA GPUs.
• Stage 2 SFT Pipeline: Established Supervised Fine-Tuning architecture targeting 100M tokens on KapInstruct-100M with Qwen ChatML formatting and assistant-only loss masking (labels = -100 on prompts).

Links & Resources

• Fine-tuned Huawei's Pangu-Weather model on approximately 3.5 years of IMD GFS data (2023-2026), adapting a globally pre-trained weather forecasting model for regional meteorological prediction.
• Developed preprocessing pipelines for large-scale meteorological datasets, including GRIB-to-NetCDF conversion, variable transformations, humidity parameter handling, dataset validation, and quality assurance.
• Optimized training and inference workflows, reducing training time from approximately 20 minutes to 8 minutes per epoch through code, data-loading, and distributed training improvements.
• Implemented auto-regressive forecasting pipelines capable of generating multi-day forecasts by recursively feeding model outputs back as inputs.
• Built model evaluation workflows to compare forecasts against observed weather data and existing operational forecasting systems across multiple variables, pressure levels, and forecast horizons.
• Developed geospatial visualization and analysis pipelines for meteorological variables using GIS datasets and weather forecasting outputs.

Links & Resources

• Contributed to the development of an internally implemented lightweight weather forecasting model trained on more than 40 years of ERA5 reanalysis data (1979-2024).
• Resolved critical training issues and refactored fragmented codebases into unified production-grade training and inference pipelines.
• Implemented Distributed Data Parallel (DDP) training using NCCL and developed PBS job scripts for efficient multi-GPU, multi-node execution on IMD's Arunika High Performance Computing cluster.
• Reduced training time from approximately 32 hours to 5 hours per epoch through distributed computing, pipeline optimization, and infrastructure improvements.
• Built end-to-end training, inference, validation, checkpointing, experiment tracking, logging, and forecasting workflows for large-scale weather prediction experiments.
• Contributed to architectural experimentation involving patch embeddings, ensemble inference, error-correction approaches, and variable cross-attention mechanisms for weather forecasting models.

Links & Resources

Project Overview

InferenceOS is an open-source, hardware-aware operating system and control plane designed for local Large Language Model (LLM) inference. Built on an architectural fork of llama.cpp, it bridges high-performance C++ tensor execution kernels with intelligent Python runtime orchestration to dynamically profile hardware, optimize layer placement, execute live hysteresis migrations under VRAM pressure, compress KV caches, and stream responses via dual OpenAI/Ollama REST APIs.

Features

• Hardware Auto-Profiler: Multi-vendor detection engine profiling CPU, NVIDIA CUDA, AMD ROCm, Apple Metal, Intel Arc, and integrated GPUs to auto-generate optimal execution plans.
• Dynamic Microbatch Scheduler: Real-time prefill batching using multi-heuristic scoring (VRAM headroom, prompt TPS, GPU occupancy, PCIe bandwidth) to eliminate out-of-memory crashes.
• Live Hysteresis Layer Migration: Real-time offload coordinator that dynamically migrates layers between dGPU, iGPU, and CPU during active inference under VRAM pressure.
• KV Cache Manager & Eviction: Memory optimizer supporting FP16/Q8_0/Q4_0 cache quantization and attention-sink eviction policies (H2O Heavy-Hitter Oracle, StreamingLLM, LRU, FIFO).
• Runtime Learning Database: Persistent SQLite telemetry recorder that tracks execution metrics (TTFT, prompt t/s, evaluation t/s) and predicts optimal model configurations via ML regression models.
• Dual REST API Server & 21-Command TUI: High-throughput FastAPI HTTP server supporting OpenAI (/v1/*) and Ollama (/api/*) endpoints with streaming SSE, accompanied by a 21-command interactive TUI CLI.

Links & Resources

Project Overview

KapCode-1B is a high-quality, 1-billion-token curated dataset designed for full-parameter Continued Pre-Training (CPT) and domain adaptation of compact Large Language Models (<1B parameters). Engineered to maximize signal density and syntactic correctness, KapCode-1B combines multi-lingual code, architecture documentation, function-level snippets, STEM web text, and mathematical proofs packed into 4096-token sequence shards.

Dataset Composition & Engineering

• Rigorous Domain Mixture: Balanced across 5 upstream partitions — 35% Stack v3 Code (350M tokens), 20% Stack v3 Docs (200M tokens), 20% The Vault Function-level code (200M tokens), 15% FineWeb-HQ (150M tokens), and 10% OpenWebMath (100M tokens).
• Multi-Language Code Distribution: Targeted coverage across 13 languages including Python (25%), TypeScript (13%), JavaScript (10%), SQL (9%), C++ (7%), Shell/Bash (6%), C (5%), Java (5%), HTML (5%), Rust (4%), Go (4%), CSS (4%), and Dockerfile/IaC (3%).
• Multi-Stage Filtration Pipeline: SHA-256 exact content deduplication, FastText language identification (en >= 0.70), alphanumeric density gating (>25% for code, >50% for docs), boilerplate removal, and line/character count bounds.
• Fill-in-the-Middle (FIM) Transformation: 50% of all code samples converted into Prefix-Suffix-Middle format with native special tokens for code infilling mastery.
• High-Throughput Sharded Distribution: Distributed as 244,140 sequence-packed shards (4096 tokens per sequence) formatted in memory-mapped Apache Arrow (.arrow) and Parquet (.parquet) for zero-copy streaming data loading.

Links & Resources

Project Overview

KapInstruct-100M is a high-fidelity, 100-million-token instruction-tuning dataset engineered for Supervised Fine-Tuning (SFT) and alignment of compact Large Language Models (<1B parameters). Formatted strictly with the Qwen ChatML schema (<|im_start|> / <|im_end|>) and tokenized using Qwen/Qwen3.5-0.8B-Base, the dataset enforces strict assistant-only loss masking—masking user prompts, system headers, and formatting delimiters to labels = -100—ensuring 100% of gradient updates focus directly on reasoning, syntax precision, and answer generation.

Dataset Composition & Engineering

• 12-Source Balanced Mixture Across 6 Domains: Code Generation (31% — Magicoder-Evol 13M, Magicoder-OSS 8M, Self-OSS StarCoder2 5M, SmolTalk Constraints 3M), General Reasoning (27% — Smol-Magpie-Ultra 18M, OpenHermes-2.5 9M), Mathematics CoT (17% — OpenMathInstruct-2 11M, NuminaMath-CoT 6M), STEM QA & Science (11% — OpenThoughts-114k 7M, WebInstructSub 4M), Code Debugging & Repair (10% — CodeFeedback-Filtered 10M), and High-Fidelity Instruction Following (6% — Tulu-3 SFT 6M).
• Assistant-Only Loss Masking Architecture: Token-level alignment where system prompts, user turns, and structural boundaries have labels = -100. In multi-turn conversations, loss is computed strictly across assistant response spans, maximizing gradient efficiency for compact sub-1B models.
• Deficit-Weighted Balanced Scheduling: Dynamic deficit-driven sampling scheduler that measures exact tokenizer tokens post-filtering, guaranteeing precise representation across all 12 domains without starvation or over-representation.
• Quality Filtering & Sanitization: FastText English identification (min_confidence = 0.65) with code-mixed technical dialogue preservation, 16-language canonical programming alias normalization, automated regex secret/API key redaction (sk-, ghp_, RSA keys), LaTeX syntax validation, and cross-source SHA-256 deduplication.
• Zero-Copy Memory-Mapped Loading: Delivered in sequence-packed 4096-token Arrow IPC (.arrow) and Parquet (.parquet) shards for zero-copy streaming directly into PyTorch SFT training loops.

Links & Resources

Project Overview

myCloud is a full-stack, high-performance distributed personal cloud platform engineered for the mySphere ecosystem. It decouples file metadata from physical storage, allowing users to connect their own PCs and servers as distributed storage nodes ("Smart Relay") to build a self-hosted cloud with centralized control, real-time synchronization, enterprise-grade security, and multi-interface access.

Features

• Multi-Client Access: Responsive Web PWA, CustomTkinter Desktop Node GUI, Python CLI (mycloudctl), official Python SDK (mycloud-sdk), and cross-platform Electron/Capacitor wrappers.
• Distributed Storage Nodes: Connect private PCs or servers as storage nodes over WebSockets ("Smart Relay") with MySQL node routing.
• Advanced Sharing Suite: Direct user invites with RBAC permissions, public token link generation, and batch share links grouping disparate files/folders.
• Stash Lifecycle: Temporary file isolation workflow with scheduled auto-cleanup (10m, 1h, 1d, or custom intervals).
• Authentication & Security: Google/GitHub OAuth, 2FA challenge flows, CSRF protection, HttpOnly/SameSite session security, and bcrypt hashing.
• Observability & Monitoring: Built-in internal dashboards monitoring system CPU/RAM, Nginx/Gunicorn service health daemons, and product analytics.
• Infrastructure & Hosting: Production-grade deployment on Oracle Cloud Infrastructure (OCI) using Nginx reverse proxy and Gunicorn Eventlet async worker processes.

Links & Resources

Project Overview

The Sign Language Interpreter is a multimodal, client-side American Sign Language (ASL) recognition, sensor fusion, and assistive translation platform. By fusing browser-based computer vision tracking with IoT Smart Glove kinematics, the system overcomes traditional monocular occlusion, motion blur, and depth ambiguity during rapid signing to provide seamless, real-time vocalized translation.

Features & Technical Highlights

• 21-Landmark Computer Vision Pipeline: Google MediaPipe Hands captures 21 3D hand keypoints at 30–60 FPS, normalized relative to the wrist into 42 spatial features for rotation and distance invariance.
• On-Device Deep Neural Network: 4-layer client-side Sequential DNN (Dense(256) -> Dense(128) -> Dense(64) -> Dense(26)) trained on 36,400+ samples and cached in IndexedDB for instant, zero-latency local classification.
• IoT Smart Glove Telemetry: Custom wearable glove using an ESP32 microcontroller and MPU-6050 6-axis IMU broadcasting roll and pitch rotational kinematics over Web Bluetooth (BLE) at 50 Hz.
• Heuristic Sensor Fusion Engine: Evaluates a sliding 10-frame IMU kinematic buffer to disambiguate dynamic, motion-dependent signs:
  • 'J': Detects pitch ramp and wrist scoop while holding 'I'.
  • 'Z': Detects dynamic index-finger pitch oscillation while holding 'D'.
  • "YES" & "HELLO": Detects dynamic fist nodding and outward salute sweeps.
  • [SPACE]: Detects horizontal flat palm gesture.
• Hybrid Active Learning (k-NN Overrides): On-the-fly calibration allowing users to capture difficult or custom hand shapes in the browser to override baseline model predictions without neural network retraining.
• Assistive Sentence Builder & Vocalization: Real-time letter stream with double-letter reset debounce, 880 Hz Web Audio confirmation chimes, browser SpeechSynthesis voice output, and cloud translation into 14 languages (Tamil, Hindi, Telugu, Spanish, French, German, Japanese, and more).
• Multimodal AI Gesture Chatbot: Integrated Puter.js GPT-4o Vision assistant for interactive gesture analysis, practice validation, and sign language learning.

Links & Resources