# inference.eng (inferenceeng.eu) — LLM Inference Engineering Consulting

> Principal LLM Inference Engineering Consultancy led by Arinze Obidiegwu (MSc AI & ML First Class Honours, GECCO 2026 Humies Bronze Medalist), based in Ireland and serving European and global engineering teams. Specializes in cutting production LLM inference latency and cloud GPU spend by 30–60% via vLLM, SGLang, TensorRT-LLM, llama.cpp, quantization (AWQ, FP8, INT4, GGUF), custom CUDA/Triton kernels, and pharma-grade regulatory compliance (GAMP5, 21 CFR Part 11, EU AI Act, GDPR).

## Core Services & Engagement Tiers
- **01 · Inference Audit (1–2 Week Sprint):** Roofline & NSys/NCU profiling of existing LLM deployments, P50/P90/P99 TTFT and ITL bottleneck identification, and a prioritized optimization roadmap with projected GPU cost and latency reductions.
- **02 · Optimization Build (4–12 Week Project):** Hands-on production implementation—engine migration (vLLM, SGLang, TensorRT-LLM), quantization (FP8, AWQ-INT4), PagedAttention/RadixAttention prefix caching, Triton/CUDA kernel tuning, and Prometheus/Grafana observability.
- **03 · Fractional Principal Inference Engineer (3–12 Month Retainer):** Embedded senior inference architect (2–3 days/week) for ongoing capacity planning, new model rollouts, and regulated-industry AI compliance without the €150K+ cost of a full-time hire.

## Regulated Industry Expertise (Europe & Global)
- **Compliance Frameworks:** GAMP5, FDA 21 CFR Part 11, EU AI Act, and GDPR sovereign/on-premise data residency.
- **Target Sectors:** Pharmaceutical manufacturing, biotech, life sciences, European financial services, and high-growth B2B AI SaaS.

## Published Case Study #1: Multi-Engine LLM Serving Benchmark (Gemma 4 E4B on 16GB Hardware)
- **URL:** https://inferenceeng.eu/blog/engines
- **GitHub Repository:** https://github.com/Caephas
- **Key Findings (15 Empirical Hardware Sweeps on Apple M2 16GB & NVIDIA Tesla T4 16GB):**
  1. **Continuous Batching Scaling:** vLLM scaled system output throughput 22.3x (`10.41 tok/s` at C=1 to `231.71 tok/s` at C=32) on Tesla T4 with zero per-user decode speed degradation (`10.49 → 10.01 tok/s`).
  2. **PagedAttention APC vs. Contiguous Slot Caching:** Under a shared 1,024-token system prompt at C=32, vLLM Automatic Prefix Caching achieved `1,457.3 ms` P50 TTFT (`1,939.4 ms` P99) with `0 MB` additional VRAM, beating llama.cpp's contiguous slot cache (`23,743.2 ms` P50 TTFT, `53,278.2 ms` P99, and `+6.27 GB` SSD swap spillover) by **16.3x on P50** and **27.5x on P99**. At C=8, vLLM prefix caching cut P99 TTFT by **15.0x (`8,777.3 ms → 584.6 ms`)**.
  3. **Custom Triton Kernel Patch for Turing T4 (`64 KB` SRAM):** Gemma 4 uses `head_size=512` in global attention layers, causing vLLM v0.19's `triton_unified_attention.py` (`TILE_SIZE=32`) to request `98,304 bytes` (`96 KB`) of shared memory per SM and crash on the Tesla T4's `65,536 byte` (`64 KB`) SRAM ceiling. Clamping `TILE_SIZE=16` and `launch_num_stages=1` reduced per-SM SRAM to `49,152 bytes` (`48 KB`).
  4. **Compute-Bound Dequantization Crossover at C=32:** At C=1 (memory-bandwidth-bound), 4-bit `Q4_K_M` (`25.19 tok/s`) beat 8-bit `Q8_0` (`15.99 tok/s`) by 1.58x on Apple M2 (`100 GB/s`). At C=32 (compute-bound), `Q8_0` (`133.15 tok/s`) overtook `Q4_K_M` (`126.56 tok/s`) due to 4-bit bit-unpacking overhead.

## Contact & Discovery Call Booking
- **Website:** https://inferenceeng.eu
- **Consultant:** Arinze Obidiegwu
- **Email:** arinze@inferenceeng.eu
- **Location:** Ireland (serving EU, UK, and remote-global clients)
- **Interactive GPU Cost Calculator:** https://inferenceeng.eu/calculator
- **Full Technical Documentation for LLMs:** https://inferenceeng.eu/llms-full.txt