Architecting High-Throughput FAST AI Platforms, Streaming Data, & DevEx Tooling
Architectural blueprint and interactive system prototype for the Fetch Rewards FAST AI Platform and Platform Foundations team. Bridging core ML models to brand managers across Fetch's 27PB Data Lake using async Python backends, type-safe API boundaries, and real-time streaming interfaces.
Ingestion Scale
600 → 10,000 rows/run with bounded memory
Annual API Savings
Redis semantic cache with 38% hit rate
Real-Time Streaming
Kafka distributed event pipeline (0 dropped)
Production Uptime
FDA 21 CFR Part 11 regulated systems
FAST Real-Time Ingestion, OCR & Agent Execution Engine
Test the live architecture under simulated high-throughput receipt traffic. Benchmarks Server-Sent Events (SSE), SKU brand matching, and background agent orchestration over the 27PB lakehouse.
Pipeline Controls
Pantry & Groceries Run (Kroger #4910)
Target Brand Partners: PepsiCo & Unilever
| SKU / Item Name | Brand Attribution | Price | Base Pts | Earned Points |
|---|
Curious how this scales to 4M req/min across a 27PB Lakehouse?
3-Tier Enterprise Architecture: Fetch FAST AI Platform
Scalable, distributed blueprint connecting mobile receipt clients, async Python streaming backends, 27PB Iceberg lakehouse analytics, and type-safe frontend dashboards.
Tier 1: Edge, Ingestion & Real-Time Event Bus
Staff-Level Architectural Trade-offs & Guardrails:
- ›Partitioning by `user_id` avoids hot-partition bottlenecks while guaranteeing per-user idempotency.
- ›Async event decoupling allows the mobile app to return immediate 202 Accepted (<45ms).
Redis Vector & Semantic Cache
Caches SKU vector embeddings and frequent B2B partner query summaries with >0.92 cosine similarity, bypassing Trino and LLM inference.
Trino & 27PB Apache Iceberg
Columnar Parquet storage on AWS S3 with partition pruning on merchant_id and receipt_timestamp, powering multi-year brand analytics.
Kafka Event Bus (64 Partitions)
Consistent hashing by user_id prevents hot partitions, maintaining consumer lag <50k peak across 4M events/min (inspired by Dylan's catalog wins).
How do we enforce zero schema drift and resilient SSE streaming in code?
Zero-Trust Sequences, Async Backpressure & Type Contracts
Detailed technical workflows solving real-time streaming timeouts, schema drift between Python and TypeScript, and multi-model cost attribution.
Server-Sent Events (SSE) Streaming with Bounded Queue & Reconnection
Solves AWS ALB 60s idle socket timeouts and client drops during heavy analytics queries.
Client Connect
GET /api/v1/receipts/stream with JWT & Last-Event-ID header.
Bounded Queue
FastAPI assigns asyncio.Queue(maxsize=100) preventing RAM bloat.
15s Heartbeats
Async task emits ': ping\n\n' every 15s preventing proxy timeout.
Redis Pub/Sub
Worker pushes OCR chunks to user stream channel id.
Auto Replay
On drop, client reconnects; Redis replays unacknowledged tokens.
# backend/services/sse_streaming_gateway.py · Async SSE Generator
async def event_generator(user_id: str, last_event_id: Optional[str] = None):
queue = asyncio.Queue(maxsize=100) # Bounded backpressure queue (prevents OOM)
redis_client = await get_redis_pool()
# Replay missed events from Redis stream if reconnecting
if last_event_id:
missed = await redis_client.xrange(f"stream:{user_id}", min=last_event_id)
for msg_id, payload in missed[1:]:
yield f"id: {msg_id}\ndata: {payload['data']}\n\n"
try:
while True:
try:
# Wait for token with 15-second heartbeat timeout
data = await asyncio.wait_for(queue.get(), timeout=15.0)
msg_id = await redis_client.xadd(f"stream:{user_id}", {"data": data})
yield f"id: {msg_id}\ndata: {data}\n\n"
except asyncio.TimeoutError:
# Keep-alive comment ping prevents AWS ALB / Cloudflare socket termination
yield ": ping\n\n"
except asyncio.CancelledError:
logger.info(f"SSE stream closed gracefully for user {user_id}")
How do these systems rebalance partitions and catch fraud in real time?
Production Algorithms: Interactive Decision Trees
Adjust real-time parameters below to trace live decision branch execution across fraud rate limiting, SKU fuzzy resolution, points knapsack, and Kafka partition rebalancing.
Sliding Window Log Velocity Limiter (Redis ZSET)
Engineering TL;DR: Atomic ZREMRANGEBYSCORE + ZADD + ZCARD in Redis prunes expired logs older than T - W in O(log N) time, preventing fixed-window boundary burst exploits.
FAST Business Purpose: Fraud Mitigation & Partner Trust. Prevents automated botnets from scraping or re-submitting synthetic receipts to drain brand partner promotional budgets. Saves an estimated $1.8M/year in fraudulent point disbursements while ensuring legitimate shoppers receive sub-10ms response times.
rate_limit:user:4910# fast/security/rate_limiter.py (Redis ZSET Sliding Window)
async def is_upload_allowed(user_id: str, now: float, window_sec: int, max_req: int) -> bool:
key, cutoff = f"rate_limit:{user_id}", now - window_sec
pipe = redis.pipeline()
pipe.zremrangebyscore(key, 0, cutoff) # 1. Prune expired logs
pipe.zcard(key) # 2. Count active elements in window
_, count = await pipe.execute()
if count < max_req:
await redis.zadd(key, {str(now): now}) # 3. Enqueue timestamp
return True # 202 Accepted
return False # 4. 429 Rate-limit exceeded
See how these distributed systems patterns performed at Airbnb, Lilly & Southwest.
Production Arsenal & Career Provenance
7+ years of battle-tested experience building high-throughput distributed backends, AI platforms, and compliance-grade architectures.
Airbnb
Current RoleSenior Software Engineer, ML Infrastructure & AI Engineering (GenAI Platform)
- ✓Architected BPI Virtual Analyst (greenfield-to-production) for 128+ daily active enterprise analysts, scaling tabular batch ingestion 16x (600 to 10,000 rows/run) with bounded memory streaming.
- ✓Built FacadeDriver multi-model Python runtime standardizing 30+ foundation models (Bedrock Claude, Azure OpenAI, Vertex AI) with OpenTelemetry cost attribution and Redis semantic caching, saving $180,000/year.
- ✓Engineered automated 23-version evaluation harness (1,690 ground-truth test cases) ensuring deterministic regression testing across all model upgrades.
- ✓Operate on-call rotations for foundational Redpen Airflow & BigAir data pipelines handling high-throughput batch uploads across 19 production configs in 11+ languages.
Eli Lilly and Company
FDA RegulatedSenior Software Engineer — Dose Management Platform (FDA 21 CFR Part 11)
- ✓Engineered mission-critical radiopharmaceutical distribution backend maintaining 99.9% uptime across 6 months under strict federal compliance.
- ✓Built dynamic gRPC `GrpcMetadataProvider` for Kubernetes ServiceAccount token rotation on disk, eliminating worker starvation and manual pod restarts.
- ✓Created self-healing `/fix-temporal` state re-arming engine to resync Temporal workflow histories with PostgreSQL after disaster recovery restores.
- ✓Engineered atomic gap-free ID generation (`CMCDOS-2031`) preventing PostgreSQL sequence skips and FDA compliance deletion flags.
Southwest Airlines
Distributed SystemsSenior Software Engineer — Backend & Data Platform
- ✓Scaled distributed Kafka event streaming pipelines processing 4,000,000 requests/minute peak with zero message drop across high-volume flight operations.
- ✓Built resilient consumer group partition rebalancing logic and dead-letter queue (DLQ) retry topologies.
Shell PLC
ML SystemsSenior Software Engineer — Backend & Data Science
- ✓Architected deep learning autoencoder anomaly detection pipelines for real-time sensor telemetry across global asset networks.
Modular Deep Learning Architecture for Cross-Domain Transfer and Incremental Learning
App. No.: 202541026299 (Indian Patent Office) · Continuous parameter isolation & zero catastrophic forgetting.
Leadership & Peer Endorsements
Senior platform impact isn't just claimed in resumes—it is substantiated by the Senior Managers, Staff Data Scientists, Product Managers, and Operations Leads who have partnered with Sai.
Ameet Shinde
AirbnbSenior Manager, Business Process Improvement (BPI)
“Sai has been an outstanding partner in the deployment of the BPIVA tool, and I want to take a moment to recognize his incredible contributions. Thanks to Sai's efforts, the BPIVA tool has had a significant impact on reducing non-value-added work, enabling the BPI team to shift their focus to high-impact, actionable tasks exactly where their energy should be. What truly sets Sai apart is his deep understanding of technology combined with his ability to quickly grasp tool requirements and translate them into real solutions. He doesn't just deliver, he continuously looks for ways to enhance and upgrade the tool's capabilities, ensuring it evolves alongside our team's needs. None of this would have been possible without Sai's dedication and expertise. He is a truly great partner who consistently goes above and beyond to deliver excellence. Thank you, Sai, for everything you do! 🙌”
Proves high-agency platform stewardship: Sai identifies operational friction, translates ambiguity into resilient primitives, and continuously elevates developer experience without being micromanaged.
Jeremy Chua
AirbnbStaff Data Scientist
“Sai is an incredibly thoughtful engineer who brings both technical depth and a genuine desire to help the team succeed. His work on the ML infrastructure side has been invaluable — he moves fast without sacrificing quality and is always thinking about how to make things better for the people using the tools he builds.”
Validates distributed systems velocity and craft: Staff-level data science and engineering peers trust Sai to architect complex ML and event-streaming pipelines that remain robust under heavy analytical query loads.
Alejandro Virrueta
AirbnbProduct Manager
“Sai has consistently demonstrated strong ownership and a solutions-first mindset throughout our collaboration. He takes feedback well, iterates quickly, and delivers results that genuinely move the needle. It's been a pleasure working with him — the kind of engineer you want on every project.”
Direct proof of product-engineering synergy: Rather than debating theoretical specs, Sai partners with product leads with a prototype-first cadence, turning user feedback into shipping features in hours.
Lori Barber
AirbnbOperations Lead
“I just wanted to call out Sai for his exceptional support on the BPI Virtual Analyst project. He has been incredibly responsive and proactive in addressing our requests, and his contributions have meaningfully improved the team's efficiency. He is a true team player and an asset to any cross-functional effort.”
Embodies Fetch's customer creed ('Listen, build, ship, tell the customer, repeat'): operational users experience zero latency when asking for enhancements, maximizing platform adoption across non-technical teams.
The Consistent Thread Across Airbnb, Lilly, and Southwest
High-velocity prototyping with human architectural governance. Whether building AI analytics tools or scaling real-time distributed pipelines, Sai combines senior systems rigor with genuine customer empathy.
High-Agency Execution: Working Prototypes & Proof of Concepts
Complex distributed systems and high-throughput data platforms are best de-risked through functional prototypes and empirical metrics rather than speculative documentation. Here is how I approach rapid discovery, system design, and platform ownership.
AI-Assisted Engineering Acceleration
High-velocity iteration with human architectural ownership.
"Leverage AI harnesses to automate test matrices, explore edge cases, and eliminate contract boilerplate—while keeping critical system invariants human-governed."
Working Prototypes & Proof of Concepts
Functional software de-risks architecture faster than theoretical proposals.
"When evaluating high-throughput distributed systems, an interactive proof-of-concept reveals empirical bottleneck thresholds that static documents overlook."
Small Teams Ship Exponentially Faster
High agency, direct stakeholder feedback, zero bureaucratic drag.
"Engage directly with power users, ship focused resilient slices, verify real metrics, and iterate rapidly."
The 4 Core Evaluation Pillars
Mapped directly to Fetch's senior engineering scope and technical expectations.
Technical Experience & Project Ownership
Spearheaded platform foundations across Airbnb GenAI Platform, Eli Lilly FDA Part 11 validation, and Southwest Airlines high-throughput event streaming.
Delivered 16x throughput scaling (from 600 to 10,000 rows/run, 40MB batch uploads) by implementing bounded memory queues and decoupled async workers.
Authored Indian Patent 202541026299 for modular deep learning architecture with cross-domain transfer learning.
Architectural Deep Dive & Open System Inquiries
Prepared high-agency inquiries exploring the FAST AI platform architecture, distributed lakehouse streaming, and developer foundations:
Background Agent Sandboxing & Skills Registry
Inquiry 1"In multi-step autonomous agent workflows, how does the FAST platform approach type-safe tool schemas, runtime execution sandboxing, and dynamic skill registration across shared microservices?"
Real-Time Streaming vs. 27PB Batch Analytics
Inquiry 2"For analytical platform UIs, what does the architectural trade-off look like between real-time streaming interfaces (WebSockets/SSE) and asynchronous background batch processing across the 27PB data lake?"
0-to-1 Velocity vs. Platform Foundations
Inquiry 3"How does the engineering org balance rapid 0-to-1 feature velocity for brand managers with long-term platform stability and shared component developer experience for the rest of Fetch's engineering teams?"
30-60-90 Day FAST Execution Roadmap
Tailored specifically for Dylan Lennard (Tech Lead Manager, FAST Foundations & Labs) and Jared Flatow (VP Engineering). A structured plan to transition from active listening and developer friction removal to shipping high-velocity platform primitives, streaming resilience, and enterprise-grade Lakehouse AI analytics.
Listen, Instrument & Eliminate Developer Friction
Deliver immediate velocity gains to FAST feature engineers before introducing foundational architectural changes.
FAST Developer Friction Audit & Shadowing
Maps exact friction bottlenecks across 4 feature teamsShadow 4 FAST feature squads (Product, Catalog, Analytics, Mobile). Trace the end-to-end PR lifecycle from local sandbox to staging to identify local dev environment spin-up delays and mocking pain points.
Automated AST Contract Linter (Pydantic V2 ↔ Zod)
Eliminates silent frontend/backend schema driftDeploy a lightweight GitHub Actions pre-merge validation step that introspects FastAPI Pydantic V2 response models and verifies 1:1 synchronization with client-side Zod definitions, preventing runtime type mismatches.
27PB Iceberg Lakehouse Partition & Query Audit
Identifies query bottlenecks across B2B partner reportsAnalyze the top 20 slowest SQL queries across the Trino/Iceberg analytics cluster. Identify missing partition pruning on high-cardinality keys (e.g., `merchant_id`, `receipt_timestamp`) to optimize query planning.
Catalog Event-Stream Deep Dive & Kafka Shadowing
Builds on Dylan's 2.3M → 50k consumer lag reduction winDeep-dive into the Kafka consumer group rebalance patterns and catalog event stream topologies that Dylan and the team optimized, ensuring platform foundations are architected to withstand 10x traffic surges.
Zero Virtual DOM Lag for High-Throughput Receipt Analytics
Fetch processes millions of receipt events per day. For live B2B partner analytics dashboards, traditional React Virtual DOM reconciliation incurs frame drops under 50+ updates/sec. By leveraging Svelte 5 Runes ($state, $derived, $effect), updates mutate exact DOM text nodes via fine-grained signals—slashing render latency to <16ms and eliminating Interaction to Next Paint (INP) bottlenecks.
Multiplier Value I Bring to Fetch
Senior platform engineers don't just ship tickets—they elevate the engineering brand, mentor talent, build scaffolding, and champion a culture of Demos > Memos.
Fetch Tech Blog & Engineering Brand
Elevating Fetch to an Airbnb & Stripe-Tier Engineering Brand
Conference Keynotes & Fetch Ambassador
Representing Fetch at Major Industry Engineering Summits
Hackathon Leadership & 0-to-1 Culture
Turning 48-Hour Experiments into Durable Platform Primitives
Internal DevEx Tooling & Scaffolding
Cutting New Service Spin-Up Time Across 300+ Engineers
Talent Magnet & Non-Zero-Sum Mentorship
Attracting Top 1% Engineers & Elevating Team Craft
Cloud FinOps & Lakehouse Efficiency
Slashing LLM Inference Costs & Protecting the 27PB Lakehouse
7+ YOE"Listen, build, ship, tell the customer, repeat."
Technical leadership is measured by product adoption and customer velocity, not lines of code. I look forward to partnering with Dylan, Jared, and Cam to establish Fetch as the benchmark for modern AI platform engineering.