Production engineering, explained through the problem it had to protect people from.
These are field notes from SpotBook and collaborative work on EPFO. Each one starts with the operational failure, follows the engineering decision, and ends with what changed for the person using the system.
01The Vanishing Cart
Two salespeople booked the last piece simultaneously. One order vanished entirely.
The Vanishing Cart
Two salespeople booked the last piece simultaneously. One order vanished entirely.
Root Cause
No concurrency control on stock checks. Classic race condition — both transactions read "1 available," both tried to decrement, second one hit a constraint violation and the entire multi-item order was rolled back.
The Fix
Implemented SELECT ... FOR UPDATE (pessimistic locking) in the OrderRepository. Added version-based optimistic locking on the Product entity as a second layer. If a single item in a draft fails, only that item is rejected — the rest of the draft persists.
Impact
Zero order loss incidents from Season 2 onwards. Staff trust in the app went from cautious to complete.
02Identity-Based Diffing for Order Edits
Editing a 60-item order meant replacing the entire thing. Dangerous and slow.
Identity-Based Diffing for Order Edits
Editing a 60-item order meant replacing the entire thing. Dangerous and slow.
Root Cause
Initial implementation used "delete all items + re-insert" strategy. This broke stock accounting and was fragile under concurrent edits.
The Fix
Built a surgical editing system using identity-based diffing. Frontend computes the delta (items added, removed, quantity-changed) by comparing old vs. new item lists by product ID. Backend validates only the delta's stock impact — not the full order — using deltaQuantity = new - old.
Impact
Order edits became atomic, safe, and fast. No more phantom stock leaks from replace-all operations.
03The Composite Loading Gate
React pages flickered between loading → empty → loaded. Users thought data was missing.
The Composite Loading Gate
React pages flickered between loading → empty → loaded. Users thought data was missing.
Root Cause
RTK Query's isLoading only fires on first fetch. Subsequent navigations use isFetching but the component was only checking isLoading, so cached-but-stale states showed empty data briefly.
The Fix
Created a composite loading check: isUninitialized || (isFetching && !data). Wrapped it in a reusable LoadingGate component. Added skeleton screens instead of spinners for perceived performance.
Impact
No more flash-of-empty-state. Staff stopped refreshing pages thinking "the app is broken."
04iPhone HEIC Image Bomb
Product images from iPhones wouldn't display. 70% of uploaded photos were broken.
iPhone HEIC Image Bomb
Product images from iPhones wouldn't display. 70% of uploaded photos were broken.
Root Cause
iPhones save photos in HEIC format by default. Browser <img> tags and createImageBitmap don't support HEIC on most platforms. 70% of uploaded images were silently broken.
The Fix
Built a two-path image processor: try native createImageBitmap first (fast, works for JPEG/PNG/WebP). If it throws, fall back to heic2any library for client-side HEIC→JPEG conversion before upload. Compressed to 80% quality + resized to max 1200px.
Impact
100% image compatibility across all devices. Upload size reduced by ~60%. Zero broken thumbnails.
05DNS Debugging on the Expo Floor
App worked on WiFi, failed on mobile data. Expo starts in 2 hours.
DNS Debugging on the Expo Floor
App worked on WiFi, failed on mobile data. Expo starts in 2 hours.
Root Cause
Custom domain DNS records hadn't fully propagated. WiFi DNS resolved (cached), but mobile carrier DNS hadn't picked up the new records yet. The SSL certificate was also mis-configured for the bare domain vs. www subdomain.
The Fix
Switched to a verified domain with pre-configured SSL. Added both A and CNAME records. Tested with dig across multiple DNS resolvers. Set low TTL for the cutover period.
Impact
App was accessible from any network 30 minutes before expo doors opened.
06WhatsApp Receipt Automation
Retailers & Distributors wanted instant order confirmation. Manual sending was impossible at scale.
WhatsApp Receipt Automation
Retailers & Distributors wanted instant order confirmation. Manual sending was impossible at scale.
The Challenge
After each order, the distributor expected a receipt. With 50+ orders per day across multiple salespeople, manual WhatsApp messages were not feasible.
The Fix
Integrated WhatsApp Business API for automated order receipts. Each confirmed order triggers a formatted message with order number, item list, quantities, and total amount — sent directly to the distributor's registered WhatsApp number.
Impact
Instant confirmation. Zero manual effort. Retailers/Distributors started trusting the digital system over paper because they had immediate proof.
07Multi-Tenant Data Isolation
Evolving SpotBook from its initial single-client deployment to multi-tenant use without data leaks.
Multi-Tenant Data Isolation
Evolving SpotBook from its initial single-client deployment to multi-tenant use without data leaks.
The Challenge
SpotBook serves multiple manufacturers. Every query, every API call, every report must be scoped to the authenticated tenant. One leaked row = catastrophic trust violation.
The Fix
Implemented tenant context at the Spring Security filter level. Every request extracts tenant ID from the JWT. Hibernate filters automatically append WHERE tenant_id = ? to all queries. Added integration tests that specifically try to access cross-tenant data and assert 403/empty results.
Impact
Airtight data isolation. Zero cross-tenant leaks in testing. Backoffice (Zone 0) is the only module that bypasses tenant filters — by explicit design.
08Size Matrix Complexity
Clothing products have size × color variants. Managing inventory per-variant at expo scale.
Size Matrix Complexity
Clothing products have size × color variants. Managing inventory per-variant at expo scale.
The Challenge
A single jeans style can have 8 sizes × 4 washes = 32 SKU variants. Inventory and ordering need to work at the variant level, but catalog browsing needs to work at the product level. Two different mental models in one system.
The Fix
Designed a two-layer product model: Product (catalog-level) contains ProductVariants (inventory-level). Orders reference variants. Catalog UI shows products with a size/wash matrix selector. Stock checks aggregate at the variant level. Prepacks bundle multiple variants into a single orderable unit.
Impact
Salespeople can browse products naturally but order at the granularity the warehouse needs. Inventory accuracy went from "roughly right" to exact.
09PWA Offline Resilience
Expo venues have terrible WiFi. App needed to not die when signal dropped.
PWA Offline Resilience
Expo venues have terrible WiFi. App needed to not die when signal dropped.
The Challenge
Trade expo venues in India are large halls with spotty WiFi coverage. Sales agents walking between booths regularly lost connectivity. A hard crash or blank screen during an order = lost sale.
The Fix
Built as a PWA with service worker for static asset caching. Cart state persists to localStorage. Network requests use a retry-with-exponential-backoff pattern. Visual indicator shows online/offline status. Critical actions (order submission) queue and retry when connection returns.
Impact
App remained usable during brief connectivity drops. No lost carts. No blank screens. Agents stopped complaining about "the app crashing."
10Facebook Ads: The Marketing Failure
Not a code problem — a business one. But it taught me more than any bug.
Facebook Ads: The Marketing Failure
Not a code problem — a business one. But it taught me more than any bug.
The Problem
SpotBook needed customers. I ran Facebook lead-gen ads targeting clothing wholesalers. Got form submissions. Called them. Most had no idea what they'd signed up for, had no budget, or thought it was a government scheme.
What I Learned
Facebook ads optimize for form fills, not qualified leads. The algorithm finds people who click buttons, not people who have a problem to solve. For B2B SaaS in a traditional industry, the right GTM is relationship-based sales — expo visits, referrals from existing users, WhatsApp group word-of-mouth.
Impact
Burned the marketing budget. But gained clarity: engineering solves the product problem, distribution solves the business problem, and they require completely different skills.
11Aadhaar eSign at Government Scale
EPFO — Integrating India's national biometric identity into a claim pipeline processing millions.
Aadhaar eSign at Government Scale
EPFO — Integrating India's national biometric identity into a claim pipeline processing millions.
The Challenge
EPFO claim processing required legally valid signatures. Physical signature verification took 7–10 days per claim. With 30.6 lakh (3M+) claims per month, the manual bottleneck was the single biggest source of delay.
The Fix
Integrated Aadhaar-based eSign via Protean's API into the claim pipeline. Built retry logic for OTP failures, handled Aadhaar service downtime gracefully with queue-and-retry, and ensured compliance with government security standards for financial data.
Impact
70% of advance claims (Form 31) processed within 3 days of filing — down from 7–10 days. Eliminated physical signature dependency for the majority of claims.
12API Performance at 10M+ Transactions/Month
EPFO — When a slow query doesn't cause a spinner, it causes a helpline to light up.
API Performance at 10M+ Transactions/Month
EPFO — When a slow query doesn't cause a spinner, it causes a helpline to light up.
The Challenge
EPFO's UAN Portal dashboards serve millions of monthly users checking claim status, transaction records, and PF balances. Query performance degraded as audit log tables grew. Index scans became full table scans on claims tables with 100M+ rows.
The Fix
Redesigned database schemas with optimised indexing for claims, audit logs, and transaction history tables. Added composite indexes on high-cardinality query patterns. Restructured heavy join queries to use indexed lookup paths. Maintained compliance-grade audit trails throughout — government financial data has strict immutability requirements.
Impact
35% query performance improvement. 99.9% API uptime maintained across 3M+ monthly transactions. Zero SLA breaches on the dashboard endpoints.