Software Engineer
Rebuilt the core integration suite delivering resident and tenant access data to smart locker hardware across thousands of residential and commercial properties. Optimized high-volume data pipelines processing over 10 million access record updates per hour, refactored critical legacy architecture, and built internal provisioning tools.
Overview
At Luxer One, I was responsible for engineering the software pipelines and integration layer that powered physical smart package lockers across multi-family residential communities, universities, and enterprise office complexes.
Smart lockers depend on continuous, accurate property management data: who currently lives in which unit, what their contact information is, when leases start or terminate, and which locker banks they are authorized to open. If an integration pipeline fails or falls behind, packages cannot be delivered, delivery couriers are blocked, and residents cannot retrieve their packages.
Core System Objective: Decouple incoming property roster imports from active locker hardware state, replacing fragile single-process scripts with a horizontally scalable, idempotent event-driven queue pipeline.
System Pipeline Architecture
[ Property Management Systems ] (Yardi, RealPage, Entrata)
│
▼ (REST Webhooks / SOAP XML / SFTP Drop)
[ Ingestion Gateway ] ──► [ Redis Ingestion Stream ]
│
▼ (500-Row Deterministic Chunks)
[ Asynchronous Worker Pool ]
│
┌──────────────────┴──────────────────┐
▼ ▼
[ MySQL Batch Upsert ] [ Zoho ERP Telemetry ]
│
▼
[ Smart Locker Fleet ]
Problems I Worked On
→ High-Volume Access Record Processing
The existing access synchronization jobs were periodically timing out, hitting database memory ceilings, and locking critical tables as the locker fleet scaled nationally. The system needed to process incoming deltas and bulk roster dumps from thousands of properties simultaneously.
| Pipeline Layer | Ingestion Mechanism | Batch Strategy | Production Throughput |
|---|---|---|---|
| Edge Ingestion | REST Webhooks & SFTP | Immediate Redis Staging | Sub-50ms HTTP Response |
| Worker Queue | Distributed Horizon Workers | Chunked 500 records | Asynchronous & Scalable |
| Database Sync | MySQL Composite Primary Keys | Deterministic Lock Ordering | 10M+ updates / hour |
| Hardware Fleet | Event Push & MQTT Telemetry | Differential State Sync | Near real-time locker updates |
- The bottleneck: Property management systems (Yardi, RealPage, Entrata) exported large, unindexed resident listings via varying channels (nightly batch files, periodic SOAP endpoints, and high-frequency REST webhooks). Processing these records sequentially inside single PHP worker processes caused worker starvation and prolonged database table locks.
- The approach: I redesigned the synchronization flow around a chunked, idempotent pipeline. Ingestion was separated from reconciliation: incoming payloads were staged into Redis queues, chunked into deterministic batches of 500 records, and reconciled against master records using composite primary keys and batch upserts.
- The result: The pipeline was able to process over 10 million access record updates per hour with predictable memory consumption and zero downtime.
→ Third-Party System Integrations & Protocol Normalization
Every property management software provider had its own data model, idiosyncratic error codes, and transport mechanism. Some supported modern RESTful webhooks, others required legacy SOAP XML envelopes with session-state cookies, and several major enterprise clients strictly provided encrypted CSV exports dumped into SFTP dropboxes.
| Attribute | Legacy Single-Script Architecture | Modern Modular Factory Architecture |
|---|---|---|
| Structure | 20,000-line single procedural file | MVC & Factory Design Pattern with clean interfaces |
| Onboarding | 3 to 4 weeks of high-risk custom code | 2 to 3 days by extending standard adapter contract |
| Failure Mode | One bad record killed entire property sync | Dead-letter queue quarantines bad rows safely |
| Operations | Required engineer intervention for debug | Self-service admin dashboard for support teams |
- The legacy state: The system had historically evolved as a monolithic single-file script containing nested conditionals checking property IDs and vendor types. Adding support for a new vendor was fraught with regression risk.
- The modular architecture: I refactored the 20,000-line monolithic codebase into an Object-Oriented architecture using the Strategy and Factory design patterns. Each external provider implemented a clean, standardized
PropertyIntegrationInterface, encapsulating vendor-specific auth, payload transformation, retry logic, and telemetry reporting. - Support tooling: I built a unified "kitchen sink" administrative dashboard allowing non-engineering support and onboarding specialists to configure API credentials, trigger on-demand sync tests, inspect failure logs, and verify tenant mappings without engineering intervention.
Selected Achievements
- 10M+ Records/Hour Pipeline: Designed and scaled the core access-record update pipeline to reliably ingest, reconcile, and synchronize over 10 million records per hour without lock contention.
- 20,000-Line Codebase Modernization: Successfully refactored a brittle, single-file legacy integration script into an extensible MVC / Factory architecture with clean unit test coverage, cutting new vendor onboarding time from weeks to days.
- Automated Telemetry & ERP Integration: Connected locker system health metrics and sync telemetry into Zoho ERP, providing automated alerting for sync anomalies and giving customer success real-time visibility into field hardware status.
- Self-Service Support Dashboard: Built administrative interfaces that empowered account managers to diagnose client sync discrepancies, reducing integration support escalation tickets by an estimated 60%.
Technical Challenges
Database Contention During Large Synchronizations
When multiple property sync workers attempted to update tenant permissions concurrently, MySQL row locks often cascaded into transaction deadlocks. To solve this, sync batches were sorted deterministically by tenant ID prior to execution, ensuring all workers acquired row locks in identical sequential order and eliminating circular deadlocks.
Fragile Upstream Vendor Endpoints
Upstream property management APIs were notoriously unreliable—frequently timing out, returning 500 errors during their internal backups, or emitting malformed XML payloads. I instituted robust circuit-breaker patterns, exponential backoff retries with jitter, and dead-letter queues to quarantine corrupted tenant records without blocking the remaining roster updates.
What I Learned
- Idempotency is the bedrock of distributed reliability. Network requests and worker threads will inevitably fail midway through execution. Designing every database mutation and hardware command to be safe for multiple executions is essential.
- Standardize vendor adapters aggressively. Writing vendor-specific one-offs creates an unmaintainable system. Investing upfront in a strict adapter boundary keeps external chaos from infecting internal domain logic.
- Decouple data ingestion from data reconciliation. Never process heavy business logic inside an active API request or incoming webhook handler. Accept the payload rapidly, stage it safely, and let worker queues process the workload asynchronously.