Large document summarization pipelines are inherently expensive. They require reading entire files, chunking the content, tokenizing strings, and passing data through LLMs or transformer models. Doing this synchronously over REST API endpoints inevitably leads to request timeouts and severe server loading.
During the build of my **AI Document Intelligence Platform**, I engineered an asynchronous worker pattern utilizing **Django**, **Flask**, and **Redis** to scale document conversions and summary generations.
The Architecture We split the system into two distinct boundary layers: 1. **Web and Auth Gate (Django)**: Serves client assets, stores document metadata in PostgreSQL, and acts as the gatekeeper. 2. **Inference Workers (Flask + Celery)**: Isolated workers hosting HuggingFace pipeline models to perform extractive and abstractive summarization.
The Semantic Cache Strategy Instead of running heavy inference for identical documents, we set up a Redis caching layer. We computed a hash of the cleaned document body, using it as a lookup key: * If the key exists, return the cached summary immediately (**5x speedup**). * If the key doesn't exist, queue the document for inference and cache the response.
This optimized CPU resource utilization and maintained an average api uptime of **99.9%** even under concurrent document loads.