Engineering Notes

Using Redis and sentence embeddings to catch similar LLM prompt queries before hitting the OpenAI API, drastically improving response times from 1200ms down to 180ms.

Why I chose to run Postgres, Redis, and Prometheus locally via Compose before spinning up full cloud resources to save costs and simulate production.