Uber
Teams across Uber ask roughly 45,000 questions each month in hundreds of Slack support channels. Answers are fragmented across Engwiki, an internal Stack Overflow, engineering requirement documents, and prior conversations. Repeated questions and multi-round waiting consume time for both internal users and on-call engineers.
Uber built Genie in-house. Spark jobs ingest internal sources; LangChain chunks content; OpenAI embeddings turn documents into vectors; and Sia plus Terrablob provide a channel-scoped index. A Slack query enters the Knowledge Service, retrieves relevant evidence, and is answered by an LLM. The system also records request-level cost, immediate user feedback, and offline evaluation results.
Genie handles questions that existing documentation can answer. Code review requests, missing context, and irrelevant answers are routed to an on-call engineer. Users label answers Resolved, Helpful, Not Helpful, or Not Relevant; the platform team uses this feedback and evaluation reports to tune retrieval and generation.
- Measure support-channel volume and identify repeated questions
- Compare fine-tuning with RAG and select the faster path to production
- Map Slack channels to knowledge-space access controls
- Pass a UUID across each request for model-cost attribution
- Join user feedback with request context for retrieval and generation evaluation
- An internal knowledge assistant must inherit source-system permissions
- Support bots need an explicit route for requests that require a human
- Cost, user feedback, and quality evaluation should share one request identifier
- Evaluate retrieval and generation separately, not only the final answer
Genie is in production within Uber’s internal support workflow and includes traceable feedback, cost, and hallucination-evaluation loops. The engineering article describes the demand baseline and architecture, but does not report resolution rate, labor savings, or ROI.
- The roughly 45,000 monthly questions are the demand baseline, not Genie’s handled volume.
- The article does not disclose post-launch resolution rate, response time, or labor savings; no ROI is inferred.