Morgan Stanley
Wealth Management Knowledge Assistant + Evals System
OpenAI
Finance
Embedded deployment
Deep case | Key delivery chain is substantially documented
Evidence level A
Large-scale production
A
Evidence level
Morgan Stanley: Wealth Management Knowledge Assistant + Evals System
Evidence level measures whether a source can be located and reviewed; it does not mean vendor-reported claims were independently audited.
Deep case | Key delivery chain is substantially documented
10 / 12
Business context1 / 2
Transformation workflow2 / 2
Technical workflow2 / 2
Human roles & governance1 / 2
Measured outcomes2 / 2
Source traceability2 / 2
Remaining gaps: Business context, Human roles & governance
Business problem
Wealth advisers need to quickly obtain credible answers from large and highly compliant research and process files.
Solution
Morgan Stanley, in conjunction with OpenAI, optimized search and Evals: expert ratings and regression tests for digests, translations, retrievals, extended to Advisor Assistant and conference Debrief.
Technical architecture & production workflow
Step 01
Enterprise knowledge/documents/data/business systems
→
Step 02
ChatGPT Enterprise/Work access context within permission
→
Step 03
Staff member or custom GPT/Agent completes analysis, generation or automation
→
Step 04
Connector/rules/business governance control data and action boundaries
→
Step 05
Staff review of key outputs and implementation
→
Step 06
HF process sedimentation as template, GPS or Agent and continuous assessment
Key technology & infrastructure components
GPT-4RAG/ResearchEvalsWhisperExpert scoringReturn Test
Human roles & accountability
Consultants assess answers with in-house experts and assume client advice responsibilities.
FDE delivery actions
- Select high-frequency and measurable production streams with the wealth management team, rather than simply deploying chat portals
- Identifying the business context, system privileges, tools and security boundaries required for the model
- Connect models to real software/data/business processes and establish tests, Evals or validation of certainty
- Design the Human-in-the-lop and failed upgrade paths to ensure clear boundaries of responsibility
- Continuous succession of Prompt, context, tools and processes based on production usage, errors and user feedback
Reusable delivery patterns
- Enterprise AI effects depend on context, tools, validation and adoption rates, and not only on model capabilities
- Quantifiable workflows first followed by product/platformization of successful models
- As far as you can, Agent has access to tests, CI/CDs and security checks to form a closed loop.
- High-risk industries must leave professional responsibilities and establish sustainable Evals
Business outcomes & delivery results
Officially, 98 per cent plus the use of a team of consultants; answerable materials expanded from approximately 7,000 questions to 100,000 documents; document accessibility increased from 20 per cent to 80 per cent.
Evidence boundaries & verification notes
- The OpenAI official case details changes in Evals, search overlaps, adoption rates and document coverage.
- The source can be directly located in the case; the value remains disclosed by the source and does not represent an independent audit.