No worries, this recent blog post from the Azure Integration Services gives you all the possible approaches and examples. Another hidden gem that can save you some time when architecting these types of challenges.

Main questions answered?

1. How many Azure Open AI (AOAI) APIs should I create?
2. How do I version AOAI APIs?
3. How do I support consumers with different content-safety and model requirements?
4. How do I restrict throughput per Consumer, per deployment?
5. How do I scale out AOAI services?
6. How do I log all prompts and responses including streaming, without disruption?
7. What other value add services should a platform offer consumers?

Find out everything here: https://techcommunity.microsoft.com/t5/azure-integration-services-blog/designing-and-running-a-generative-ai-platform-based-on-azure-ai/ba-p/4223092

My personal note, don’t underestimate the power of Azure API Management. During my recent projects in terms of B2B and/or B2C interactions and having Dataverse/Dynamics 365 in the back, it was already a productivity saver and an essential proposed component of the architecture. With the introduction of new Generative AI services, it can play even a more central internal role.