The challenge
Five months ago, your team at a global bank's capability centre in Bengaluru launched an LLM gateway. It masks customer data, logs every call and tracks spend. Leadership funded it so 14 product teams would stop building their own.
Your dashboard says “11 teams onboarded”, so nobody is asking why the platform sits unused. They will at the budget review in 8 weeks.
Meanwhile, at least 3 teams call models through their own cloud AI subscriptions, so customer data reaches models unmasked and unlogged. In a bank, that’s an open control issue. The teams say your gateway is slower and gets new models 6 weeks late. And you have no revenue line, and you can’t order anyone to switch.
What is really going on
Most of the time, what you’re really competing with is the direct API call, or the chatbot people already use. In Microsoft’s 2024 survey, 78% of AI users brought their own AI tools to work, and 72% in India [1][2]. The habit was there before your platform arrived.
When teams go around you, it’s usually for one of three reasons. Each one needs something different from you.
If they say “Nobody told us which tools we can use with customer data”, they’re not sure what’s allowed. Without a stated rule, each team keeps what it started with, because nobody wants to guess wrong with customer data. Ask three team leads which route customer data must take to a model, and you’ll get three answers, or none.
Turn your company’s existing AI policy into a one-page guide for product teams, and start now, because sign-off in a regulated firm takes weeks.
If they say “Moving to your gateway means rework and a queue, and our roadmap gets nothing from it”, the cost is theirs and the benefit is the company’s. Their lead usually agrees in principle but can’t name a date to move. That’s a fair position for a team judged only on its own launches.
- Masking
- Logging
- Cost control
- Reuse
- Migration work
- Added delay
- Less control
Tie the move to something their launch needs, like audit logs a client or regulator is asking for. Or pay the switching cost yourself.
And if they say “We tried it, but the model we need isn’t on your gateway yet”, they’re worried about being stuck on an old model. Product teams move to newer models as they appear, so a gateway that adds them weeks late looks like a brake. You’ll see teams leave within weeks of a model release, with requests that name a model, not a feature.
In a regulated firm, each new model goes through a third-party and model risk review that you don’t own. So promise when the review starts, not when the model arrives, and invite the team back when it clears. There doesn’t seem to be a published survey that asks product teams why they leave a platform, so check this reason with them directly.
How do we get more teams onboarded?
What did this team call last week instead of us, and why?
The mistake most teams make
The natural move is to ask leadership to mandate the platform for every AI project. More than a third of platform teams in a December 2025 community survey still rely on mandates [3]. But then teams take API keys and file exceptions, while their live traffic stays on direct routes. Your numbers go up and you stop hearing why.
Ask leadership for something narrower instead, a rule on which route sensitive data must take. Then win the rest of the traffic by being the easier route.
The options you have
Most teams combine two or three of these, matched to the reasons they found.
| Option | Best when | What it costs you | Watch out |
|---|---|---|---|
| Stop building the full platform and serve one team’s launch | No team runs production traffic through it yet, and one has a deadline and a problem you can solve. | Pausing funded work needs a formal re-plan with your sponsor, and other teams wait a quarter. | It fails if that team’s needs are unusual, so pick a problem at least three other teams share. |
| Make the governed route the fastest route | Teams would use the platform, but steps, approvals or delay stop them. | Your roadmap shifts from features to onboarding, documentation and model upkeep, which a steering committee seldom gets excited about. | It fails if the gateway adds delay users can feel, which easy onboarding can’t make up for. |
| Close the direct route for sensitive data, then let teams opt in | Regulation or client contracts already demand the data rule, and your platform is as easy as the alternatives. | Security spends its authority on the rule, and teams with non-sensitive data stay outside it. | Naming your platform as the only route while it lags on models invites exceptions, so allow approved vendor tools that meet the logging rule. |
| Pay the switching cost from the centre | Team leads agree in principle but have no roadmap time to move. | The money comes from your run budget or the sponsor’s, so agree it and a spending cap with the sponsor and finance first. | A cost statement nobody pays is easy to ignore, so announce the billing date and rate on the first day. |
To be the fastest route, start with self-serve access and starter templates for common jobs. It also helps to review the platform’s controls once, so each use case needs a short attestation instead of a full risk assessment. Uber did something useful here. When it built its GenAI Gateway, it copied the OpenAI API’s interface, so teams kept their code libraries and got personal-data masking and audit logs on top [4]. For a team, joining could mean changing where their calls went, without rewriting their code.
PM tools to put in place
Keep a list of your internal customers. Each team building with AI gets a row with its named lead, the data it handles, the next model it needs and its reason for going around you. One column records what it called last week. Update it after every conversation.
| Team | Data it handles | Called last week | Reason |
|---|---|---|---|
| Team A | Customer data | Its own cloud AI subscription | Not sure what is allowed |
| Team B | Internal documents | A direct API | The cost is theirs |
| Team D | Customer data | A direct API | Fear of an old model |
Write a one-page guide to the AI policy. It says which models and tools suit which data, which route customer data must take, and how to request a new model. Security, model risk and compliance sign it off, and the platform’s onboarding page links to it.
And publish a service promise, reported monthly, with a public queue that shows where each requested model sits. Publish the misses as well as the hits, because teams count them anyway.
- Access
- Within one working day
- New model review
- Started within a set number of days
- Delay the gateway adds
- Measured on each call
- Uptime
- Reported with every miss
Your first three steps
- Ask every team lead building with AI, direct callers first, what they called last week, with what data, and why. Mark each team’s reason, and the mix tells you whether to fix the rules, the funding or the model delay. Raise any direct route carrying customer data with operational risk or security as an open control issue, so it gets an owner and a fix date before an audit finds it.
- Onboard yourself the way a new team would, and time every step until your first successful call, plus the delay the gateway adds. You’ll know which steps to cut before a team tells you.
- Book 30 minutes with your sponsor to replace “teams onboarded” with “teams in production” as the number the platform reports. If you change it now, it’s simply a decision. If you wait until the budget review, it’ll look like an excuse.
What to measure, and how
Check these every week, in this order.
How many teams run live features through the platform?
A team in production depends on you, and vouches for you to other teams. Count the teams whose live feature sent real traffic through production keys in the last 7 days, leaving out your own demos and health checks. Track the teams that stopped separately.
If the count is flat while onboarding rises, or a live team’s traffic falls, call that team’s lead within the week. Ask what it calls now, and why.
How much of the company’s AI spend goes around the platform?
Any AI spend outside the platform and outside approved logged tools is traffic nobody masks or logs. Each week, compare the AI lines in your cloud billing export, including teams’ own AI subscriptions, with the spend the platform logs.
If direct spend grows faster than platform spend, find out which reason is holding back the three largest direct spenders.
How long does a new team take to reach its first production call?
The longer a new team waits, the easier a direct call starts to look. For each new team, record the days from its access request to its first successful production call, which API providers call time to first call [5]. Record the days from a new-model request to the model’s arrival on the platform too.
Both should be measured in days, and they should be coming down. If either runs to weeks, find the slowest step for the last three teams and remove or automate it before you add any feature.
Real examples
Spotify
Spotify built Backstage, a developer portal, for its own engineers. It started with a real pain. Spotify had so many microservices that it was hard to know who owned what.
Nobody at Spotify has to use it. Even so, after about a year, Helen Greul, who heads engineering for Backstage, reckoned 99% of Spotify’s developers had adopted it, “because it was convenient to do so” [6].
Then other companies picked it up, more than 2,300 of them. By Spotify’s own estimate, they average about 10% adoption.
Spotify also went after the cost of getting started. Setting Backstage up took some 70 steps, and in 2023 Spotify previewed a tool to bring that down to between three and five. Greul called ease of use “the main theme of this year” [7]. For the companies stuck near 10%, she said, “Developers have to see how this is beneficial for their day-to-day” [6].
The lessonTeams won’t use a platform just because it’s there. Solve a pain they feel every day, and count the steps a new team takes before you add another feature.
Sources
- Microsoft and LinkedIn release the 2024 Work Trend Index on the state of AI at work — Microsoft, 8 May 2024. https://news.microsoft.com/source/2024/05/08/microsoft-and-linkedin-release-the-2024-work-trend-index-on-the-state-of-ai-at-work/
- 92% of Indian knowledge workers use AI in the workplace — Microsoft India, 16 May 2024. https://news.microsoft.com/en-in/92-of-indian-knowledge-workers-use-ai-in-the-workplace-finds-microsoft-and-linkedin-2024-work-trend-index/
- Platform engineering maturity in 2026: What the data tells us — platformengineering.org, 13 Jan 2026. https://platformengineering.org/blog/platform-engineering-maturity-in-2026
- Navigating the LLM Landscape: Uber's Innovation with GenAI Gateway — Roopansh Bansal and Tse-Chi Wang, Uber Engineering Blog, 11 Jul 2024. https://www.uber.com/us/en/blog/genai-gateway/
- Why Time To First Call Is A Vital API Metric — Art Anthony, Nordic APIs, 9 Nov 2021. https://nordicapis.com/why-time-to-first-call-is-a-vital-api-metric/
- How Spotify Achieved a Voluntary 99% Internal Platform Adoption Rate — Jennifer Riggins, The New Stack, 5 Sep 2023. https://thenewstack.io/how-spotify-achieved-a-voluntary-99-internal-platform-adoption-rate/
- Behind the scenes, Spotify's Backstage a work in progress — Beth Pariseau, TechTarget, 7 Nov 2023. https://www.techtarget.com/searchitoperations/news/366558592/Behind-the-scenes-Spotify-Backstage-a-work-in-progress