Clever Cloud was born out of a desire to automate repetitive infrastructure management tasks, while ensuring that developers benefit from the right level of reliability and security. From the outset, we have focused on simplicity and intelligence, putting them at the service of our teams and our customers in a transparent way. That’s how we built our platform and our in-house orchestrator: all you need is a git push and we’ll take care of the rest.
With the emergence of needs in terms of AI, we took the same approach. We knew that our added value could not lie in simply making GPUs available in instances. We had already done this a few years ago, and we know the limits. So the question we asked ourselves was: how could we offer something better?
Clever AI: a range of solutions to simplify your life
Our platform enables you to rapidly prototype and deploy AI-enabled applications in production. Whether through its GitOps nature or its features, such as the Configuration Provider for providing different applications with similar elements such as API keys. To accompany certain use cases, we have, for example, used an add-on provider to generate and distribute tokens by service. We’ll come back to this later.
But during our discussions with various customers and prospects, we realised that the needs were elsewhere, and that they were many and varied. Of course, the request for tens (sometimes hundreds) of thousands of euros of free GPU instance credits to do what some hyperscalers do came up regularly, but we also had requests for minor modifications to the platform to adapt it to certain specific needs, to offer more complex but meaningful solutions, and to facilitate links between different tools. That’s what we focused on.
One of the first essential building blocks in our strategy was Otoroshi. We quickly identified recurring needs and requests for API control and management, which are at the heart of AI applications. After a few internal PoCs and a discussion with its lead developer Mathieu Ancelin, we agreed that Otoroshi had everything we needed to meet the challenges we had identified. The work we’d already started on the LLM extension increased the range of possibilities. We decided to join forces to take things further, together.
50 shades of Otoroshi
All that remained was to choose the form in which we were going to offer it. Because that’s the strength and weakness of a complete, detailed and extensible open source product like Otoroshi: it can sometimes be complex to understand and use, requiring specific expertise.
This is what some of our customers are looking for and asking for. That’s why we first offered this API Gateway as a Clever Cloud add-on that can be deployed in just a few clicks, in a pre-configured, easy-to-update version that takes advantage of our features such as auto-scaling and the diversity of our offering. Otoroshi is available in our public, private, air gap, edge or on-premises deployment zones.
So you can have your own Otoroshi that comes with extensions such as LLM management, WAF Coraza or the recent Biscuit studio. But this didn’t meet the other needs expressed, which were for more turnkey solutions, particularly for managing the various AI services with APIs linked to LLMs.
Unifying access to AI services
That’s why we decided to offer Otoroshi in what we call ‘serverless’ form: you don’t have to create an instance, configure CPU/RAM/storage or adapt it to the load. You create a service, adapt it to your needs and use it. It’s as simple as that.
Clever AI‘s first tool allows you to declare a multitude of AI providers that you wish to use, ‘the Clever way’: we offer the major players in the sector such as Anthropic, DeepSeek, Google Gemini, OpenAI, x.AI, but also those that have developed on our side of the Atlantic such as Hugging Face, Mistral, OVHcloud or Scaleway. And of course, any Ollama instance or API that is compatible with the OpenAI ‘standard’.
Anyone familiar with this market will know that this is not a trivial matter to set up. While everyone agrees on this way of exposing APIs, everyone comes with their own set of complexities and functionalities that we smooth over through our implementation. Some have a specific way of managing authentication, declaring models, exposing reasoning tokens, managing or not function calling and MCP, and so on.
And if you simply want instant access to the multitude of configured models without having to set up anything else, you can just open the chat interface of your endpoint, created automatically, which can be customized to your colors. You can do this using your AI services API keys, or we can provide them on request.
With Clever AI, once your service has been created, you have just one endpoint, one API key, and everything responds to the same format managed by a multitude of frameworks and clients on the market. So all you have to do is use it in existing code or in the application of your choice, as you would with a Mistral or OpenAI endpoint, for example.
And if you ever just want direct access to the multitude of configured models without having to do anything else, you can simply access your endpoint’s chat interface, created automatically, which can be customised to your colours. You can do this using your API keys or we can provide them on request.
Not just another endpoint, your endpoint
This unification is not the only strength of our tool. For each element created, we allow users to add their own rules. This makes it possible to adapt the use of AI to the needs and constraints of the business: organising access to certain suppliers/models by team, reducing shadow IT, avoiding data leaks, ensuring resilience and controlled management.
Here, we are taking advantage of Otoroshi’s mechanisms, which enable us to multiply API keys by assigning them usage limits by period, reduce costs via semantic caching, add limits by number of tokens used, specific context, define and activate moderation rules, guardrails, response functions or templates, and define model load balancing/failover policies. And all this on the fly, via our API or Clever Tools.
This means that you can, for example, script how to ask a model a question in one context, continue the exchange with another model in a different context, and so create a flow of agents with a single endpoint, a single access point, but an infinite number of possibilities.
Clever AI: at the heart of the Clever Cloud ecosystem
One of the key points of this offering, as we are building it, is that it benefits from the full range of Clever Cloud services. Secrets can be stored in Clever KMS, which was announced at the Clever Cloud Fest, chat is based on Materia KV for storing conversations, authentication is carried out via a Keycloak instance for our serverless offering, functions can use our FaaS, and so on. It demonstrates the value of this diversity of solutions.
It benefits from the expertise of all our teams and partners, just as you could with your own tools and applications. While Clever AI is a whole, it is also a brick on top of which you can imagine, create and build services, by reusing all or some of its elements. Available to the imagination of our customers.
Would you like to discuss it? Don’t hesitate to contact us.
Let’s build the story together to meet your needs
Otoroshi, the chat service and serverless endpoint, are just the first step. Our vision of AI that is multi-vendor, multi-model, open and simple to use, and tailored to the needs of businesses, goes far beyond the LLMs and tools we have today.
If you have any ideas, particular requirements, or even just questions, please don’t hesitate to talk to us about them, online, at the conferences we attend, or in more direct discussions with our teams. We’re here to listen. As we have done up to now, we will continue to create Clever AI, like our other products, for you and with you.