Otoroshi with LLM is available as an add-on on Clever Cloud
After some weeks of testing, the Clever Cloud’s Otoroshi with LLM add-on, is available in public beta. Thus, you can deploy the service, from Console, API or Clever Tools, and use it in minutes.
We developed this product with its creator and core developer: Mathieu Ancelin from Cloud APIM. Otoroshi is an open source reverse proxy that allows you to create your own routes, manage authentication, authorization, and rate limiting. It helps you to expose services with enterprise needs in mind, as you can manage organizations, teams, service groups, with event management, data export, multiple secret stores support, etc. And it can be managed from a web interface or as an API.
On Clever Cloud, it comes batteries included, pre-configured with features such as Coraza Web Appplication Firewall or LLM extension. The latter allows you to manage AI services from many providers (Anthropic, Groq, Hugging Face, Mistral, OpenAI, OVHcloud), Ollama instances or any OpenAI API compatible endpoints.
All this with a unique interface, adding token management, rate limit, context, validation, moderation, etc. Create your own agents and flows with an only URL, distributing tokens to your team or customers, for cURL requests or integrations to many services and tools… or your own applications hosted on Clever Cloud.
It’s based on a Java application and a Redis® database. Once deployed you’ll get a management URL with credentials. Want to learn more? Feel free to let us know what you think and ask your questions in our GitHub Community.