At work I was tasked to build an enterprise-level AI environment as an alternative to platforms like OpenAI.

Let me first state the use cases: chatting with codebases, technical documentation, knowledge bases, chatlogs, social media posts, public business data, search results from the web (independent from Google).

I went with a Debian-based server equipped with an Nvidia H100 GPU. For the interface, I chose Open WebUI, an open-source solution with functionalities similar to proprietary AI interfaces. The installation was easy thanks to a pre-configured Docker container that includes Open WebUI and Ollama, an LLM runner. One minor downside currently is the outdated Ollama version in the container. I've contacted the creator and an update is expected soon.

I'm now using Command A, an enterprise-level LLM, on one single 80GB GPU, though two GPUs would be optimal. Very soon, the open-sourcing of Qwen 2.5 Max and Meta's upcoming Llama 4 release will make it even more interesting.

That said, what about those use cases I mentioned earlier? Here’s a quote from Linus Torvalds, the creator of Linux:

AI is going to change the world... but right now it's 90% marketing and 10% reality... In 5 years we'll see what of the AI is getting used everyday for real workloads.

So, what's next? I'll try out Retrieval-Augmented Generation.

That's all I have for now, bye.