Buying More or Keep On Using: How an AI Assistant Became Ready for Production

I'll show you how to count memory and KV-cache, how inference layer solutions change the load profile, and then we'll move on to our implementation in Deckhouse.