SAP: How to Implement AI Applications On-Premise
Many companies continue to run their SAP systems on-premises and are faced with the question of how they can harness the potential of artificial intelligence without fundamentally overhauling their existing IT landscape. Bridge technologies make it possible to connect AI applications and autonomous agents directly to SAP data and business processes today. This opens up new opportunities for the gradual and cost-effective automation of complex business processes.
The DSAG Investment Report 2026 shows that many companies continue to run SAP on-premises. The transition to the cloud is proceeding much more slowly than SAP had planned. SAP’s AI strategy relies on Joule as a generative AI assistant. As a cloud service, however, Joule still falls short of the reality faced by many companies. In a highly dynamic environment, users are faced with the question of whether to wait and see or risk ending up in a dead end by implementing AI services within their tried-and-true systems. AI solutions still rarely utilize SAP data. At the same time, the DSAG study also shows that 43 % of companies have already implemented their first AI use cases. These primarily process knowledge from documents, which can be searched more quickly and provide answers in natural language. The direction is set by the use of AI in business processes: Business AI applications have the potential to significantly increase productivity. The goal is clear: processes should be embedded in agents that can initiate and execute changes largely autonomously using operational business data. Routine tasks should then be mapped into AI-driven workflows. However, the use of SAP data is essential for this.
AI-Driven Workflows: Between a Pipe Dream and Reality
The initial euphoria surrounding AI in the SAP community has now given way to questions about its added value. While queries and image generation are already standard, the focus in the future will be on autonomous AI-driven workflows. At the same time, the view has taken hold in the SAP community that AI applications can only be used in the cloud. In reality, however, SAP users do not have to wait for S/4HANA or the cloud. With bridge technologies, such as the Billion With the AI Construction Kit, AI applications and agents can already be implemented on-premises using ABAP-based technology and leverage SAP data. This enables SAP customers and SAP partners to make their solutions AI-ready and bridge the gap between highly innovative and volatile AI environments and their proven on-premise landscape. In these solutions, AI functions run outside the core ERP system and securely access SAP data via defined interfaces using authorization controls. Business objects and process data are utilized via standardized APIs, allowing the AI applications to operate independently of the ERP system. They are explicitly designed to comply with SAP’s restrictive API policy and can be seamlessly integrated into Joule, migrated to the cloud, or used in a hybrid environment via an OData service. The framework allows for the development of on-premises agents that can trigger change processes based on AI analyses. Agents can thus be used today and continue to evolve as technology advances.
Cost-Benefit Analysis Slows Down AI Adoption in Companies
The biggest hurdles to AI adoption lie less in the architecture and more in scaling to complex business processes. The use of large language models can quickly lead to high operating costs, particularly in complex and data-intensive SAP processes, even though the additional benefits may initially be limited. Language models do not process content directly as words, but rather break down text into so-called tokens using a tokenizer. Even extensive documents or large data sets can generate several hundred thousand to over a million tokens. Since many AI services charge for both input and output tokens, fees can quickly add up with large data sets. This results in high costs even in the early stages of implementation, before any corresponding economic value is realized. For this reason, highly complex SAP processes cannot be transferred one-to-one to AI models. Saving on tokens is therefore the order of the day. Milliarum therefore relies on on-demand function calls: Instead of continuously feeding large data sets to a language model, SAP data and business logic are queried only when they are actually needed for the task at hand. The AI therefore does not need to „know“ the SAP processes itself, but can access existing functions and information in a targeted manner. This reduces token consumption and operating costs while preserving the proven SAP business logic.
Flexible, Open, and Scalable: Implementing AI Applications in SAP
By retrieving Customizing data and utilizing standard SAP functions, the Milliarum AI Construction Kit can be used to map even complex, chained business use cases. Companies can freely choose the appropriate LLM provider or operate LLMs themselves and deploy the most cost-effective model depending on the use case. A needs-based function call ensures that only the data relevant to the respective use case is passed to the model. Batch functions enable the efficient processing of large volumes of data and lay the groundwork for AI-driven process automation. The open Model Context Protocol (MCP) also enables integration with other AI platforms, such as ChatGPT or Claude AI. Over 1,000 predefined functions for core SAP modules such as FI, SD, MM, CO, PS, and CS form the basis for implementing automated use cases. Typical use cases range from conversational SAP access—where users can, for example, retrieve delivery statuses, prices, or FI documents using natural language, to process automation, in which AI specifically executes SAP functions—such as document creation, procurement process handling, or bulk processing of business data. For example, invoices can be generated as PDFs and then automatically sent via email. In addition, there are document-based use cases in which emails, invoices, or delivery notes are extracted, validated, and transferred into SAP structures, as well as support and development use cases, such as providing key users or developers with context-specific SAP information. This allows even users without in-depth SAP knowledge to quickly and easily access SAP data and processes—for example, in call centers or customer service when handling inquiries about delivery orders or prices. There is no need for time-consuming training, and in addition to reducing onboarding time, significant time is saved in handling customer inquiries. Real productivity gains lead to a rapid return on investment.
The Bridge to Self-Organizing Business Processes
By automating complex workflows, these bridge technologies lay the foundation for AI-based agent systems that will increasingly execute processes autonomously in the future. In this way, manufacturing companies are already generating tangible added value in existing on-premises environments while simultaneously laying the technological foundation for the next generation of AI-powered agent systems and agent-to-agent communication. In the future, this will enable the integration of specialized AI agents, each of which will handle clearly defined tasks within a process. In this context, multiple agents interact with one another—for example, a procurement agent, a logistics agent, and a finance agent. Each agent accesses its own data, tools, and SAP functions. These agents can pass tasks to one another, validate interim results, and, if necessary, integrate additional agents into the process. This will result in dynamic, self-organizing process chains in the future, in which a central system no longer makes every decision, but rather specialized AI units work together in a coordinated manner. This enables significantly greater flexibility in complex, highly variable business processes, reduces the effort required for integration between systems, and lays the foundation for adaptive, context-dependent process control across system and organizational boundaries.
Originally published at https://industrieanzeiger.industrie.de/management/it/sap-so-lassen-sich-ki-anwendungen-on-premise-realisieren/
