SAP TechEd 2024: What's new on the latest AI developments

In this TechEd we see that a great deal of innovations are focused on Joule, SAP’s digital AI assistant that is steadily being implemented across their entire portfolio. One big announcement is Joule Studio, which is planned to be available during Q1 2025. Joule Studio allows you to customize and extend the built-in functionalities of Joule in accordance with your business needs. For example, you can build a custom task in Joule (also called ‘skills’) for automatically retrieving consultant rates from external data sources. On top of that, Joule Studio allows you to build autonomous AI agents for solving complex workflows requiring planning or reasoning capabilities, complementing more repetitive workflows built in SAP Build Process Automation. These AI agents are able to collaborate with Joule and each other, allowing them to solve cross-domain business cases in an automated fashion.

In another announcement, Joule can also provide more grounded responses by showing which documents were used as source material. This makes Joule more transparent, as you can trace back the knowledge being that is being leveraged for its responses. Here, document grounding is a service that can be activated in the SAP Business Technology Platform (BTP). Other innovations of Joule include its integration in SAP’s mobile apps, the first iteration of which is planned for the SAP mobile Start app in Q4 2024. Furthermore, Joule’s ABAP developer capabilities, such as code completion and unit test generation, will be enhanced with a Large Language Model (LLM) trained specifically on ABAP code. This feature is planned for Q1 2025.

Besides Joule, other major announcements have been made regarding the Generative AI Hub, which is the part of SAP AI Core that delivers Generative AI capabilities. The biggest announcement is the orchestration functionality, allowing you to customize AI-workflows with prebuilt features. Some of these orchestration features include data masking for anonymizing sensitive data, prompt templating for streamlining the definition of prompts, content filtering for filtering undesirable inputs and outputs, and grounding for reducing hallucinations. Each of these orchestration features can be turned on or off and are universally accessed by all Large Language Models (LLMs) through an harmonized API.

Another major announcement regarding the SAP AI core in general is the availability of new Software Development Kits (SDKs). Specifically, a JavaScript SDK is now available for developing custom (Generative) AI-workflows in the SAP AI Core. SDKs in Java (Q4 2024) and in ABAP (Q1 2025) will also be available in the near future. In a smaller announcement, SAP announced new partner-built models that will become available in Generative AI Hub, such as the Large 2 model provided by Mistral AI. All currently available LLMs in the Generative AI Hub are managed and maintained by SAP, meaning that the models live within the SAP Infrastructure.

Moving on from the Generative AI Hub, SAP has announced the SAP Knowledge Graph, a graphical network that provides semantic context about SAP’s structured data artifacts, including ABAP tables, CDS views and fields. In essence, the SAP knowledge Graph forms a bridge between structured data, such as tables and views, and natural language. The Knowledge Graph can be leveraged by other LLM to prevent hallucinations when generating responses about structured data. SAP has done this themselves by developing SAP Foundation Model, an LLM that uses the Knowledge Graph for retrieving insights from structured data. This is an interesting take of SAP, as it shows that LLMs can also be used in use cases that normally would require Narrow AI solutions (e.g. regression or classification models). The SAP Knowledge Graph can be accessed and maintained with the SAP HANA Cloud Knowledge Graph Engine. This technology complements the existing Graph Engine and also provides built-in graph algorithms for retrieving relevant data in conjunction with other data types, such as vector embeddings (i.e. Vector Engine).

Finally, let’s look at some smaller announcements in the area of AI. Firstly, SAP announces the kaleidoscope functionality for the embedded AI functionalities based on SAP Central Business Configuration (SAP CBC). This functionality gives you an overview of all embedded AI features and allows you to activate or deactivate them for each application in each tenant. Secondly, SAP mentions an ethics.