AI solution for the compliance domain

Brief

Previously, Compliance Aspekte didn’t make use of artificial intelligence. All algorithms were rule-based, and the platform had a simple chatbot that could only answer a small predefined list of FAQ questions.

Generally, it worked like this: the client described their assets (premises, digital assets, employees, processes, etc.), each of which would have certain requirements for a given standard. For example, a user might provide assets to the system for IT-Grundschutz certification. Then, according to the asset type, the system would match relevant sections of IT-Grundschutz — for example, sections that correspond to buildings. The General Building section of IT-Grundschutz has a set of divisions with certain requirements (such as General Building Planning Security). A user preparing for compliance would create a list of tasks pertaining to each section of IT-Grundschutz and transfer it to DevOps, admins, and managers to complete them.

AI Implementation

The Compliance Aspekte platform has a Helga chatbot. Previously, the chatbot was built on top of one of the major SaaS Chatbot providers. It didn’t use an advanced LLM, and the machine learning capabilities were primitive and limited. The chatbot was only for support, such as if a user did not understand how to create a new asset or task. It answered questions, but not always correctly and relevantly.

We proposed turning the chatbot into a co-pilot system that would not only answer questions but also interact with the platform’s entities and allow users to manage them through the chatbot interface. The co-pilot system sees the entire context and all requirements, understands where exactly the user is on the page, and gives them tasks to complete in order to obtain certification. A user can also upload documentation linked to an asset into the system.

Features

Compliance Aspekte organizes company assets and documentation to prepare for compliance, creating tasks and synchronizing them with Jira and other task managers.

The chatbot can generate tasks according to the requirements of a specific standard or regulation. The user supplies assets and requirements, and the system creates an action plan to fulfil those requirements.

If a company has any policies and described processes for a certain asset, they can be uploaded to the system as a set of documents. The system indexes these documents and allows the user to search them and ask questions. The chatbot then finds relevant parts of documents and summarizes the content or issues a list of actions that must be taken to address compliance. Users can ask certification-related questions through the chat; support for links, attachments, and screenshots will be added soon.

The chatbot system is also a universal compliance assistant, which means it can answer questions about various standards. For example, if a company has mobile devices, the user may want to understand which regulatory requirements are relevant to mobile devices. The system is able to infer relevant regulatory requirements and suggest actionable tasks to address compliance problems.

In essence, we created a co-pilot system that, to a certain extent, takes the responsibilities of a compliance consultant but at a much lower cost.

A user can ask an AI chatbot questions they would usually ask an expert and get relevant and insightful answers. And even if our AI cannot handle certain scenarios yet, Compliance Aspekte is always ready to suggest skilful and experienced advisors. In fact, the co-pilot system also serves the role of first-line support, reducing the amount of manual work and entry-level personnel.

Services Provided

The solution design phase that included a series of workshops with the client that resulted in a refined solution architecture and specifications

AI co-pilot development, integration, and tuning

Solution support and maintenance

Technologies

Challenges

Onboarding into the product

A major challenge was to onboard into the product and build a clear picture of its components and relations. In order to design and implement the solution, it was necessary to understand how compliance documentation is usually arranged, used, and updated. Our team learned a lot while working on this project and is using the gained knowledge in our own ISO 27001/27002 certification.

Technical limitations of the LLM provider

The second challenge was addressing the technical limitations of the LLM provider (in our case, OpenAI). Some of the prompts and request chains are pretty bulky, and it takes a long time to process these requests. This affects the system’s performance, and therefore, it’s always important to establish real-time performance monitoring with alerts and metrics to make sure that the service works as expected. More and more users are getting into OpenAI and other LLM SaaS solutions, which significantly increases their load and occasionally drains the compute resources, making it difficult to scale. Taking this into account, it’s vital for the business to select the LLM provider, not only considering the quality but also the performance and to secure decent usage quotas. Also, we did some optimizations to improve performance by caching some of the already processed data. So it can be simply reused without the need to engage with the LLM again. And, of course, we did extensive prompt tuning.

Results

AI guides users in tracking and analyzing compliance progress

Helps users create task checklists

Simplifies platform navigation and aids users in document searches.

We created a comprehensive AI solution for the compliance domain that has already proven to work efficiently with diverse sectors such as telecommunications, engineering, manufacturing, and banking. The AI co-pilot designed and implemented by S-PRO extends Compliance Aspekte’s potential, offering new possibilities to users. AI guides users in tracking and analyzing compliance progress while assisting them in creating task checklists.

Also, as a knowledgeable advisor, the AI asks insightful questions to comprehend a company’s processes and later leverages that understanding to generate suggestions on how to implement the requirements or address threats. Additionally, it simplifies platform navigation and aids users in document searches.

The co-pilot was released to production in September 2023, presented at the well-known it-sa conference, and has already received some interest from big companies.