In June, S-PRO team hosted an expert webinar “How AI Chatbots are revolutionizing the Modern Workplace”. We focused on the growing adoption of AI chatbots and assistants, its evolution, integration challenges and trends, and the opportunities that AI technologies offer to modern workplaces.
As a cherry on top, our speakers shared their own practical advice on adoption and usage of AI chatbots, alongside with the recent case studies by S-PRO that enabled smoother operations for a couple of client businesses. This event was a fantastic opportunity to engage with experts in the field, because the S-PRO team has been working with ML & AI solutions for almost 5 years already, before the hype times of ChatGPT.
Get access to the webinar recording here.
Meet the speakers:
Dmytro Voitekh
Dmytro Voitekh, CTO at S-PRO, an engineering & data solutions consultant with a key focus on AI and ML. He has successfully led many startups, including the EdTech startup Chozy that received funding from Google and reached a valuation of $1 million. He was a Machine Learning Lead in one of the major media platforms that provided animated content to biggest social platforms and messengers. Dmytro frequently speaks at conferences and workshops, runs mentorship sessions, sharing his expertise in machine learning and artificial intelligence.
Maksym Golovan
Max Golovan, Head of S-PRO Netherlands and the Amsterdam branch, has over 10 years experience in business development in top management roles and 7 years in the hospitality industry. As a co-founder and CEO of Edison Space Hub, Max brings a wealth of experience in event planning, hosting, and startup consulting. In his current role, he focuses on business development and facilitating AI adoption journeys for numerous clients. Feel free to connect with Maxim Golovan on LinkedIn.
Chatbots, Large Language Models, and Assistants
Imagine you’re trying to get quick support for a task at work, and instead of navigating through endless FAQs, you simply ask an assistant. That’s where assistants, chatbots, and LLMs come into play.
Chatbots: More Than Simple Scripts
AI chatbots, like the ones you're familiar with in customer support, have evolved. They no longer rely solely on predefined scripts and decision trees like the older rule-based systems did. Instead, today’s artificial intelligence bots, such as OpenAI chatbots, leverage AI to understand the intent behind user queries. This shift allows them to provide more context-aware, personalized responses.
In essence, chatbots can classify intent, connect with databases, and offer solutions based on real-time information, making them ideal for repetitive yet critical tasks. For instance, if you’re interacting with an AI chatbot app, it can immediately retrieve data or perform actions like updating records without human intervention.
Large Language Models: The Brain Behind ChatGPT
Large language models (LLMs) are the powerhouse behind many modern AI chatbots, including ChatGPT and its alternatives. These models are pre-trained on vast amounts of data, allowing them to generate human-like responses. But they don’t stop there. With instruction tuning, like what we see in ChatGPT, these models can be fine-tuned to perform specific tasks.
They can analyze context, detect sentiment, and even predict the next steps in a conversation. For example, using a few-shot prompting technique, you can provide examples of desired outcomes, and the AI chatbot app can generate responses that fit your particular needs. This capability turns them into valuable assets for any business looking to enhance customer service or internal communication.
Assistants and Agents: More Than Just Chatbots
AI-powered assistants have taken the idea of chatbots to a whole new level. These systems can do more than just respond to inquiries – they can plan, reason, and even carry out complex tasks based on your behavior. Imagine an assistant that not only answers your queries but also orchestrates multiple actions simultaneously.
Assistants based on AI go beyond traditional AI chatbot apps. They incorporate advanced features such as memory, reasoning, and task decomposition, which allows them to perform complex, multi-step processes. Whether it’s handling queries across various platforms or triggering a series of actions, AI assistants are becoming the go-to tool for optimizing workflows in the modern workplace.
Key Chatbots and Assistants Features
Knowledge Reasoning vs. Safety and User Experience
A critical balance in AI chatbots is between knowledge reasoning and maintaining safety and user experience. This involves ensuring that chatbots provide intelligent responses while safeguarding against inappropriate manipulations. There were many techniques to collapse ChatGPT and bypass its guardrails and security limitations at the beginning. Users were able to ask unsafe questions, triggering some abusive responses from ChatGPT.
“This really shows how prone those technologies are to different manipulations and tricks. For cases that we haven't yet explored, the behavior won’t be consistent. At the same time, we don't want to sacrifice the quality of the model, the way it can be creative to handle our problems”, highlights Dmytro.
Tools and Integrations
A major strength of any AI chatbot app lies in its ability to integrate with various tools and platforms. OpenAI chatbots, including ChatGPT alternatives, offer integrations that allow users to connect with web searches, APIs, and third-party systems like Wolfram Alpha or scheduling tools. To book a flight or to create some diagram based on the description you provide – the more plugins and integrations you have, the more high quality results you get.
Retrieval Augmented Generation (RAG)
Retrieval Augmented Generation (RAG) is a powerful feature that combines traditional information retrieval with AI generation to provide more accurate and contextually relevant responses. Initially, this process required extensive manual coding and setup. However, AI providers now offer no-code solutions, enabling businesses to easily integrate vast knowledge bases into chatbots. RAG allows AI chatbots to efficiently pull from existing data sources to generate high-quality answers that feel more natural and precise.
Personalization
Personalization allows AI chatbots like ChatGPT to tailor their responses based on stored user preferences and past interactions, making each interaction more relevant. By using these stored conversations, chatbots can adapt over time, offering more user-specific assistance without needing manual adjustments.
Multi-modality
Multi-modality enables AI chatbots to handle inputs and outputs across various formats – text, audio, images, and even video. For instance, users can upload a PDF for summarization or use voice commands. Or you have an image with some bill that you want to do OCR on.
No-code and Low-code Tools
No-code and low-code tools, such as Azure AI Studio, allow users without coding skills to design and deploy chatbots. These platforms enable the creation of complex workflows and integrations using visual interfaces. Pre-built intent classifiers help to trigger the existing AI modules, which are likely also based on LLM.
AI Chatbots Use Cases at S-PRO
There are many use cases for AI chatbots, but we'd like to highlight those we use at S-PRO and which are battle-tested.
Internal Information Retrieval
We have a couple of internal tools and AI chatbots for internal information retrieval, leveraging retrieval-augmented generation. These tools help find specific details in documentation, internal processes, and projects. They also assist our pre-sales team in finding similar past opportunities to take as a reference, improving efficiency and outcomes.
Productivity Tools
AI chatbots enhance productivity by connecting to data sources like Google Calendar, email, and meeting recordings. They provide recommendations, summarize activities, and suggest tasks. For example, using an AI chatbot app like Gemini natively integrates these services while ensuring privacy and compliance.
General AI Assistant
For general assistance, AI chatbots like GPT-4 can answer questions about various tools and domains. They can compensate for a lack of precise keywords by transforming abstract queries into accurate search results. This capability helps users find articles or information they might struggle to locate through traditional search engines.
Media Listening Tools
AI chatbots can monitor media for mentions of your brand, competitors, or industry trends. They analyze sentiment, track engagement, and provide insights to inform marketing strategies. This real-time monitoring helps companies stay ahead of the competition and respond quickly to public sentiment.
Business Processes Planning
AI chatbots assist in business process planning by analyzing data, identifying bottlenecks, and suggesting improvements. They can simulate different scenarios to predict outcomes and recommend the best course of action. This capability helps businesses optimize their operations and make informed decisions.
Coding AI Assistant
In our company, engineers use AI chatbots like GPT-4 for coding assistance. Team leads and developers prefer on-demand AI support for tasks like refactoring code or summarizing file contents. This approach contrasts with continuous auto-completion like GitHub Copilot, providing more control over when and how AI assistance is utilized.
Content Generation and Copywriting
AI chatbots are invaluable for content generation and copywriting, especially in marketing. Tools like Gemini and ChatGPT are popular for creating text content. Some team members also use AI to generate images for posts, utilizing platforms like DreamStudio from Stability AI for more detailed and customizable image creation.
Sales Outreach Automation
AI chatbots streamline sales outreach by automating the process of contacting potential clients. They can personalize messages, schedule follow-ups, and analyze responses to improve engagement strategies. This automation saves time and increases the effectiveness of sales efforts.
"But I specifically use more specialized tools for that, like DreamStudio from Stability AI or other dedicated AI-based tools. These allow for more editing and granular prompting than what's possible with the text bar provided by ChatGPT", says Dmytro.
Advice on Adoption and Usage
To successfully and pain-free integrate AI chatbots for your organization, our CTO Dmytro has prepared a set of 6 best practices to follow.
Choose Trusted Solutions. Start with reliable platforms like OpenAI or Azure OpenAI. These providers prioritize privacy and do not use your data for analytics or fine-tuning, ensuring a secure foundation for your AI chatbot app.
Educate Your Team. Conduct workshops or encourage your team to take popular prompt engineering courses, such as those from Deep Learning AI. This empowers employees to effectively use AI tools like ChatGPT and ChatGPT alternatives.
Collect Feedback. Implement feedback forms and establish clear points of contact for employees to share their experiences. Regularly collect and process this feedback to improve future iterations of your AI systems.
Monitor Operational Costs. Keep an eye on the costs associated with AI APIs. For large organizations with high request volumes, consider self-hosting solutions using publicly available LLMs, fine-tuned with your custom datasets.
Customize When Necessary. Only after establishing a solid foundation should you consider customization for specific tasks. Use more specialized services or develop custom solutions to address unique requirements.
Diversify Providers. To mitigate scaling issues and improve stability, make your workflows compatible with multiple providers. This reduces risk and enhances the reliability of your AI chatbot operations.
Challenges and Future Trends
Challenges
Hallucination, limited capacity
One of the most significant challenges AI chatbots face is hallucinations, where they generate incorrect, false information or nonexistent objects. The hallucination rate varies with different tools and problems but can be mitigated using AI guardrails, entity recognition, explicit post-processing, manual and automatic fact-checking, or external APIs.
Dmytro Voitekh emphasized, “If fine-tuning models isn't possible, provide as many different knowledge bases or just databases as an entry point for your chatbot, so it will be able to contextualize the queries as much as possible.”
Moreover, not all tasks should be given to AI chatbots, especially simple ones. If you want to find the closest city to the given one, AI will definitely do that for New York. But for smaller towns, the risk of hallucinations is huge.
Trustworthiness in LLMs
Trustworthiness in AI chatbots, especially those using large language models (LLMs) like OpenAI chatbots, is another challenge. Ensuring reliability involves outlining key risk, formalizing LLM policies, workflows, standards and following best practices.
Also, the metrics are crucial to measure AI performance. These can be text smoothness, language adherence, and specific outcomes like user engagement and conversions. On top of this, feedback mechanisms are vital – without it, we can't be sure the AI chatbot app performs well.
AI Regulation
Pretty soon, artificial intelligence bots will face AI regulations issues. Recently, the European AI Act was accepted and will take effect soon. This Act addresses the risks associated with AI applications. Each AI service, including AI chatbot apps, will be classified based on metrics and policies outlined by European authorities. Also, there’s the Blueprint for an AI Bill of Rights by White House which is pivotal for the AI world as well.
Businesses must comply with these regulations. While we await detailed instructions, companies that adhere to standards like GDPR, SOC 2, or ISO 27000 are already on the right track. It’s a matter of a few months before the AI code of conduct will appear.
Trends
On-device Assistants and SLMs
In our opinion, one of the most intriguing trends in AI chatbots is the development of Small Language Models (SLMs) and on-device AI assistants. Unlike large language models, SLMs are designed to run efficiently on devices with limited resources. Recent advancements have enabled the compression of models, allowing smaller models to achieve performance levels comparable to larger ones, such as the Llama 70 billion-parameter model, but with only 10 billion parameters. This is achieved by using curated, clean datasets tailored to specific tasks.
On-device machine learning isn't new, but its adoption has been slow. Simple apps like auto-completion tools on smartphones demonstrate its potential, but more sophisticated uses, like AI-driven chatbots built into apps, are just emerging. Apple's recent presentation at WWDC highlighted new on-device AI capabilities, which could significantly boost the adoption and performance of on-device ML.
These on-device AI chatbots bring many advantages, including faster interactions due to the elimination of network latency and the potential for offline functionality. However, for more complex queries that require the expertise of larger models like GPT-4, the system can still route these to the cloud. This hybrid approach balances efficiency and capability.
S-PRO Case Studies
Compliance Aspekte
The Compliance Aspekte tool automates user interactions within GRC solutions, enhancing efficiency and accuracy. It provides actionable advice by encoding compliance standards and examples, allowing users to understand their compliance status. The tool features a flexible interface that offers summaries, links, and suggestions tailored to specific compliance queries.
TravelPlanBooker
TravelPlanBooker utilizes AI to create personalized itineraries based on user preferences, such as family needs or activity interests. Integrates with external APIs to book flights, accommodations, and activities, streamlining the entire travel planning process. It enhances user experience by allowing natural language queries for trip suggestions, making it more intuitive and user-friendly.
E-commerce AI Assistant
This AI-powered assistant (NDA client) is capable of searching, recommending, and comparing products from various eCommerce platforms in one interface. It utilizes NLP to understand user queries and provide tailored product suggestions based on preferences and previous interactions. Integration with various eCommerce APIs enables real-time price comparisons, availability checks, and seamless booking or purchasing options.
