<iframe src="https://www.googletagmanager.com/ns.html?id=GTM-5MNKFGM7" height="0" width="0" style="display:none;visibility:hidden">
New call-to-action

CIONET Trailblazer: RAG as a Service (RaaS) – Deloitte’s approach to AI-driven knowledge management in the enterprise

Published by Daniel Eycken
April 30, 2025 @ 9:04 AM

How can enterprises transform scattered knowledge into strategic intelligence? As data volumes explode and information silos multiply, digital leaders face a growing challenge: making enterprise knowledge accessible, reliable, and actionable. Deloitte’s answer? RAG as a Service is a cutting-edge approach to AI-powered knowledge management built on Retrieval Augmented Generation.

In this CIONET Trailblazer, we sit down with Jan Van Looy, Director and (Gen)AI Specialist at Deloitte, to explore how RAG is reshaping enterprise decision-making. From breaking down silos to embedding AI into workflows, Jan shares Deloitte’s vision for smarter, safer, and more scalable knowledge strategies, and what digital leaders should know before taking the leap.

What are the biggest challenges that digital leaders face today in managing and utilising enterprise knowledge effectively?

Digital leaders today grapple with a myriad of challenges in the realm of knowledge management. One of the foremost issues is the fragmentation of knowledge across various departments. With teams such as HR, Legal, R&D, and IT often pursuing their own independent strategies and solutions, organisations risk creating silos of information. This not only results in duplicated efforts and investments but also makes it difficult to access comprehensive information quickly. Moreover, there are significant concerns regarding data security and compliance, particularly when sensitive information is involved. Digital leaders must navigate the delicate balance between enabling knowledge sharing and maintaining robust data governance. Additionally, ensuring that the knowledge management solutions are user-friendly and can be effectively adopted by all employees, regardless of their technical proficiency, presents another layer of complexity.

How can AI-powered knowledge management solutions like Retrieval Augmented Generation (RAG) help address these challenges and improve decision-making?

Retrieval Augmented Generation (RAG) is a technique that combines information retrieval (think ‘search’) with generative language models to increase response quality by first retrieving relevant documents and then generating contextually appropriate answers based on that information. This approach improves the accuracy and relevance of generated content by grounding it in factual data. RAG presents a formidable answer to the challenges that digital leaders face. It enhances the capabilities of a Large Language Model (LLM) by referencing external data sources that extend beyond the model's training data. This means that organisations can access up-to-date and relevant information to inform their decision-making processes. By integrating RAG into their knowledge management systems, businesses can enhance information retrieval, ensuring that employees across departments can quickly find the data they need without having to sift through isolated silos. Furthermore, RAG equips leaders with the ability to maintain control over their data sources, ensuring compliance and security. This can lead to improved decision-making as teams can base their strategies and actions on a more robust and comprehensive understanding of available knowledge.

Can you provide specific examples of how RAG can be applied in various enterprise scenarios to drive tangible business outcomes?

Deloitte Certainly, RAG can be applied across a wide array of enterprise scenarios to yield significant business outcomes. For instance, in the realm of HR, RAG can enhance employee onboarding processes by providing new hires with immediate access to the latest policies, training materials, and team-specific knowledge, all in a conversational format. In Legal departments, RAG can assist legal teams by rapidly retrieving relevant case law or compliance regulations, enabling them to build stronger arguments and make informed decisions more efficiently. In Research and Development, teams can leverage RAG to synthesise complex data from various sources, enabling faster innovation cycles and keeping pace with industry changes. Moreover, for IT teams, RAG can streamline incident management by pulling from a knowledge base to quickly diagnose and resolve issues, thus minimising downtime. Each of these applications not only maximises the utility of existing information but also drives tangible improvements in efficiency and productivity.

What are the key considerations for digital leaders when evaluating and implementing a RAG solution?

When digital leaders consider implementing a RAG solution, there are several key considerations to bear in mind. First and foremost, they should assess the integration capabilities of the RAG system with existing enterprise tools and workflows. A seamless integration is crucial for enhancing user adoption and ensuring that employees can easily access the RAG capabilities within their daily operations. Additionally, evaluating the scalability of the solution is vital. As businesses grow and evolve, their knowledge requirements may change, and a scalable RAG system must allow for the addition of new data sources and models. Data security and governance are also paramount; leaders should ensure that the RAG system adheres to company policies and compliance regulations. Lastly, they should consider the user experience; a user-friendly interface can significantly impact the successful adoption of the technology across the organisation.

How does RAG as a Service integrate with existing enterprise systems and workflows?

RAG as a Service (RaaS) is designed to seamlessly integrate with existing enterprise systems and workflows, thereby minimising disruption during implementation. By offering a centralised platform for knowledge management, RaaS can interface with various data sources and existing applications within the organisation, such as CRM systems, document management solutions, and intranets. This integration allows teams to ingest their data easily and access RAG capabilities directly from their familiar environments. Furthermore, the modular nature of RaaS enables organisations to choose specific components—such as different chunking and search algorithms—tailoring the solution to fit their unique requirements. By embedding RAG within existing workflows, organisations can enhance their knowledge management without the need for extensive retraining, ultimately leading to higher utilisation rates and better outcomes.

What are the future trends and developments in the field of AI-powered knowledge management that digital leaders should be aware of?

As we look to the future of AI-powered knowledge management, several trends and developments are emerging that digital leaders should pay close attention to. One significant trend is the evolution of Deep Research capabilities, which combine web search with advanced reasoning models. These models can conduct sophisticated reasoning throughout the research process, enabling more comprehensive understanding and insights. Unlike traditional knowledge assistants, which may simply retrieve information from internal sources based on one round of searching, these advanced models can plan and execute multiple search and reasoning steps, leading to more nuanced conclusions. Additionally, there is a growing focus on agentic applications. Agents are more than simple knowledge assistants; they can autonomously perform tasks, make decisions based on complex inputs, and engage in proactive problem-solving. This shift towards agentic systems signifies a move towards more autonomous and intelligent solutions that can not only retrieve information but also contextualise and act upon it. As these technologies continue to develop, they will offer organisations unprecedented capabilities in knowledge management.

What advice would you give to digital leaders who are looking to leverage their enterprise knowledge using GenAI?

For digital leaders aiming to leverage Generative AI effectively within their enterprise, my foremost advice is to prioritise a strategic approach. It is essential to understand the specific knowledge challenges faced within the organisation and align the implementation of GenAI solutions with these needs. Stakeholder engagement is another critical aspect; involving users in the design and implementation phases can lead to higher adoption rates and better results. They should also invest in education and training to ensure that employees feel comfortable and confident in using these new tools. Additionally, it's crucial to establish clear governance and compliance frameworks to safeguard sensitive information while facilitating knowledge sharing. Lastly, leaders should remain agile and open to iterating on their GenAI strategies based on feedback and evolving organisational needs. This adaptability will be key to realising the full potential of GenAI in enhancing enterprise knowledge management.

How can digital leaders measure the ROI of implementing GenAI knowledge services?

Measuring the ROI of implementing GenAI knowledge services requires a multifaceted approach. Digital leaders should start by defining clear metrics aligned with their strategic objectives. This could include metrics related to efficiency gains, such as time saved in information retrieval or decision-making processes. Additionally, it would be beneficial to track user engagement and satisfaction levels post-implementation, as these factors often correlate with productivity improvements. Financial metrics, such as reductions in training costs or increased revenue from faster innovation cycles, can also provide valuable insights into ROI. Furthermore, qualitative measures, such as case studies or user testimonials, can help illustrate the value realised from the GenAI initiatives. By employing a combination of quantitative and qualitative metrics, digital leaders can gain a comprehensive understanding of the overall impact and return on their investment in GenAI knowledge services.

 --

No Comments Yet

Let us know what you think

You May Also Like

These Stories on CIONET Belgium

Subscribe by Email