Custom Enterprise Data with ChatGPT + Azure OpenAI and Azure Cognitive Search

24 May

In the world of AI, OpenAI’s ChatGPT has made a remarkable impact, reaching over 100 million users in just two months. The technology’s potential is vast, and users worldwide are exploring its application across a broad range of scenarios. One question that often arises is, “How can I build something like ChatGPT that uses my own data as the basis for its responses?” Today, I will demonstrate and guide you through the process of creating a ChatGPT-like experience using your own data with Azure OpenAI and Cognitive Search.

Note – In this blog post I am not going to explain each of the Azure services. The best source is to grab all those details.

What You’ll Need

To get started, you’ll need an Azure subscription with access enabled for the Azure OpenAI service. You can request access here. Additional the user should have Owner and Cognitive Services Contributor (Note if you dont add this role to your account the azd up command will come with a error why processing the files)

  • Github Codespaces – I am using this as its a preconfigured enviornment with all pre-requsites to run the code from the VScode IDE.
  • Install locally on your device – You’ll also need to have Azure Developer CLI, Python 3+, Node.js, Git, Powershell 7+ (pwsh) installed on your local machine.

Azure OpenAI and Cognitive Search offer an effective solution for creating a ChatGPT-like experience using your own data. Azure Cognitive Search allows you to index, understand, and retrieve the right pieces of your data across large knowledge bases, while Azure OpenAI’s ChatGPT offers impressive capabilities for interacting in natural language to answer questions or engage in conversation.

Getting Started with the sample project

To begin, you’ll need to click on the Github Codespaces and login with your Github account.

The project gets cloned and a new one gets created under Codespaces with everything preconfigured for you. Note this step takes approx. 15 mins to complete as its a 4 CPU, 8 GB RAM compute environment been created.

Next Step create a folder in my case DellAz and further cd into the newly created folder within the terminal. Further login to the Azure Subcription.

Initialize the Azure-search-openai-demo (In my case I already did that earlier)

Uploading Your Data

To upload your data, follow these steps:

  1. Upload Your Data: Within VScode do to you intialized project folder DellAz –> ./data and right click and select upload files and upload a few files. In my secnario I am uploading few Dell Azure Stack HCI pdf files. Azure Cognitive Search will be breaking up larger documents into smaller chunks or summarizing content to fit more candidates in a prompt.

You can do this by running the azd up command, which will provision Azure resources and deploy the sample application to those resources, including building the search index based on the files found in the folder. I had recieved the error intially but after overcoming the permissions mentioned above it went smoothly as expected.

Resources within Azure Subcription

All the resources required will be deployed within the resource group. (App Service, Form recongnizer, Azure OpenAI, Search Service, App Service Plan and Storage Account)

Storage Account (PDFs getting chunked)

App Service (front end portal)

Search Index using Azure Congnitive Search

Interaction, Trustworthy Responses and User Experience

One of the key aspects of creating a successful ChatGPT-like experience is ensuring that the responses generated by the model are trustworthy. This can be achieved by providing citations and source content tracking, as well as offering transparency into the interaction process. As you can see all the citations are from the documents I have uploaded to the ./data folder along with LLM making it easily consumable for an user.


Creating a ChatGPT-like experience using your own data with Azure OpenAI and Cognitive Search is a powerful way to leverage AI in your enterprise. Whether you’re looking to answer employee questions, provide customer support, or engage users in conversation, this combination of technologies offers a flexible and effective solution. So why wait? Start revolutionizing your enterprise data today!


Following are the references and important links worth going through for more details:

Microsoft Sample Code whcih we are usingGitHub – Azure-Samples/azure-search-openai-demo
Microsoft Blog on the same topicRevolutionize your Enterprise Data with ChatGPT: Next-gen Apps w/ Azure OpenAI and Cognitive Search
[YouTube] ChatGPT + Enterprise data with Azure OpenAI and Cognitive Search
[YouTube] Use ChatGPT On Your Own Large Data – Part 2

I hope you will find this helpful information for creating a custom ChatGPT with your own enterprise data. Please let me know if I have missed any steps or details, and I will be happy to update the post.

Aresh Sarkari

One Response to “Custom Enterprise Data with ChatGPT + Azure OpenAI and Azure Cognitive Search”

  1. sadaf Zaheer July 30, 2023 at 10:03 pm #

    Thankyou for this information.New to chatgpt n cogintive search and this post really helpful

Leave a Reply