Possibly the final video in our Using Generative AI in Techcomm series is “Unlocking ChatGPT’s Full Potential: The Impact of Source Content Quality”.
Technical writers prioritise high-quality content. However, with the advent of tools like ChatGPT, a question arises: Does writing high-quality content still matter? ChatGPT and similar chatbots derive their answers from large language models. These models have been trained on vast datasets and are adept at tasks such as summarising information, answering questions, translation, and pattern matching.
About Retrieval Augmented Generation
There’s a method we’ve discussed previously called “Retrieval Augmented Generation”. In this approach, chatbots can use a company’s unique knowledge base or source content. Instead of sourcing answers from vast language models, the chatbot draws from this specialised content.
The quality of source content matters
While chatbots can decipher and make sense of messy content, the quality of the source material plays a significant role in the accuracy and quality of their responses. For instance:
- Examples have shown that ChatGPT can effectively interpret complex slides from the American Department of Defense’s Afghanistan campaign.
- It can also clarify convoluted parking signs in America.
However, from our experiments:
- Chatbots that offer a personalized experience for users and those relying solely on a company’s knowledge base tend to produce better responses when the source content is of high quality.
- There’s a noticeable difference in the quality of answers when the information is drawn from YouTube videos with chapters versus those without.
- Personalised chatbots can provide tailored information (e.g., specific to Mac users) only if that information is well-defined in the knowledge base.
Personalising information
If you want chatbots to offer personalised information to different users based on parameters like device type, user proficiency, software version, etc., the content in the knowledge base should reflect these nuances. For instance, if your knowledge center lacks information specific to Macs, the chatbot might only give generic responses applicable to both Mac and Windows users.
Recommendations
Documentation from entities like Google and OpenAI suggests that for content to be effectively ingested by large language models, it should adhere to good technical writing practices. This includes structured headings, self-contained topics, numbered lists, and tables.
Conclusion
While large language models and chatbots can interpret messy content, better results are achieved with high-quality source content.
In the future, organisations will likely aim to offer personalised user experiences via chatbots, be it in the form of learning paths, support, or knowledge centres.
Well-structured and differentiated content will facilitate this migration towards a personalised user experience. Thus, maintaining high-quality source content is crucial.
For those interested in using generative AI in technical communications, we offer an e-learning course available on the subject.
Leave a Reply