The Early AI Scramble
Immediately professionals from all different walks of life naturally thought, “How can this help me with my work?”. Companies also formed tactical teams to consider embedding it in their solutions while scrambling to find the most effective ways for employees to use it for certain tasks from automating customer responses, author assist, business analytics and of course writing code. It’s a grand experiment, a petri dish of organisms fighting for survival and trying to gain that evolutionary edge.
In mid 2023, a trend in AI emerged with the rise of domain-specific or vertical-specific AI models. Technically this preceded solutions like ChatGPT commercially with examples like Github Copilot, released in October of 2021, allowing software developers to multiply their productivity. Still, the explosion of vertical specific solutions is at its height, smack dab in the middle of the AI hype cycle, making it difficult to predict which solutions have staying power.
After the release of chatGPT, 303 Software was also hustling to leverage the latest AI advancements. By January the 303 Software team had worked through the approach and created user stories around applying a generative large language model towards the mining industry in the MSHAWise app.
There were 3 big problems
1) ChatGPT was REALLY bad at answering specific questions about mining compliance.
2) At the time, there was no API access to chatGPT which is necessary to enable 303 Software to solve problem #1.
3) The AI hallucination problem turned out to be even more egregious in regulated industries
OpenAI launched API access on March 1st, opening the floodgates and unblocking the 303 R&D team. The API allowed the team to start prototyping. This took shape by experimenting with vector embeddings, implementing cosine scoring, programmatic prompt engineering and, most importantly, scoping the queries specifically to our MSHA body of knowledge.
Tom (Monty) Montgomery, CTO of 303 Software & co-founder of MSHAWise, states “We made the decision to train the AI across two different bodies of knowledge for the same regulatory sphere because our users needed two distinct points of view. One in the context of regulations and the other in the context of case law.”
These two domain-specific AI models are aptly named the MSHA Copilot and the MSHA Case Law Copilot. The system uses a technique called embeddings-based search where it essentially gives chatGPT guidance on which documents probably contain the best answer for a given prompt, leveraging our curated taxonomy and search engine to increase the accuracy of the AI responses.
Eden Corbin, Lead Architect, notes: “We do embeddings through OpenAI, then perform a vector search with specific tunings to help improve the quality of the answers. Our MVP started with a giant JSON file of vectors and it quickly became clear we needed to pivot to a vector database like Milvus configured and optimized for our use case.”
The marriage of enterprise search and vertical specific AI wins the day
Unveiled in October 2023 at the Western MSHA Conference, the customer response to MSHAWise was overwhelming. Safety Managers, safety trainers and mining attorneys marveled at the level of accuracy and overall utility. You could see the wheels turning on how much time and money could be saved on safety compliance as conference attendees asked tough questions and got correct answers or were pointed to the exact case law in seconds that otherwise would have taken hours.
One attendee ran over to the MSHAWise conference booth and said, “Ask it about mercury vapor best practices!” after debating the topic with another attendee. The AI models and the search engine results in MSHAWise returned precisely what she needed, including the specific documents and case law referenced.
The reality is, generative AI by itself is not enough in the use case of building a compliance knowledge solution aimed at federally regulated industries like mining. The user experience needs to be targeted at the body of knowledge, yet flexible enough to handle very specific questions, sometimes leveraging documents that don’t exist on the public internet, while excluding public data known to not be correct. This curation and related guard rails is how we solved the hallucination issue.
It is this coupling between a search based solution atop a curated taxonomy of data with multiple AI Copilots that creates the trifecta for the next generation AI-powered compliance tool.
