Elevating Enterprise Search With AI, ML, NLP, and LLMS

PROMPT COMPRESSION

Users can control the costs of employing language models for enterprise search by employing prompt compression.
With this construct, a language model reduces the number of tokens in a prompt before sending it to the language model that answers the question.

Prompt compression is pivotal because organizations “pay in terms of the performance of the LLM, and the cost you pay to the provider is based on the number of tokens that get sent,” Fulkerson explained.

The language model responsible for prompt compression “strips away any unnecessary tokens without damaging the quality of the response,” Fulkerson observed. “It improves the performance of the LLM, so customers get faster responses, and improves their cost profiles by decreasing the cost, because you’re sending less unnecessary tokens.”

LEXICAL SEARCH

The hybrid search capabilities Harutyunyan mentioned are vital to the success of enterprise search—even in the wake of other forms of AI. Keyword search continues to be apropos in scenarios “where you have lower amounts of data to search through and you care about the specific accuracy in terms of the search term,” Harutyunyan disclosed. For instance, life sciences organizations may need to search through an assortment of information to find a particular compound, or name, of a formula that might be “CA1B5.”

“Vector search will decide a word that starts with CH and ends with 5 is very similar, but what you really need is for it to be that exact term,” Harutyunyan said. In this case, keyword search would return the correct search results immediately, which is why many vector databases incorporate measures for hybrid search involving this older search method.

THE LONG TERM

Enterprise search is certainly more expansive than ever with the advent of language models, their ML-based NLP, and generative capabilities.

Enterprise search is no longer circumscribed to documents and taxonomies, but is applicable to almost any type of data or use case. There are indicators that its long-term trajectory will evolve beyond single applications of language models at a time to encompass what Das envisions as many models, each with its own areas of specialization, for more accurate and germane results.

This approach is typified by what’s called a compound AI system, which Das mentioned has proponents from the University of California—Berkeley, and is loosely structured along a process in which “You will ask the model, the model will respond with some augmentation to the next model, and then to the next model,” Das said. Eventually, those model responses will be combined to answer incredibly intricate queries, further broadening the scope and utility of enterprise search in an enterprise AI world.