New Open Source Elastic Search Tool for building Generative AI Apps

Open-source search engine Elastic is disrupting the field with its latest offering, the Elasticsearch Relevance Engine, which is available now. This innovative tool empowers businesses to harness both structured and unstructured data, enabling them to build custom generative AI applications without compromising data security or investing in expensive Large Language Models (LLMs).

Matt Riley, Elastic’s GM for enterprise search, highlighted the limitations of large and costly language models like GPT-3 or Bard, emphasizing their lack of access to private information. In contrast, Elastic’s insight engines combine search capabilities with composite AI, providing context-enriched analysis. These engines gather data from diverse sources, such as repositories, websites, and databases, into a centralized and queryable index.

The potential of insight engines extends beyond basic search functionality. They can answer queries, provide contextual recommendations, and derive actionable insights. As a result, organizations can make informed decisions and leverage data for automation purposes.

Elastic operates in the global enterprise search market, competing with prominent players like Micro Focus, Squirro, IBM, Microsoft, Sinequa, Coveo, and others. Industry projections indicate that this market is expected to reach $8.8 billion by 2030, driven by the growing need for effective data management and operational capabilities among organizations.

Elastic Strengths

With the Elasticsearch Relevance Engine, Elastic aims to elevate AI-powered search capabilities to new heights. This tool enables enterprises to leverage generative AI while ensuring the security of their proprietary business data. Users can ask questions without exposing company-specific information to the public internet.

The engine’s power lies in its unified APIs for vector search and transformer models, which capture meaning and context. It employs the BM25f search ranking function to estimate document relevance, and utilizes hybrid search techniques that combine multiple algorithms to enhance accuracy and relevance. Additionally, enterprises have the flexibility to incorporate their own transformer models or integrate with third-party options.

James Governor, co-founder of analyst firm RedMonk, noted the value of Elasticsearch Relevance Engine in simplifying the adoption of transformers and LLM models, building upon Elastic’s core strengths in search capabilities. Several Elastic customers, including Relativity, have already embraced the Elasticsearch Relevance Engine. Relativity, for example, is leveraging the tool in conjunction with the Azure OpenAI Service to improve the relevance of results in their e-Discovery product.

Chris Brown, Chief Product Officer at Relativity, expressed enthusiasm for Elasticsearch Relevance Engine and its potential to deliver powerful AI-augmented search outcomes to their customers. Elastic remains committed to providing industry-leading search capabilities that enable organizations to organize data effectively, uncover truths, and take decisive action.