Amrit Sarkar is Search Engineer and Consultant at Lucidworks Inc, California-based enterprise search technology company, with 4+ years of experience in the search domain and big data, e-commerce, and product. He is working primarily on running search-based applications on Kubernetes, and developing and improving core components of Apache Solr.
Apache Solr is an open-source search platform built on an Apache Lucene library. It offers Apache Lucene's search capabilities in a user-friendly way. We, at Lucidworks, runs over a thousand distributed-mode Apache Solr clouds spread across several machines for our clients for a plethora of use-cases around Search and Analytics. The traffic demands a massive scale which creates scenarios of in-depth micro-management of resource allocation. Factors that are beyond sheer Solr cloud management like complimentary services communication, tasks w.r.t machines like operating systems upgrade, maintenance window required by cloud providers affect the overall search experience.
Kubernetes is fast becoming the operating system for the cloud and brings ubiquity which has the potential for massive benefits for technology organizations. Applications/microservices are moved to orchestration tools like Kubernetes to leverage features like horizontal autoscaling, fault tolerance, CICD and more. However, Kubernetes has room for improvement with respect to Solr cloud deployment.
In this talk, we will take a quick journey on how the Managed Search Team at Lucidworks addressed the respective challenges and therefore running Solr cloud cluster at scale on Kubernetes.
We further discuss enhancements done for overall experience by leveraging the advantages of features available in Solr and Kubernetes, like the abstraction of services, autoscaling, monitoring, etc. The session concludes with listing down the open-source contributions and future roadmap by Lucidworks.