Appendix E. Turning search upside down with the percolator

The Elasticsearch percolator is typically defined as “search upside down” for the following reasons:

You index queries instead of documents. This registers the query in memory, so it can be quickly run later.

You send a document to Elasticsearch instead of a query. This is called percolating a document, basically indexing it into a small, in-memory index. Registered queries are run against the small index, so Elasticsearch finds out which queries match.

You get back a list of queries matching the document, instead of the other way around like a regular search.

The typical use case for percolation is alerting. As shown in figure E.1, you can notify users when new documents (matching their interests) appear.

Figure E.1. Typical use case: percolating a document enables the application to send alerts to users if their stored queries match the document.

As the figure shows, using the get-together site example we’ve used throughout the book, you could let members define their interests, and you’d save them as percolator queries. When a new event is added, you can percolate it against those queries. Whenever there are matches, you can send emails to the respective users to notify them of new events relevant to their interests.

Next, we’ll describe how to implement those alerts using the percolator. After that, we’ll explain how it works under the hood, and then we’ll move on to performance and functionality tricks.