Designs for making your searches faster - Part 4

Let us quickly recap the earlier part.
We looked at improving the performance throughput through use of an efficient caching solution. Then we further helped efficient cache management through indexed and segmented caching to strike a good balance between page size selection and cache access.
this part we will look at pushing the performance envelope further. When users run searches their focus is on fast and accurate response. In cases where the user specified criteria returns results that span more than a few pages, here is how we can make them faster.
Let us divide the searching process in two parts.

1. The first batch comprising data for the first few pages
2. The second batch containing the complete set of results

Now, if we could divide the search program to run in such a way that a first batch of results are returned with just about enough data that can be used to display the first few pages, the performance will improve several notches.
This is because, in cases where the number of results returned are large, the time taken to compile results for just a few pages would be significantly lesser in comparison to the time taken to complete the entire search.
The odds of users looking beyond the first few pages are also lesser. So, the program could be written such that it stops searching for results after fetching a finite set of rows after which, the program forks. A new process is initiated that keeps looking for more results while the main process returns with the initial batch of results.
For instance a user searches the product database for products starting with letter 'a'. This could lead to results spanning several pages.

The program returns immediately with 60 rows which amounts to 3 pages of results while a background process is gathering all the rows. This behaviour presents a perception of extremely fast response times and user is able to view results almost instantaneously.
Clicking next for the first few pages would result in the cached data for the first few pages being displayed from the cache as explained in part 3 of this blog series.
By the time user reaches the third page, several seconds are likely to have elapsed, by which time, the background process would have completed its job and cached all the results without the user realizing it.

In order to maximize the performance gain in this approach, it is essential that

1. Maximum number of results thrown by any search is limited to a finite and logical limit. Say, 1000 rows. Some might even argue that this number is higher. Good then, reduce it further.
2. Avoid sorting search results by default. Encourage users to use filters for seeking transactions and encourage use of reports for reporting. Some users tend to use searches with sorting and grouping as interim reporting solutions
3. Use thread pools and thread queuing to ensure that background threads do not block critical CPU resources under heavy load.

In this part we have seen how multi-threading can be used to improve perceptive performance of searches. This approach is very suitable when chances of large number of results are very high.

Consideration	Compliance	Remarks
Search must be fast	Complies	Much Improved performance over earlier approach
Search must be accurate	Complies
Search must use minimal system resources	Complies	This is much better to the earlier DB only approach.
Search must avoid redundant queries	Complies	still executes redundant queries when paginating
Search must provide current data	Partly Complies	Cached results do not reflect updates that happen after the search is executed.
Pagination must be fast	Complies	Optimized Cache Access
Must facilitate on demand sorting	Complies	Requires requerying the db unless sorting API is available in the programming language. eg.LINQ
Must facilitate on demand result filtering	Complies
Must be multi-lingual friendly	Complies
Solution must be cluster friendly	Complies*	Subject to support from Caching solution

In the next part, we will look at taking search performance to the next level, pushing the performance envelope to its fathest.