Apache SOLR in .NET Applications

In one of our latest projects there was a need to implement a full-text search engine. We had several search requirements such as filtering by categories, searching near a given location and, of course, searching for a specific keyword. We already had experience using Lucene.Net so we chose to use Apache Solr, which uses the Lucene library to provide fast searches and full-text indexation to give better performance.

To build the search server we downloaded the latest release of Apache Solr and configured it for our database model and specifications. We installed the Solr instance with Jetty running in a Windows Service.

Although Apache Solr provides a REST API for search queries and index updates, for security reasons we decide to use middleware to perform all the index and search operations. The technology we used on the project was based on .Net MVC 4 and SQL Server so we chose to use SolrNet client which provides an abstraction layer using Solr in a .NET application. The following image shows the high level architecture of the application:

undefined

Using SolrNet

As an example on how to use SolrNet, let's say we have a School Entity:

public class SolrSchoolModel
{
 [SolrUniqueKey("id")]
 public string Id { get; set; }

 [SolrField("Name")]
 public string Name { get; set; }

 [SolrField("Description")]
 public string Description { get; set; }

 [SolrField("NumberStudents")]
 public string NumberStudents { get; set; }

 [SolrField("Postcode")]
 public string Postcode { get; set; }
      
 [SolrField("LatitudeLongitude")]
 public string Coordenates { get; set; }

 [SolrField("Type")]
 public string Type { get; set; }
}

The SolrField attribute indicates the name of the property configured in Solr. The Id field is the unique identifier of a school entity in the database and Solr index. We need to specify the SolrUniqueKey attribute in order to index this field as a unique key. Other fields like Name or Description provide the data to search for.

To use search by distance, we need to find the coordinates for each school location and store it in a Solr field in the format "51.467389,-0.28926".

 

Index

When a school is created or edited in the application we need to index or update the index in that object. For that we use the following operations in our middleware to add or update a generic entity:

public interface ISolrIndexService<T>
{
    bool AddUpdate(T document);
    bool Delete(T document);
}

public class SolrIndexService<T, TSolrOperations> : ISolrIndexService<T>
        where TSolrOperations : ISolrOperations<T>
{
  private readonly TSolrOperations _solr;

  public SolrIndexService() 
  {
    _solr = ServiceLocator.Current.GetInstance<TSolrOperations>();
  }
  public void AddOrUpdate(T document)
  {
    try
    {
      // If the id already exists, the record is updated, otherwise added                         
      _solr.Add(document);
      _solr.Commit();
    }
    catch (SolrNetException ex)
    {
     //Log exception    
    }
  }
  public void Delete(T document)
  {
    try
    {
      //Can also delete by id                
      _solr.Delete(document);
      _solr.Commit();
    }
    catch (SolrNetException ex)
    {
    //Log exception
    }
  }
}

 

Search

To build a multiple criteria search we use theSolrMultipleCriteriaQuery function as in the following example:

public ISolrQuery BuildQuery(SearchParameters parameters)
{
  var queryList = new List<ISolrQuery>();

  //Search for a given keyword in all configured Solr fields.
  queryList.Add(new SolrQueryByField("all", parameters.Keyword));

  //Search for a given string in a specific field
  queryList.Add(new SolrQueryByField("Name", parameters.Name));

  return new SolrMultipleCriteriaQuery(queryList, "OR");
}

If we want to search for a given keyword in all indexed fields we can configure the "all" field in the Solr configuration file.

Now let's say we want to find Schools near a given City name or Postcode. First we need to calculate the coordinates for that location and then use the SolrQueryByDistance criteria to find Schools within the given radium (in Km).

//Search by location
//Given the latitude, longitude and a distance (Km)
var queryByDistance = new SolrQueryByDistance("LatitudeLongitude", latitude, longitude, distance));

After building the search query, we use the Query method to return the query results:

private readonly ISolrReadOnlyOperations<SolrJobModel> _solr;

public SearchService()
{
  _solr = ServiceLocator.Current.GetInstance<ISolrOperations<SolrJobModel>>();
}

public SolrQueryResults<SolrSchoolModel> Search(SearchParameters parameters){

  var solrQueryResults = _solr.Query(BuildQuery(parameters), new QueryOptions
  {
    FilterQueries = new Collection<ISolrQuery> { Query.Field("Type").Is(parameters.Filter) }, 
    Rows = parameters.PageSize,
    Start = parameters.PageIndex,
    OrderBy = new Collection<SortOrder>{ SortOrder.Parse("Name asc") },
    Facet = new FacetParameters
    {
      Queries = new Collection<ISolrFacetQuery>{ new SolrFacetFieldQuery("Type") { MinCount = 1 }}
    });
  }
  return solrQueryResults;
}

We can use several query options into the Query method to help with pagination, ordering results and setting other filters.

Pagination can be done by passing the number of records to list in the page (Rows) and the current page (Start). We can order by any fields we want, using the SortOrder class to specify a field name and the ordering type (asc or desc).

The faceting feature allows refining the search results into categories. For example, let's say we want to show how many school types have been returned from the search. We would have a list of types like:

  • Primary (10 schools)
  • Secondary (5 schools)
  • Public (10 schools)
  • Private (5 schools)

Using the Facet property in the search options we can easily return the results list to the user through solrQueryResults.FacetFields and allow the user to see the number of schools for each type and filter by type.

To apply a filter by type on a resultset, we use the FilterQueries property and when the user clicks in a type of school, all the previous results will be filtered by that specific filter query.

var resultsModel = new SchoolResultsModel
       {
         Search = parameters,
         TotalCount = solrQueryResults.NumFound,
         Facets = solrQueryResults.FacetFields,
         Schools = solrQueryResults
       };

By using Apache Solr and SolNet we have included search capabilities with useful features like faceted or refined search and search by location in .NET applications.

For more information, see the following references:


Ines 27 Aug 2014