Using Redis with ASP.NET Web API

In this article I am going to show how to use Redis as a data store in a ASP.NET Web API application. I will implement a basic scenario that leverages ServiceStack.Redis library and its strongly typed Redis client, show how to model and store one-to-many relationships and how to use Web API dependency injection capabilities along with Autofac to inject repositories into controllers.

Client libraries

At the time of writing there are two popular and actively developed C# client libraries for Redis available:

• ServiceStack.Redis - created by Demis Bellot of ServiceStack fame and based on Miguel de Icaza's redis-sharp project,
• BookSleeve - mantained by Mark Gravell and as I understand used by Stack Exchange.

Before making a choice I would suggest trying both of them and deciding which API and capabilities better suit your project.
BookSleeve has non-blocking (asynchronous) API, provides thread-safe connection object, while ServiceStack implementation provides JSON serialization, connection pool like client factory and uses convention to simplify POCO object persistence.

In this article I will use ServiceStack.Redis, but remember that BookSleeve has been proved in a big real-world web application and is also very capable.

Redis in a nutshell

If you are reading this article then very likely you already know what Redis is. If you are an experienced Redis user interested in ASP.NET Web API integration you can safely jump to the next part.

In order to use Redis efficiently and avoid potential pitfalls one needs to understand a little bit about how it works and how different it is from relational databases.I strongly recommend reading one of books or online materials available on the topic.

Simply put Redis is an in-memory key-value data store that supports durability.
In-memory and key-value sounds much like a memory cache - and indeed you can think of Redis as of a specialized and more advanced memory cache. Unlike other caches (such as memcached) Redis delivers richer feature set including things like sorted sets and even Lua scripting capabilities.

It's main advantage over 'traditional' databases comes from the fact that it stores and retrieves data directly to / from operating memory - which means it is really fast.

Redis is simple and specialized - unlike relational databases it does not provide any table-like abstractions nor relational capabilities. Instead, it provides five fundamental data types along with specialized operations that can manipulate those types (stored values). This is why it is sometimes refered as a data structure server:

• strings - the most basic and atomic type that can be used to store any data (integers, serialized POCO objects, etc.),
• lists - lists of strings that are sored by insertion order,
• sets - logical sets of strings,
• hashes - maps between string-only keys and string values,
• sorted sets - similar to sets, but each element is associated with a score that is being used to sort.

Examples of specialized commands:

• strings - SET, INCR, APPEND, INCRBY, STRLEN, SETBIT,
• lists - LPUSH, LPOP, LTRIM, LINSERT,
• sets - SADD, SDIFF, SINTER, SUNION, etc.

Hopefully this should give you a basic feel for what Redis is about :)

Why would I use it?

Whether and how easily your application could benefit from Redis depends on its architecture, data volume, data complexity and experienced loads. When used correctly Redis can be bring major performance improvements and may help scale application out.

Here are some use cases I can think of:

• as a main data store,
• as one of multiple data stores, for example storing small, but frequently accessed information,
• as a highly performant read-only view over your domain model,
• as a cache.

Bearing in mind that Redis operates in memory, the first option is quite extreme and viable only if your data sets are small (or you can afford to have lots of RAM).
Because in this article I want to focus on ASP.NET Web API integration not architectural aspects I will choose this option.

Using Redis in a ASP.NET Web API application

I will use an empty ASP.NET Web API application as my starting point along with two third party libraries:

• ServiceStack.Redis - C# Redis client,
• Autofac - dependency injection container with Web API integration.

Obviously we will also need a working Redis server instance. If you don't have one running already you can download Windows port provided by MS Tech. Please note that the port is not considered production ready yet (you need to use one of the official packages for that), but is good for development scenarios.

Model

For the sake of this example lets consider the following requirements:

• the API should provide capability to store Clients, retrieve Client details and retrieve list of all Clients in the system,
• Clients may place orders that consist of multiple items,
• API should expose a list of N best selling items.

Here is how we could design the model:

public class Customer
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public IList<Guid> Orders { get; set; }
    public Address Address { get; set; }
}

Properly defining your data model will help you use Redis in an efficient way. Redis stores values as byte blobs internally and *ServiceStack.Redis* will serialize the whole object graph for us. Thus it is important that we define aggregate boundaries. As you can see Address is a *value object* and will be persisted and retrieved as a part of Customer *aggregate*, while *Orders* property is a list of ids.

public class Order
{
    public Guid Id { get; set; }
    public Guid UserId { get; set; }
    public IList<OrderLine> Lines { get; set; }
}

public class OrderLine
{
    public string Item { get; set; }
    public int Quantity { get; set; }
    public decimal TotalAmount { get; set; }
}

public class Address
{
    public string Line1 { get; set; }
    public string Line2 { get; set; }
    public string City { get; set; }
}

Now let's define repository contracts:

public interface ICustomerRepository
{
    IList<Customer> GetAll();
    Customer Get(Guid id);
    Customer Store(Customer customer);
}

public interface IOrderRepository
{
    IList<Order> GetCustomerOrders(Guid customerId);
    IList<Order> StoreAll(Customer customer, IList<Order> orders);
    Order Store(Customer customer, Order order);
    IDictionary<string, double> GetBestSellingItems(int count);
}

The implementation can look like this:

public class CustomerRepository : ICustomerRepository
{
    private readonly IRedisClient _redisClient;

    public CustomerRepository(IRedisClient redisClient)
    {
        _redisClient = redisClient;
    }

    public IList<Customer> GetAll()
    {
        using (var typedClient = _redisClient.GetTypedClient<Customer>())
        {
            return typedClient.GetAll();
        }
    }

    public Customer Get(Guid id)
    {
        using (var typedClient = _redisClient.GetTypedClient<Customer>())
        {
            return typedClient.GetById(id);
        }
    }

    public Customer Store(Customer customer)
    {
        using (var typedClient = _redisClient.GetTypedClient<Customer>())
        {
            if (customer.Id == default(Guid))
            {
                customer.Id = Guid.NewGuid();
            }
            return typedClient.Store(customer);
        }
    }
}

public class OrderRepository : IOrderRepository
{
    private readonly IRedisClient _redisClient;

    public OrderRepository(IRedisClient redisClient)
    {
        _redisClient = redisClient;
    }

    public IList<Order> GetCustomerOrders(Guid customerId)
    {
        using (var orderClient = _redisClient.GetTypedClient<Order>())
        {
            var orderIds = _redisClient.GetAllItemsFromSet(RedisKeys
                        .GetCustomerOrdersReferenceKey(customerId));
            IList<Order> orders = orderClient.GetByIds(orderIds);
            return orders;
        }
    }

    public IList<Order> StoreAll(Customer customer, IList<Order> orders)
    {
        foreach (var order in orders)
        {
            if (order.Id == default(Guid))
            {
                order.Id = Guid.NewGuid();
            }
            order.CustomerId = customer.Id;
            if (!customer.Orders.Contains(order.Id))
            {
                customer.Orders.Add(order.Id);
            }

            order.Lines.ForEach(l=>_redisClient
                .IncrementItemInSortedSet(RedisKeys.BestSellingItems,
                                                                 (string) l.Item, (long) l.Quantity));
        }
        var orderIds = orders.Select(o => o.Id.ToString()).ToList();
        using (var transaction = _redisClient.CreateTransaction())
        {
            transaction.QueueCommand(c => c.Store(customer));
            transaction.QueueCommand(c => c.StoreAll(orders));
            transaction.QueueCommand(c => c.AddRangeToSet(RedisKeys
                .GetCustomerOrdersReferenceKey(customer.Id),
                orderIds));
            transaction.Commit();
        }

        return orders;
    }

    public Order Store(Customer customer, Order order)
    {
        IList<Order> result = StoreAll(customer, new List<Order>() { order });
        return result.FirstOrDefault();
    }

    public IDictionary<string, double> GetBestSellingItems(int count)
    {
        return _redisClient
            .GetRangeWithScoresFromSortedSetDesc(RedisKeys.BestSellingItems, 
            0, count - 1);
    }
}

As you can see repositories expose specialized operations. We make use of Redis sorted set type to efficiently store and retrieve best selling products list.

It is worth noting how we implemented Customer -* Orders relation. We store customer's orders (their ids) in a dedicated set so that they can be retrieved quickly without the need for pulling out entire Customer entity.

Client and connection lifecycle management

One of the challenges we will face is connection/client lifecycle management. As you may already know Web API ships with an extensible dependency injection mechanism that can be leveraged to inject and dispose dependencies on per request basis. Instead of writing custom IDependencyResolver implementation from scratch (which is also an option) we can use of of .NET DI libraries such as Ninject, StructureMap, Unity, Windsor or Autofac. The last one is my personal favorite and has good Web API integration that is why I am going to use it in this example.

ServiceStack.Redis ships with IRedisClient factories called client managers:

• BasicRedisClientManager - client factory with load-balancing support,
• PooledRedisClientManager - client factory with load-balancing and connection pooling - useful when working,
• ShardedRedisClientManager - provides sharding of client connections using consistent hashing.

Because these classes are thread-safe we can use one factory instance across all requests.

public class ApiApplication : System.Web.HttpApplication
{
    public IRedisClientsManager ClientsManager;
    private const string RedisUri = "localhost";

    protected void Application_Start()
    {
        ClientsManager = new PooledRedisClientManager(RedisUri);

        AreaRegistration.RegisterAllAreas();

        WebApiConfig.Register(GlobalConfiguration.Configuration);
        ConfigureDependencyResolver(GlobalConfiguration.Configuration);

        FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
        RouteConfig.RegisterRoutes(RouteTable.Routes);
    }

    private void ConfigureDependencyResolver(HttpConfiguration configuration)
    {
        var builder = new ContainerBuilder();
        builder.RegisterApiControllers(Assembly.GetExecutingAssembly())
            .PropertiesAutowired();

        builder.RegisterType<CustomerRepository>()
            .As<ICustomerRepository>()
            .PropertiesAutowired()
            .InstancePerApiRequest();

        builder.RegisterType<OrderRepository>()
            .As<IOrderRepository>()
            .PropertiesAutowired()
            .InstancePerApiRequest();

        builder.Register<IRedisClient>(c => ClientsManager.GetClient())
            .InstancePerApiRequest();

        configuration.DependencyResolver
            = new AutofacWebApiDependencyResolver(builder.Build());
    }

    protected void Application_OnEnd()
    {
        ClientsManager.Dispose();
    }
}

We are using pooled connection manager as IRedisClientsManager implementation. Every time a request is made a new client instance will be retrieved, injected into repositories and disposed at the end of request.

Controllers

Now that we have repositories let's implement the controllers - one for adding and retrieving the customers and one for managing orders.

public class CustomersController : ApiController
{
    public ICustomerRepository CustomerRepository { get; set; }

    public IOrderRepository OrderRepository { get; set; }

    public IQueryable<Customer> GetAll()
    {
        return CustomerRepository.GetAll().AsQueryable();
    }

    public Customer Get(Guid id)
    {
        var customer = CustomerRepository.Get(id);
        if (customer == null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }

        return customer;
    }

    public HttpResponseMessage Post([FromBody] Customer customer)
    {
        var result = CustomerRepository.Store(customer);
        return Request.CreateResponse(HttpStatusCode.Created, result);
    }

    public HttpResponseMessage Put(Guid id, [FromBody] Customer customer)
    {
        var existingEntity = CustomerRepository.Get(id);
        if (existingEntity == null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        customer.Id = id;
        CustomerRepository.Store(customer);
        return Request.CreateResponse(HttpStatusCode.NoContent);
    }
}

public class OrdersController : ApiController
{
    public IOrderRepository OrderRepository { get; set; }
    public ICustomerRepository CustomerRepository { get; set; }

    public HttpResponseMessage Post([FromBody] Order order)
    {
        var customer = CustomerRepository.Get(order.CustomerId);
        var result = OrderRepository.Store(customer, order);
        return Request.CreateResponse(HttpStatusCode.Created, result);
    }

    [ActionName("top")]
    [HttpGet]
    public IDictionary<string, double> GetBestSellingItems(int count)
    {
        return OrderRepository.GetBestSellingItems(count);
    }

    [ActionName("customer")]
    [HttpGet]
    public IList<Order> GetCustomerOrders(Guid id)
    {
        return OrderRepository.GetCustomerOrders(id);
    }
}

That's about it. We are now using Redis as our data store and dependencies should be auto-wired.

Source is available on Bitbucket.