MySQL-Memcached or NOSQL Tokyo Tyrant – part 3

This is part 3 of our series. Inpart 1we talked about boosting performance with memcached on top of MySQL, inPart 2we talked about running 100% outside the data with memcached, and now in Part 3 we are going to look at a possible solution to free you from the database. The solution I am going to discuss here is Tokyo Cabinet and Tyrant.

I am not going to give you a primer or Tutorial on Tyrant and Cabinet, there are plenty of these out there already. Instead I want to see what sort of performance we can see compared to MySQL and Memcached, and later on other NoSQL solutions. Tokyo actually allows you to use several types of databases that are supported, there are hash databases which are very similar to memcached, a table database which is similar to your classic database tables where you can add a where clause and search individual columns, and a ton more “database options” beyond just those two. Again my goal is to not make this a Tokyo Tyrant tutorial but rather show one potential role it can play.

More details can be read about here:
http://1978th.net/tokyotyrant/
http://1978th.net/tokyocabinet/

So if we can get performance similar to memcached with Tokoyo Tyrant when using disk based hash tables it would be a compelling replacement for our application here. It should provide the interface and the same access we saw in memcached but with disk persistence. So let’s look at the numbers:

Tyrant’s disk based hash was almost 2x faster then combining memcached and MySQL, and about 20% slower then the all memory memcached approach. So for this particular application I would have been much better off not storing my data in MySQL and instead looking outside the database for an answer. Now sure there are other reasons you may want to keep data in the database… but I am trying to get you to think about your application and if those reasons are really valid. Helping clients pick the right solution is one of the things we do here at Percona. If an application requires a database great, but if there is a better solution we want to suggest it. It’s our goal to make your application perform optimally.

Finally, one concern you have to have is the scalability of your storage solution. As load, number of threads, and data size increases how does performance differ or change? One knock on Tokyo -vs- Memcached is Tokyo is not distributed by default. Now that’s not to say we could not shard it based on a hash, or even build an api with the capability built in ( or use the memcached clients which works! )… but native support is lacking. It does support replication which could make some rather interesting architectures in the future.

So lets look at some scalability benchmarks, my server resources are rather limited but I thought I should try throwing more threads and work at the server until it hit its limit and fell over dead. It’s interesting to see the number of transactions that occur with a given number of threads. let’s look at some of these:

As expected the smaller buffer pool struggled ( why a smaller buffer pool? This simulates a much larger data set. A BP of 256M with 1GB of data, can give similar performance to 20GB of data and a 5GB BP ). So with 256M BP and 4GB of memcached we were well off the numbers we hit with a 4GB BP+4Gb of memcached ( which is expected ).  Adding more threads even up to 128 threads increased overall throughput but my load average on the server hit 40 and my CPU was pegged. At 128 threads I was pegging out my CPU across the board. Also interesting is I started to hit bottlenecks in MySQL/Innodb when I had enough memory but I increased the threads from 64 to 128. As time permits I should revisit this and look at increased datasets, and look for area’s where Tyrant may stumble a bit.

Bottom line given a specific application and data pattern sometimes a relational database is not the appropriate place for storing data. A tool like Tokyo Tyrant may not be for everyone or every application, but neither is a relational database. Before building your next application try and understand whether an RDBMS is really needed or not.

How did I do these tests:

The above number were run with 32 Threads, Tyrant was started with 8 threads and 128M of memory, memached was started with 16 threads ( 1.4 memcached ), mysql was 5.1 XtraDB. Each environment had 2 tables each with 2 million rows. The data was identical. memcached and Tyrant stored a comma delimited string to represent the row. Mysql was running with 256M allocated to the innodb buffer unless otherwise noted.

What’s next? Well next I am going to try and continue this series by exploring and benchmarking other NOSQL options and comparing them to database based solutions. I think showing the performance of a couple of different Tokyo database formats would also be interesting. What other solutions are people interested in? I know I have gotten a lot of requests for cassandra #’s, but what else? Drop a comment and let me know!