mcrouter-hub: An HTTP companion to Facebook’s mcrouter

Gophercon 2015 is right around the corner and that sets up the mood for another open source project.

So, what is mcrouter? Mcrouter is a memcache router that helps Facebook scale their Memcache infrastructure to 5 billion requests per second.

It has a dozen or so, read and write strategies for customized scaling experience: Replication, hot-cold warmup, shard by prefix to a separate backend pool, and live reload config file.

The mcrouter daemon itself is tiny, on cold boot consumes 11MB of RAM, high performance C++ project.

For more information, visit mcrouter wiki page.

Live reload config file

This one particular feature is ripe to have a complementary HTTP agent helper.

Imagine the ability to live update hundreds of mcrouter, how cool is that?

So today I am please to release mcrouter-hub, an HTTP companion to mcrouter. It allows you:

  • In agent mode:

    • to expose the mcrouter config JSON via HTTP or HTTPS.

    • to modify the mcrouter config JSON. The POST method can be protected behind token-based security.

    • to expose mcrouter stats gathered from both telnet interface and file.

    • to export stats data to New Relic Insights.

  • In central mode:

    • to receive config JSON from all agents via HTTP or HTTPS.

    • to receive stats JSON from all agents.

    • Both features can be protected behind token-based security as well.

For more information, visit mcrouter-hub README page.

Getting started

Download the binary from this list and run it.

# Agent mode
# Prerequisites:
#   * It must be run on the same box where mcrouter is running.
#   * netcat command line tool must be installed.
#
# MCRHUB_CENTRAL_URLS is only useful if you want to have central daemons.
# That setting is optional.
MCROUTER_ADDR=localhost:5000 \
MCROUTER_CONFIG_FILE=/path/to/mcrouter.json \
MCRHUB_CENTRAL_URLS=http://localhost:5002 \
/path/to/mcrouter-hub

# Central mode
MCRHUB_MODE=central /path/to/mcrouter-hub

For those of you who run considerable Memcache infrastructure, I heartily recommend mcrouter. It provides greater flexibility when provisioning extra capacity to your cluster.

And when you do try mcrouter, feel free to try mcrouter-hub as well. See if you like it.