Memcachedに大量のJSONを保存していたのですが．Memcachedサーバの容量を超えてしまい，データ容量の削減が必要になってしまいました．いろいろ調べているうちに，クライアントとして使っているCache::Memcached::Fastに圧縮オプションがあることを発見したので，試してみました．デフォルトではオフになっているのですが，newする際にcompress_thresholdを設定することで有効になります．
Cache::Memcached::Fast->new( my $memcached_comp = Cache::Memcached::Fast->new({( servers => [qw/localhost:11211/], compress_threshold => 500, compress_ratio => 0.9, )});
この設定の便利な点は，圧縮オプションを後から設定しても，既存の非圧縮のキャッシュの読み書きが普通に行える点です．さらに，圧縮してある値に対して，圧縮設定をオフにしているクライアントからアクセスしても読み書きが行えます．これにより，既に動いているサービスに対しても圧縮を追加することが可能になります．ソースを見てみたところ，どうやら毎回getの際に解凍メソッドに投げているようです．
もちろん速度的なオーバーヘッドはつきまとうので，使うかどうかは場合によるのですが，私の場合は特に問題ない範囲だったのでそのまま採用しました．なお，圧縮メソッドを入れ替えることも可能なのですが，IO::Compress::Gzip::gzipよりもデフォルトのCompress::Zlib::memGzipの方がわずかに速い程度でした．
ベンチマーク結果は以下の通りです．
容量
- compress 1338 byte
- raw 9219 byte
速度
set
Benchmark: timing 10000 iterations of compress set, raw set... compress set: 15 wallclock secs (11.95 usr + 2.45 sys = 14.40 CPU) @ 694.44/s (n=10000) raw set: 0 wallclock secs ( 0.11 usr + 0.30 sys = 0.41 CPU) @ 24390.24/s (n=10000)
生データに対してのget
圧縮設定の有無にかかわらず同じ処理を行っている模様．
Benchmark: timing 10000 iterations of compress_get raw_data, raw_get raw_data... compress_get raw_data: 1 wallclock secs ( 0.11 usr + 0.31 sys = 0.42 CPU) @ 23809.52/s (n=10000) raw_get raw_data: 1 wallclock secs ( 0.11 usr + 0.30 sys = 0.41 CPU) @ 24390.24/s (n=10000)
圧縮データに対してのget
圧縮設定の有無にかかわらず同等の速度でgetできていることから，設定に関係なく解凍メソッドが毎回走っていると思われる，
Benchmark: timing 10000 iterations of compress_get comp_data, raw_get comp_data... compress_get comp_data: 3 wallclock secs ( 2.60 usr + 0.29 sys = 2.89 CPU) @ 3460.21/s (n=10000) raw_get comp_data: 4 wallclock secs ( 2.59 usr + 0.29 sys = 2.88 CPU) @ 3472.22/s (n=10000)
コード
#!perl use strict; use warnings; use Benchmark qw(timethese cmpthese);; use Cache::Memcached::Fast; my $memcached = Cache::Memcached::Fast->new({( servers => [qw/localhost:11211/], )}); my $memcached_comp = Cache::Memcached::Fast->new({( servers => [qw/localhost:11211/], compress_threshold => 500, compress_ratio => 0.9, )}); my $data = "[[{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/417K6qXAgvL._SL75_.jpg\",\"media\":\"大型本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/417K6qXAgvL._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4798119172&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/417K6qXAgvL._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4798119172/-/ref=nosim\",\"asin\":\"4798119172\",\"isadult\":\"\",\"publisher\":\"翔泳社\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/417K6qXAgvL._SL160_.jpg\",\"ourprice\":\"￥ 2,940\",\"image\":\"http://ecx.images-amazon.com/images/I/417K6qXAgvL._SL75_.jpg\",\"title\":\"モダンPerl入門 (CodeZine BOOKS)\",\"price\":\"￥ 2,940\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/417K6qXAgvL._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/417K6qXAgvL.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/51C59GNSC3L._SL75_.jpg\",\"media\":\"単行本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/51C59GNSC3L._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4873111269&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/51C59GNSC3L._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4873111269/-/ref=nosim\",\"asin\":\"4873111269\",\"isadult\":\"\",\"publisher\":\"オライリージャパン\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/51C59GNSC3L._SL160_.jpg\",\"ourprice\":\"￥ 3,780\",\"image\":\"http://ecx.images-amazon.com/images/I/51C59GNSC3L._SL75_.jpg\",\"title\":\"初めてのPerl\",\"price\":\"￥ 3,780\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/51C59GNSC3L._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/51C59GNSC3L.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/51p4CXKu1JL._SL75_.jpg\",\"media\":\"単行本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/51p4CXKu1JL._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4797336803&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/51p4CXKu1JL._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4797336803/-/ref=nosim\",\"asin\":\"4797336803\",\"isadult\":\"\",\"publisher\":\"ソフトバンククリエイティブ\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/51p4CXKu1JL._SL160_.jpg\",\"ourprice\":\"￥ 2,835\",\"image\":\"http://ecx.images-amazon.com/images/I/51p4CXKu1JL._SL75_.jpg\",\"title\":\"新版Perl言語プログラミングレッスン入門編\",\"price\":\"￥ 2,835\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/51p4CXKu1JL._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/51p4CXKu1JL.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/514Mpb4q4XL._SL75_.jpg\",\"media\":\"大型本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/514Mpb4q4XL._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4873113970&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/514Mpb4q4XL._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4873113970/-/ref=nosim\",\"asin\":\"4873113970\",\"isadult\":\"\",\"publisher\":\"オライリージャパン\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/514Mpb4q4XL._SL160_.jpg\",\"ourprice\":\"￥ 3,780\",\"image\":\"http://ecx.images-amazon.com/images/I/514Mpb4q4XL._SL75_.jpg\",\"title\":\"マスタリングPerl\",\"price\":\"￥ 3,780\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/514Mpb4q4XL._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/514Mpb4q4XL.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/410B5%2BzbIxL._SL75_.jpg\",\"media\":\"大型本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/410B5%2BzbIxL._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4774135046&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/410B5%2BzbIxL._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4774135046/-/ref=nosim\",\"asin\":\"4774135046\",\"isadult\":\"\",\"publisher\":\"技術評論社\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/410B5%2BzbIxL._SL160_.jpg\",\"ourprice\":\"￥ 3,780\",\"image\":\"http://ecx.images-amazon.com/images/I/410B5%2BzbIxL._SL75_.jpg\",\"title\":\"すぐわかる オブジェクト指向 Perl\",\"price\":\"￥ 3,780\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/410B5%2BzbIxL._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/410B5%2BzbIxL.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/51EK7E5K5FL._SL75_.jpg\",\"media\":\"単行本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/51EK7E5K5FL._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4873110963&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/51EK7E5K5FL._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4873110963/-/ref=nosim\",\"asin\":\"4873110963\",\"isadult\":\"\",\"publisher\":\"オライリー・ジャパン\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/51EK7E5K5FL._SL160_.jpg\",\"ourprice\":\"￥ 5,565\",\"image\":\"http://ecx.images-amazon.com/images/I/51EK7E5K5FL._SL75_.jpg\",\"title\":\"プログラミングPerl〈VOLUME1〉\",\"price\":\"￥ 5,565\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/51EK7E5K5FL._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/51EK7E5K5FL.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/51T110CBWTL._SL75_.jpg\",\"media\":\"大型本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/51T110CBWTL._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4798109029&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/51T110CBWTL._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4798109029/-/ref=nosim\",\"asin\":\"4798109029\",\"isadult\":\"\",\"publisher\":\"翔泳社\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/51T110CBWTL._SL160_.jpg\",\"ourprice\":\"￥ 1,764\",\"image\":\"http://ecx.images-amazon.com/images/I/51T110CBWTL._SL75_.jpg\",\"title\":\"Perl の絵本\",\"price\":\"￥ 1,764\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/51T110CBWTL._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/51T110CBWTL.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/51EHNQ64V6L._SL75_.jpg\",\"media\":\"単行本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/51EHNQ64V6L._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4873110971&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/51EHNQ64V6L._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4873110971/-/ref=nosim\",\"asin\":\"4873110971\",\"isadult\":\"\",\"publisher\":\"オライリー・ジャパン\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/51EHNQ64V6L._SL160_.jpg\",\"ourprice\":\"￥ 4,935\",\"image\":\"http://ecx.images-amazon.com/images/I/51EHNQ64V6L._SL75_.jpg\",\"title\":\"プログラミングPerl〈VOLUME2〉\",\"price\":\"￥ 4,935\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/51EHNQ64V6L._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/51EHNQ64V6L.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/51PBFWZSTBL._SL75_.jpg\",\"media\":\"大型本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/51PBFWZSTBL._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4873113059&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/51PBFWZSTBL._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4873113059/-/ref=nosim\",\"asin\":\"4873113059\",\"isadult\":\"\",\"publisher\":\"オライリー・ジャパン\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/51PBFWZSTBL._SL160_.jpg\",\"ourprice\":\"￥ 3,360\",\"image\":\"http://ecx.images-amazon.com/images/I/51PBFWZSTBL._SL75_.jpg\",\"title\":\"続・初めてのPerl 改訂版\",\"price\":\"￥ 3,360\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/51PBFWZSTBL._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/51PBFWZSTBL.jpg\"},{\"catalog\":\"Book\",\"img\":\"http://ecx.images-amazon.com/images/I/51FHW5B8GQL._SL75_.jpg\",\"media\":\"単行本\",\"imagefilesmall\":\"http://ecx.images-amazon.com/images/I/51FHW5B8GQL._SL75_.jpg\",\"mobile_url\":\"http://www.amazon.co.jp/gp/aw/rd.html?url=/gp/aw/d.html&lc=msn&dl=1&a=4873112028&at=-m-&uid=\",\"mobile_image\":\"http://ecx.images-amazon.com/images/I/51FHW5B8GQL._SL75_SX52_.jpg\",\"url\":\"http://www.amazon.co.jp/exec/obidos/ASIN/4873112028/-/ref=nosim\",\"asin\":\"4873112028\",\"isadult\":\"\",\"publisher\":\"オライリージャパン\",\"imagefile\":\"http://ecx.images-amazon.com/images/I/51FHW5B8GQL._SL160_.jpg\",\"ourprice\":\"￥ 5,145\",\"image\":\"http://ecx.images-amazon.com/images/I/51FHW5B8GQL._SL75_.jpg\",\"title\":\"Perlクックブック〈VOLUME1〉\",\"price\":\"￥ 5,145\",\"mobile_img\":\"http://ecx.images-amazon.com/images/I/51FHW5B8GQL._SL75_SX52_.jpg\",\"imagefilelarge\":\"http://ecx.images-amazon.com/images/I/51FHW5B8GQL.jpg\"}],\"2009-04-29T14:10:44+09:00\"] "; my $key = 'hoge'; timethese(10000, { 'raw set' => sub { $memcached->set($key, $data); }, 'compress set' => sub { $memcached_comp->set($key, $data); }, }); ## 生データに対してのget $memcached->set($key, $data); timethese(10000, { 'raw_get raw_data' => sub { $memcached->get($key, $data); }, 'compress_get raw_data' => sub { $memcached_comp->get($key, $data); }, }); ## 圧縮データに対してのget $memcached_comp->set($key, $data); timethese(10000, { 'raw_get comp_data' => sub { $memcached->get($key, $data); }, 'compress_get comp_data' => sub { $memcached_comp->get($key, $data); }, });
