Neal's quick hack at Memory management inside Tokyo Cabinet

Essentially I added:

• rssgoal parameter as peer of lcnode/ncnode
• uses libproc to get RSS memory size
• change to tcbdbcacheadjust() to proportionally adjust down leafc and nodec

• if actual memory usage is 10% higher than the goal via standard methods in tcbdbcacheadjust()

  Example usage w/ 2.3 Gig:

  nmemb=256#ncnum=10240#lmemb=512#lcnum=65536#bnum=100000000#xmsiz=536870912#rssgoal=2300000000#opts=l

  Still in testing. Results inconclusive due to repeated server crashes .. I attempted to substitute this version of tc for a current older running version. That might cause crashes.

This is a work-in-progress so the build modification are not integrated into 'configure' inputs.

Build instructions 1) wget http://1978th.net/tokyocabinet/tokyocabinet-1.4.34.tar.gz 2) wget http://1978th.net/tokyotyrant/tokyotyrant-1.1.35.tar.gz 3) locate proc/readproc.h

• if it doesn't exist install it
• standard headers for libproc.so 4) tar xvfz both tarballs 5) apply the patch to tokyocabinet-1.4.34 patch -p0 < tc_1_4_34_rss_management.patch 6) configure and make as normal - will error on link 7) make -f Makefile.TC.libproc 8) configure and make tokyotyrant as normal - will error on link 9) make -f Makefile.TT.libproc 10) Install ttserver_rsshack, libtokyotyrant_rsshack.so.3 & libtokyocabinet_rsshack.so.8 in the normal places 11) ldd libtokyocabinet_rsshack.so.8 libbz2.so.1 => /usr/lib64/libbz2.so.1 (0x00002ba0f91f9000) libz.so.1 => /usr/lib64/libz.so.1 (0x00002ba0f9409000) librt.so.1 => /lib64/librt.so.1 (0x00002ba0f961d000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ba0f9827000) libm.so.6 => /lib64/libm.so.6 (0x00002ba0f9a41000) libc.so.6 => /lib64/libc.so.6 (0x00002ba0f9cc4000) libproc-3.2.7.so => /lib64/libproc-3.2.7.so (0x00002ba0fa013000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) 12) ldd ttserver_rsshack libtokyotyrant_rsshack.so.3 => /usr/local/lib/libtokyotyrant_rsshack.so.3 (0x00002ab055cd6000) libtokyocabinet_rsshack.so.8 => /usr/local/lib/libtokyocabinet_rsshack.so.8 (0x00002ab055eed000) libbz2.so.1 => /usr/lib64/libbz2.so.1 (0x00002ab05616c000) libz.so.1 => /usr/lib64/libz.so.1 (0x00002ab05637d000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002ab056591000) libnsl.so.1 => /lib64/libnsl.so.1 (0x00002ab0567a6000) libdl.so.2 => /lib64/libdl.so.2 (0x00002ab0569be000) librt.so.1 => /lib64/librt.so.1 (0x00002ab056bc2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ab056dcb000) libm.so.6 => /lib64/libm.so.6 (0x00002ab056fe6000) libc.so.6 => /lib64/libc.so.6 (0x00002ab057269000) libproc-3.2.7.so => /lib64/libproc-3.2.7.so (0x00002ab0575b7000) /lib64/ld-linux-x86-64.so.2 (0x00002ab055abb000)

It should be trivial to extract the code/functions from libproc to remove the dependency on librproc. libproc's get_proc_stats() reads and parses the /proc/[PID] file to fill a proc_t struct. This would be a bit cleaner...