Posts

Showing posts from December, 2011

Export HDFS over CIFS (Samba3)

Three weeks ago I played with libhdfs and NFS, but I did not get the results I expected. Then my next idea was, why not to use Samba? Samba3x is stable and most OS can mount an exported share.
The main task was to research the performance and setup of this scenario, because samba has a lot of tuning mechanisms inside. Let's go!

I used a RHEL 5.7 and the delivered RPMs:
#> rpm -qa|grep samba
 samba-3.0.33-3.29.el5_7.4.x86_64
 samba-common-3.0.33-3.29.el5_7.4.x86_64

Like I described in "NFS exported HDFS" I mounted hdfs over fuse into the directory /123 via /etc/fstab:

#> cat /etc/fstab
 [..]
 hadoop-fuse-dfs#dfs://NAMENODE:9000 /123/hdfs fuse usetrash,rw 0 0

and checked it:
#> mount
 [..]
 fuse on /123/hdfs type fuse (rw,nosuid,nodev,allow_other,default_permissions)

#> ls -la /123
 total 16
 drwxr-xr-x  3 root root   4096 Dec  9 16:36 .
 drwxr-xr-x 27 root root   4096 Dec  9 12:11 ..
 drwxr-xr-x  5 hdfs nobody 4096 Dec  9 02:14 hdfs

The first step afterwards is to configur…

OSX improved shell environment

Image
There is my favorite environment for all reasons on my MacBook and come with an improved zsh and extended .vimrc with highlighting, checking, TextMate, Solarize. Some tools are to install, so here a recipe for it.
Features are direction keys navigation through directories or files, dev friendly colors, command highlighting, improved history, option autocomplete, ssh autocomplete (if the keys are known) and a lot more useful things.

Get Xcode:
 AppStore => Xcode => Install Xcode

From now we use a terminal window.

Install Brew
/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

Install git and wget:
brew install git
brew install wget

Install oh-my-zsh:
wget --no-check-certificate https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | sh

The script want to change your shell from /bin/bash into /bin/zsh, here you have to provide your password.

To change the theme edit ~/.zshrc and edit line 8:
ZSH_THEME="YOUR_FAV_THEME"

Install and en…