Daily Archives: July 11, 2014


Using HAWQ/Pivotal Hadoop Virtual Machine 1

Yesterday I was asked by someone… Can I play with PivotalHD and HAWQ? How? What do I need?

This prompted me to think about going through this process myself afresh using my home computer to make sure everything works as smoothly I expected to work when we put together PivotalHD/HAWQ VM.  As I was downloading and trying to run the VM I realized that though running VM, thereby PivotalHD/HAWQ is straight forward there is no single place the entire process is laid out.  Therefore, hoping this blog post will guide someone step by step, from scratch to run PivotalHD and HAWQ.  In particular, in addition to running VM I wanted to show using HAWQ with pgAdminIII a popular open source tool used by Postgres community.
Install VMPlayer
Step 1: Skip this step if you  have either VMWare Fusion (for Mac) or VMWare Workstation/VMPlayer (for windows) already installed in your computer . If not, proceed to download one of these tools  from https://my.vmware.com/web/vmware/downloads.

Tip: VMWare Fusion (for Mac) and Workstation (for Windows) are paid tools. If you are running windows and don’t want to pay for Workstation, you’ve the option of using VMPlayer which available only for Windows or Linux, there is no Mac VMPlayer. If you are using Mac/OS X you would’ve to use other free alternatives.  VirtualBox is a free Mac alternative to VMPlayer you could try. Be warned, I’ve not tried this option myself, but should work. I’m sure you can google  http://smallbusiness.chron.com/open-vmdk-virtualbox-28847.html

Scroll down to “Desktop & End-User Computing” section and download VMPlayer.

VMPlayer_Download

 

Download Pivotal HD.

Download Pivotal HD VM from http://gopivotal.com/big-data/pivotal-hd.

 

pidHD_download

VM is pretty big 1.9GB, be patient. At my home, I’ve reasonable fast AT&T uverse. It took approximately 45min to download the entire file.

If you don’t have unzipping software, you can download 7z windows version from http://www.7-zip.org/download.html

Once unzipped, right click on the file PIVHDSNE110_VMWARE_VM and select “Open with VMWare Player” option to open using VMWare Player. Once launch happened you’ll see PivotalHD Desktop as shown below.

 

PivHD_Desktop

HAWQ administration documentation can be found here: http://docs.gopivotal.com/pivotalhd/HAWQAdministration.html

Double click on “start_all.sh” to start HAWQ services in the VM.

 

Before connecting pgAdminIII to HAWQ host based security should be enabled for the local host.

 

[pivhdsne:~]$ echo $MASTER_DATA_DIRECTORY
/data/1/hawq_master/gpseg-1
[pivhdsne:~]$

cd to “/data/1/hawq_master/gpseg-1” and open the file “pg_hba.conf” in your favorite editor. Add your local host where pgAdminIII runs as shown below.

 

pg_hba

After saving the file, run the following command to reload the configuration file.

[pivhdsne:~]$ gpstop -u

 

Now configure

 

Connecting pgAdmin3 to HAWQ.

Click on the connect icon. In the dialog box, enter your pivotal HD virtual machine ip address. To find virtual machine IP address enter the following command in a terminal window

pgAdmin_connection

 

[pivhdsne:~]$ ifconfig
eth1  Link encap:Ethernet HWaddr 00:0C:29:D8:10:3B
inet addr:192.168.2.104 Bcast:192.168.2.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fed8:103b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
…..

……

In the above example, virtual machine address is 192.168.2.104. Enter this in the host field of pgAdminIII. Enter username as “gpadmin” and password as “gpadmin”.

Click “ok” to connect to HAWQ. Ignore following warning message.

 

pgAdmin_warning

Download pgAdmin3 from this link: http://www.postgresql.org/ftp/pgadmin3/release/v1.18.1/win32/

Once successfully connected you should be able to see pgAdminIII screen as shown below.

 

pgAdmin_screen

PivotalHD tutorial can be found here: http://pivotalhd.cfapps.io/index.html

PivotalHD samples can be found here: https://github.com/gopivotal/pivotal-samples