Mid-Week Readings : June 18, 2014

  • 10 Hot Startup Websites (FoxBusiness)
  • Your Company is not a Family  (HBR)
    • Reid Hoffman of LinkedIn hits the point home very well. A company is a team, not a family.
  • Every company should dream of getting this kind of endorsement from a VC that has invested in it (a16Z)
    • A blog post by Steven Sinofsky about Tanium.
  • Breakthrough Technologies every analyst should know about (analyst one)
  • Big Data & Business Intelligence Forecasts and Market Estimates 2014 (Forbes)

Last Updated on July 26, 2014 by SK

Using HAWQ/Pivotal Hadoop Virtual Machine

Yesterday I was asked by someone… Can I play with PivotalHD and HAWQ? How? What do I need?

This prompted me to think about going through this process myself afresh using my home computer to make sure everything works as smoothly I expected to work when we put together PivotalHD/HAWQ VM.  As I was downloading and trying to run the VM I realized that though running VM, thereby PivotalHD/HAWQ is straight forward there is no single place the entire process is laid out.  Therefore, hoping this blog post will guide someone step by step, from scratch to run PivotalHD and HAWQ.  In particular, in addition to running VM I wanted to show using HAWQ with pgAdminIII a popular open source tool used by Postgres community.
Install VMPlayer
Step 1: Skip this step if you  have either VMWare Fusion (for Mac) or VMWare Workstation/VMPlayer (for windows) already installed in your computer . If not, proceed to download one of these tools  from https://my.vmware.com/web/vmware/downloads.

Tip: VMWare Fusion (for Mac) and Workstation (for Windows) are paid tools. If you are running windows and don’t want to pay for Workstation, you’ve the option of using VMPlayer which available only for Windows or Linux, there is no Mac VMPlayer. If you are using Mac/OS X you would’ve to use other free alternatives.  VirtualBox is a free Mac alternative to VMPlayer you could try. Be warned, I’ve not tried this option myself, but should work. I’m sure you can google  http://smallbusiness.chron.com/open-vmdk-virtualbox-28847.html

Scroll down to “Desktop & End-User Computing” section and download VMPlayer.



Download Pivotal HD.

Download Pivotal HD VM from http://gopivotal.com/big-data/pivotal-hd.



VM is pretty big 1.9GB, be patient. At my home, I’ve reasonable fast AT&T uverse. It took approximately 45min to download the entire file.

If you don’t have unzipping software, you can download 7z windows version from http://www.7-zip.org/download.html

Once unzipped, right click on the file PIVHDSNE110_VMWARE_VM and select “Open with VMWare Player” option to open using VMWare Player. Once launch happened you’ll see PivotalHD Desktop as shown below.



HAWQ administration documentation can be found here: http://docs.gopivotal.com/pivotalhd/HAWQAdministration.html

Double click on “start_all.sh” to start HAWQ services in the VM.


Before connecting pgAdminIII to HAWQ host based security should be enabled for the local host.


[pivhdsne:~]$ echo $MASTER_DATA_DIRECTORY

cd to “/data/1/hawq_master/gpseg-1” and open the file “pg_hba.conf” in your favorite editor. Add your local host where pgAdminIII runs as shown below.



After saving the file, run the following command to reload the configuration file.

[pivhdsne:~]$ gpstop -u


Now configure


Connecting pgAdmin3 to HAWQ.

Click on the connect icon. In the dialog box, enter your pivotal HD virtual machine ip address. To find virtual machine IP address enter the following command in a terminal window



[pivhdsne:~]$ ifconfig
eth1  Link encap:Ethernet HWaddr 00:0C:29:D8:10:3B
inet addr: Bcast: Mask:
inet6 addr: fe80::20c:29ff:fed8:103b/64 Scope:Link


In the above example, virtual machine address is Enter this in the host field of pgAdminIII. Enter username as “gpadmin” and password as “gpadmin”.

Click “ok” to connect to HAWQ. Ignore following warning message.



Download pgAdmin3 from this link: http://www.postgresql.org/ftp/pgadmin3/release/v1.18.1/win32/

Once successfully connected you should be able to see pgAdminIII screen as shown below.



PivotalHD tutorial can be found here: http://pivotalhd.cfapps.io/index.html

PivotalHD samples can be found here: https://github.com/gopivotal/pivotal-samples

Last Updated on December 5, 2014 by SK

YouTube Suggestion: Five (hundred) little monkeys jumping on the bed!

YouTube server displayed this message this morning at 8am PST. My 2 year old found this, couldn’t understand what was going on and started screaming!!:-) May be time to show a kid friendly 500 message (with a video of 500 monkey’s working on something important), considering early morning during summer holidays people who use youtube are kids?  Just saying….


Last Updated on July 3, 2014 by SK