A portable data analyst’s environment
Posted by Adrian on 2010-03-30
In my paying job, I am an information management consultant. My work includes data analysis, data profiling and data quality improvement. It can take some time to get access and resources necessary to perform what I’ve been engaged to do.
Every client (and employer for that matter) has different tools and utilities, and usually takes a slow route through procurement.
My technique to quick results invokes a simple mantra: portable open source applications on Windows.
The basis for my Windows toolkit is PortableApps and XAMPP.
If I can install my toolkit to a hard drive, I’ll install Portable Apps to C:\Portable\... and XAMPP to C:\xampp. Since I already have done this on an external hard drive, I can just copy the Portable and xampp directories across from that without running an install program.
I have portable versions of Java, Firefox, 7zip, WinMerge, PuTTY, GIMP, CommandPrompt, FileZilla, ClamAV and a host of other utilities installed under the \Portable\PortableApps\ directory. Just click on the \Portable\StartPortableApps.exe executable for a menu in the taskbar. This works no matter which medium it is on.
XAMPP gives me MySQL, Apache web server, Tomcat, PHP and Perl, bundled together. Using the PortableApps XAMPP Control panel, I can fire up a full backend web/db server, point my browser to http://localhost and presto chango, have a web app running without installation.
So, in summary I have a copy of my full environment on my laptop, a mirror of that on an external hard drive, a minimal version on an 8Gb USB key and a client subset on the client’s PC. Each can run a suite of open source applications and utilities, as well as a fully functional database and webserver, from the medium I choose with installation. (From the machine’s hard drive is fastest, though, and so the preferred mode of operation.)
I also have a few other applications installed under \Portable\apps, for those apps not catered for by the PortableApps stack. I have a \Portable\apps\bin directory holding utilities like bzip2, robocopy and for any utility scripts I write. When I need to run these, I run them inside a PortableCommandPrompt, which has paths & environment variables set up for the portable environment.
Perl, Ruby on Rails, DataCleaner (a data profiling tool), Talend (Open Profiler and Open Studio) and JasperSoft iReport are installed under \Portable\apps, using zip files to install where possible (in preference to running an install exe). JasperServer is installed as a Tomcat webapp.
All this rolled together allows me to deliver data work with little delay, assuming the client allows the connection to a USB drive. Look for more posts on this subject in the near future.
