iPowow – Analytics

ABOUT THE PROJECT

iPowow Analytics is the reporting platform I built for iPowow over the course of 4 months that allows producers in the company to fetch analytical data from multiple sources and build rich PDF reports with full HTML markup, charts and images.  Producers are able to pick any time range, slice the data based on a weekly or daily view and add their own touches to the pages using the built-in rich text editor Redactor.  As I was the sole architect and engineer on this project I saw the project through from concept to finished product.

The Analytics platform required a simple job system to be crafted, which I built using Node.js and MongoDB.  We called this project “Mura”, from the Japanese word meaning “unevenness; irregularity; lack of uniformity; nonuniformity; inequality” which has processing roots to FIFO queue based systems.  The Mura queue was constructed with one master “Hive” server, and many “Drone” servers.  The Hive is responsible for managing moving jobs into the task queue when it’s time to schedule them, and reaping expired or failed jobs from the queue and rescheduling them.  Drones were the workers of the cluster, they are responsible for receiving work from the Hive server and processing the work using the associated module for that specific task.

Mura is used in a variety of ways in house:  Report generation to PDF using a headless Phantom.js browser, scraping of metrics from the Google Analytics API, sanity checks on the IoT platform ensuring appliances are healthy and not experiencing any problems.

Technology Profile
  • Node.js for core systems and the communication platform
  • Phantom.js for headless report rendering to PDF
  • Angular.js, Grunt, WebSockets for the front end UI layer
  • MongoDB, Redis for the main databases and message passing layer
  • EC2 for server infastructure