your datalake needs structure

By | August 10, 2021

A typical datalake contains raw data from many sources. The data can be in various forms: unstructured, semi-structured or structured. UnStrucutured TXT CSV Semi-Structured XML JSON Structured MySQL Parquet This post will be about improving your datalake by using parquet files for better downstream processing. Why use Parquet files? Structured files are preferred for data… Read More »

take a spark trip with Zeppelin notebooks

By | July 28, 2021

In a previous post we used a jupyter notebook to play with spark. In this post we will use a zepellin notebook. Both can be used for big data and it is personal preference which one you want to use. Create a directory and create a csv file First create a directory on your machine… Read More »

add a little spark to your toolbelt

By | July 27, 2021

This tutorial will be about playing around with spark using a jupyter notebook. The Jupyter notebook allows you to create documents that contain live code, equations, visualizations and narrative text.  We will use this notebook to become familiar with spark and demonstrate some big data concepts. To keep costs low it will be installed on… Read More »

promises promise what?

By | June 16, 2021

This post will be on promises in javascript. Promises are used for asynchronous processes where something will take time and it will prevent blocking your code. Promises return a resolve for a successful operation or a reject for a failure. Define a promise as follows and then use .then for the success and .catch for… Read More »

let’s play in the CoderPad sandbox

By | June 7, 2021

After a successful call with a recruiter the next step typically is a technical interview. Technical interviews are an opportunity to show off how you code and think. I have frozen up in a lot during these. I am not a fan of toy problems, but it is a necessary evil in the interviewing process.… Read More »

what’s the story with StoryBook

By | February 2, 2021

Storybook is a great UI testing tool and is simple to install in your react project. To install enter the following. Now check that is running by typing Storybook will compile and present a page like the following. If not navigate to ‘http://localhost:6006/’. The storybook example also installs some react components that you can play… Read More »

react with typescript

By | January 21, 2021

Why use typescript with react? If you like strongly typed languages you will welcome using typescript in your react projects. This simple project will show you how to create a simple react app with a typescript template already installed. To create a react project with typescript it is pretty simple to do. All you need… Read More »

who owns the code, we do

By | January 6, 2021

Code ownership is a debated subject with two opposing trains of thought. Single person ownership or team ownership. I believe a hybrid of the two is the best choice. Open source In an open source project a person or a small group owns the code. Many people can contribute to the project by doing their… Read More »

off to AWS we go with Terraform

By | December 11, 2020

Running terraform locally is great for local development but the real power of Terraform is when you provision your code on the cloud. This post will discuss how we can do that. First you will need a AWS account, click on this link for information on the free tier account. Next you need to install… Read More »

on a tear with terraform

By | December 11, 2020

Imagine if you could write code that would set up your infrastructure on any cloud system and would not have change if you switch providers. This is the concept of infrastructure as code. Terraform is one of the most popular tools to do this and will be the subject of this post. Let’s get started… Read More »