A typical datalake contains raw data from many sources. The data can be in various forms: unstructured, semi-structured or structured. UnStrucutured TXT CSV Semi-Structured XML JSON Structured MySQL Parquet This post will be about improving your datalake by using parquet files for better downstream processing. Why use Parquet files? Structured files are preferred for data… Read More »
In a previous post we used a jupyter notebook to play with spark. In this post we will use a zepellin notebook. Both can be used for big data and it is personal preference which one you want to use. Create a directory and create a csv file First create a directory on your machine… Read More »
This tutorial will be about playing around with spark using a jupyter notebook. The Jupyter notebook allows you to create documents that contain live code, equations, visualizations and narrative text. We will use this notebook to become familiar with spark and demonstrate some big data concepts. To keep costs low it will be installed on… Read More »
This post will be on promises in javascript. Promises are used for asynchronous processes where something will take time and it will prevent blocking your code. Promises return a resolve for a successful operation or a reject for a failure. Define a promise as follows and then use .then for the success and .catch for… Read More »
After a successful call with a recruiter the next step typically is a technical interview. Technical interviews are an opportunity to show off how you code and think. I have frozen up in a lot during these. I am not a fan of toy problems, but it is a necessary evil in the interviewing process.… Read More »
Storybook is a great UI testing tool and is simple to install in your react project. To install enter the following. Now check that is running by typing Storybook will compile and present a page like the following. If not navigate to ‘http://localhost:6006/’. The storybook example also installs some react components that you can play… Read More »
Why use typescript with react? If you like strongly typed languages you will welcome using typescript in your react projects. This simple project will show you how to create a simple react app with a typescript template already installed. To create a react project with typescript it is pretty simple to do. All you need… Read More »
Code ownership is a debated subject with two opposing trains of thought. Single person ownership or team ownership. I believe a hybrid of the two is the best choice. Open source In an open source project a person or a small group owns the code. Many people can contribute to the project by doing their… Read More »
Running terraform locally is great for local development but the real power of Terraform is when you provision your code on the cloud. This post will discuss how we can do that. First you will need a AWS account, click on this link for information on the free tier account. Next you need to install… Read More »
Imagine if you could write code that would set up your infrastructure on any cloud system and would not have change if you switch providers. This is the concept of infrastructure as code. Terraform is one of the most popular tools to do this and will be the subject of this post. Let’s get started… Read More »