Intro to Automated Package and Template Generation with BIML In order to simplify and automate the template creation and package creation process, we will create a Proof-of-concept (POC) that uses the Business Intelligence Markup Language (BIML, pronounced bim-ul). BIML provides a language with which to generate SSIS packages using highly configurable metadata. Business Intelligence Markup … Continue reading Biml Tutorial 1 – Introducing BIML
The Common ETL Framework Part 2
Auditing/Logging Each package is responsible for logging at two different levels: Audit and Debug. The Audit level of logging needs to record the following for each package execution: Start Day and Time Executing Machine name Executing as Username Records Affected (if appropriate) – Can include Records Inserted, Updated, Deleted, Archived, Errored, etc. Error data if … Continue reading The Common ETL Framework Part 2
The Common ETL Framework Part 1
Defining a Common ETL Framework SQL Server Integration Services (SSIS) provides a mechanism to Extract data from some source, Transform it as necessary, and Load it into the tables in a target database (ETL). Each SSIS package should implement a common framework for logging, executing, monitoring and error handling so that these details are abstracted … Continue reading The Common ETL Framework Part 1
Redirecting Web Traffic from HTTP to HTTPS
So I need to redirect the incoming traffic to my website so that it always uses https/SSL. I am running IIS 7.5 on my Windows 7 machine. I googled for "how do I make sure that all my web traffic uses https". The second link showing was to my web host, GoDaddy.com. They have a page … Continue reading Redirecting Web Traffic from HTTP to HTTPS
Hadoop Follow Up – Hortonworks HDP Sandbox
The Hortonworks Hadoop Sandbox download got corrupted the first time. It worked fine the second time. Installation I installed Oracle VirtualBox first. Then, in the Oracle VM VirtualBox Manager, I select the File | Import Appliance... option, selected the HDP_2.4_virtualbox_v3.ova file and clicked Next and Import. A few seconds later, the box was installed, so I … Continue reading Hadoop Follow Up – Hortonworks HDP Sandbox
My First Foray into Hadoop
So I have a big dataset (1.7 billion rows) that I want to analyze. I figured, "Hey, Hadoop is all over this Big Data thing, I wonder if I can do a Proof of Concept?" Compiling Hadoop on Windows (Ugh!) So, first, I tried to follow some instructions on how to get the Hadoop source … Continue reading My First Foray into Hadoop
Oops…I Broke the SQL Server
So this happened. In an attempt to give my SQL Server Instance access to more memory, I set the Max Memory to zero, expecting that to mean infinite. No luck, SQL Server Management Studio (SSMS) set the max memory to 16MB instead and broke the instance. I could not do anything more in SSMS because … Continue reading Oops…I Broke the SQL Server
Undoing Someone Else’s Changes in TFS
Do a google search for "how to check-in another user's changes in TFS" and you will find a couple of pages of MSDN documentation and a whole lot of forum articles questioning how to do this. Unfortunately, there is no UI option to do this, so you must resort to the command-line. Here's the command … Continue reading Undoing Someone Else’s Changes in TFS
Creating User Stories
The 5 W's of an Issue When an issue gets raised in a production environment, a project planning meeting, or a requirements-gathering meeting, we ask the Who, What, When, Where, and Why of the issue to figure out how to solve it. Typically, the Where is implied as "In the System" and the When is … Continue reading Creating User Stories
The 3 Things a Project Needs to Accomplish – Highest Level Requirements Document for an Agile Project
Often, in an agile software development project, there can be a “fly by the seat of your pants” feel to the development effort as stories are created, prioritized and scheduled into Sprints without formal Requirements. One way to handle this lack, is to create a “3 Things My Agile Project Needs to Accomplish” document.
One Man went to Mow... 