Pretty much everyone has a drawer full of tools somewhere in their house. In that drawer are some tools you pull out for use when you need to do something. You might have 2 screwdrivers, or 50. I have 50:

My father, who passed away 30 year ago this year, had been a master mechanic for 20+ years and had accumulated many of these tool. He understood why they were different. I inherited them, and I kept his toolbox because it was his (and it looks cool in the garage):

I don’t know how to use 1/3 of the tools that are still in this box, and the rest are rarely used. My toolbox is this keyboard I am kicking right now. But he knew how and why he bought all of these tools. He knew why he needed two screwdrivers I can barely tell apart.

My tools are query engines

I have recently been working with Fabric Data Warehouse more and more. This week I am starting a series of posts on the differences between the Data Warehouse product and the relational SQL Server and Azure SQL DB. It is mind-bending at times because after you have things set up, you can pretty much treat at least the Fabric Data Warehouse just like any SQL Server database.

But you shouldn’t. Because they are different. They aren’t even built for the same purpose exactly. But they have a lot of shared qualities.

I start most of my database design sessions with a little discussion of why we build databases. The input is simply “Record Activity”, which I note anyone can do really well using about any medium. HUGE businesses were run on paper years ago. Small businesses today still run on a paper, or QuickBooks, or Excel spreadsheets. You can just save every transaction to a text file and have all the activity recorded.

Then I have a bunch of arrows in the output side to note all the different purposes that data serves from processing orders, analysis, audits, and our new frenemy: AI.

Different query engines have different purposes

I had never professionally used a lakehouse style database before, but I had done some editing on articles about Snowflake and had a bit of idea how different it was going to be when I started using Fabric. I was not wrong.

The coding and patterns you can use in a lakehouse versus a relational databases have a lot of similarities and a lot of differences. And reminding myself (and getting reminded by Jason) of that regularly has helped, but it is hard.

Keep on learning

Every screwdriver had a purpose in that box. There are many different screw sizes on a vehicle (especially 40 years ago…ugh, I am getting old), and every tool has its purpose. The same can be said about platforms when programming. Data Warehouses, Lakehouses, Relational Databases, all have differing purposes. Every screwdriver had a purpose in that box. There are many different screw sizes on a vehicle (especially 40 years ago…ugh, I am getting old), and every tool has its purpose.

The same can be said about platforms when programming. Data Warehouses, Lakehouses, Relational Databases, all have different uses. Each edition of SQL Server is there for a different use. DIfferent languages have different purposes.

Whether you are a specializer like me, or a generalist/full-stack developer, you can get comfortable. Specializers need both depth in their specialty, but also enough breadth to understand their environments. You just have to understand the differences.