As the host, I don’t actually have to answer this one, and I won’t lie, I considered it. This might just have been too much to ask of anyone, much less the one who set the whole question up. I am answering because this could be a really powerful thing to let other people of all experience levels know that mistakes will be made. In my case, a bit of a rookie mistake.

I hope other people admit to a mistake they made that can really show that all the newbies that they are not alone. I know my mistake was.

Types of Mistakes

Some mistakes are choices. For example, you were around when VCRs were coming out and you chose BetaMax even though your local Blockbuster was all VCR. As a DBA you may have chosen a backup/restore strategy that didn’t work in the situation that actually occurred. These are mental errors that often hurt the most because perhaps there is a lot of data loss that you have to deal with, or you have all your home videos on a format that you can’t find a way to view.

Others are physical mistakes. These concepts came about for me when I used to play tennis. Lobbing the ball right to your opponent is a choice, but missing the shot and lobbing it where they can smash it down your throat was a physical mistake. I knew what I was trying, but I missed and now am picking yellow fuzz from my teeth.

In computing, a physical mistake data programmers might make occasionally is to plan to highlight an entire DELETE statement. It had a WHERE clause that deleted just one customer, but you missed the WHERE clause and every customer in your database was deleted and then you had to test restoring your database to get that data back… only to realize it was actually only backing it up weekly. (Mistakes from bad choices hurt worst of them all!)

Note too that mistakes are things that are theoretically preventable. It is not a mistake if someone asks you to choose a number between 1 and 10 unless you answer 375.2 (or any number that isn’t between 1 and 10, but 375.2 is possibly THE worst thing to choose).

You could not have studied longer, practiced harder, or implemented safeguards to make sure you chose the exact random number. If your server needs to be restored and you haven’t been backing it up because you “haven’t needed it before”, well, then you are squarely in the mistake zone.

A Mistake I Learned from (too) Recently

Most of my big mental mistakes that I have made and noticed the effects of as a DBA are database design errors. Some are just annoying (like without spell check I cannot spell solicitation about 2/3 of the time and there may be a column or two spelled wrong. Designing databases is a rewarding task, but the mistakes you make will live with you (or the people you leave it to) practically forever.

But the most recent really dumb thing I did was a physical mistake. I had just started back at my current company last summer. We have databases that can be tricky to build from a script for one reason or another, so pulling them down from source control is a lot. As I was working through this process, trying to document the process, well…. I was tired of how it was working, so I started creating a script to make it easier.

To make that first load easier, I was dropping the database, recreating it, all in a lovely repeatable manner, adding to the script each time until I could pre-create anything needed so the “normal” script would work. Then I needed to do some work in a DEV/QA database on another server. I researched the problem, got it done, and moved back to the process to get that database loaded on my local server.

Of course, you can probably see where this is going.

And you are right. I dropped the database because I was inadvertently in the wrong environment context.

I have been doing DBA like tasks for 30 years. This was the first time I had ever dropped a database I didn’t mean to (a few tables, some data, a few times, not saying I am perfect, but never a database.) Though it is no excuse, I had not been doing DBA work for a few years, and I had lost my DBA “check everything multiple times, then one last time when you are doing potentially dangerous actions” mindset. As an editor, a typo is bad, but nothing compared to dropping an object.

Since this is a database that is loads data nightly for testing, it was easy enough to restore and reload thankfully. It did require a few hours of work/distraction for people other than me.

And it was super embarrassing 🙂

My Biggest Takeaways

Do everything you can to make sure you have as much of a safety net as possible when working with server resources.

In my case, a few things:

• Be safe. It was important to get my brain back in the safety/security mode of working with data that matters (even non-production data!)
• Backups. These were there, the database was restored, and the data re-added but it did cost several hours for several people since this wasn’t a database that had any high-availability features available. But it could have been much much worse, even in DEV.
• Clear indications of environment context. I turned on coloring of tabs in SSMS. Being able to see the difference in environments is an outstanding safeguard when working on multiple environments.
• Separation of Duties. When possible, have security set up in a way that you can’t make changes except in one environment. Not completely appropriate to this scenario, because it was a DEV database.
• Limit mixing environments. Don’t connect to multiple environments in SSMS unless you need to (like you are comparing two sets of data.
• Use preventative measures. In my script to drop the database, I added a line that checked for the environment to make sure that even if somehow the script was executed on a production database, it would not be dropped and would simply throw an error.
• Use (and don’t automatically commit) transactions. Not exactly applicable here, since DROP DATABASE is not transactional, but most other things are. So whenever writing manual, risky code, open a transaction automatically, and don’t commit it until you know you are ready.

In conclusion, I hate learning lessons like this

Sometimes it is good to admit that we make mistakes. What matters is how we handle the mistakes and how we work to make sure we don’t make them again.

It is especially delightful relearning basic things like “be careful”. But learning lessons is how we learn lessons.

Sometimes you need a refresher, and as much as I hope the entries in this series will, in fact, save you from a physical mistake or two, there are so many more ways to do accidental stuff that you may be the first to achieve. And you can share them next time.