Something that every writer needs to be careful of is doing too much benchmarking-type work. In many of the software licensing agreements you have signed, you promise not to do that. But at the same time, you can generally give out performance numbers if you aren’t making claims about particular software, especially compared to another.

So, if you come up with an algorithm to do something in a better way than you have seen, it is nice to show the software, give the reader access to the code you are showing the performance of, and include the computer you are running it on.

This gives people transparency, and the ability to try your code on their machines and give feedback.

In this vlog, I want to cover the importance of providing context when discussing performance metrics. When you say something executes in 10 seconds, you also need to know if that is on an i7 or that 486 computer with the 1MB of RAM that you have under your bed. Without transparency regarding the hardware used for testing, your performance claims could mean anything.

So, if you need to tell readers your reference architecture for any performance metrics, along with the necessity of running tests multiple times, in different orders sometimes, to ensure accuracy in performance comparisons.

Takeaways

• You need to explain what you are executing code on when you say it is 10 times faster.
• Run any performance tests a few times, doing what you can to eliminate outside costs
• What do you mean by that? Is it 10 times faster because it’s disk-bound?
• Use tools like CrystalDiskInto, CPUz, etc., to provide stats to readers when it might matter. (Note, use at your own risk. These links look reputable, but I have not downloaded from them.)
• Tell people if your test machine is active, doing other kinds of work, like my machine has Dropbox and OneDrive on it. I typically test processes several times when I can to make sure I get the same performance multiple times.