Reproducibility in testing results is of huge importance, since it allows any individual to verify (or to challenge) our findings. For someone to be able to take similar readings, he has to have access to similar and calibrated equipment. In today’s article I will shed more light in this hot topic.
I engaged in a conversation in the HWbuster discord server, and I realized that most people don’t understand the importance of having not only the proper equipment for testing various PC hardware (and other stuff, too) but equipment that other individuals or labs can get, and repeat your measurements and either verify or challenge them. It goes without saying that the equipment has to be well maintained and calibrated yearly on most occasions.
To give you an example of how important this is, let’s say that I am a genius and can build testing equipment for everything I want to measure on my own. This means that nobody else can access the same tools and cannot repeat the same procedures and get results to check the validity of my results. Moreover, custom-made equipment is not checked against any standard to verify its accuracy, and in most cases, it cannot be calibrated too. But let’s also explain what calibration means. Simply put, let’s say that I want to calibrate a bench power supply. I measure its output with a multimeter which I know is accurate, my reference instrument, and by using this multimeter, I compare the accuracy of the bench PSU’s output. If I find significant deviations, I tune the PSU to be as accurate as possible. In this case, the calibrator, the instrument used as a reference, was the multimeter. As a side note, the reference instrument has to have higher accuracy than the device under test. If the bench PSU’s resolution was 1mV, the multimeter’s volt resolution has to be higher (e.g., 0.1mV)
Every lab that respects itself and its customers has to operate under the ISO/IEC 17025 guidelines, which are tough but fair, leading to the ideal operation and lab results output. This ISO helps laboratories generate trustworthy results, which other labs can verify. This is why test reports and certificates from ISO 17025 labs are widely accepted, without any doubt, by everyone. One of the fundamental ISO 17025 requirements is for a lab to find another lab that has similar capabilities and areas of expertise and to verify the results with each other. For example, Cybenetics, which follows the ISO 17025 guidelines, must frequently verify its results with another lab. This is easily done. Cybenetics runs complete tests on a power supply or another PC part and then ships the product to another lab to take the exact measurements, using the same or similar capabilities equipment. Afterward, both reports are compared to find potential issues or high deviations.
The table above shows the differences in vampire power on the same sample between the Cybenetics lab and a TUV lab. The difference at 115V is minimal, but there is a notable difference at 230V, around 8.86%, which needs further investigation since both labs used calibrated but different power analyzers and different testing fixtures.
From the moment Cybenetics uses widely available and calibrated equipment, its customers can use another lab to verify its results. This couldn’t be the case if Cybenetics used custom-made equipment so that most brands wouldn’t accept its results. This is precisely the case for reviewers. When you use custom-made and/or non-calibrated equipment, the results can be easily challenged by anyone. This is why I decided from the start of my review career to invest in commercial load testers and not continue building my machines since I got tired of brands doubting the results of my first custom loader, Faganas. But I couldn’t blame them since I used something nobody could access.
I know firsthand how tough it is for most reviewers to get lab-grade equipment and that they try to do their best with the resources that they have in hand. Moreover, the vast number of reviewers, especially on YouTube, makes it super tough to make a decent profit from reviewing and consequently invest in proper equipment. In other words, the Return on investment (ROI) is absent. This is something that recently, a fellow reviewer told me about the Powenetics v2 kits that I sell. That the price is too high for him to get his money back at some point, he probably exaggerated a bit since most high-end graphics cards cost more than Powenetics, but you get the meaning.
The purpose of today’s article is not to make me feel like the reviewer hero with all that fancy equipment, which actually belongs to the Cybenetics lab, but to make you aware that when you hear someone using the word “laboratory,” they should be careful since many vital things are hiding behind that word, with the most important being the ISO 17025 accreditation, which verifies that a lab works appropriately in its section. I barely scratched the surface of this important matter. If you want to read more, look at this article by MIT.
