Testing Procedures

Testing Procedure

Hey guys, what’s up!

I try my best to provide you with an objective perspective on each device I review to allow you to make informed purchasing decisions. I ensure that the tests are fair, give real results, and are constantly refined as technology evolves. On this page, I’m offering a detailed look at the specifics of the testing process.

Phase 1 – Daily Driver

When I receive the device, I immediately start using it as my primary device. I set up the device as if I’m going to be using it in the near future. As I use it, I take notes on various aspects of the device, the build quality, the interface, camera, etc. I also ensure that my usage of the device matches how I use previous devices (if I’m testing a new phone, I ensure I play the same games in the same amount of time as the old phone) so that it stays consistent.

This part of the test is not as scientific as the next phase, but allows me to build a general idea of how I’m going to write the review.

Phase 2 – Detailed Tests

After I’ve used the device for a week give or take, I begin detailed tests. I move back to using my old device and start intensively testing.

I take a good look at the display, compare it side by side with the other devices I have, and compare the maximum brightness as well as sunlight legibility.

I test the speaker quality, both in loudness as well as quality. Its rather easy to test the loudness of the speakers, playing the same audio track side by side with other phones. However, quality is a little trickier. Basically, I play a detailed audio track at the loudest volume and listen for any distortion, muddling, and noise present in the audio tracks and rate them accordingly.

I snap numerous pictures in controlled indoor settings, and also controlled outdoor settings (as much as I can control the weather), and compare the sharpness, colour reproduction, and noise of each photo. However, I do not spend 20 minutes setting up for the perfect picture because that is not how most people use smartphones, I take maybe a few seconds to position myself and take the photo in order to best represent how the user would use the camera.

Finally, I perform intensive battery tests. There are primarily two tests that I perform, the WiFi browsing test and the video playback test.

In the WiFi browsing test, I used to automatically reload a few webpages that have different elements, for example, the mobile version of CNN, and websites like www.gearbest.com and www.focalprice.com. I then tried doing the exact same test, but just with one webpage, and the results were very close. Since then, I’ve changed my WiFi testing procedure to just reloading one webpage (www.slickdeals.net, desktop version) every 10 seconds until the phone dies. I set the screen brightness to 250 nits and installed Web Reloader from the Google Play Store and used it to reload the www.slickdeals.net webpage over WiFi every 10 seconds.

In the video playback test, the screen brightness is again set to 250nits and I loop a 480p video until the phone dies.

In addition, during Phase 1 of the testing procedure, I record daily battery statistics that I add to the battery section of the review as well.

Phase 3 – Wrap Up

In this phase, the detailed testing is finished and I use this phase to perform retests on parts of the phone that I feel weren’t done as well or needed a second test. I also record down what I didn’t like about the phone after the two previous phases.

Hopefully this gave you a little insight on the testing procedure, and if you have any questions, hit the comments below!