Statement regarding PCMark05 culmanitive cleanup and rules update

You may have noticed some recent changes to the our PCMark05 rankings, this is the due to culmanitive cleanup effort of the HWBOT staff. For some time now the PCMark ranks have become increasingly out of control and this is our attempt at bringing a once florishing benchmark back to a competitive and enjoyable state.

The first and most important point to stress is there have been only minimum rule changes (listed below). Most of the removed submissions were violating two fundamental HWBOT general rules:

  • "Using software, performing hardware modifications or by human interaction altering the perceived speed of the benchmark program, tricking it to believe it ran faster and thus producing a better result. (adjusted 31st of August 2010)"
  • Update 20th July 2010: Any software or human interaction causing the benchmark to use different calculation algorithms as intended by the original software creator. E.g. replacing the codec or browser in PCMark2005.

Most of the "super tweaks" consisted of tricking PCMark into giving higher scores using deceptive techniques. It should be clear that this conduct is not tolerated by HWBOT and will result in appropriate action.

There are some specific rule clarifications for performance enhancements that fall between the current HWBOT general rules and PCMark rules.

  • 1. Sector size changes are banned. PCMark doesn't have an appropriate internal mechanism to deal with the sector changes and resultantly gives incorrect scores. This does fall under altering the percieved speed of the benchmark, but we believe it also needs to be individually clarified.
  • 2. AMD Raid Xpert and Intel RST are allowed. After long debate weve decided that while there is some ram disk caching involved in these utilities they have both been long accepted and removing them would cause more harm than help.

Furthermore the arguement has been raised that allowing RST/Raid Xpert is penalizing those with expensive raid controller setups and allowing it goes against our promotion of PCMark as a storage based benchmark. Just to clarify this, those with raid setups will still have a significant advantage in the general access test.

We acknowledge that in future performance enhancements may arise that are not covered in our current rules and we will make a strong effort to ensure these issues are dealt with in a prompt and fair fashion.

While HWBOT will always try to outline rules as clearly as possible, please use common sense when determining if something is deemed legal or not. An easy way to determine if your latest tweak is legal is to apply the "does my tweak alter the percieved speed of the benchmark", if the answer is yes, then in most cases it will be disallowed. If you are unsure if a tweak is allowed you can refer to the Clarification in PCMark thread or contact a moderator.

HWBOT staff hope that these small changes will allow PCMark to return to a balanced benchmark of which storage is a key component.


1

Slovenia tiborrr says:

Good call, thank you! :nana:

Please log in or register to comment.

Leave a Reply: (BBCODE allowed: [B], [QUOTE], [I], [URL], [IMG],...)