HWBOT Y-Cruncher Pi-25m Benchmark Validation Regulations

  • Benchmark Rules
  • 9
  • HWBOT

HWBOT Y-Cruncher Pi-25m Benchmark Validation Regulations

Allowed benchmark versions
Allowed operating systemsAll Windows based (Platform Clock must be enabled for Windows 8/8.1/10)
Default benchmark settingsPi-25M
ScoreThe score is the time the calculation took
Allowed optimisations
  • Run CMD as admin and paste “bcdedit /set useplatformclock yes”
  • Enabling/Disabling HyperThreading
  • Adjust priority and affinity
  • The benchmark requires Java
Disallowed tweaks/cheats
  • Altering benchmark files or the rendering
  • Any software or human interaction altering the perceived speed of the benchmark program, tricking it to believe it ran faster
Verification screenshot requirementsThe screenshot must be obtained using the integrated screenshot functionality

  • Y-Cruncher window (with all information visible)
  • CPU-Z CPU tab
  • CPU-Z Memory tab
Verification linkNo verification link required
Verification datafileVerification datafile is mandatory
Example screenshot

Red: Mandatory Green: Optional

Additional notesThis page contains the benchmark result regulation specifications for Y-Cruncher Pi-25m. The rules as described in our General Rules section are still applicable even if they are not specifically mentioned on this page.

It is your responsibility, and the responsibility of your team captain(s) that these rules are followed properly. Your submission may be blocked by HWBOT moderators without notice if your submission is in violation of one of these rules. Multiple infractions may lead to a warning, and eventually an account ban.

Changelog
  • Sep 17, 2016: first draft

Christian Ney says:

Reserved for tweaks and tips

 

Link to support thread

Denmark mr.wulten says:

There's no good reason why the screenshot must be obtained using the integrated screenshot functionality. A screenshot is a screenshot. I don't think my score should be disqualified if the integrated screenshot functionality causes my computer to crash. As it sometimes does. And is why I always make another screenshot before using it. But whatever. I'm going to post my scores with whatever screenshot I have. If that ain't good enough .

United States Mr.Scott says:

LOL

It ain't good enough, and there is a reason.

Datafile required.

Nice attitude and first post BTW.

United States MrGenius says:

Well...there's 3 things funny about that.

1. I posted that. And my name isn't mr.wulten. Nor was it my first post. BTW...I was using an older version that didn't have the "Make Datafile" option. And trying to submit scores using "Submit to HWBOT!" was not working. I eventually got that figured. But anywho...

2. There's no need for obtaining a screenshot of any kind(using whatever functionality) when the datafile already contains one. IF the datafile saves itself properly. I keep getting datafiles that are missing the screenshot. So I try submitting them first to see if it's there. Then resave it until it is...if it isn't. Learned that lesson the hard way...:/

EDIT: Ok...now I get it. You HAVE to use the integrated screenshot functionality FIRST. Or the datafile won't have a screenshot. Which is stupid. It should take the screenshot automatically when you click "Make Datafile". "Take Screenshot" is worthless/pointless, and only adds unnecessary complexity. Everything else, not struck through, I've said below is still valid.

EDIT 2: Wait...you can point the app to whatever screenshot(via the "Screenshot Path"), obtained using whatever functionality, and have the datafile contain that screenshot when it's made. So...unstrike everything I said below...IT'S ALL VALID!!!

EDIT 3: Evidently the screenshot needs to be named using the specific numeric naming scheme. YearMonthDay-HourMinuteSecond. Might need to be a PNG image too(but I doubt it). Don't know...don't care(see EDIT 4). But you most certainly CAN use ANY PNG image with a suitable name and have it work just as well as the one obtained with the integrated screenshot functionality.

EDIT 4: It does NOT need to be a PNG image. JPG will also work(as will many other formats I presume).

3. In actuality...it's a just PNG image. One PNG image is the same as another PNG image. There's nothing special about it. Thus I reiterate...if a screenshot were to be required...THERE IS ABSOLUTELY NO REASON WHY IT MUST BE OBTAINED USING THE INTEGRATED SCREENSHOT FUNCTIONALITY. Because, if it's a full screenshot in PNG format, there's no way to tell if it was or wasn't obtained using the integrated screenshot functionality(or some other way). REGARDLESS...that clause needs removed from the rules. As it DOES NOT APPLY when the datafile is required. As the datafile SHOULD already contain the mandatory screenshot. Just having the screenshot obtained using the integrated screenshot functionality(or any other full screenshot in PNG format) is not proof enough for validation of a score. And, in fact, will get you nowhere if that's all you have to submit. Nor can it be submitted under "verification", when submitting your score via the datafile(since that isn't available when supplying the datafile...see note below). You can only add it as a "picture of your system"

Also of note...if you do submit a datafile without a screenshot...it will say "Using supplied data file. No further verification is required". And there is no place to add a "verification" screenshot(like I said...it can only be added as a "picture of your system"). So...what also needs fixed is that. If we're going to be fussy about requiring a verification screenshot. As in make sure the datafile contains the required screenshot before it can be verified. How can it not tell if there's a screenshot in the datafile or not? And/or...does it really even matter? I like the idea of requiring the screenshot. But I shouldn't be able to submit a datafile that's missing one and have it be verified. Which I've done numerous times just to see if it would work. And every time it has. Don't make me prove it...

United States Mysticial says:

To clear a few things up here from a technical standpoint.

tl;dr Version:

  • The validation file by itself (and thus datafile) is sufficient to verify that the computation happened with the associated hardware.
  • A screenshot is only useful for capturing additional information such as CPUz frequency - which itself isn't that useful anymore.

All the important information is captured in the validation file - thus a screenshot is not necessary from a technical standpoint. But if HWBOT wants to require a screenshot, that's up to them. It isn't really that useful for the purpose of validation. But it's still nice to have for illustration purposes.

Long Version:

The validation file captures the following information (among others):

  • Benchmark score
  • Hardware involved (CPU model, cores enabled, ram, etc...)
  • Computational settings
  • Program version
  • The reference clock. (TSC, HPET, etc...)

This information is protected with a hash so it cannot be tampered with without breaking the protection. This information is sufficient to verify that the computation did happen and that it wasn't faked or tampered with.

This validation file is then used to generate the datafile that is submitted to HWBOT. Embedded in the datafile is the original validation file itself. So moderators can examine it if they suspect anything.

On the other hand, the validation file does not record the following:

  • True CPU frequency

For all practical purposes, it cannot do that. This requires a kernel mode driver. So it's not practical for all but the most hard-core benchmark writers.

Thus the only thing that a screenshot can do that the validation file cannot is to capture CPUz output. But that's actually kind of useless:

  • CPU frequency fluctuates too much to be useful.
  • A screenshot is easily faked.

When you're taking the screenshot, the CPU is probably idling at maybe 1.2 GHz. Or perhaps you locked it to 5.0 GHz, but because of the AVX(512) offsets, the benchmark actually ran at 4.5 GHz instead of the 5.0 GHz that CPUz is showing. Or perhaps it's a power-throttled laptop so the frequency was fluctuating between 2.2 - 3.2 GHz for the whole benchmark. IOW, CPU frequency is becoming useless and days of low-clock benchmarks are over as they are neither practical nor enforceable.

Screenshots are easily faked.

  • You can run the computation on one computer, transfer the validation to another and submit from there with a completely different CPUz window.
  • The console window output is also easily faked by writing a small program that gives the same output - and it would be accurate to the pixel. Thus screenshots of the console output of y-cruncher are 100% useless.
  • Screenshot of the HWBOT submitter UI are also useless. Not only can those be faked, the score and timer information are already captured in the validation file + datafile. So it's not even needed.

Other Comments:

There's no point in requiring that the screenshot be taken using the built-in functionality. There is no difference at all. And it doesn't provide any additional security. It is there only for convenience.

 

Austria _mat_ says:

I hear you and you are completely right. If you think it to the end, and that's what we devs usually have to do, a screenshot is for viewing pleasure only and should have nothing to do with validation.

Did you have a look at my latest project, BenchMate? It would provide your benchmark with the necessary hardware detection like frequency, voltage, temperatures and so on. It also adds another protection layer to the benchmark process (and child processes) - specifically DLL injection for Timer API hijacking. Last but not least you would not have to bother with timer reliability issues, there are functions available by a low-level driver that give access to HPET without enabling it for the whole OS (as System Clock Source).

What language is your wrapper written in? I am aiming to provide a SDK for benchmark developers to add BenchMate's features directly into it. It will still take some time before that's finished, but there is the possibility to add integration externally (by hijacking API functions).

Let me know if you are interested.

United States Mysticial says:

45 minutes ago, _mat_ said:

I hear you and you are completely right. If you think it to the end, and that's what we devs usually have to do, a screenshot is for viewing pleasure only and should have nothing to do with validation.

Did you have a look at my latest project, BenchMate? It would provide your benchmark with the necessary hardware detection like frequency, voltage, temperatures and so on. It also adds another protection layer to the benchmark process (and child processes) - specifically DLL injection for Timer API hijacking. Last but not least you would not have to bother with timer reliability issues, there are functions available by a low-level driver that give access to HPET without enabling it for the whole OS (as System Clock Source).

What language is your wrapper written in? I am aiming to provide a SDK for benchmark developers to add BenchMate's features directly into it. It will still take some time before that's finished, but there is the possibility to add integration externally (by hijacking API functions).

Let me know if you are interested.

Hey! We haven't spoken in a while.

I'm aware of BenchMate. Nice work! But I've been way to busy to even look at it beyond reading the forums.

I'm glad someone has the time to attack this. Thank you!

 

Quote

What language is your wrapper written in? I am aiming to provide a SDK for benchmark developers to add BenchMate's features directly into it. It will still take some time before that's finished, but there is the possibility to add integration externally (by hijacking API functions).

Let me know if you are interested.

Java. But there's a possibility it will involve C++ as well depending on where it needs to interface with the benchmark.

Yeah, I'm interested. Let's start a new thread for this. I have a LOT of questions and potential ideas.

 

Austria _mat_ says:

Let's do it! :)

United States Mysticial says:

25 minutes ago, _mat_ said:

Let's do it! :)

Link me to the new thread once you've made it. And I'll start with some questions. :)

Please log in or register to comment.