| |
Main
Date: 19 Sep 2008 22:18:24
From: Guest
Subject: Random number mobility scores
|
Some time back it was mentioned that Hyatt had mentioned a test by Don Beal
on random mobility scores for a material only program.
I finally got curious enough to dig out an old chess program and try it.
The program is crap. It is a mailbox program that was going through a lot
of modification when I packed it away.
But it seems to be working, and it has enough Winboard ability that works
for me to run the Win-At-Chess test through it.
The program used only material and a random number generator as a mobility
score. No piece tables, no pawn scoring, etc. Just material and a random
number generator from Numerical Recipes (I didn't want to use the one that
comes with the compiler, of course.) I did a random number of 128 and put
half negative and half positive. So about a half a pawn worth of mobility
balance.
The move ordering was done MVV/LVA with History Heuristic. No SEE move
ordering.
The capture search does only captures, and all of them. No checks or
promotions, etc. are included.
There are no search extensions like Null Move, check extensions, etc.
There are no pruning methods beyond alpha-beta. No futility moves, no LMR,
etc. etc.
No transposition tables.
Pretty much a basic, niave alpha-beta chess program with an eval of only
material and random numbers. I didn't even bother to compile it with
optimization enabled. I left it in debug mode.
I used epd2wb and ran the Win-At-Chess test through my program, giving it 10
seconds to think. I used a 2ghz laptop. Only one core was used.
(Note: This is the original WAC test, not the revised / fixed version.)
I got +203 / -77 = 67%. The later ones were where most of the failures
occured.
As another test, I ran WAC through the same program without the random
number generator. Scoring only material. I got +197 / -103 = 65%
As a lark, I also tried it with the original evaluator I had in the program.
This eval is junk. It was stuff I had cut & pasted from some other simple
public domain programs. No effort was made to tune it. I ran very few
tests with it. I just added stuff so I had an eval that did more than
material.
It got +197 / -103 = 65%... (grin) Pretty bad....
I then decided to try the WAC tests again, this time without the mates. The
idea being the search itself is finding the mates, not the evaluator.
Random: +148 / -80=64%.
Material: +148 / -80=64%
Odd. Maybe the mobility helped it find a few mates? Or at least picked the
right move for the wrong reason? Or maybe it provided enough extra guidence
in the search that it was able to go to the next ply and see the mate?
I also tried the Bratko-Kopec test.
With the random one. It got a total of 11 right. That gives a BK rating of
2000-2199. (And no, Sanny, that's not ELO, even though it's supposed to
be.)
For the Material only version, it got only 8 right. That's a 1800-1999 BK
Rating.
The one with the old evaluator, it got a score of 16.5, which is around
2400. (The score table doesn't handle fractional points. For 16, it'd be
2000-2399 and for 17 it'd be 2400+. So around 2400.)
Finally, note that these results are just one test. Due to the nature of
random numbers, a different see could conceivably cause all of them to give
the right answer or wrong answer.
But it does appear that a random number generator mobility score is at least
as good as plain material and in some cases better.
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
|
| |
Date: 23 Sep 2008 15:28:09
From: David Richerby
Subject: Re: Random number mobility scores
|
Guest <[email protected] > wrote:
> [By using an evaluator that returned material score plus a random
> number in the range +/- 50 centipawns,] I got +203 / -77 = 67%. The
> later ones were where most of the failures occured.
>
> As another test, I ran WAC through the same program without the random
> number generator. Scoring only material. I got +197 / -103 = 65%
This doesn't really surprise me. As has been mentioned, minimax on
the randomly perturbed scores approximates mobility. But mobility
itself approximates material, since the very reason that a queen is
more powerful than a knight is that it has more mobility in typical
positions. So, your two tests are, on the one hand, material and, on
the other hand, material plus something that approximates material.
Now, you'd expect material plus mobility to do slightly better than
just material (which it does) because mobility factors in how well the
available material can be deployed.
So, I'm not surprised by your results. But it's really good that
somebody actually sat down and did the experiment. Thank you.
Dave.
--
David Richerby Psychotic Voodoo Cheese (TM):
www.chiark.greenend.org.uk/~davidr/ it's like a lump of cheese that
has mystical powers but it wants to
kill you!
|
| |
Date: 22 Sep 2008 13:18:31
From: Kenneth Sloan
Subject: Re: Random number mobility scores
|
The more interesting test is: actual mobility - with no other term,
not even material.
--
Kenneth Sloan [email protected]
Computer and Information Sciences +1-205-932-2213
University of Alabama at Birmingham FAX +1-205-934-5473
Birmingham, AL 35294-1170 http://KennethRSloan.com/
|
| | |
Date: 23 Sep 2008 13:21:11
From: Simon Krahnke
Subject: Re: Random number mobility scores
|
* Guest <[email protected] > (06:02) schrieb:
> "Simon Krahnke" <[email protected]> wrote in message
> news:[email protected]...
>>* Guest <[email protected]> (03:48) schrieb:
>>
>>>> Compare
>>>>
>>>> rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - KkQq 0 1
>>>>
>>>> with
>>>>
>>>> 1n2k1n1/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - - 0 1
>>>
>>> Compare them how?
>>
>> Well, using the pure mobility scoring Kenneth supposed.
>>
>> In the second position white has two more possible moves than black,
>> giving it a better score than the first position, where in reality the
>> first position is far better in white's perspective.
>
> I don't think it would show anything.
>
> It may have a different score, but that would be based on the search from
> that point.
>
> So whatever score or 'best move' would be based on the starting position,
> rather than comparing the two.
>
> And comparing the scores or best moves among the programs for the two
> positions wouldn't really show a lot since there's nothing to judge their
> result by. No 'best move to make' or 'move to avoid', etc.
That might not be the best example positions, but they are easy to come
up with. Surely there are lots of real positions where one or two rooks
have little or no mobility and changing that is no immediate concern.
Your tests also show that taking out material from the equation does no
good.
mfg, simon .... l
|
| | |
Date: 23 Sep 2008 04:25:55
From: Simon Krahnke
Subject: Re: Random number mobility scores
|
* Guest <[email protected] > (03:48) schrieb:
> "Simon Krahnke" <[email protected]> wrote in message
> news:[email protected]...
>>* Kenneth Sloan <[email protected]> (20:18) schrieb:
>>
>>> The more interesting test is: actual mobility - with no other term, not
>>> even material.
>>
>> Compare
>>
>> rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - KkQq 0 1
>>
>> with
>>
>> 1n2k1n1/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - - 0 1
>
> Compare them how?
Well, using the pure mobility scoring Kenneth supposed.
In the second position white has two more possible moves than black,
giving it a better score than the first position, where in reality the
first position is far better in white's perspective.
mfg, simon .... beware of errors in the handcrafted fen
|
| | | |
Date: 22 Sep 2008 23:02:50
From: Guest
Subject: Re: Random number mobility scores
|
"Simon Krahnke" <[email protected] > wrote in message
news:[email protected]...
>* Guest <[email protected]> (03:48) schrieb:
>
>>> Compare
>>>
>>> rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - KkQq 0 1
>>>
>>> with
>>>
>>> 1n2k1n1/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - - 0 1
>>
>> Compare them how?
>
> Well, using the pure mobility scoring Kenneth supposed.
>
> In the second position white has two more possible moves than black,
> giving it a better score than the first position, where in reality the
> first position is far better in white's perspective.
I don't think it would show anything.
It may have a different score, but that would be based on the search from
that point.
So whatever score or 'best move' would be based on the starting position,
rather than comparing the two.
And comparing the scores or best moves among the programs for the two
positions wouldn't really show a lot since there's nothing to judge their
result by. No 'best move to make' or 'move to avoid', etc.
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
| | |
Date: 22 Sep 2008 21:25:26
From: Simon Krahnke
Subject: Re: Random number mobility scores
|
* Kenneth Sloan <[email protected] > (20:18) schrieb:
> The more interesting test is: actual mobility - with no other term, not
> even material.
Compare
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - KkQq 0 1
with
1n2k1n1/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - - 0 1
mfg, simon .... l
|
| | | |
Date: 22 Sep 2008 20:48:53
From: Guest
Subject: Re: Random number mobility scores
|
"Simon Krahnke" <[email protected] > wrote in message
news:[email protected]...
>* Kenneth Sloan <[email protected]> (20:18) schrieb:
>
>> The more interesting test is: actual mobility - with no other term, not
>> even material.
>
> Compare
>
> rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - KkQq 0 1
>
> with
>
> 1n2k1n1/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - - 0 1
Compare them how?
The score returned?
Do a tournament of the different mobility versions using thos positions as a
starter? Too much trouble.
>
> mfg, simon .... l
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
| | |
Date: 22 Sep 2008 14:13:30
From: Guest
Subject: Re: Random number mobility scores
|
"Kenneth Sloan" <[email protected] > wrote in message
news:[email protected]...
> The more interesting test is: actual mobility - with no other term, not
> even material.
I said I wasn't going to run any more tests....
But alright. That's somewhat interesting.
I'll rerun some of the tests with the material removed, leaving just the
various forms of mobility.
It'll take a day or so to get around to it.
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
| | | |
Date: 22 Sep 2008 15:16:00
From: Kenneth Sloan
Subject: Re: Random number mobility scores
|
Guest wrote:
> "Kenneth Sloan" <[email protected]> wrote in message
> news:[email protected]...
>> The more interesting test is: actual mobility - with no other term, not
>> even material.
>
> I said I wasn't going to run any more tests....
>
> But alright. That's somewhat interesting.
>
> I'll rerun some of the tests with the material removed, leaving just the
> various forms of mobility.
>
> It'll take a day or so to get around to it.
>
Also - it's worth noting that a random term (don't know why you want to
*call* it "mobility") is, in practice, a good *indirect* measure of
actual mobility (back one ply).
--
Kenneth Sloan [email protected]
Computer and Information Sciences +1-205-932-2213
University of Alabama at Birmingham FAX +1-205-934-5473
Birmingham, AL 35294-1170 http://KennethRSloan.com/
|
| | | | |
Date: 22 Sep 2008 20:49:33
From: Guest
Subject: Re: Random number mobility scores
|
"Kenneth Sloan" <[email protected] > wrote in message
news:[email protected]...
> Guest wrote:
>> "Kenneth Sloan" <[email protected]> wrote in message
>> news:[email protected]...
>>> The more interesting test is: actual mobility - with no other term, not
>>> even material.
>>
>> I said I wasn't going to run any more tests....
>>
>> But alright. That's somewhat interesting.
>>
>> I'll rerun some of the tests with the material removed, leaving just the
>> various forms of mobility.
>>
>> It'll take a day or so to get around to it.
>>
>
> Also - it's worth noting that a random term (don't know why you want to
> *call* it "mobility") is, in practice, a good *indirect* measure of actual
> mobility (back one ply).
Right. That's why I simplify and call it 'mobility'.
The random number is taking the place of the mobility score. As far as the
sarch is concerned, it really is the mobility score. Hence, random mobility
score.
For this discussion, we all know what's being tested.
Although it's only 'accurate' at the ply above, rather than the current ply,
the closer to root, the greater the chance of it being closer to being
accurate. I don't know what the math would be to calculate how many plies
would be needed for it to be close enough as to make no statistical
difference. Figuring 35 moves per position.... (frown) I'd say at least 8
plies.
So I'm nto sure I'd call it "a good *indirect* measure of actual mobility".
Better than expected, tolerable, etc. But not 'good'.
What it does have going for it is that it's faster than a regular mobility
evaluation, and therefor you can search a little deeper. My 10 second tests
aren't super deep, but a longer 3 minute search would get better results.
Speaking of which, I wonder how much of the BK test is influenced by the
deeper search, over the 10 second search I'm doing for the WAC tests.
These tests aren't meant to be super accurate, either. I'm not doing nearly
enough tests, and for the random mobility, I'd have to try various ranges of
random numbers, etc. (Doing the -64..+63 is likely too large. A smaller
number would be more realistic, but I didn't think about that when I ran
that first test. I was thinking more about making sure I was getting enough
randomness in it to have a noticable effect.)
I'm not interested enough to redo the tests. It's just a curiosity.
I'm not actually planning on using a random number generator in my program.
This was just a novelty.
>
> --
> Kenneth Sloan [email protected]
> Computer and Information Sciences +1-205-932-2213
> University of Alabama at Birmingham FAX +1-205-934-5473
> Birmingham, AL 35294-1170 http://KennethRSloan.com/
>
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
| |
Date: 20 Sep 2008 12:21:43
From: Guest
Subject: Re: Random number mobility scores
|
I have some more results for the random mobility score.
I was originally interested in how it compared to just material alone, but
then I wondered how it compared to real mobility scoring.
Here are the program versions. Same junk program as before.
Mtl is material scoring only.
Rand: Addition of -64..+63 random value to simulate mobility. I just lopped
off some bits and made half negative. The asymetry might cause some issues,
but I don't care enough to repeat it with balanced results or a reduced
range.
Eval: The original eval in the program. Primative. A little pawn
knowledge, a little mobility, queen/king closeness, a tiny bit of king info.
Just stuff hacked together, mostly
Mob: Material and two calls to GenAllMoves(Side).
Mob2: Same as "Mob" but I don't bother to store the moves, pawn promotions
are countd as 1 instead of 4, and I don't do castling. Faster.
PMob: Material and pawn based mobility. The mobility the sliders would have
if there were only pawns on the board. Only the sliders are scored. (This
was alread in the full eval, so I just cut out the unrelated stuff.)
Bratko-Kopec test. (2=1/2 4=1/4)
Mtl : 100000100001011001001100 = 8 = 1800-1999
Rand: 100100100101011100101001 = 11 = 2000-2199
Mob : 100100110001011001101000 = 10 = 2000-2199
Mob2: 100110140001011001001000 = 9.25= 2000-2199
PMob: 101110110001111101101001 = 15 = 2200-2399
Eval: 101111110201011111101100 = 16.5 = 2400'ish
Tact: T---T-T--T-T-TTT-TT-TT--
Pos : -PPP-P-PP-P-P---P--P--PP
(The BK test are 12 tactical & 12 positional / Levers, as indicated above.)
Tactical Bias = 12*(T-L)/S
Tact: T---T-T--T-T-TTT-TT-TT--
Mtl : 100000100001011001001100 = 8 Bias = 12
Rand: 100100100101011100101001 = 9 Bias = 7.6
Mob : 100100110001011001101000 = 8 Bias = 7.2
Mob2: 100110140001011001001000 = 8 Bias = 8.75
PMob: 101110110001111101101001 = 11 Bias = 5.6
Eval: 101111110501011111101100 = 11.5 Bias = 4.7
Tact: T---T-T--T-T-TTT-TT-TT--
So we can see that just material is entirely matierially biased. No
surprise.
Adding any form of mobility improves the bias. It's still very materially
biased, but it helps.
It is a little odd that Mob2 did worse than Mob. It could be sheer
statistical randomness, though. More tests might have different results.
Classic Win-At-Chess test.
Mtl +197 / -103 = 65%
Rand +203 / -77 = 67%
Mob +179 / -121 = 59%
Mob2 +189 / -111 = 63%
PMob +194 / -106 = 64%
Eval +197 / -103 = 65%
The WAC tests are very tactically oriented. Very simple tests, too.
In this case, the random mobility provides just a little guidence with
minimal impact on the speed of the search.
A better quiscient search would probably help these tests, regardless of the
eval.
So how do I interpret the results...
Well, the mobility ones and the full eval do run slower. WAC is very
tactical so a slow prgram is at a little disadvantage. Mine is particularly
slow (compared to modern programs), and it's dumb (by nearly any standard),
so anything that slows down the program is going to be at a disadvantage for
a tactically niave / slow program.
Hence, the material only does pretty well. And the random one does pretty
well too because it provides a tiny bit of smarts to the program at very
little cost.
The BK positions are a combination of tactics and positional. In this case,
the full evaluator makes up for its slowness by being smart enough to find
the right move anyway.
Even mobility, in any form, helps in the BK tests. Although they do slow
down the program, it helps over all.
So, my final conclusion is this....
Random mobility is a good thing if you are doing just material. It's quick,
it's easy to add, and it actually helps some. But it's no substitute for
real mobility scoring, especially if you can do that fast, rather than just
doing calls to GenAllMoves().
It looks like pawn based mobility is a good compromise. Quick to generate
but decent results. If you don't have time to program that.... then go for
random mobility.
And of course, your results may vary depending on the random generator you
use, the speed of the program, the number of tests you do, etc. etc.
I have no plans to run more tests to see if piece square tables would
benefit from random moblity, etc.
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
| | |
Date: 22 Sep 2008 20:48:16
From: Guest
Subject: Re: Random number mobility scores
|
At Kenneth's suggestion I've run some more tests. He wondered how things
would perform without material balance. Just mobility.
The program is the same as the one used before. Primitive alpha-beta,
MVV/LVA & History Heuristic, capture search only, no trans table, simple
eval. Possibly buggy, even. Same 2ghz computer.
The WAC (old version, not new version) is done at 10 seconds per position.
The Bratko-Kopec test is done for 2 minutes, as prescribed.
Here are the program versions.
Mtl is material scoring only.
Rand: Addition of -64..+63 random value to simulate mobility. I just lopped
off some bits and made half negative. The asymetry might cause some issues,
but I don't care enough to repeat it with balanced results or a reduced
range.
Eval: The original eval in the program. Primative. A little pawn
knowledge, a little mobility, queen/king closeness, a tiny bit of king info.
Just stuff hacked together, mostly
Mob: Material and two calls to GenAllMoves(Side).
Mob2: Same as "Mob" but I don't bother to store the moves, pawn promotions
are countd as 1 instead of 4, and I don't do castling. Faster.
PMob: Material and pawn based mobility. The mobility the sliders would have
if there were only pawns on the board. Only the sliders are scored. (This
was alread in the full eval, so I just cut out the unrelated stuff.)
Kenneth suggested the following tests, without the material score.
0eval: The original evaluator but with the material removed. For
reference.
0mob: Two calls to GenAllMoves(Side). No material score.
0mob2: faster, slightly modified GenAllMoves, but no material score.
0pmob: Pawn based mobility. No material score.
0rand: random number mobility score. No material score.
I also ran:
eval-NoMob: The original evaluator, but with the mobility (pawn based
mobility for the bishop & rook) removed. Just to test how much the P-Mob
helps the evaluator.
Bratko-Kopec test. (2=1/2 3=1/3 4=1/4)
Mtl : 100000100001011001001100 = 8 = 1800-1999
Rand : 100100100101011100101001 = 11 = 2000-2199
Mob : 100100110001011001101000 = 10 = 2000-2199
Mob2 : 100110140001011001001000 = 9.25= 2000-2199
PMob : 101110110001111101101001 = 15 = 2200-2399
Eval : 101111110201011111101100 = 16.5 = 2400'ish
0rand: 100110000001000000001000 = 5 = 1600-1799
0mob : 100001010011000100000000 = 6 = 1600-1799
0mob2: 100004000011000000004000 = 3.5 = 1300-1599
0pmob: 101101010000000000001001 = 7 = 1800-1999
0eval: 101011010301000000000000 = 6.3 = 1700-1800
EvNoM: 101111110101011100101100 = 15 = 2200-2399
111111111122222
123456789012345678901234
Tact : T---T-T--T-T-TTT-TT-TT--
Pos : -PPP-P-PP-P-P---P--P--PP
(The BK test are 12 tactical & 12 positional / Levers, as indicated above.)
Tactical Bias = 12*(T-L)/S
Tact : T---T-T--T-T-TTT-TT-TT--
Mtl : 100000100001011001001100 = 8 Bias = 12
Eval : 101111110501011111101100 = 11.5 Bias = 4.7
EvNoM: 101111110101011100101100 = 11 Bias = 5.6
0eval: 101011010301000000000000 = 3.3 Bias = 0.5
Rand : 100100100101011100101001 = 9 Bias = 7.6
Mob : 100100110001011001101000 = 8 Bias = 7.2
Mob2 : 100110140001011001001000 = 8 Bias = 8.75
PMob : 101110110001111101101001 = 11 Bias = 5.6
Tact : T---T-T--T-T-TTT-TT-TT--
0rand: 100110000001000000001000 = 4 Bias = 7.2
0mob : 100001010011000100000000 = 3 Bias = 0.0
0mob2: 100004000011000000004000 = 2.25 Bias = 3.4
0pmob: 101101010000000000001001 = 2 Bias = -5.1
Tact : T---T-T--T-T-TTT-TT-TT--
The 0pmob (pawn based slider mobility only, no material) gets a bias of -5.1
which says it's pretty positionally oriented, rather than tactially.)
Classic Win-At-Chess test.
Mtl +197 / -103 = 65%
Rand +203 / -77 = 67%
Mob +179 / -121 = 59%
Mob2 +189 / -111 = 63%
PMob +194 / -106 = 64%
Eval +197 / -103 = 65%
0rand +133 / -167 = 44%
0mob +132 / -168 = 44%
0mob2 +141 / -159 = 47%
0pmob +155 / -145 = 51%
0eval +131 / -169 = 43%
evNoMob +205 / -95 = 68%
For the BK tests, it's obvious that a lack of material makes a significant
difference. In spite of that, the full eval manages to do pretty well.
And pawn based mobility still does better than common pseudo-legal move
mobility.
For the WAC, all of them are also effected by lack of material. Not too
surprising considering WAC is mostly tactics.
Again, no surprises there. Unless you consider that they did as well as
they did without any material balance....
The eval with no mobility got a higher WAC score than with mobility. Could
be the eval is smart enough it doesn't matter much, or it could be the
mobility scoring is taking enough extra time that it's slowing the search.
Without the extra cost, it can do better tacticall. If that supposition is
true, a better, smarter Q-Search would likely be an advantage Considering
the performance in the BK test, I suspect a better Q-search is what's
needed.
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
| |
Date: 20 Sep 2008 17:28:15
From: Simon Krahnke
Subject: Re: Random number mobility scores
|
* Guest <[email protected] > (05:18) schrieb:
> Some time back it was mentioned that Hyatt had mentioned a test by Don Beal
> on random mobility scores for a material only program.
>
> I finally got curious enough to dig out an old chess program and try it.
Cool!
> The program used only material and a random number generator as a mobility
> score. No piece tables, no pawn scoring, etc. Just material and a random
> number generator from Numerical Recipes (I didn't want to use the one that
> comes with the compiler, of course.) [...]
Of what kind is that random generator? A Mersenne Twister?
> Pretty much a basic, niave alpha-beta chess program with an eval of only
> material and random numbers. I didn't even bother to compile it with
> optimization enabled. I left it in debug mode.
>
>
> I used epd2wb and ran the Win-At-Chess test through my program, giving it 10
> seconds to think. I used a 2ghz laptop. Only one core was used.
>
> (Note: This is the original WAC test, not the revised / fixed version.)
Which one did you post here?
> I got +203 / -77 = 67%. The later ones were where most of the failures
> occured.
>
> As another test, I ran WAC through the same program without the random
> number generator. Scoring only material. I got +197 / -103 = 65%
>
>
> As a lark, I also tried it with the original evaluator I had in the program.
> This eval is junk. It was stuff I had cut & pasted from some other simple
> public domain programs. No effort was made to tune it. I ran very few
> tests with it. I just added stuff so I had an eval that did more than
> material.
>
> It got +197 / -103 = 65%... (grin) Pretty bad....
It would be interesting to know if all they solved the same 197
positions. Do you have detailed logs?
> I then decided to try the WAC tests again, this time without the mates. The
> idea being the search itself is finding the mates, not the evaluator.
The evaluator should help. :-)
> Random: +148 / -80=64%.
>
> Material: +148 / -80=64%
>
> Odd. Maybe the mobility helped it find a few mates? Or at least picked the
> right move for the wrong reason? Or maybe it provided enough extra guidence
> in the search that it was able to go to the next ply and see the mate?
Answers a log could give.
Finalfun scored 188 in 5s each on a Pentium 4 at 3.4GHz. It must be
doing something right. Transposition table?
> But it does appear that a random number generator mobility score is at least
> as good as plain material and in some cases better.
Thanks for testing.
mfg, simon .... l
|
| | |
Date: 21 Sep 2008 17:54:06
From: Simon Krahnke
Subject: Re: Random number mobility scores
|
* Guest <[email protected] > (19:17) schrieb:
> "Simon Krahnke" <[email protected]> wrote in message
> news:[email protected]...
>>* Guest <[email protected]> (05:18) schrieb:
>>
>> Of what kind is that random generator? A Mersenne Twister?
>
> Numerical Recipes 3rd ed., RanLim32. It's the only 32 bit random generator
> they recommend and provide code for.
Well, that's an investment. 50.95 Euros at Amazon.
>> Finalfun scored 188 in 5s each on a Pentium 4 at 3.4GHz. It must be
>> doing something right. Transposition table?
>
> That could certainly be it.
>
> This program has no trans table at all, and poor move ordering.
The best move from the TT is all the move ordering Finalfun has. I never
got to implement MVV/LVA.
> It's entirely possible that if you ran these tests, you'd get very different
> results than what I do.
Let's see if I can get my new engine to actually generate moves. Right
now, it knows what a board is, and that there are piece on it. That's
not really much. I'm kind of working on preparation for chess960
compatible castling now.
mfg, simon .... slow pace
|
| | | |
Date: 21 Sep 2008 11:40:08
From: Guest
Subject: Re: Random number mobility scores
|
"Simon Krahnke" <[email protected] > wrote in message
news:[email protected]...
>* Guest <[email protected]> (19:17) schrieb:
>
>> Numerical Recipes 3rd ed., RanLim32. It's the only 32 bit random
>> generator
>> they recommend and provide code for.
>
> Well, that's an investment. 50.95 Euros at Amazon.
You can read it online for free on their website. They make it available as
PDFs.
They don't provide seperate code to download, but it's only a few lines to
type in.
Plus, if you look hard enough, you can find places online where people have
already typed it in for you.
I did some random number testing some time back, and it's actually pretty
hard to find a good random number generator even for things like the hash
keys.
A lot of the simplier ones on the web have very poor hamming distances for
the hash keys, even if they are okay for normal random numbers.
On the other hand, even some of the classic simple ones can do okay for hash
keys if you do things right. They aren't random in the mathematical sense,
but the hamming distances can be good enough for a hash table.
I think many people just end up using Hyatt's method. You make a few
throw-away calls until you get to the point where you get the best hamming
distances for your hash table.
Hyatt says it's based on one in NumRec, which was based on one in Knuth, but
I haven't seen the sources. The only thing I don't like about it is it uses
precomputed data.
>>> Finalfun scored 188 in 5s each on a Pentium 4 at 3.4GHz. It must be
>>> doing something right. Transposition table?
>>
>> That could certainly be it.
>>
>> This program has no trans table at all, and poor move ordering.
>
> The best move from the TT is all the move ordering Finalfun has. I never
> got to implement MVV/LVA.
Wow.
Most people do MVV/LVA & history heuristic long before they do a trans
table.
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
| | |
Date: 20 Sep 2008 12:17:27
From: Guest
Subject: Re: Random number mobility scores
|
"Simon Krahnke" <[email protected] > wrote in message
news:[email protected]...
>* Guest <[email protected]> (05:18) schrieb:
>
> Of what kind is that random generator? A Mersenne Twister?
Numerical Recipes 3rd ed., RanLim32. It's the only 32 bit random generator
they recommend and provide code for.
I could have done a 64 bit using long long, but I wanted to keep the program
in 32 bit clean.
>> (Note: This is the original WAC test, not the revised / fixed version.)
>
> Which one did you post here?
I posted the original one.
When I ran my tests, that was handy and I wasn't expecting to run so many
tests.
Plus, some of those 'corrected' moves actually take quite a bit of effort.
Not much chance my program would get them right anyway.
>> As a lark, I also tried it with the original evaluator I had in the
>> program.
>> This eval is junk. It was stuff I had cut & pasted from some other
>> simple
>>
>> It got +197 / -103 = 65%... (grin) Pretty bad....
>
> It would be interesting to know if all they solved the same 197
> positions. Do you have detailed logs?
Yeah. I dumped the results of each test to seperate files.
But I don't really have the urge to look into them. I was just curious
about the end results, not in tuning my evaluator.
My guess is the poor results are due to a slow down in the program and a
poor Q-search. It's doing just a capture search.
And there could even be bugs in the program. It was a program 'in
transition'. It had never really been finished and had been in a constant
state of change.
> Finalfun scored 188 in 5s each on a Pentium 4 at 3.4GHz. It must be
> doing something right. Transposition table?
That could certainly be it.
This program has no trans table at all, and poor move ordering.
It's entirely possible that if you ran these tests, you'd get very different
results than what I do.
----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= - Total Privacy via Encryption =---
|
|
