Main
Date: 21 Feb 2007 00:01:57
From: Dave (from the UK)
Subject: limits on ICC history database
If you try to obtain too many games from the history of players on ICC,
the database will respond with

"Game database temporarily unavailable."

Does anyone here know what parameters cause this to happen, and what
causes it to become available again?

I can get 100 PGNs quite quickly with something like:


aics% spgn chesscoach -3
[Event "ICC 15 0 u"]
[Site "Internet Chess Club"]
[Date "2007.02.20"]
[Round "-"]
[White "ronlad"]
[Black "ChessCoach"]
[Result "0-1"]
[ICCResult "White resigns"]
[WhiteElo "796"]
[BlackElo "1877"]
[Opening "French defense"]
[ECO "C00"]
[NIC "FR.01"]
[Time "06:07:16"]
[TimeControl "900+0"]

1. e4 e6 2. Ne2 d5 3. exd5 exd5 4. d4 Nf6 5. f3 Be7 6. h3 O-O 7. g4 Re8 8.
g5 Nh5 9. h4 Bd6 10. f4 Bxf4 11. Bxf4 Nxf4 12. Kf2 Bg4 13. Qd2 Nxe2 14. Bxe2
Rxe2+ 15. Kf1 Rxd2 16. Nxd2 Qd6 17. a3 Qf4+ 18. Ke1 Qe3+ {White resigns} 0-1


(I can repeat for various players and/or various game numbers).

But after 1000 it will respond with "Game database temporarily unavailable."

I'm trying to find out what I can do to get the histories in as short a
time as possible.

Could for example downloading them with a 5 sec gap between them allow
you to do it without the database blocking one?


--
Dave (from the UK)

Please note my email address changes periodically to avoid spam.
It is always of the form: [email protected]
Hitting reply will work for a few months only - later set it manually.

http://chessdb.sourceforge.net/ - a Free open-source Chess Database




 
Date: 22 Feb 2007 11:19:55
From: mike
Subject: Re: limits on ICC history database
I have noticed the same limitation after 80-100 game downloads. I *kind of*
understand the necessity to restrict game downloads so it won't bog down the
server -- but doing it for histories seems a little bit paranoid, especially
as few as 100.

ICC has this incredible resource -- millions of FM/IM/GM games where titled
players try out new ideas. Also a ton of games where a titled player plays
an untitled player. These kinds of games are enormously instructive for
most patzers. But YOU CAN'T GET AT THEM! ICC puts so many limits on its
search and download capabilities that this resource is virtually unusable.

Seems such a waste.





  
Date: 23 Feb 2007 01:15:20
From: Dave (from the UK)
Subject: Re: limits on ICC history database
mike wrote:
> I have noticed the same limitation after 80-100 game downloads. I *kind of*
> understand the necessity to restrict game downloads so it won't bog down the
> server -- but doing it for histories seems a little bit paranoid, especially
> as few as 100.

Sorry, it was games in their history. I've never tried requesting a lot
of histories. I'm using 'spgn' to download a specific player. I thought
the limit was 100.

Do you know anything about what happens after than 100? For example, if
you wait 5 mins, can you get another hundred quickly? I've found that
changing the rate to about 1 every 70 seconds seems to be OK, but that
is a lot slower than the 1/second or so before the 100 is reached.


> ICC has this incredible resource -- millions of FM/IM/GM games where titled
> players try out new ideas. Also a ton of games where a titled player plays
> an untitled player. These kinds of games are enormously instructive for
> most patzers. But YOU CAN'T GET AT THEM! ICC puts so many limits on its
> search and download capabilities that this resource is virtually unusable.

Also, I think there is a lot to be said for getting them from weaker
players too. There are plenty of resources for GM games, but less so for
weaker players.

A version of ChessDB

http://chessdb.sourceforge.net/

I hope to release in the next week has the ability to download from
specific players from ICC, so can be used for any player - not just GMs.
It will put them directly into the database (same format as Shane's
Scid). The code works - screenshot here:

http://www.althorne.org/chessdb/website2/tutorial/img/icc-fics-window.png

but I want to make the downloads a bit more robust before releasing a
version.

Interested parties can check out the database with CVS. Doing so on a
UNIX/Linux box should be as easy as


$ cvs -z3
-d:pserver:[email protected]:/cvsroot/chessdb co -P
chessdb

$ cd chessdb
$ ./configure
$ make
$ su
# make install

Then this is on the Tools - > Downloads -> ICC

> Seems such a waste.

Yes, like you I feel that better searches and more sensible limits would
be useful.


--
Dave (from the UK)

Please note my email address changes periodically to avoid spam.
It is always of the form: [email protected]
Hitting reply will work for a few months only - later set it manually.

http://chessdb.sourceforge.net/ - a Free open-source Chess Database


   
Date: 24 Feb 2007 10:01:17
From: mike
Subject: Re: limits on ICC history database
> Do you know anything about what happens after than 100? For example, if
> you wait 5 mins, can you get another hundred quickly? I've found that
> changing the rate to about 1 every 70 seconds seems to be OK, but that is
> a lot slower than the 1/second or so before the 100 is reached.
>
>

I don't know what the formula is, but it takes longer than 5 minutes to
reset. More like 20 or 30 minutes.




 
Date: 21 Feb 2007 02:08:04
From: Dave (from the UK)
Subject: Re: limits on ICC history database
Dave (from the UK) wrote:

> I can get 100 PGNs quite quickly with something like:

> But after 1000 it will respond with "Game database temporarily
> unavailable."


It is after 100 the database restricts access, not after 1000 as my
message said at one point.



--
Dave (from the UK)

Please note my email address changes periodically to avoid spam.
It is always of the form: [email protected]
Hitting reply will work for a few months only - later set it manually.

http://chessdb.sourceforge.net/ - a Free open-source Chess Database


  
Date: 21 Feb 2007 13:26:40
From: Thomas T. Veldhouse
Subject: Re: limits on ICC history database
"Dave (from the UK)" <[email protected] > wrote:
> Dave (from the UK) wrote:
>
>> I can get 100 PGNs quite quickly with something like:
>
>> But after 1000 it will respond with "Game database temporarily
>> unavailable."
>
>
> It is after 100 the database restricts access, not after 1000 as my
> message said at one point.
>

What are you doing, mining history? Why not send your question off to ICC and
ask them?

--
Thomas T. Veldhouse
Key Fingerprint: D281 77A5 63EE 82C5 5E68 00E4 7868 0ADC 4EFB 39F0