Some quick spot checks (ie picking a few random spots to look at in the manifest) , make it abundantly clear that the issue is the MD5 chunk data, which is causing the huge manifests.

I'm not sure what to do about that, it's not possible to spit it out to a downloading directory at update time because you don't know where that'll be, but there's definitely people that want it.

Maybe emitting it directly into a MD5 folder in the route dir, is the best way to handle it ? Gogrepoc itself never reads this data (and has no reason to , there's no actual point in doing offline MD5 chunk comparison when you'd need to be online to do the repairs anyway), so it'd just be a matter of writing it and deleting any outdated ones when updates are done.

Also making it optional I suppose.

Umm.. if it's temporary data, shouldn't the files be saved in a temporary folder and not in the actual installers archive?
Or maybe make it a subfolder of "!downloading"?

They aren't really "temporary" files more like information that some people want.

GOGRepoC does *use* this information but it always fetches a fresh copy whenever needed (because the primary use cases is either allowing download resumes or repairing a file that failed it's MD5 check but in either case it's pointless attempting to do with data that's out of sync with what's on the GOG servers) so there's no need to save it for functionality purposes.

They can't go in !downloading because !downloading is a subfolder of the install directory which you don't know during an update command. I also don't really think they belong there, since these are more info provided for someone who wants it (like the !info.txt) whereas !downloading is more for stuff that gogrepo is doing.

Okay, I've stopped it saving the MD5 chunk data for now, while I think about how to handle this better.

And here's an in progress version for testing for those who want to.

It won't work on anything except Python 3.11 or greater, which is why I'm not uploading it to the github dev branch.

Please do not try using it with Python < 3.11 (or Python 2.7) because it does absolutely no checking at the moment because I intend to move to dateutils for python <3.11 before I release this for realsies, so what it will do in that scenario is unknown to me but probably involves turning your manifest into hot garbage.

Also make sure any resumes in progress are finished before you use this, it'll delete old resume manifests for incompatibility when it encounters them. I'll implement a nicer thing that just asks if you want to abort and use an older version to finish the in process resumes or trash it when it detects this scenario.

That said:
https://www.dropbox.com/scl/fi/3vjpklrehs89994399epu/gogrepoc.py?rlkey=mtixrng5up32gim9izo18hlbt&amp;dl=0

I updated my Python to 3.12.3, doing a fresh install.
I think I installed everything that's needed as per the instructions here: https://github.com/Kalanyr/gogrepoc/tree/dev .
(html5lib, requests, pyOpenSSL, html2text, caffeinate)

I ran the script as a test, and got 504 errors for all updated games.
Log link: https://pastebin.com/4JFHHcDL

Did I miss anything, or is it just a hiccup on GOG's side?

(I think my gogrepoc version is up-to-date with a github one, the gogrepoc.py file shows its last modification date as 26th November 2023.)

504 is a server error, and in theory indicates that the (server side) proxy you're connected to timed out trying to reach a server upstream of it.

In practice GOG sometimes fucks up their response codes so I'd check what happens if you use the login command again to try and refresh your token.

I recreated the token, and it seems to be working, thanks. :)

@Kalanyr
I tried the new WIP-Version, so far I didn't stumble across any problems. But I didn't test everything.

Slugify and unidecode are new hard requirements?

Pardon me if it was already explained elswhere: For what are the !images folder? They're all empty in my case.

Keep up your good work!

The Slugify function is actually include in gogrepoc since it (the slugify funciton) is BSD licensed, decode and encode should be built in's of the unicode string and thus native to Python 3 but it's possible that the implementation here doesn't work on python 2.7, Could you let me know what the error is and what your python version is ? I'll dig up the ppython 2.7 slugify code path and include that if that's the problem.

!images only get's filled in if you use the -covers and/or -backgrounds flags on the download command, in which case it pulls down the appropriate image files and stores them in there so you can use them for whatever. It's off by default since it can be a pretty slow process on the first run if you have a large collection of games.

I used Python 3.12.3 for the tests. Slugify does indeed work as intended. My fault, I mixed it up with dateutil ;)