Page 1 of 2

AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 2:26 pm
by MusicallyInspired
So this has been making the rounds in retro video game circles. It's been used with great effect for Doom sprite and texture upscaling and the Duke3D community has also been experimenting with it. I found a thread that featured upscaled backgrounds from Monkey Island 2 and the results are quite incredible (here's the thread with even more examples that are simply astounding). There are various models for filters available. One is waifu2x but that one makes things look only slightly better than your standard SuperEagle or hq2x filters. This one called ESRGAN (which uses a data set called Manga109 as a base) however....well, see for yourself.

Upscaled (no, seriously. this is an honest-to-god upscale without any post-editing):
Image

Original:
Image

And for the sake of comparison, here's the official MI2 Special Edition background:
Image

The results are pretty fantastic. Here's another one from MI2:
Image

Imagine seeing a filter like this generated in real-time in programs like ScummVM and DOSBox!

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 3:14 pm
by MusicallyInspired
A GIF comparison. I mean, just look at this. It's obviously not perfect but pretty darn close enough I'd say:
Image

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 4:24 pm
by MusicallyInspired
Curse of Monkey Island:
Image

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 7:21 pm
by notbobsmith
The comparison of the first two images are really quite impressive. I looks like it turned the background back into the original artwork. You can even see the brushstrokes.

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 7:54 pm
by MusicallyInspired
On the King's Quest 6 CD-ROM there were a collection of hi-res versions of backgrounds from the game in 512x365 resolution (game resolution is 320x200). Using this upscaling method I converted one to 2048x1460 and it looks phenomenal!!!

Upscaled w/ESRGAN (click to download at full res!):
Image
Hi-res scan on KQ6 CD-ROM:
Image
In-game low-res original background:
Image

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 8:46 pm
by Collector
Now if this could be done in real time in DOSBox with the pixel perfect patch...

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 8:55 pm
by MusicallyInspired
Someone in the thread mentioned it took him 8 seconds to render an image with his new RTX video card. Give that 10 years (if that) and we'll have a silky smooth 60fps framerate out of that.

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 9:15 pm
by Semi-Happy Partygoer
Wow. This is incredible. I'll have to give it a go, though I'm not sure if my PC is fully compatible. I guess I'll just have to try and see.

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 9:40 pm
by MusicallyInspired

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 9:41 pm
by MusicallyInspired
The Lord of the Dead (2048x1460):
Image

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 10:42 pm
by Collector
Love it. Well, if someone wanted to take all of the graphics from KQ6 and pre-render them to remake the game it would be great. Of course it would need to be under the radar until after the release. What is the upper resolution that AGS will deal with?

Re: AI Neural Network image upscaling filters

Posted: Tue Jan 08, 2019 10:56 pm
by Semi-Happy Partygoer
MusicallyInspired wrote: Tue Jan 08, 2019 9:40 pm Here's a good starting point
Thank you for that. This is a bit beyond anything I've tried before, but I will make some time to give it a whirl. I always like learning new techniques.

Re: AI Neural Network image upscaling filters

Posted: Wed Jan 09, 2019 5:11 pm
by Tawmis
These are amazing.

Re: AI Neural Network image upscaling filters

Posted: Wed Jan 09, 2019 11:21 pm
by Jules
Wow those are so nice! The Lord of the Dead one is awesome.

Re: AI Neural Network image upscaling filters

Posted: Thu Jan 10, 2019 11:19 am
by MusicallyInspired
I took most of yesterday and wrote a script that extracts background resources from a Sierra game, converts to PNG, and upscales with ESRGAN. I'm doing all the Sierra games and uploading them to a Google Drive folder. I'll upload more as time goes on: https://drive.google.com/drive/folders/ ... sp=sharing

For PQ4 (and some screens in PQ3) I'm going to try a different model that's more accustomed and trained to restore low resolution live photos. It's coming out more like a painting, and while that's cool, I think it can do these better.

For QFG4, all the backgrounds are in pieces with foreground objects as different picture resources entirely so I'm going to take more time with that one and stitch them all together before upscaling.